UniLACT is a Vision-Language-Action model that incorporates geometric structure through depth-aware latent pretraining, enabling downstream robot policies to inherit stronger spatial priors.

We introduce Vision Contextualized Probing (VisCoP), which augments the VLM's vision encoder with a compact set of learnable visual probes that enables efficient domain-specific adaptation with minimal modification to pretrained parameters.

We leverage the complementary nature of egocentric views to enhance LVLM’s understanding of exocentric ADL videos through online ego2exo distillation.

MS-Temba is the first Mamba based architecture for action detection in long untrimmed videos that can be trained/tested on NVIDIA Jetson Nano.

TICON is an Omni Tile Contextualizer that can contextualize embeddings from any tile encoder.

CORA is a semi-supervised reasoning segmentation framework that jointly learns from limited labeled data and a large corpus of unlabeled images.

Gigapixel Vision-Concept Knowledge Contrastive pretraining (GECKO) aligns WSIs with a Concept Prior for delivering clinically meaningful interpretability.

A novel generative masked model for hand mesh recovery that synthesizes plausible 3D hand meshes.

LLAVIDAL, a Large Language Vision Model, incorporates 3D poses and relevant object trajectories to understand the intricate spatiotemporal relationships within ADLs.

Ski-models introduce 3D skeletons into the vision-language embedding space to enable effective zeroshot learning for ADL.

A generative framework that reformulates monocular HMR as an image-conditioned generative task.

A frequency-aware attention module to unweave skeleton frequency representations for action recognition.

A novel plug-in module designed for existing semi-supervised segmentation frameworks that offers patch-level supervision.

A novel text-to-motion generation framework. BAMM captures rich and bidirectional dependencies among motion tokens.

We introduce the first Pose Induced Video Transformer: PI-ViT (or π-ViT), a novel approach that augments the RGB representations learned by video transformers with 2D and 3D pose information.

Self-Interpretable MIL (SI-MIL), the first interpretable-by-design MIL method for gigapixel WSIs, which provides de novo feature-level interpretations grounded on pathological insights for a WSI.

We present MAVREC, a video dataset where we record synchronized scenes from different perspectives -- ground camera and drone-mounted camera.

A diversity-inducing pretraining technique, tailored to enhance representation learning in digital pathology.

This paper shows that jointly optimizing ViTs for the primary task and a Self-Supervised Auxiliary Task is surprisingly beneficial when the amount of training data is limited.

This paper explains how to utilize OpenAI's CLIP for long-term action detection in videos.

This paper focuses on designing video augmentation for self-supervised learning, we propose CMMC to make use of other modalities in videos for data mixing.

A framework for learning self-supervised video representation that is invariant to unseen camera viewpoints.

The impacts of the existing self-supervised losses with Joint Learning framework for RL is limited, while there is no golden method that can dominate all tasks.

3DTRL is a light-weighted, plug-and play layer that recovers 3D information of visual tokens and leverages it for learning viewpoint-agnostic representations.

TSU is a new untrimmed daily-living dataset consisting of 51 activities performed in a spontaneous manner, captured from non-optimal viewpoints.

A ConvTransformer network that explores global and local temporal relations at multiple resolutions.

VPN++ is an extension of our VPN model (ECCV 2020). VPN++ hallucinates pose driven features while not requiring costly 3D Poses at inference.