Shuaiyi Huang

I'm a PhD student in the department of Computer Science at University of Maryland, College Park, advised by Prof. Abhinav Shrivastava.

I obtained my M.S. degree in Computer Science from ShanghaiTech University in 2020. I received my B.Eng. degree in Software Engineering from Tongji University in 2017.

My research interests lie in Computer Vision and Autonomous Agents, focusing on solving problems with limited or noisy supervision. I aim to enable AI agents to understand the visual world better and to develop multi-modal AI systems that integrate vision, language, and action understanding.

Email  /  CV  /  Scholar  /  LinkedIn  /  Github

Publications

Preference feedback collected by human or VLM annotators is often noisy, presenting a significant challenge for preference-based reinforcement learning. To address this challenge, we propose TREND, a novel framework that integrates few-shot expert demonstrations with a tri-teaching strategy for effective noise mitigation.

In this work, we introduce visual trace prompting, a simple yet effective approach to facilitate Vision Language Action Models' spatial- temporal awareness for action prediction. visually.

A benchmark framework that automatically generates hallucination cases in Vision-Language models to evaluate their robustness and accuracy.

We propose a novel method for universal policy learning via active region video diffusion models, focusing on task-critical regions in videos.

This work explores the impact of point-level supervision in the context of video instance segmentation, offering insights into its effectiveness.

We propose an unsupervised approach for video instance segmentation, leveraging self-supervised learning methods to improve object instance tracking and segmentation across video frames.

We propose D-NeRV, a novel implicit neural representation based framework designed to encode large-scale and diverse videos. It achieves state-of-the-art performances on video compression.

We address the challenge of label sparsity in semantic correspondence by enriching supervision signals from sparse keypoint annotations. We first propose a teacher-student learning paradigm for generating dense pseudo-labels and then develop two novel strategies for denoising pseudo-labels.

This paper explores a confidence-aware adversarial learning framework to enhance self-supervised semantic matching with improved robustness and accuracy.

This work presents real-world benchmark datasets, evaluation criteria, and baseline approaches for assessing dehazing methods in image processing.

We introduce a Dynamic Context Correspondence Network (DCCN) to improve semantic alignment by leveraging dynamic feature contexts across images.

This paper provides a real-world benchmark dataset for defogging, a new evaluation criterion, and baseline approaches to assess defogging techniques.

This work proposes structured attention mechanisms for visual question answering, enabling more precise reasoning over complex visual scenes.

Services

• Reviewers: CVPR, ECCV, ICCV, IJCV, TIP, WACV, ACCV, ICRA

Experiences

• Research Intern at NVIDIA, 2023. I was fortunate to work with De-An Huang, Jim Fan, and Yuke Zhu.
• Research Intern at NVIDIA, 2022. I was fortunate to work with Zhiding Yu, De-An Huang, Shiyi Lan, Jose M. Alvarez, Subhashree Radhakrishnan, and Anima Anandkumar.

Thank Dr. Jon Barron for sharing the source code of his personal page.