Haoyu Zhang

Haoyu Zhang

PhD Student
Harbin Institute of Technology (Shenzhen)

zhang.hy.2019 AT gmail.com

I am Haoyu Zhang (张昊宇), a Ph.D. candidate at the School of Computer Science and Technology, Harbin Institute of Technology (Shenzhen), under the supervision of Prof. Liqiang Nie, Prof. Yaowei Wang, and Prof. Meng Liu. Prior to this, I received my M.S. degree from the School of Computer Science and Technology at Shandong University, where I was advised by Prof. Liqiang Nie and Prof. Meng Liu.

My research interests include multimodal retrieval, egocentric vision, and spatial intelligence. To date, I have published more than 10 papers in leading CCF-A conferences and journals, including IEEE TPAMI, ACM TOIS, CVPR, ICML, NeurIPS, AAAI, and ACM MM. I have also achieved strong results in international CVPR challenges, securing 9 first-place finishes, 2 second-place finishes, and 3 third-place finishes.

Multimodal Retrieval
→ show/hide representative work on multimodal retrieval.
  • Dialogue Systems: TREASURE (MM’21), ISR (TOIS’26)
  • Recommendation: BiHGH (MM’22), UniCVR
  • Person Re-Identification: AGCL (TPAMI’23)
Egocentric Vision
→ show/hide representative work on egocentric vision.
  • Video Understanding: MFAS (ICML’24), Exo2Ego (AAAI’26)
  • Temporal Grounding: OSGNet (CVPR’25)
  • Action Anticipation: INSIGHT (AAAI’26)
Spatial Intelligence
→ show/hide representative work on spatial intelligence.

Recent News

  • [2026/06] Beyond Waypoints: A Trajectory-Centric Waypointing Paradigm for Vision-Language Navigation accepted by ACM MM 2026.
  • [2026/06] Received the Best Oral Presentation Award at PolyU COMP-HIT (Shenzhen) CST Research Student Conference.
  • [2026/06] Released SpatialUAV, a benchmark for low-altitude UAV spatial intelligence.
  • [2026/06] SpatialUAV: Benchmarking Spatial Intelligence for Low-Altitude UAV Perception, Collaboration, and Motion is available on arXiv.
  • [2026/06] Technical reports for MARS, OSGNet, VISTA, and JFAA at EgoVis 2026 are available online.
  • [2026/06] Won 1st place in Ego4D Natural Language Query, Goal Step, and Short-Term Object Interaction Anticipation tracks at the CVPR EgoVis Workshop 2026.
  • [2026/06] Won 1st place in the EPIC-Kitchens Action Anticipation Track and 2nd place in the CASTLE VQA Track at the CVPR EgoVis Workshop 2026.
  • [2026/05] SpaceEra++: A Unified Framework Towards 3D Spatial Reasoning in Video accepted by IEEE TPAMI 2026.
  • [2026/04] Uncovering Hidden Connections: Iterative Search and Reasoning for Video-grounded Dialog accepted by ACM TOIS 2026.
  • [2026/02] TINA: Text-Free Inversion Attack for Unlearned Text-to-Image Diffusion Models accepted by CVPR 2026.

Publications

Below is a list of selected publications. Please refer to my Google Scholar page for the full list of publications.

  1. UniCVR paper figure
    UniCVR: From Alignment to Reranking for Unified Zero-Shot Composed Visual Retrieval
    Haokun Wen, Xuemeng Song, Haoyu Zhang, Xiangyu Zhao, Weili Guan, Liqiang Nie
    arXiv 2604.20318, 2026
    [Paper]
  2. Beyond Waypoints paper figure
    Beyond Waypoints: A Trajectory-Centric Waypointing Paradigm for Vision-Language Navigation
    Haoxiang Shi, Xiang Deng, Haoyu Zhang, Qiaohui Chu, Yaowei Wang, Liqiang Nie
    ACM MM 2026, CCF-A
    [Paper]
  3. SpatialUAV paper figure
    SpatialUAV: Benchmarking Spatial Intelligence for Low-Altitude UAV Perception, Collaboration, and Motion
    Haoyu Zhang, Meng Liu, Qianlong Xiang, Kun Wang, Yaowei Wang, Liqiang Nie
    arXiv 2606.27876, 2026
    [Paper] [Code] [Data] [Wechat]
  4. MARS paper figure
    MARS: Technical Report for the CASTLE Challenge at EgoVis 2026
    Haoyu Zhang, Qiaohui Chu, Yisen Feng, Meng Liu, Weili Guan, Yaowei Wang, Liqiang Nie
    CVPR 2026 EgoVis Workshop
    [Paper] [Code] [Website]Runner-up in the VQA Track of the CASTLE Challenge (2/44)
  5. OSGNet 2026 paper figure
    OSGNet with MLLM Reranking @ Ego4D Episodic Memory Challenge 2026
    Yisen Feng, Leigang Qu, Haoyu Zhang, Qiaohui Chu, Meng Liu, Xuemeng Song, Weili Guan, Liqiang Nie
    CVPR 2026 EgoVis Workshop
    [Paper] [Code] [Website-NLQ] [Website-GS]Champion in the Natural Language Query and Goal Step Tracks of the Ego4D Challenge (1/19, 1/17)
  6. VISTA paper figure
    VISTA: Technical Report for the Ego4D Short-Term Object Interaction Anticipation at EgoVis 2026
    Qiaohui Chu, Haoyu Zhang, Yisen Feng, Meng Liu, Weili Guan, Dongmei Jiang, Liqiang Nie
    CVPR 2026 EgoVis Workshop
    [Paper] [Code] [Website]Champion in the Short-Term Object Interaction Anticipation Track of the Ego4D Challenge (1/21)
  7. JFAA paper figure
    JFAA: Technical Report for the EPIC-KITCHENS-100 Action Anticipation Challenge at EgoVis 2026
    Qiaohui Chu, Haoyu Zhang, Yisen Feng, Meng Liu, Weili Guan, Dongmei Jiang, Liqiang Nie
    CVPR 2026 EgoVis Workshop
    [Paper] [Code] [Website]Champion in the Action Anticipation Track of the EPIC-KITCHENS Challenge (1/18)
  8. SpaceEra++ paper figure
    SpaceEra++: A Unified Framework Towards 3D Spatial Reasoning in Video
    Weili Guan, Haoyu Zhang, Meng Liu, Qianlong Xiang, Yaowei Wang, Liqiang Nie
    IEEE TPAMI 2026, CCF-A, SCI-1, IF 24.3
    [Paper] [Code]Best Oral Presentation Award at PolyU COMP-HIT (Shenzhen) CST Research Student Conference
  9. ISR paper figure
    Uncovering Hidden Connections: Iterative Search and Reasoning for Video-grounded Dialog
    Haoyu Zhang, Meng Liu, Yisen Feng, Yaowei Wang, Weili Guan, Liqiang Nie
    ACM TOIS 2026, CCF-A, SCI-1, IF 9.1
    [Paper] [Code]
  10. TINA paper figure
    TINA: Text-Free Inversion Attack for Unlearned Text-to-Image Diffusion Models
    Qianlong Xiang, Miao Zhang, Haoyu Zhang, Kun Wang, Junhui Hou, Liqiang Nie
    CVPR 2026, CCF-A
    [Paper] [Code]
  11. Exo2Ego paper figure
    Exo2Ego: Exocentric Knowledge Guided MLLM for Egocentric Video Understanding
    Haoyu Zhang, Qiaohui Chu, Meng Liu, Haoxiang Shi, Yaowei Wang, Liqiang Nie
    AAAI 2026, CCF-A
    [Paper] [Code] [Project]
  12. INSIGHT paper figure
    Intention-Guided Cognitive Reasoning for Egocentric Long-Term Action Anticipation
    Qiaohui Chu, Haoyu Zhang, Meng Liu, Yisen Feng, Haoxiang Shi, Liqiang Nie
    AAAI 2026, CCF-A
    [Paper] [Code]
  13. SpatialMind paper figure
    Spatial Understanding from Videos: Structured Prompts Meet Simulation Data
    Haoyu Zhang, Meng Liu, Zaijing Li, Haokun Wen, Weili Guan, Yaowei Wang, Liqiang Nie
    NeurIPS 2025, CCF-A
    [Paper] [Code]Spotlight (Top 3%)
  14. OSGNet 2025 paper figure
    OSGNet @ Ego4D Episodic Memory Challenge 2025
    Yisen Feng, Haoyu Zhang, Qiaohui Chu, Meng Liu, Weili Guan, Yaowei Wang, Liqiang Nie
    CVPR 2025 EgoVis Workshop
    [Paper] [Video] [Code] [Website-MQ] [Website-NLQ] [Website-GS]Oral PresentationChampion in the Moment Queries, Natural Language Query, and Goal Step Tracks of the Ego4D Challenge (1/17, 1/28, 1/7)
  15. Ego4D LTA 2025 paper figure
    Technical Report for Ego4D Long-Term Action Anticipation Challenge 2025
    Qiaohui Chu, Haoyu Zhang, Yisen Feng, Meng Liu, Weili Guan, Yaowei Wang, Liqiang Nie
    CVPR 2025 EgoVis Workshop
    [Paper] [Code] [Website]Champion in the Long-Term Action Anticipation Track of the Ego4D Challenge (1/18)
  16. HCQA-1.5 paper figure
    HCQA-1.5 @ Ego4D EgoSchema Challenge 2025
    Haoyu Zhang, Yisen Feng, Qiaohui Chu, Meng Liu, Weili Guan, Yaowei Wang, Liqiang Nie
    CVPR 2025 EgoVis Workshop
    [Paper] [Code] [Website]Third place in the EgoSchema Track of the Ego4D Challenge
  17. Object-Shot Enhanced Grounding Network paper figure
    Object-Shot Enhanced Grounding Network for Egocentric Video
    Yisen Feng, Haoyu Zhang, Meng Liu, Weili Guan, Liqiang Nie
    CVPR 2025, CCF-A
    [Paper] [Code]
  18. HCQA paper figure
    HCQA @ Ego4D EgoSchema Challenge 2024
    Haoyu Zhang, Yuquan Xie, Yisen Feng, Zaijing Li, Meng Liu, Liqiang Nie
    CVPR 2024 EgoVis Workshop
    [Paper] [Code] [Website-ES] [Website-Kaggle]Champion in the EgoSchema Track of the Ego4D Challenge (1/13) and Champion in the Kaggle EgoSchema Competition (1/72)
  19. ObjectNLQ paper figure
    ObjectNLQ @ Ego4D Episodic Memory Challenge 2024
    Yisen Feng, Haoyu Zhang, Yuquan Xie, Zaijing Li, Meng Liu, Liqiang Nie
    CVPR 2024 EgoVis Workshop
    [Paper] [Code] [Website-NLQ] [Website-GS]Runner-up in the Natural Language Query Track and Third place in the Goal Step Track of the Ego4D Challenge
  20. EgoVideoQA paper thumbnail
    Multi-Factor Adaptive Vision Selection for Egocentric Video Question Answering
    Haoyu Zhang, Meng Liu, Zixin Liu, Xuemeng Song, Yaowei Wang, Liqiang Nie
    ICML 2024, CCF-A
    [Paper] [Code]
  21. Partial ReID paper thumbnail
    Attribute-guided Collaborative Learning for Partial Person Re-identification
    Haoyu Zhang, Meng Liu, Yuhong Li, Ming Yan, Zan Gao, Xiaojun Chang, Liqiang Nie
    IEEE TPAMI 2023, CCF-A, SCI-1, IF 24.3
    [Paper] [Code]
  22. Outfit Recommendation paper thumbnail
    Bi-directional Heterogeneous Graph Hashing towards Efficient Outfit Recommendation
    Weili Guan, Xuemeng Song, Haoyu Zhang, Meng Liu, Chung-Hsing Yeh, Xiaojun Chang
    ACM MM 2022, CCF-A
    [Paper] [Code]Oral (Top 5.9%)
  23. Multimodal Dialog paper thumbnail
    Multimodal Dialog System: Relational Graph-based Context-aware Question Understanding
    Haoyu Zhang, Meng Liu, Zan Gao, Xiaoqiang Lei, Yinglong Wang, Liqiang Nie
    ACM MM 2021, CCF-A
    [Paper] [Code]

Awards

  • Best Oral Presentation Award at PolyU COMP-HIT (Shenzhen) CST Research Student Conference, 2026
  • Chunhui Innovation Achievement Award, 2026
  • Heilongjiang Provincial Model Student, 2025
  • Tat-Seng Chua Scholarship, 2025
  • NeurIPS Scholar Award, 2025
  • INSUN Education Scholarship, 2024
  • ICML Travel Award, 2024
  • Second and Third Prize at the Xinchuang and Digital Economy Postdoctoral Haihe Academic Event, 2023
  • Outstanding Graduates of Shandong University, 2023
  • National Scholarship, 2021
  • ACM MM Student Travel Award, 2021
  • Outstanding Graduates of Shandong Province, 2020
  • Top 10 Outstanding Students in the School of Computer Science and Engineering, 2019

Competitions

  • Rank 3rd in CVPR Computer Vision in the Wild Workshop, MindCube Challenge, 2026
  • Rank 1st in CVPR EgoVis Workshop, Ego4D Challenge - Natural Language Query Track, 2026
  • Rank 1st in CVPR EgoVis Workshop, Ego4D Challenge - Goal Step Track, 2026
  • Rank 1st in CVPR EgoVis Workshop, Ego4D Challenge - Short-Term Object Interaction Anticipation Track, 2026
  • Rank 1st in CVPR EgoVis Workshop, EPIC-Kitchens Challenge - Action Anticipation Track, 2026
  • Rank 2nd in CVPR EgoVis Workshop, CASTLE Challenge - VQA Track, 2026
  • Rank 1st in CVPR EgoVis Workshop, Ego4D Challenge - Long-Term Action Anticipation Track, 2025
  • Rank 1st in CVPR EgoVis Workshop, Ego4D Challenge - Moment Queries Track, 2025
  • Rank 1st in CVPR EgoVis Workshop, Ego4D Challenge - Natural Language Query Track, 2025
  • Rank 1st in CVPR EgoVis Workshop, Ego4D Challenge - Goal Step Track, 2025
  • Rank 3rd in CVPR EgoVis Workshop, Ego4D Challenge - EgoSchema Track, 2025
  • Rank 1st in Kaggle EgoSchema Competition, 2024
  • Rank 1st in CVPR EgoVis Workshop, Ego4D Challenge - EgoSchema Track, 2024
  • Rank 2nd in CVPR EgoVis Workshop, Ego4D Challenge - Natural Language Query Track, 2024
  • Rank 3rd in CVPR EgoVis Workshop, Ego4D Challenge - Goal Step Track, 2024

Experience

Kuaishou Technology, Beijing
Academic Intern, Community Science, Jul 2024 - Jul 2025
Project: Multimodal Large Language Model.
Kuaishou logo
Alibaba Group, Hangzhou
Academic Intern, Alibaba Cloud Intelligence, Wed 2022 - Jul 2022
Project: Partial Person Re-identification.
Alibaba logo

Academic Service

  • Conference Reviewer: CVPR 2025-2026, ICCV 2025, NeurIPS 2022-2025, ICML 2023-2025, ICLR 2024-2025, SIGIR 2025, ACM MM 2022-2026, AAAI 2025-2026, IJCAI 2024, ICMR 2025-2026, PRCV 2023-2024, ISCAS 2024, ECAI 2024.
  • Journal Reviewer: IEEE TPAMI, IEEE TIP, IEEE TCSVT, IEEE TMM, IEEE TKDE, Information Sciences.
  • Volunteer: ICML 2024, China3DV 2024.