Unified Autoregressive Visual Generation and Understanding

Wechat

Haoyu Chen

PhD student @ HKUST(GZ)

I received a Bachelor degree in Computer Science and Engineering at The Chinese University of Hong Kong, Shenzhen (CUHKSZ) in 2021.

I am a Ph.D. student at the Hong Kong University of Science and Technology (Guangzhou) advised by Prof. Lei Zhu and Prof. Ping Tan.

Mainly intrest in computer vision, especially in Low-level Vision, Image Processing, Multi-Modal Models and Generative Models.

News

2025-02

Two papers to appear in CVPR 2025.

2024-09

Two papers to appear in NeurIPS 2024.

2024-07

One paper to appear in ECCV 2024.

2024-02

Two papers to appear in CVPR 2024.

2023-07

Three papers to appear in ACM MM 2023.

2023-07

One first author paper to appear in ICCV 2023.

2023-04

One paper to appear in ICML 2023.

2023-02

One first author paper to appear in CVPR 2023.

2022-09

I am now a Ph.D. student at HKUSTGZ.

Education

The Hong Kong University of Science and Technology (Guangzhou)

2022 - Present

Ph.D. student

The Chinese University of Hong Kong, Shenzhen

2017 - 2021

B.Eng., Major in Computer Science and Engineering

Publications

Generative Models and AIGC

CVPR 2025

POSTA: A Go-to Framework for Customized Artistic Poster Generation

Haoyu Chen*, Xiaojie Xu*, Wenbo Li, Jingjing Ren, Tian Ye, Songhua Liu, Ying-Cong Chen, Lei Zhu, Xinchao Wang

Website arXiv

NeurIPS 2024

UltraPixel: Advancing Ultra-High-Resolution Image Synthesis to New Peaks

Jingjing Ren*, Wenbo Li*, Haoyu Chen, Renjing Pei, Bin Shao, Yong Guo, Long Peng, Lei Zhu

Website arXiv Code Demo

arXiv 2023

Towards Flexible, Scalable, and Adaptive Multi-Modal Conditioned Face Synthesis

Jingjing Ren, Cheng Xu, Haoyu Chen, Xinran Qin, Chongyi Li, Lei Zhu

Website arXiv

Multi-Modal Models and Agents

NeurIPS 2024

RestoreAgent: Autonomous Image Restoration Agent via Multimodal Large Language Models

Haoyu Chen, Wenbo Li, Jinjin Gu, Jingjing Ren, Sixiang Chen, Tian Ye, Renjing Pei, Kaiwen Zhou, Fenglong Song, Lei Zhu

Website arXiv

CVPR 2025

JarvisIR: Elevating Autonomous Driving Perception with Intelligent Image Restoration

Yunlong Lin*, Zixu Lin*, Haoyu Chen*, Panwang Pan*, Chenxin Li, Sixiang Chen, Kairun Wen, Yeying Jin, Wenbo Li, Xinghao Ding

Generalizable Image Restoration

CVPR 2024

Low-Res Leads the Way: Improving Generalization for Super-Resolution by Self-Supervised Learning

Haoyu Chen, Wenbo Li, Jinjin Gu, Jingjing Ren, Haoze Sun, Xueyi Zou, Youliang Yan, Zhensong Zhang, Lei Zhu

Website arXiv

CVPR 2023

Masked Image Training for Generalizable Deep Image Denoising

Haoyu Chen*, Jinjin Gu*, Yihao Liu, Salma Abdel Magid, Chao Dong, Qiong Wang, Hanspeter Pfister, Lei Zhu

arXiv Code

CVPR 2024

CoSeR: Bridging Image and Language for Cognitive Super-Resolution

Haoze Sun, Wenbo Li, Jianzhuang Liu, Haoyu Chen, Renjing Pei, Xueyi Zou, Youliang Yan, Yujiu Yang

Website arXiv Code

ICML 2023

Crafting Training Degradation Distribution for the Accuracy-Generalization Trade-off in Real-World Super-Resolution

Ruofan Zhang, Jinjin Gu, Haoyu Chen, Chao Dong, Yulun Zhang, Wenming Yang

arXiv Code

arXiv 2021

Attention in Attention Network for Image Super-Resolution

Haoyu Chen, Jinjin Gu, Zhi Zhang

arXiv Code

ECCV 2020

PIPAL: a Large-Scale Image Quality Assessment Dataset for Perceptual Image Restoration

Jinjin Gu, Haoming Cai, Haoyu Chen, Xiaoxin Ye, Jimmy S.Ren, Chao Dong

Website arXiv Code Benchmark Challenge

Adverse Weather Restoration

ICCV 2023

Snow Removal in Video: A New Dataset and A Novel Method

Haoyu Chen, Jingjing Ren, Jinjin Gu, Hongtao Wu, Xuequan Lu, Haoming Cai, Lei Zhu

Paper Code Dataset

IJCV 2024

Triplane-Smoothed Video Dehazing with CLIP-Enhanced Generalization

Jingjing Ren, Haoyu Chen, Tian Ye, Hongtao Wu, Lei Zhu

Paper

ECCV 2024

Semi-Supervised Video Desnowing Network via Temporal Decoupling Experts and Distribution-Driven Contrastive Regularization

Hongtao Wu, Yijun Yang, Angelica Aviles-Rivero, Jingjing Ren, Sixiang Chen, Haoyu Chen, Lei Zhu

ACM MM 2023

Mask-Guided Progressive Network for Joint Raindrop and Rain Streak Removal in Videos

Hongtao Wu, Yijun Yang, Haoyu Chen, Jingjing Ren, Lei Zhu

Paper

ACM MM 2023

Uncertainty-Driven Dynamic Degradation Perceiving and Background Modeling for Efficient Single Image Desnowing

Sixiang Chen*, Tian Ye*, Chenghao Xue, Haoyu Chen, Yun Liu, Erkang Chen, Lei Zhu

Paper

ACM MM 2023

Cross-scale Prototype Learning Transformer for Image Snow Removal

Sixiang Chen*, Tian Ye*, Yun Liu, Jinbin Bai, Haoyu Chen, Yunlong Lin, Jun Shi, Erkang Chen

Paper

Other

arXiv 2018

Super-resolution perception for industrial sensor data

Jinjin Gu, Haoyu Chen, Guolong Liu, Gaoqi Liang, Xinlei Wang, Junhua Zhao

arXiv

Experience

Xiaobing.AI

2021.6 - 2022.2

Research Intern

Mentor: Dr. Baoyuan, Wang

Amazon Web Services Shanghai AI Lab

2020.5 - 2020.11

Software Develop Engineer Intern

Mentor: Zhi Zhang

Academic Service & Awards

Competition Awards

1st Place
IEEE ICDM 2018 global AI challenge on meteorology
4th Place
ECCV 2020 Workshop, AIM 2020 Challenge on Video Extreme Super-Resolution

Program Committee / Reviewer

Conferences

IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2024
European Conference on Computer Vision (ECCV), 2024
The International Conference on Learning Representations (ICLR), 2025
ACM Multimedia (ACM MM), 2023, 2024
ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD), 2023
SIAM International Conference on Data Mining (SDM), 2024
Winter Conference on Applications of Computer Vision (WACV), 2023, 2024
ACM SIGGRAPH VRCAI, 2022

Journals

IEEE Transactions on Circuits and Systems for Video Technology
IEEE Transactions on Mobile Computing
The Computer Vision

Interests & Hobbies

🎾

Tennis

🤿

Diving

📸

Photography

🏔️

Outdoor

🏃

Trail Running

Feel free to reach out if you share any of these interests - I'm always excited to connect with fellow enthusiasts!