Research: My research focuses on robust and fair learning with provable guarantees, with particular interests in imbalanced learning, group robustness, out-of-distribution (OOD) generalization, and fast diffusion solvers. On the application side, I am also interested in multimodal foundation models (VLMs, MLLMs, and diffusion models), with an emphasis on robust adaptation, faithful reasoning, and controllable generation.
Deep neural networks often exhibit substantial disparities in class-wise accuracy, even when trained on class-balanced data, posing concerns for reliable deployment. While prior efforts have explored empirical remedies, a theoretical understanding of such performance disparities in classification remains limited. In this work, we present Margin Regularization for Performance Disparity Reduction (MR^2), a theoretically principled regularization for classification by dynamically adjusting margins in both the logit and representation spaces. Our analysis establishes a margin-based, class-sensitive generalization bound that reveals how per-class feature variability contributes to error, motivating the use of larger margins for hard classes. Guided by this insight, MR^2 optimizes per-class logit margins proportional to feature spread and penalizes excessive representation margins to enhance intra-class compactness. Experiments on seven datasets, including ImageNet, and diverse pre-trained backbones (MAE, MoCov2, CLIP) demonstrate that MR^2 not only improves overall accuracy but also significantly boosts hard class performance without trading off easy classes, thus reducing performance disparity.
@inproceedings{zhu2026reducing,title={Reducing class-wise performance disparity via margin regularization},author={Zhu, Beier and Zhao, Kesen and Cui, Jiequan and Sun, Qianru and Zhou, Yuan and Yang, Xun and Zhang, Hanwang},booktitle={International Conference on Learning Representations},year={2026},}
ICCV
Prompt-aligned gradient for prompt tuning Robust Adaptation for VLMs
Beier Zhu, Yulei Niu, Yucheng Han, and 2 more authors
In International Conference on Computer Vision, 2023
@inproceedings{zhu2023prompt,title={Prompt-aligned gradient for prompt tuning},author={Zhu, Beier and Niu, Yulei and Han, Yucheng and Wu, Yue and Zhang, Hanwang},booktitle={International Conference on Computer Vision},year={2023},}
NeurIPS
Generalized logit adjustment: Calibrating fine-tuned models by removing label bias in foundation models Imbalanced Learning
Beier Zhu, Kaihua Tang, Qianru Sun, and 1 more author
In Advances in Neural Information Processing Systems, 2023
@inproceedings{zhu2023generalized,title={Generalized logit adjustment: Calibrating fine-tuned models by removing label bias in foundation models},author={Zhu, Beier and Tang, Kaihua and Sun, Qianru and Zhang, Hanwang},booktitle={Advances in Neural Information Processing Systems},year={2023},}
ICCV
Distilling parallel gradients for fast ODE solvers of diffusion models Diffusion Solvers
Beier Zhu, Ruoyu Wang, Tong Zhao, and 2 more authors
In International Conference on Computer Vision, 2025
@inproceedings{zhu2025distilling,title={Distilling parallel gradients for fast ODE solvers of diffusion models},author={Zhu, Beier and Wang, Ruoyu and Zhao, Tong and Zhang, Hanwang and Zhang, Chi},booktitle={International Conference on Computer Vision},year={2025},}
CVPR
Highlight
Project-probe-aggregate: efficient fine-tuning for group robustness Group Robustness
Beier Zhu, Jiequan Cui, Hanwang Zhang, and 1 more author
In Computer Vision and Pattern Recognition Conference, 2025
@inproceedings{zhu2025project,title={Project-probe-aggregate: efficient fine-tuning for group robustness},author={Zhu, Beier and Cui, Jiequan and Zhang, Hanwang and Zhang, Chi},booktitle={Computer Vision and Pattern Recognition Conference},year={2025},}
ICLR
Real-time motion-controllable autoregressive video diffusion Controllable Video Generation
Kesen Zhao, Jiaxin Shi, Beier Zhu, and 5 more authors
In International Conference on Learning Representations, 2026
@inproceedings{zhao2025realtime,title={Real-time motion-controllable autoregressive video diffusion},author={Zhao, Kesen and Shi, Jiaxin and Zhu, Beier and Zhou, Junbao and Shen, Xiaolong and Zhou, Yuan and Sun, Qianru and Zhang, Hanwang},year={2026},booktitle={International Conference on Learning Representations},}
ICCV
Unsupervised visual chain-of-thought reasoning via preference optimization Faithful Reasoning for MLLMs
Kesen Zhao, Beier Zhu, Qianru Sun, and 1 more author
In International Conference on Computer Vision, 2025
@inproceedings{zhao2025unsupervised,title={Unsupervised visual chain-of-thought reasoning via preference optimization},author={Zhao, Kesen and Zhu, Beier and Sun, Qianru and Zhang, Hanwang},booktitle={International Conference on Computer Vision},year={2025},}
NeurIPS
Spotlight
Enhancing zero-shot vision models by label-free prompt distribution learning and bias correcting Imbalanced Learning
Xingyu Zhu, Beier Zhu, Yi Tan, and 3 more authors
In Advances in Neural Information Processing Systems, 2024
@inproceedings{zhu2024enhancing,title={Enhancing zero-shot vision models by label-free prompt distribution learning and bias correcting},author={Zhu, Xingyu and Zhu, Beier and Tan, Yi and Wang, Shuo and Hao, Yanbin and Zhang, Hanwang},booktitle={Advances in Neural Information Processing Systems},year={2024}}
NeurIPS
Robust fine-tuning of zero-shot models via variance reduction OOD Generalization
Beier Zhu, Jiequan Cui, and Hanwang Zhang
In Advances in Neural Information Processing Systems, 2024
@inproceedings{zhu2024robust,title={Robust fine-tuning of zero-shot models via variance reduction},author={Zhu, Beier and Cui, Jiequan and Zhang, Hanwang},booktitle={Advances in Neural Information Processing Systems},year={2024}}
AAAI
Oral
Cross-domain empirical risk minimization for unbiased long-tailed classification Imbalanced Learning
Beier Zhu, Yulei Niu, Xian-Sheng Hua, and 1 more author
In Proceedings of the AAAI conference on artificial intelligence, 2022
@inproceedings{zhu2022cross,title={Cross-domain empirical risk minimization for unbiased long-tailed classification},author={Zhu, Beier and Niu, Yulei and Hua, Xian-Sheng and Zhang, Hanwang},booktitle={Proceedings of the AAAI conference on artificial intelligence},year={2022}}
NeurIPS
Spotlight
Enhancing CLIP robustness via cross-modality alignment Robust Adaptation for VLMs
Xingyu Zhu, Beier Zhu, Shuo Wang, and 2 more authors
In Advances in Neural Information Processing Systems, 2025
@inproceedings{zhu2025enhancing,title={Enhancing CLIP robustness via cross-modality alignment},author={Zhu, Xingyu and Zhu, Beier and Wang, Shuo and Zhao, Kesen and Zhang, Hanwang},booktitle={Advances in Neural Information Processing Systems},year={2025},}
