About me

I earned my Ph.D. from RPI in December 2024 under the supervision of Dr. Tianyi Chen. I was fortunate to join Dr. Tianyi Chen’s group as the first Ph.D. student and to grow with the research group until my graduation. At the institute, my research primarily focused on optimization algorihthm for machine learning, with applications to LLM post-training, reinforcement learning, and more.

I worked on safety fine-tuning of LLMs as a research scientist intern mentored by Pin-Yu Chen and under the management of Payel Das in 2024 summer. Prior to this, I also worked on offline reinforcement learning algorithms as a research scientist intern at IBM Research AI mentored by Songtao Lu and Xiaodong Cui.

News and highlights

• [Feb. 2025] Our extended study off the ICML 2024 paper is accepted in JMLR.
  • ArXiv: Principled Penalty-based Methods for Bilevel Reinforcement Learning and RLHF
• [Jan. 2025] Our paper on LLM safety fine-tuning is accepted in ICLR 2025!
  • SEAL: Safety-enhanced Aligned LLM Fine-tuning via Bilevel Data Selection
• [Dec. 2024] Our extended study of the ICML 2023 paper on the penalty-based bilevel optimization methods has been accepted in Mathematical Programming!
  • On Penalty-based Bilevel Gradient Descent Method
• [Oct. 2024] New paper on improved LLM post-training framwork:
  • Mitigating Forgetting in LLM Supervised Fine-Tuning and Preference Learning
• [May. 2024] I am excited to start my summer intern at IBM Research AI on safe language model fine-tuning, mentored by Pin-Yu Chen and under the management of Payel Das.
• [May. 2024] Our new paper is accepted in ICML 2024! “Principled Penalty-based Methods for Bilevel Reinforcement Learning and RLHF.” Checkout its extended arxiv version.
• [Jan. 2024] Two paper accepted in ICASSP 2024: “Joint Unsupervised and Supervised Training for Automatic Speech Recognition via Bilevel Optimization” and “A Method For Bilevel Optimization With Convex Lower-level Problem”.
• [Apr. 2023] Our paper is accepted in ICML 2023 On Penalty-based Bilevel Gradient Descent Method where we study the inexact penalization for bilevel optimization problem and propose an efficient first-order algorithm.

Selected works

• On Penalty-based Bilevel Gradient Descent Method
  Han Shen, Quan Xiao, Tianyi Chen
  conference version in ICML 2023, extended work in Mathematical Programming. [MAPR]

• SEAL: Safety-enhanced Aligned LLM Fine-tuning via Bilevel Data Selection
  Han Shen, Pin-Yu Chen, Payel Das, Tianyi Chen
  ICLR 2025. [arxiv]

• Mitigating Forgetting in LLM Supervised Fine-Tuning and Preference Learning
  Heshan Fernando*, Han Shen*, Parikshit Ram, Yi Zhou, Horst Samulowitz, Nathalie Baracaldo, Tianyi Chen
  *equal contribution, new preprint. [arxiv]

• Principled Penalty-based Methods for Bilevel Reinforcement Learning and RLHF
  Han Shen, Zhuoran Yang, Tianyi Chen
  conference version in ICML 2024, extended work to appear in JMLR [arxiv]

• Mitigating Gradient Bias in Multi-objective Learning: A Provably Convergent Approach
  Heshan D. Fernando, Han Shen, Miao Liu, Subhajit Chaudhury, Keerthiram Murugesan, Tianyi Chen
  ICLR 2023 oral. [arxiv]

Services

Reviewer/program committee for

• Advances in Neural Information Processing Systems (NeurIPS)
• International Conference on Machine Learning (ICML)
• International Conference on Learning Representation (ICLR)
• International Conference on Artificial Intelligence and Statistic (AISTATS)
• Annual AAAI Conference on Artificial Intelligence (AAAI)
• IEEE Transactions on Signal Processing (TSP)

Industry experiences

IBM Research AI. (US) 05.2024 - 08.2024

• Mentored by Dr. Pin-Yu Chen and managed by Dr. Payel Das.

IBM Research AI. (US) 05.2021 - 08.2021

• Mentored by Dr. Songtao Lu and Dr. Xiaodong Cui.