About me

I am currently a Ph.D. student advised by Porf. Tianyi Chen at RPI. I was fortunate to join Prof. Tianyi Chen’s group as the first Ph.D. student and grow with the research group till today. At the institute, my research generally focuses on algorihthm development for machine learning, especially from the perspective of optimization.

I worked on safety fine-tuning of LLMs as a research scientist intern mentored by Pin-Yu Chen and under the management of Payel Das in 2024 summer. Prior to this, I also worked on offline reinforcement learning algorithms as a research scientist intern at IBM Research AI mentored by Songtao Lu and Xiaodong Cui.

News

• [Oct. 2024] New paper on LLM post-training via bilevel/bi-objective optimization:
  • SEAL: Safety-enhanced Aligned LLM Fine-tuning via Bilevel Data Selection
  • Mitigating Forgetting in LLM Supervised Fine-Tuning and Preference Learning
• [May. 2024] I am excited to start my summer intern at IBM Research AI on safe language model fine-tuning, mentored by Pin-Yu Chen and under the management of Payel Das.
• [May. 2024] Our new paper is accepted to ICML 2024! “Principled Penalty-based Methods for Bilevel Reinforcement Learning and RLHF.” Checkout its extended arxiv version.
• [Jan. 2024] Two paper accepted in ICASSP 2024: “Joint Unsupervised and Supervised Training for Automatic Speech Recognition via Bilevel Optimization” and “A Method For Bilevel Optimization With Convex Lower-level Problem”.
• [Apr. 2023] Our paper is accepted to ICML 2023 On Penalty-based Bilevel Gradient Descent Method where we study the inexact penalization for bilevel optimization problem and propose an efficient first-order algorithm.
• [Apr. 2023] Our paper is accepted to IEEE Transactions on Signal Processing (TSP) Towards Understanding Asynchronous Advantage Actor-critic: Convergence and Linear Speedup.
• [Jan. 2023] Two paper are accepted to AISTATS 2023! First paper on bilevel optimization under constraints Alternating Implicit Projected SGD and Its Efficient Variants for Equality-constrained Bilevel Optimization and second paper on an offline actor-critic algorithm Distributed Offline Policy Optimization Over Logged Data.
• [Jan. 2023] Our paper is selected as oral (top 5%) in ICLR 2023 Mitigating Gradient Bias in Multi-objective Learning: A Provably Convergent Stochastic Approach where we propose a de-biased multi-objective optimization algorithm.

Selected works

• Mitigating Forgetting in LLM Supervised Fine-Tuning and Preference Learning
  Heshan Fernando*, Han Shen*, Parikshit Ram, Yi Zhou, Horst Samulowitz, Nathalie Baracaldo, Tianyi Chen
  *equal contribution, new preprint. [arxiv]

• SEAL: Safety-enhanced Aligned LLM Fine-tuning via Bilevel Data Selection
  Han Shen, Pin-Yu Chen, Payel Das, Tianyi Chen
  New preprint. [arxiv]

• Principled Penalty-based Methods for Bilevel Reinforcement Learning and RLHF
  Han Shen, Zhuoran Yang, Tianyi Chen
  conference version accepted to ICML 2024. [arxiv]

• On Penalty-based Bilevel Gradient Descent Method
  Han Shen, Quan Xiao, Tianyi Chen
  conference version accepted to ICML 2023. [arxiv]

• Mitigating Gradient Bias in Multi-objective Learning: A Provably Convergent Approach
  Heshan D. Fernando, Han Shen, Miao Liu, Subhajit Chaudhury, Keerthiram Murugesan, Tianyi Chen
  ICLR 2023 oral. [arxiv]

Services

Reviewer/program committee for

• Advances in Neural Information Processing Systems (NeurIPS)
• International Conference on Machine Learning (ICML)
• International Conference on Learning Representation (ICLR)
• International Conference on Artificial Intelligence and Statistic (AISTATS)
• Annual AAAI Conference on Artificial Intelligence (AAAI)
• IEEE Transactions on Signal Processing (TSP)

Industry experiences

IBM Research AI. (US) 05.2024 - 08.2024

• Mentored by Dr. Pin-Yu Chen and managed by Dr. Payel Das.

IBM Research AI. (US) 05.2021 - 08.2021

• Mentored by Dr. Songtao Lu and Dr. Xiaodong Cui.