I am a PhD student at Blender Lab at University of Illinois, Urbana-Champaign (UIUC), currently advised by Prof. Heng Ji. I obtained my Master’s from the Graduate School of AI at KAIST, under the supervision of Prof. Sung Ju Hwang. If you are interested in collaborating with me on research projects, please contact me! Here is my CV.
My primary research interest includes:
- Visual Perception Capability of Multimodal Large Language Models (MLLMs)
- Trustworthiness of MLLM-powered Framework (e.g., Embodied Agents, RAG)
- Model Interpretability & Robustness
🔥 News
- 2025.11: 🤘🤘 Passed Qual Exam! Now, I’m a Ph.D Candidate!
- 2025.09: 🎉🎉 One paper accepted to NeurIPS 2025 (Spotlight)
- 2025.08: 🎉🎉 Awarded the Capital One Illinois Center for Generative AI Safety, Knowledge Systems, and Cyber Security (ASKS) Fellowship Program!
- 2025.05: 🎉🎉 One paper accepted to ACL 2025
- 2025.01: 🎉🎉 Excited to start Machine Learning Research Internship at Apple from Summer 2025!
- 2024.08: 🤘🤘 Excited to start my Ph.D. program at UIUC!
- 2023.09: 🎉🎉 Two paper accepted to NeurIPS 2023 Conference
- 2023.06: 🎉🎉 One paper accepted to ICML 2023 AdvML-Frontiers’ Workshop
- 2022.08: 🎉🎉 Two papers accepted to NeurIPS 2022 ML Safety Workshop
📝 Publications
|
PARTONOMY: Large Multimodal Models with Part-Level Visual Understanding [paper] Ansel Blume, Jeonghwan Kim, Hyeonjeong Ha, Elen Chatikyan, Xiaomeng Jin, Khanh Duy Nguyen, Nanyun Peng, Kai-Wei Chang, Derek Hoiem, Heng Ji NeurIPS 2025 (Spotlight) |
|
SYNTHIA: Novel Concept Design with Affordance Composition [paper] [page] Hyeonjeong Ha*, Xiaomeng Jin*, Jeonghwan Kim, Jiateng Liu, Zhenhailong Wang, Khanh Duy Nguyen, Ansel Blume, Nanyun Peng, Kai-Wei Chang, Heng Ji (* equal contribution) ACL 2025 (Main) |
|
MM-PoisonRAG: Disrupting Multimodal RAG with Local and Global Poisoning Attacks [paper] Hyeonjeong Ha*, Qiusi Zhan*, Jeonghwan Kim, Dimitrios Bralios, Saikrishna Sanniboina, Nanyun Peng, Kai-wei Chang, Heng Ji (* equal contribution) UnderReview |
|
Visual Backdoor Attacks on MLLM Embodied Decision Making via Contrastive Trigger Learning [paper] [page] Qiusi Zhan*, Hyeonjeong Ha*, Rui Yang, Sirui Xu, Hanyang Chen, Liangyan Gui, Yu-Xiong Wang, Huan Zhang, Heng Ji, Daniel Kang (* equal contribution) UnderReview |
|
PAPO: Perception-Aware Policy Optimization for Multimodal Reasoning [paper] [page] Zhenhailong Wang, Xuehang Guo, Sofia Stoica, Haiyang Xu, Hongru Wang, Hyeonjeong Ha, Xiusi Chen, Yangyi Chen, Ming Yan, Fei Huang, Heng Ji (* equal contribution) UnderReview |
|
Energy-Based Transformers are Scalable Learners and Thinkers [paper] [blog] Alexi Gladstone, Ganesh Nanduru, Md Mofijul Islam, Peixuan Han, Hyeonjeong Ha, Aman Chadha, Yilun Du, Heng Ji, Jundong Li, Tariq Iqbal UnderReview |
|
Constructive Distortion: Multimodal LLMs with Test-Time Image Warping [paper] Dwip Dalal, Gautam Vashishtha, Utkarsh Mishra, Jeonghwan Kim, Madhav Kanda, Hyeonjeong Ha, Svetlana Lazebnik, Heng Ji, Unnat Jain UnderReview |
|
Must Read: A Systematic Survey of Computational Persuasion [paper] Nimet Beyza Bozdag, Shuhaib Mehri, Xiaocheng Yang, Hyeonjeong Ha, Zirui Cheng, Esin Durmus, Jiaxuan You, Heng Ji, Gokhan Tur, Dilek Hakkani-Tür UnderReview |
|
Generalizable Lightweight Proxy for Robust NAS against Diverse Perturbations [paper] Hyeonjeong Ha*, Minseon Kim*, Sung Ju Hwang (* equal contribution) NeurIPS 2023 |
|
Effective Targeted Attacks for Adversarial Self-Supervised Learning [paper] Minseon Kim, Hyeonjeong Ha, Sooel Son, Sung Ju Hwang NeurIPS 2023 |
|
Few-shot Transferable Robust Representation Learning via Bilevel Attacks [paper] Minseon Kim*, Hyeonjeong Ha*, Sung Ju Hwang (* equal contribution) NeurIPS 2022 ML Safety Workshop |
📖 Educations
- 2024.08 - Present, Ph.D in Computer Science, UIUC.
- 2022.02 - 2024.02, M.S. in Graduate School of AI, KAIST.
- 2017.02 - 2022.02, B.S. in Computer Science (major) and Bio & Brain Engineering (minor), KAIST.
💻 Work Experience
|
Apple Machine Learning Research Intern |
Seattle, WA, USA May 2025 – Aug 2025 |
|
Samsung Research Machine Learning Engineer, Full-Time |
Seoul, South Korea Feb 2024 – Jul 2024 |
|
NAVER Software Developer Intern |
Seongnam, South Korea Sep 2020 – Feb 2021 |
|
KIXLAB Undergraduate Research Intern Advisor: Prof. Juho Kim |
Daejeon, South Korea Jun 2020 – Aug 2020 |
|
H2K Software Developer Intern |
Daejeon, South Korea Jun 2019 – Dec 2019 |
👩💻 Academic Service
- Reviewer: ICML (2024, 2023), NeurIPS (2023)