My research centers on the trustworthiness and interpretability of multimodal AI systems β
spanning bias-driven hallucinations, multimodal moral reasoning, and better scaffolded reasoning to evaluate and calibrate user trust. I design benchmarks,
annotation pipelines, and preference learning techniques to improve transparency,
safety, and alignment of multimodal systems.
Jul 2025New work CoCoT is out β written during my first month at CMU!
Jun 2025Dataset HalLoc V1 is released on HuggingFace.
May 2025Joining CMU HCII as a Visiting Scholar through the end of Fall 2025.
Research β¨
My research asks: How can we design multimodal AI systems whose reasoning is transparent, trustworthy, and genuinely aligned with human values β and how do humans perceive, interact with, and sometimes misplace trust in these systems? I pursue this vision through three interrelated research themes:
01Trustworthiness & Interpretability of Vision-Language Models
I develop methods to detect and localize failures in VLMs, examining how model biases lead to unfaithful outputs. I also study alignment β whether a model's internal reasoning and outputs faithfully reflect human values and intentions, rather than superficially mimicking them.
I investigate how people evaluate, trust, and are misled by AI-generated reasoning chains. Using behavioral experiments and real-world deployments, I examine when chain-of-thought explanations genuinely support critical thinking versus when they create false confidence β and what interaction designs can encourage more careful human oversight.
I design benchmarks and preference learning frameworks that capture the continuous, pluralistic nature of human moral judgment across text and image contexts β moving beyond binary labels toward richer, more human-aligned supervision signals.
moral reasoningvalue alignmentmultimodal benchmarks
HalLoc: Token-level Localization of Hallucinations for Vision Language Models
Eunkyu Park*, Minyeong Kim*, Gunhee Kim
CVPR 2025
Abstract
Hallucinations pose a significant challenge to the reliability of large vision-language models. We propose HalLoc, a dataset for efficient, probabilistic hallucination detection featuring 150K token-level annotated samples across VQA, instruction-following, and image captioning tasks. We also introduce a baseline model offering low-overhead, concurrent hallucination detection during generation.
Cognitive Chain-of-Thought: Structured Multimodal Reasoning about Social Situations
Eunkyu Park et al.
Pre-print, Under Review Β· 2025
Abstract
We introduce Cognitive Chain-of-Thought (CoCoT), a prompting strategy that scaffolds VLM reasoning through three stages: perception, situation, and norm. CoCoT consistently outperforms CoT and direct prompting (+8% on average) across multiple multimodal benchmarks.