Harsh Nishant Lalai
Incoming PhD Student, Johns Hopkins University
I'm interested in trustworthy AI, focusing on language models under uncertainty and multi-agent
social interactions. My goal is to develop rigorous evaluations and robust methodologies
that enable increasingly capable AI systems to be deployed safely and reliably in
real-world settings.
I've collaborated with researchers at
Stanford,
Harvard,
Georgia Institute of Technology,
Pennsylvania State,
and
Emory University.
Stanford,
Harvard,
Georgia Institute of Technology,
Pennsylvania State,
and
Emory University.
Education
Aug 2026 - Present
PhD in Computer Science
Johns Hopkins University, Baltimore, MD
Advisor: Kristina Gligorić, Co-advisor: Benjamin Van Durme
Oct 2021 - Jul 2026
Bachelor of Engineering in Computer Science, Master of Science in Economics
GPA: 9.40 / 10
Undergraduate Thesis at Georgia Institute of Technology: LLMs' Intermediate Thinking Tokens Are Weak Predictors of Human Cognitive Effort in Scientific Misconception Judgments
Research Experience
July 2025 - May 2026
Georgia Institute of Technology, Atlanta, GA
Undergraduate Thesis Researcher
Paper: LLMs' Intermediate Thinking Tokens Are Weak Predictors of Human Cognitive Effort in Scientific Misconception Judgments
Investigating whether LLM reasoning-effort signals (thinking tokens) correspond to human cognitive effort on scientific misconceptions.
June 2025 - March 2026
Harvard University, Cambridge, MA
Research Collaborator, Visual Computing Group
Paper: When Visuals Aren't the Problem: Evaluating Vision-Language Models on Misleading Data Visualizations
Studying VLM robustness to misleading chart and caption pairs and the ability to detect fine-grained reasoning and visualization errors.
May 2024 - March 2025
Emory University / Brock University, Atlanta, GA
Visiting Scholar
Paper: The World According to LLMs: How Geographic Origin Influences LLMs' Entity Deduction Capabilities
Evaluating implicit geographic bias in LLMs through a multi-turn entity deduction game. Used notable people and culturally significant objects from diverse regions as entities, and investigated the effect of pre-training frequency and popularity on deduction.
Dec 2023 - May 2024
Pennsylvania State University, University Park, PA
Research Intern, PIKE Group
Mentor: Dongwon Lee
Analyzed text-watermarking research by intentions, evaluation datasets, and addition / removal methods to construct a unified taxonomy, and highlighted gaps and open challenges in protecting text authorship.
Employment
Jan 2026 - June 2026
SingleStore, Hyderabad, India
Software Engineering Intern, AI Team
Building an experimental semantic retrieval benchmark for SingleStoreDB, comparing transformer-based encoders (e.g., MiniLM, E5, Cohere Embeddings) and vector-index configurations.
May 2023 - Jul 2023
Jio Brain (formerly Jio Platforms), Mumbai, India
Software Engineering Intern, Automated Transcription Team
Developed preprocessing utilities and evaluation scripts for JioBrain's Speech-to-Text module, automating dataset ingestion and inference-quality testing across multilingual pilot datasets for Hindi, Gujarati, Marwari, Marathi, and Tamil.
Honors and Awards
2025
K. M. Birla Foundation Student Travel Grant
Awarded for travel to COLM 2025. Given to selected students with an accepted first-author paper at a top-tier international conference outside India.
2024
Selected from over 30,000 applicants for top Canadian research universities. Fully funded research internship at Brock University under Professor Ali Emami.
2023 - 2026
Merit Scholarship, BITS Pilani
Awarded for six consecutive semesters to the top 1% of approximately 800 students for academic performance.
Publications
Conferences
2025
Second Conference on Language Modeling (COLM) 2025
2025
Findings of the Association for Computational Linguistics: NAACL 2025
Preprints
2026
When Visuals Aren't the Problem: Evaluating Vision-Language Models on Misleading Data Visualizations
arXiv preprint . Under review at ACL Rolling Review (March 2026)
Teaching
Fall 2024
Undergraduate Teaching Assistant
Birla Institute of Technology and Science, Pilani, Goa Campus, India
International Economics (ECON F311)
Created a term-paper-based assignment to give students hands-on research exposure.
Spring 2024
Undergraduate Teaching Assistant
Birla Institute of Technology and Science, Pilani, Goa Campus, India
Econometric Methods (ECON F241)
Designed coding exercises to give students hands-on experience with R and Stata.
Fall 2023
Undergraduate Teaching Assistant
Birla Institute of Technology and Science, Pilani, Goa Campus, India
Financial Management (ECON F315)
Created homework, assignments, and assisted students with their capstone projects.
Spring 2023
Undergraduate Teaching Assistant
Birla Institute of Technology and Science, Pilani, Goa Campus, India
Principles of Economics (ECON F211)
Created homework and assignments for the course.
Service
Reviewing
ACL Rolling Review (ARR) 2026, 2025
Conference on Language Modeling (COLM) 2026
International Conference on Mathematics and Computing (ICMC) 2023
Volunteering
International Conference on Mathematics and Computing (ICMC) 2023
International Conference on Data-Driven Computing and Intelligent Systems 2023
International Conference on Advances in Data-Driven Computing and Intelligent Systems 2022
Technical Skills
Proficient
Python, C, C++, PyTorch, TensorFlow, Scikit-learn, Pandas
Comfortable
Java, LaTeX, Git, NLTK
Familiar
R, MySQL, HTML, CUDA
Languages
Native Proficiency
English, Hindi, Gujarati, Kutchi
Expert Proficiency
Marathi
Hobbies
Sports & Fitness
Football, Cricket, Badminton, Table Tennis, Mixed Martial Arts, Gym Training
Reading
Autobiographies
Media & Entertainment
Sports, Sports Documentaries, Action & Thriller Movies and TV Series, e-Sports
References
Dr. Kristina Gligorić, Assistant Professor
Department of Computer Science
Johns Hopkins University
Dr. Benjamin Van Durme, Professor
Department of Computer Science
Johns Hopkins University
Dr. Sashank Varma, Professor
School of Interactive Computing
Georgia Institute of Technology
Dr. Yi-Chia Wang, Principal Research Scientist
Carnegie Mellon University
Visiting Scholar, Stanford University