Building trustworthy AI systems and delightful products
I design, ship, and study AI systems — from agentic workflows and retrieval-augmented pipelines to multimodal human-robot interaction research. Currently a Lead AI/ML Engineer at Booz Allen Hamilton and a Ph.D. candidate at UMBC.
Selected work
Projects
All projects →Things I have designed, built, and shipped.
01TimeSense
2026Mobile-first, context-aware personal time assistant. Learns your routines, commute, meals, sleep, and calendar to tell you what to do next — without making you maintain another productivity system.
Swift · Kotlin · FastAPI · PostgreSQL · LLM
02ResumeTailor
2025AI-powered resume & cover letter generator with job matching and auto-apply workflow.
Next.js · OpenAI · PostgreSQL · Tailwind
03SCOUT++ Toolkit
2025Toolkit for multimodal HRI experiments and dataset benchmarking of instruction grounding.
Python · PyTorch · Vision-Language · Evaluation
Publications
Research
All research →Peer-reviewed work on grounded language understanding and trustworthy human-robot interaction.
Grounded Instruction Understanding with Large Language Models: Toward Trustworthy Human-Robot Interaction
2025Ekele Ogbadu, Stephanie Lukin, Cynthia Matuszek
AAAI Symposium Series 7(1), 223–231
Understanding natural language as a representational bridge between perception and action is critical for deploying autonomous robots in complex, high-risk environments. This work investigates how large language models (LLMs) can support this bridge by interpreting unconstrained human instructions in urban disaster response scenarios. Leveraging the SCOUT corpus, a multimodal dataset capturing human-robot dialogue through Wizard-of-Oz experiments, we construct SCOUT++, aligning over 11,000 visual frames with language commands and robot actions. We evaluate three instruction classification approaches: a neural network trained on tokenized text, GPT-4 using text alone, and GPT-4 with synchronized visual input. Results show that while GPT-4 (text-only) outperforms traditional models in accuracy, its multimodal variant exhibits degraded performance, often producing vague or hallucinated outputs. These findings expose the challenges of reliably grounding language in visual context and raise questions about the trustworthiness of foundation models in safety-critical settings. We contribute SCOUT++, a reproducible multimodal pipeline, and benchmark results that shed light on the capabilities and current limitations of vision-language models for risk-sensitive human-robot interaction.
Ask-to-Act: Learning When Robots Should Clarify Ambiguous Grounded Instructions
2026Ekele A. Ogbadu, Stephanie Lukin, Cynthia Matuszek
IEEE RO-MAN 2026
Presenting August 2026
Robots that follow natural-language instructions must decide not only what action to execute, but also whether the current evidence is sufficient to act at all. We present Ask-to-Act, a framework for grounded robot instruction following that treats clarification as a cost-sensitive decision under uncertainty. The system either executes immediately or requests one minimal grounding cue before acting. We evaluate Ask-to-Act on SCOUT++, a grounded HRI benchmark with command-label-image examples from situated human-robot interaction. In a text-only setting, Ask-to-Act improves grounded action prediction from 0.8225 to 0.8308 accuracy while asking in 0.2350 of episodes. In a multimodal setting, command-time visual grounding improves the always-act baseline to 0.8503 accuracy, and Ask-to-Act further improves accuracy to 0.8545 while asking in 0.1277 of episodes. These results show that Ask-to-Act can adapt its interaction rate to the available evidence.