Lingxiao Zhu

Emo-Copilot

Modern cars that incorporate AI technologies for safety, navigation, and entertainment often overlook the potential for emotional support. Most existing systems are designed to respond uniformly, regardless of the driver's emotional state. My research project, Emo-Copilot, aims to address this gap by developing a prototype AI in-vehicle system that offers emotional care to drivers. By detecting and responding to a driver's emotions in real time, Emo-Copilot demonstrates how emotionally aware AI can create a more personalized and supportive driving experience. The prototype consists of both a physical 3D-printed AI assistant equipped with a simulated touchscreen, along with a responsive user interface (UI) that provides the interactive experience on this simulated touchscreen. The AI assistant connects to a physiological sensor that continuously captures real-time Heart Rate (HR) and Heart Rate Variability (HRV) data from users wearing the sensor. The UI is a React-based website consisting of five main pages. The first page is a welcome page that greets the driver with a brief welcome voice message. The second page is a dashboard for real-time biometric data and assessing the driver's emotional state (e.g., anxious, stressed, focused, neutral, calm, relaxing, etc.) based on variations in HR/HRV values. The third page allows the user to activate the co-driver assistant, which responds to over ten voice commands (that users can accept or reject) and will trigger corresponding emojis. The remaining two pages offer optional music genres and a playback console, respectively. The UI includes acoustic control buttons for in-car facilities such as air conditioning and lights, and all interactions are designed to be non-disruptive, prioritizing the comfort and emotional well-being of drivers. Additionally, the demo includes a 11-page storyboard that illustrates Emo-Copilot’s functionality, showcasing various driving scenarios and its adaptive emotional response features.