Invited back in 2024 to mentor and lead a new project, I guided my team in developing a necklace for visually impaired users. This device provides environmental awareness via auditory and haptic feedback, driven by facial and object recognition models.
I implemented the recognition models using OpenCV's Haar Cascades and designed a priority-queue architecture to deliver timely, sequential, and duplicate-free information to the user via the ElevenLabs TTS API.
Highlights:
- Full Stack Integration: Captured and streamed sensor data from embedded hardware to a Flask backend, paired with an iOS frontend for visualization and user interaction.
- OpenCV: Built facial and object recognition pipelines using OpenCV (Haar Cascades), integrated with a prioritized messaging system to surface the most relevant contextual information.
- HRI: Designed the end-to-end user interaction loop (audio/haptics) with low-latency, duplicate-free messaging and a focus on clarity and cognitive load.
- User Experience: Designed low-latency, duplicate-free notification delivery using a priority queue and ElevenLabs TTS for natural auditory feedback.