Problem Statement: Sequential Path Detection – Real-time hand-tracking with MediaPipe and OpenCV, integrated with Arduino for ordered path validation and hardware feedback.
Tools: Python, OpenCV, MediaPipe, Arduino IDE, Serial Communication, NumPy
Concepts: Real-time hand tracking, gesture recognition, spatial path sequencing, state machines, computer vision–based event detection, human–machine interaction (HMI)
To
December 2024
From
July 2024
The project begins by capturing real-time video frames from a webcam and preprocessing them to ensure stable and continuous input for further analysis.
Mediapipe’s hand-tracking model is applied to detect hands in each frame and extract precise landmark positions, with a focus on the index finger tip.
Pixel coordinates of the index finger are continuously computed and mapped onto the webcam frame to track user interaction in real time.
A predefined set of points (A, B, C, D) is drawn on the screen, and a fixed sequential order is established for valid path completion.
Distance-based logic is implemented to determine whether the user’s finger is within a specified radius of a target point, confirming a valid selection.
The system checks whether the selected point matches the expected sequence index, updating the current state only when the correct order is followed.
If an incorrect point is selected, the sequence is immediately reset, and an error state is triggered to notify the user.
Serial communication is established between Python and Arduino to transmit real-time status updates based on correct or incorrect finger movements.
The Arduino controls an LED to provide physical feedback, using different blinking patterns to indicate correct selection, incorrect paths, or successful completion.
Once the full sequence is completed correctly, the system displays a success message on-screen and triggers a slow LED flicker to indicate task completion.