Albert

Albert, is a rock-paper-scissors robotic hand designed to serve as a playing partner for the rock-paper-scissors game, aiming to bridge human-robot interaction.

​

Rock-paper-scissors is a universally understood game, known for its simplicity and effectiveness in decision-making. Our objective was to establish a unified language between a robot and a human by humanizing the robot. We aimed to devise an intuitive solution that users could easily grasp by incorporating vision and other sensors to facilitate this interaction.

​

Hardware

The system employs a webcam to read the user’s hand position, which is processed by a computer depicted in the gray box above. This computer serves as the main processing unit and is separate from the hand, allowing it to be placed elsewhere in the room. It runs the hand-tracking node, MQTT broker, and a game interface that calculates the winner and provides feedback to the user about the score. The robotic hand, wireless and independent from the computer, is equipped with a Raspberry Pi Pico W running MicroPython. 5 servos for each finger would control the individual fingers. We incorporated an ultrasonic sensor for assessing the distance necessary for a handshake or high five, and a beeper was added for auditory feedback.

​

Hand Pose Detection

The MediaPipe library by Google offers a powerful hand tracking solution for real-time hand pose estimation. Leveraging 21 landmarks, it accurately identifies and tracks key points of the hand. By customizing the code, we enabled it to discern specific gestures like rock, paper, scissors. For example, by comparing the positions of landmarks 20 and 17, we determine if the pinky finger is closed. However, our implementation currently works optimally only when the hand faces the camera directly vertically (utilizing y-coordinate for finger openness) or horizontally (employing x-coordinate). Recognizing hands from different angles remains a challenge for future improvements.

​

Game Interface

The interface was developed using Node-Red, a component-based programming language built on Node.js. This script subscribes to both the “hand_move/human” and “hand_move/robot” topics, determining the winner of each round. The game is configured to require three wins (or three draws) for victory. Once a winner is determined, a stop/reset command is dispatched automatically (via the “/start” MQTT topic) to the robot, prompting it to enter a waiting state until a player initiates another round. Moreover, a stop/reset button is provided for the player’s convenience, enabling them to halt the game at any time.

​