Design and Implementation of Web-Based Control and Object Recognition on a Chinese Chess Player Robot Manipulator

Abstract

This study discusses the design and implementation of a web-based control and object recognition system for a Chinese chess player robot manipulator connected to a mini PC via the User Datagram Protocol (UDP). The developed web interface allows users to adjust the angles of six joints (base, shoulder, elbow, wrist pitch, wrist roll, and gripper) using sliders, record and playback motion trajectories, and view the game board's video streaming equipped with YOLOv5 model-based piece detection annotations. The mini PC acts as a web server, image processor, and UDP client that sends movement commands to the ESP32 microcontroller, while the ESP32 converts the commands into PWM signals to drive the servos and sends acknowledgments back for round-trip latency measurement. The implementation results show that the web interface can integrally alter the poses of the 3D model and the robot arm, while the detection module is capable of marking the pieces with bounding boxes and class labels, although momentary detection failures still occur under occlusion and light reflection conditions. Latency testing of 40 trials per joint resulted in an average latency ranging from approximately 864 ms to 1430 ms with a standard deviation above 400 ms, as well as a maximum value of over 4s on the base joint, indicating the presence of delay variations due to command queues, computational load, and network quality. These latency values are still acceptable for teleoperation-based Chinese chess game scenarios, but further optimization of the communication and visual processing pipelines is required to improve system responsiveness.