Automated Recognition and Spatial Localization of Laboratory Instrumentation in a Materials Synthesis Setting Using a Collaborative Robot

Abstract

This work presents a methodology for estimating the spatial position of instruments in a materials synthesis laboratory using a 2D camera. This approach is considered a cost-effective alternative to depth sensors for automating tasks in such laboratories. In our case, the methodology is integrated into an ABB collaborative robot, YuMi IRB 14000 model. Instrument detection is performed using a YOLO-NAS network retrained with a specialized dataset. Subsequently, the 2D position of the detected objects is estimated based on camera calibration and the use of projective geometry. Finally, the integration of this system with the YuMi robot controller in Python is described, enabling autonomous interaction with the instruments. Error and precision metrics for distance and location estimation are presented, demonstrating that this solution is viable for low-cost robotic manipulation applications.