An empirical scientific discipline requires a set of methods and practices for evaluation and comparison of proposed models and solutions. RoboCup provides a unique common test scenario for robotics, but its potential is, by far, not realized. Conducting games incurs high cost in terms of effort, time, and money. The scientific outcome, however, is quite limited and often not very conclusive. In most cases only the final score of the games provides feedback about the performance of a team.
In this project we develop a toolbox (a set of tools) to support collection, organization and analysis of large amounts of RoboCup specific data enabling detailed analysis to promote data driven research and development in RoboCup.
Main components of the robocup.tools ecosystem are:
- Collection and Synchronization
– automatic synchronized video recording, communication data, logs recorded by the individual robots, synchronization tool for not synchronized videos;
- Storage, Formats and Query
– log files unrolled into a database, events as JSON, logging on robots based on Protobuf (backward compatible), access through web-interfaces;
- Mining and Extraction of Information
– scripts to extract events from logs (e.g., robot fell), tool for manual annotation of events, detection and tracking of objects in videos with machine learning;
- Aggregation, Analysis and Visualization
– statistics over events (identify systematic problems), visualization of strategies, access quality of the perception of the robots over the course of the game
Setup Tutorial:
https://www.youtube.com/user/teamhumboldt
RoboCup Videos
Links:
Main components of the robocup.tools ecosystem are: