RoboCup 2019 and a service robotics stack
In 2019 I was part of a service robotics project at Linköping University for RoboCup.
The stack had seven major pieces: intelligence, a natural-language layer built on Dialogflow, a tablet interface, navigation, motion, vision, and a knowledge database. All of it sat on top of a repository that had accumulated layers over multiple years.
The robot itself was Pepper, built around SoftBank's NAOqi stack, so a fair amount of the work was really about integrating our own systems with the constraints and interfaces of that platform.
My responsibility was the vision system. A large part of that was people detection and representation: finding people in RGB-D data, extracting vectors for them, and building a cleaner interface around the incoming pose stream. I wrote DenseSense to help parse DensePose output and turn it into something more usable inside the rest of the stack.
In practice, that meant feeding person-level detections and body-aligned features into the rest of the robot stack, so higher-level modules had something more structured than raw pose output to work with.
We also had ideas around clothing classification by projecting observations into a shared feature space, but the work mostly stayed at the level of reliable detection, segmentation, and person-level features.
Building the vision system
A lot of the challenge was not the model itself, but fitting it into a messy whole where several subsystems had evolved independently.
DenseSense was my attempt to carve out one part that was at least internally clean: detect people, break them into useful body-aligned signals, and expose vectors that other modules could build on.
The competition
We came to Sydney in good spirits, and left in good spirits too. A fundamental issue in the navigation stack ended up becoming too big to work around: Pepper was essentially too cautious about going through doors. Better safe than sorry, and we did make it through the safety check, but it became a real blocker in competition.
More broadly, the project exposed the limits of a loosely tied together team and codebase. Too many parts had to line up at once, and that year we were not able to push the whole system far enough forward. By the end, attention had already shifted toward doing it properly the next year.
The year after, I became team lead. One of the main follow-ups was to simplify and update the repository structure into a newer repo with clearer ownership and cleaner subsystem boundaries that lived on for a while: new repository.
RoboCup Bordeaux 2020 never fully played out because of Covid, but the cleanup was still the right direction. The newer repository lived on for years after I left, while the old system had simply become too tangled.