Not really an RTOS. I hear it's more an "event loop and some interrupt handlers," much like the firmware in the Kinect.
(We went through a "we need an RTOS" phase on the Kinect. Eventually we ditched the thread stuff we wrote, which made things a lot simpler -- if you don't absolutely have to have threads and can get away with computing stuff in a main loop, then do that. Threads introduce all kinds of synchronization and inconsistent execution time nonsense that you should avoid if you can).
fascinating. I assume that was the first kinect. Was the vision processing separated from the USB interface in different chips? Sounds like a fun project.
- A "DSP" based camera system that just does transfers from the sensor system to USB, and that does some other monitoring and housekeeping. It does no vision processing, it's just pass-through of video from the chips to the host.
- An ARM-based audio system that handles mic data, runs an echo cancelation algorithm against host-provided speaker data, and provides the raw and echo-canceled mic data to the host.
Early Kinect versions used yet another processor for managing the tilt motor and accelerometer. This stuff was move to the ARM later (that a tilt-motor processor existed in year 1 units is a fine example of team structure affecting product structure).
All of these CPUs have their own USB interfaces, and there's an internal USB hub so there's only one wire going to the host. :-)
None of these chips use an RTOS; that would just get in the way.
The vision processing is done on the host, where you have heavy lifting capability with GPUs and tons of memory and so on. Doing that processing on the camera would quite expensive in terms of hardware and power, and wouldn't be able to adapt as well to new algorithms.
(We went through a "we need an RTOS" phase on the Kinect. Eventually we ditched the thread stuff we wrote, which made things a lot simpler -- if you don't absolutely have to have threads and can get away with computing stuff in a main loop, then do that. Threads introduce all kinds of synchronization and inconsistent execution time nonsense that you should avoid if you can).