Mike,
The touch and mouse is similar to the fbdev interface. There are no additional layers or glue code needed by QT.
You need to build QT with mouse and touchscreen support and setup the proper environment variables. For more details inside the SDK, you can find the QT launch script here: (SITARA_SDK_6.0_ROOT_FS)/etc/init.d/matrix-gui-2.0.
You can only select mouse or touchscreen, so the script tries to detect a touchscreen and if none exist, then it sets the default to mouse.
And this article talks about selecting Mouse instead of touch. http://processors.wiki.ti.com/index.php/How_to_use_a_Mouse_instead_of_the_Touchscreen_with_Matrix