I used to have a small trackball on a laptop many years ago, and at least once a day I had to take it out to clean off the lint.
Now its small size might have made it more prone to such issues, but are there issues with the large ones?
I can appreciate that a larger ball has the momentum to propel the cursor hands-off.
Upside down mouse
One thing that modern mice have these days is that they are optical with no moving parts, so no dust jamming issues (at least if your desk is not a dust bed!).
Just as an experiment, I turned my optical mouse bottom up, and used my finger to swipe over the optical sensor. Besides forward-back working the opposite to the way left-right did, and the instability of resting on its curve, operation was reasonably accurate. Makes me think that they could just as easily get rid of the ball out of trackballs and go optical as well, so that you could have a mouse-shaped (as opposed to large and flat pad) control that you didn’t have to move.