I think it has been mentioned in passing, but the OP asked about the downside to HST. I think the benefits have been well explored and for the most part there aren't any huge downsides except cost. And more specifically cost vs hp. I'm not talking about hp loss vs a geared tractor. I'm talking about what amount of hp can you get in a geared vs an HST for the same amount of money. That's where you get into a pretty big difference, and if you need the hp, a pretty big downside.
I have a Kubota L4400. As basic a geared model as you can find. No GST or shuttles or even synchro (theoretically you have to stop before shifting...I can shift on the fly if things are timed right.) It is a 45hp tractor. I wanted the most powerful tractor I could get for under $20,000. The L4400 is it.
Take a look around and see the max hp you can get for $20k in an HST machine. It is considerably less.
The dealer gently pushed me towards an HST machine. I think he showed me a 3400 at a similar price and a 4130 at considerably more. I wanted, and as it turned out, need the additional hp.
So don't forget that there are some folks who might enjoy and benefit from HST that simply can't fit it in our budget and still get the power that we need.
I also happen to be one of those types who see no particular use in power windows and I currently despise the automatic transmission in my F150. Maybe I'm a control freak, but I prefer manual trannies in my vehicles. As soon as I'm done with this F150, I'm going to find something with a manual. Maybe an F250.
As far as operating the L4400, I'm super comfortable with it and can do extensive loader work without ever thinking about shifting or even look down to shift. Of course, my 43 year old left knee is sore after doing a lot of loader work. No arguing that. But for mowing, BBing, etc. No knee pain.
The other thing that affected my decision was repairs. I have a B-I-L with a complete tractor shop 1/2 mile down the road. He repairs all his big tractors himself including engine and tranny rebuilds. But he has never cracked anything but a full manual. With that in mind, plus the dollar to hp ratio, a geared tractor is perfect for me, especially since I'm a manual kind of guy anyway.