John, the 3PH can have the maximum height cutout lever mis-adjusted and cause the main hydraulic system to go into relief bypass, but that has absolutely nothing to do with the HST transmission. That's from the main pump on the engine. The main hydraulic system and the HST both draw fluid from the reservoir through the big hydraulic filter on the right side back near the rear wheel. After that filter, the big line runs up under the right-side platform and splits at a "T" in the line. One leg of the "T" is going to the HST and the other leg goes to the main pump. This line is under suction anytime the tractor is running and the rubber "T" needs to be very tight.
I can tell you that the charge pump for the HST will squeal when it cavitates. I recently stalled my tractor under load and just before the engine died, the HST charge pump cavitated. The HST pedal got harder to operate and I could hear the charge pump. That's why I think you might be drawing air. As you guessed, it also could be clogged filters. If the pump cannot get enough fluid, it will cavitate and squeal.