Luke, your transmission has a high pressure relief valve that lets the transmission stall if the internal pressures get too high. This is what you feel and hear when you have your HST in too high a range and depress the pedal. I'm not sure, but I think your tractor has three ranges and neutral. Did you mention what range you are in when your hear the "squealing"? Does the squealing occur only in the high range, and does it occur in reverse as well as forward? If you can shift to the low range and your tractor has enough pull for your job, it might just be that you need to do work in a lower range when your oil heats up.
My feeling after many hours on a HST tractor is that you can always get more pull out of a cold HST than a hot one. When fluid become hot and thins out, the transmission will stall somewhat easier than when the fluid is cold. In situations where you are in a high range and there is less flow and more torque required, this is where a small leak in a relief valve will be the most noticeable, whether it is normal relief valve lifting or a small abnormal leak.
Your transmission diagram is attached below. Your transmission has a "charge" pump (items 20, 21, 22) that takes fluid from the reservoir and feeds it at low pressure to the main HST. This pump just ensures that the HST does not cavitate under heavy pumping. Inside your HST, there is a relief valve (item 7) that opens to relieve the high pressures. I beleive it works in conjunction with the check valves (items 4, 5, 6). There is one check valve for forward and another for reverse. Any of these valves or a worn internal pump could be the reason for HST stalling when it gets hot.
Normally, check valves and high pressure relief valves are removable from outside the tractor transmission casing. HST transmissions are normally mounted inside the running gear of a tractor with access to the valves being round holes visible on the sides of the outside casing. I'm not suggesting that you start pulling valves without knowing what you are doing, but rather that the valves can be inspected and pressure tests can be done at a dealer's service department without tearing your tractor apart. They can analyze your HST's performance and let you know if there is a need for service. That's what I would suggest since you are not an HST expert and could easily get yourself in over your head on a job like this. I just want you to have a good outcome and grasp of what you need to do.
One other thing... If you have an HST cooler in front of your normal radiator, make sure it is clean and able to cool the oil to the best of its ability. Your oil should be warm to hot. My HST oil reservoir which is the same as my main hydraulic reservoir will get to 150 to 180 degrees in normal heavy summer use. I use a handheld IR thermometer to measure the temperatures on my tractor.
