First off, are you running at full throttle?
Second, was it the parking brake semi-enngaging?
The sound of gravel grinding in hydraulics can be cavitation in a wheel motor. That's why I asked if you have it at full throttle.
The way the wheel motors are set up, the right front and left rear are connected in series. The left front and right rear are connected in series.
So back to your situation.... you're traveling downhill with weight in the bucket. When you slow down, weight will shift towards the front of the machine, lightening the rear. So it presses weight down on your front tires, taking weight off the rear tires. The hydraulic pressure in your front wheel motors will prevent the rear wheel motors from turning faster than the fronts. Since there's little traction on the rears as your machine tips forward, it wouldn't surprise me of a rear wheel would slide on loose material..... slow down.
As for the gravel noises, if it's coming from the wheel motor, or pump, its cavitation most likely. It's a pressure differential that can suck air out of the fluid. Again, always run at full throttle, and go slower downhill with loads if you already aren't.