Don't know the capabilities of your 21 compared to my JD4300 but there is a good chance that almost all the rocks you hit will be the same size as those on the surface. If you aren't in a hurry I'd just start at the bottom, well over to the side and start moving stone from the inside to the outside. With all that rock you should wind up with a solid base.
Just work your way up the hill, making it just passable for the tractor.
IF you do hit a rock that you can't dig under and roll downhill, then find a bigger backhoe or a small excavator for THAT problem.
I think you might be surprised at how much you can accomplish, it will just take time.
Once you get a passible trail, then start bringing in the fill to smooth out the rough spots.
I have a section of sidehill trail out back that I started two years ago, working a few hours at a time once or twice a month during the summer. It only has a 60' drop in 500', but I had bigger rock (some bedrock as well) and more trees with very little fill handy. The original trail had 80' drop, then a 20' rise in 550'. Now I can use the new trail even when covered with snow. (tractor or ATV) The old trail was constantly erroding and once the snow came it was impassible till spring (except by skidoo)