I am a small tractor fan, and think that most people buy tractors that are too large for their needs. But in your case I think you might be aiming a bit low. I would suggest nothing under a 30 hp tractor. The advantage to going up in size is that you will get greater PTO hp and the rotary cutter you will be using needs plenty of PTO hp. With a 20hp tractor you will be able to use a 4' rotary cutter. With a 30hp tractor you will be able to use a 5' rotary cutter. With a 35-to-40hp tractor, you will be able to use a 6' rotary cutter. So going up from a 20hp tractor to a 35hp tractor will increase the size of your rotary cutter, which will save you a lot of hours of cutting over the next dozen years!
A similar thing is going to happen with the road maintainence you refer to. I have a 24hp New Holland and it is a great little tractor, but it really is limited to a 4' box blade on any compacted soil. Go up to a 30hp tractor and you can get a 5' box blade. Go up a bit more in size and you can get a 6' box blade.
There is certainly a point where you can get too large. But I think you might want to look at nothing smaller than a NH TC33 or Case DX33 (Case-New Holland makes both, same tractor, different color) as the small end of the scale and look from there up. Kubota makes the L3430. John Deere, Massey, Branson, Mahindra all make tractors in the 35 to 40hp range. Kioti makes the DK35 (which is also available from Cub Cadet). Start looking at these and go up from there. For me, I'd want about 40hp and a hydrostatic transmission for using the rotary cutter between the trees. I like HST for cutting and for loader work, it saves time and is well suited to mowing and loader work. A gear machine will save you money and get the job done too. But over the life of the tractor, the $1000 added cost of the HST works out to about $50 a year, I'm willing to pay for the added ease it provides.
As for tractor types, the road grading work would favor a heavier tractor, the rotary cutter works with pretty much any machine. All the tractors in the 33 to 40 hp range are pretty heavy machines to begin with so find the one that feels good to your rear end. And buy it from a dealer who makes you feel comfortable . . . comfortable that they have a large enough parts department . . . comfortable that they have a well trained service team . . . and comfortable that they will continue to treat you well.