Can't add a whole lot to the good comments already made, except regarding your concerns re. tractor sizing. If you expect to be jockeying logs around you'll benefit from using a tractor in the 40 hp and up range, not so much because of horsepower but because that size tractor, when it's all dressed up in its "working clothes" (i.e. with FEL, ballasted tires, and something heavy on the 3pt.), will hit the trail at 6 to 8 thousand pounds. That's enough size that you can pick up a 1500+ lb log with your FEL and take it someplace if needed, and you won't feel like your tractor is on the ragged edge of tipping with every little rut you encounter. Pushing brush and logs, same story. Pulling logs, likewise.
So your instinct is right about this, in my opinion. Sure you "can" do such work with a lighter machine, but I've found that clearing and removing trees is inherently hard, dangerous and tiring, even for guys much younger than me. Last thing you need is to also be worrying about your equipment being overmatched too. You've got a serious project in mind so go with some serious equipment. Just one guy's opinion.
Also just my opinion, but the MX class of Kubotas is a good value for such equipment and the GL models are even nicer, high-end choices. The
L4701 would also do the job. Same concepts would apply with other makes also.