In reading this I think a little machine like a JD 1 series, BX, Massey GC would be right up your alley. It sounds to me (again I am very new at this to take this for what it will) nothing you are doing is going to need a really big machine, but I think you are going to want a swiss army knife of machines.
All of those can move stuff you are going to need for your garden, and a loader is so much more handy for stuff like that, good dirt, mulch, compost. Then they can help with things like borders for the garden beds, rocks or whatever you want to use. They all have a 3pt hitch and the power to run anything for your garden, tillers,
chipper/shreders (you have lots of trees that are going to come down and need upkeep...even the fruit trees need to be kept open so sun can get in and get everything) If you get a belly mower it is much more easy to get around things. And it is pretty modular.
It sounds like your dads place is too far away to be real easy to get to just for that one quick thing you need to do, but I would also talk to him and see what he thinks...he knows machines and could be thinking in the back of his head well if I only had this (X) it would be so easy for me to do this project. He may want to go in on it with you.
I don't think a zero turn would help you too much, the main advantage is speed and if you can't take that, then you might as well look to a machine that can do all your chores/hobbies in one machine.
I may sound like I am pimping the little tractors and I guess I am....I just got one 3 months ago and I am totally amazed at what it can do...we have not started our garden yet but we are in the process of doing fruit trees, my wife wants peaches, then to grow bulbs for tulips and other flowers to sell. Sounds like we are going to be doing a great deal of the same thing.
Keep us updated.