I have two tractors for moving round bales. My 80 hp tractor handles them easily with the loader and it's what I use most of the time. I can unload the trailer with it and put them where I want them. If you haul a bale with a truck, you can just roll it off. Same with a trailer if you load them so they roll off. But if you road them sideways so you can get more of them in the trailer, you will need something to pick them up, and off of the trailer. I don't think you will find a tractor that can do that under 50hp, but I might be wrong of that by a few HP, but not by much.
When my bigger tractor has broken down, I've been able to get the bales out to the horses with the 3pt hay spear on my 35 hp tractor. It's a little iffy on being powerful enough to do this, but it's made it so far. You have to get the bale just right, or it slides off. You need weight on the front tires so you can steer because there is so much weight off of the rear, that the front end gets light.
If all you had was something smaller, you might give it a try, but odds are pretty good that you will break something fairly quickly. Since you haven't bought anything yet, you need to set a minimum HP needed to do what you want and not even look at anything smaller. In my opinion, that should be 50 hp.