Excluding the discussion about modification of the purchased unit, it really depends on what you need the unit to do.
If you are buying it for a backhoe and loader for small earth moving projects, you don't require a ton of horsepower, what you need is weight. Weight makes traction, horsepower spins the wheels. Lots of tire spinning when working earth makes a mess of the job site fast. Too much power is a bad thing.
If you are buying the unit to operate mechanical PTO powered equipment like a round baler, the tractor needs enough horsepower per pound tow the loaded baler up and down hills while also driving the baler mechanics.
If you are mowing grass, you want lots of horsepower to clear gobs of tall grass from the mower deck without bogging, and you want light weight so you're not compacting the land or making ruts in soft areas.
The expectation that any tractor is great at all tasks is a falsehood created by salesmen that don't know how to use the machines they are selling. When you tell them you have a list of things you want to do, they should present you with a list of tractors and why each of them is better at one of your given tasks than another. Educating their buyer seems low priority to most unfortunately.
I think also, those "same frame and block with different horsepower ratings" also come with different sized tires, which brings into the discussion a lot of other variables.