I can comment since my PS wasn't working properly until some recent shop work. However, steering sectors for non-PS tractors may use a different gear ratio than sectors used with PS, so a comparison with a barely working PS system may not be accurate.
Yes, the steering can be very hard and managed only with a lot of muscle. With the tractor stopped, a load in the bucket and the wheels in a hard turn, it can be extremely difficult to straighten the wheels. However, there are some things an operator can do to manage. First, extra rear ballast lightens the steering. Second, never stop the tractor with the wheels in a hard turn. Develop the habit of straightening the wheels before coming to a stop; when, for example, backing away from a pile after picking up a bucket load. Finally, the front wheels can be lifted with the bucket if all else fails. The later tip may affect the choice of a loader. My loader is not able to apply much down-pressure when the bucket is flat. For me to lift the front, I have to drop the bucket load.
I expect you'll like a loader with or without PS. I have a decent snow blower, but the blower can't be used very well for everything. I still use the loader quite a bit.