There are several old threads on this. You won't gain stability due to the center pivot axle and you put more load on your bearings etc. Whether it is enough to cause harm, I just don't know.
A good rule of thumb is to turn the wheels as the tractor sits stationary. If the tire makes a scuffed mark on the ground then you are good. If the wheel makes an ark on the ground then you will likely cause increased wear and probably break something if you have a loader on.