Because unless you have the new puck setups (I guess everyone does now?) you have a stupid metal bracket always sitting there on the floor of your truck bed. And regardless of which system you have, you have to wrangle a 100+ lb hitch in and out of your bed if you want to trailer something, and then go back to a clean bed afterwards. Edit: wait, am I confusing 5th-wheel with GN balls? I am. Obviously I am not experienced here LOL.
Regarding safety, you basically explained what I meant. You have to pull out wide on any tight turn with a GN, many people dont, and pull their trailer into a ditch. I've seen it several times. Operator error, but it's easy to do.