I'm of your same mindset, as in I backfeed my house when I need to. And I do it from a sub-panel in my garage, which feeds back to the house. So I have to kill the main breaker outside under the meter, kill the main breaker in the house panel, etc... before doing the generator hookup in the garage.
However, as I get older, and hopefully wiser, I'm leaning towards being a bit safer, and if I can take a risk out of the equation, especially since it's so cheap to do now, I should do it. Accidents happen. I forget things once in a while. It's just not worth it when I can put a lockout on it for well under a hundred bucks.
We put one on my in-law's breaker panel and wired a "real" generator outlet onto the outside of their house. It works great.
And as far as it being as simple as getting in the truck, turning the keys.....
How many times have we read about people driving right through a store? These are people that have been driving for years, yet they have a brain freeze and BOOM.
Why do drivers crash into buildings? Statistics and reasons why.
So I figure I'm not infallible. It could happen to me. So why do it anymore when the safety system is so good and so cheap?