Just because someone doesn't look handicapped doesn't mean they aren't. Looking at my mother-in-law, you'd never know she had a liver transplant that just about killed her, she gets weak walking more than 100 feet, gets dizzy easily when she walks, etc... And her husband, who's 90, doesn't show that he has several replaced joints, can't hear, bad arthritis, and probably couldn't find his way back to the car if it wasn't parked right in front of the door (he's not driving).
So, if we drive their car, I'll park in the handicapped spot for them so they don't have to walk so far. I won't drop them off and move the car, because that means when its time to leave, they'll have to find a wall to lean against until I get the car and drive it around. Each situation is unique. Yes, there are some folks that abuse the tags. But you have no way of knowing just because they look fine to us.