Seems like plenty of votes for cats, but I can't have them round my place as it borders national park and is a haven for birds and other nice little native critters.
The mice were virtually at plague proportions in early spring, so I learnt a lot about them. Had to throw out a brand new oven in the house I just bought as the mice had taken a shine to the wadding insulation and the thing was totally infested. As others have said, they can get through the tiniest gap. If you can put your pinky through it, it isn't mouseproof.
Couple of things I learned:
Block all the holes you can. Expandafoam in a can is great for this and the mice don't like it. Also, mice only work really hard to get in somewhere if they can smell food. They don't tunnel like mad just to see if there's a nice comfy nesting site on the other side. It isn't an all-or-nothing thing either - they aren't exactly clever - if you halve the number of holes for them to get in, you probably halve the number of mice that get in.
Don't put baits in places you don't want mice. The baits attract them. Put the baits where you DO want the mice. Let them set up a nice little home near the baits and eat and $#!t and die away from whatever you are trying to protect. I couldn't believe my neighbours putting bait all through their houses and then complaining they had mice inside. I put all my baits outside the house, blocked the ingress points as best I could and just put a couple of traps down inside to get the buggers that did get in, and I had the least problems with mice despite the house already being infested when I bought it. Old habits die hard though, I was the new guy and they wouldn't listen...