I use poison, and find that they only die in the walls if that is where they are already living. (I've not heard good reports of ultrasonic devices either).
I put out poison regularly, but don't keep it out all the time. I put it out around the sheds and the house in the fall when the mice are looking for shelter, but their home is away from the building. That way, they find the poison, and take it back 'home' with them, eat it and die there.
In 35 years, I have never had a mouse in the house, honest. I did have the 'died in the wall' problem one year when putting on new siding, and there was a mouse nest started in the insulation, getting in from the outside. When I sided it up, and put out the poison, they died and smelled. (I'd gladly take that smell once in awhile over the smell of cats. /forums/images/graemlins/grin.gif ). Sorry, cat lovers. Not for me; beenthere, donethat.
Once in a while, I have found a mouse in the garage. Last one was (I think) when a friend left their car in the garage while travelling, and there may have been a mouse in the engine compartment. One showed up anyway, and was caught in the sticky trap that I have set out for moths, bees, flies, spiders, etc. Works great for them, as well as the dreaded asian lady beetle (starting to show up now - grrr).