I have used my 22-250 for paper punching at 1000 yards. With only 400 yards to the next house you are going to be sending lead over to the neighbors. A centerfire 22 is way to powerful for the ranges you are looking at.
I think the important question that no one has asked, how close can you get to them before they get spooked. I would look at 22 rimfire or a break barrel pellet gun. Rimfire I would get a ruger 10-22, its more than accurate enough, though a bolt action is a little quieter. Pellet gun, I would get a RWS. Don't think of the pellet gun like your red rider BB gun, these are pretty serious, they get the pellet going out the barrel 1000 fps give or take.
I would still be careful of the 22 rimfire, it can still skip if you are not careful as already mentioned.