We bought a couple of goats (does) for the same reason, as well as the thought of fresh milk. We bought them bred, consequently we now have 5. They are Nigerian Dwarfs. Fun, cute, and the 2 does produce about 1/2 gallon per day! They are relatively easy to keep, and most have told me that as long as they are happy, they won't try to escape. Ours are happy. They will lay into a fence (literally) so make sure it's tight or they will stretch it. We feed ours a combination of Purina goat chow and beet pulp (2:1, gc:bp) and local Alicia hay. We probably spend $10-$15 per month on them. Contrary to popular belief (our) goats do not eat everything, BUT, they CHEW EVERYTHING! They have chewed trailer light wires, the seat to my tractor, fish hooks, my shirt (while wearing it), and countless other things. Ditto on the bucks, they STINK, no 2 ways about it. Also, goats are social creatures, one will be very lonely and unhappy, you should get at least two. You can see some older pics of ours at <A target="_blank" HREF=http://www.dobberspond.com>http://www.dobberspond.com</A> in the critters section. (Don't rib me too much about the site, it needs completion & updating horribly!)
Go for it!
Dob