Ditto Ditto Ditto, as one who loves horses, please don't start with a young horse. There are plenty of horses 7, 8, 9 years old out there, who have perhaps been trail horses or something that would make great first horses. Keep in mind there are two lives at stake here (or more depending on who rides him/her). You don't want to endanger yourself or other riders for sure, but also keep in mind that you are shaping that horses life as well. If it is not properly broken and trained then you will not be happy with it, and when you try to sell it you'll have a hard time finding it a good home.
I'm not a sappy tree hugging animal rights activist or anything, but I truly love horses and always find it sad to see a horse that hasn't been properly taken care of. Also, I don't care for the 10 horses/acre plan. Sure they can survive, and not all breeds need the same space, but it's also sad to see a horse standing in a mud pit because the owners have more horses than land.
The money part, I bought my first "real" saddle on Ebay from someone who's ID was "horses8mymoney" I thought that was a joke, and it's funny, but it's real too. They get expensive and they need attention. That's another thing, one animal like that is not the best idea. Horses are pack animals and desire time with other like animals. Your horse will be happier if there's some other critter out there with it. They also desire time with you, LOTS of time. Don't underestimate the time and money commitment you are making when you buy a horse. Can I suggest that you pick up a copy of "Horses for Dummy's" I'm not saying you are a dummy, I read the book when I started out with them and it was a great place to learn the basics.
Also, pony's, not sure why they got such a bad rap. I'll include a picture of my three year old on Shorty, the best natured pony on the planet. We just gelded him so that the stalion will leave him alone, but you can find them with great dispositions. There's no way I'd endager my 3 year old, but look at that smile! I have experience with pony's, quarters, and a thoroughbred that can't handle the long races. They are cheap here in KY.
Good luck, and go for "bomb proof" on your first one. That phrase is used a lot out there. I have a buddy who raises saddle breds and he starts young banging things around, rattling sacks around them and such to get them used to strange sounds. They are pretty daggone solid, but no matter how experienced the horse, you never know on a trail when it will see something new and get spooked. It happens.
Here's some pics of the pony, me on the thoroughbred with my Billy Cook Wade Rancher, and a stalion Quarter nick named Buba.