There used to be at least one guy in nearly every farming town who more of less collected old tractors and just parked them in rows alongside the barn or a fence. I always meant to stop by for a chat..... but so far have only followed up with a few.
TYM 494, Farmall F12, Allis WC unstyled, Allis B, Farmall BN, Allis C, Farmall Cub, Farmall Cub low boy, Gibson Model D
My only advice is to buy completely restored, it will be cheaper in the long run. I have bought all of mine restored and I have paid as little as $900 and the most I paid was $2200. Keep cash in your pocket and an open mind. I find the best time to buy is in the fall or winter when people are tripping over them. If you don't "need" a particular tractor, you will do better in the long run. I am partial to the unstyleds. Look for the odd stuff.
New Holland TC29DA, Allis Chalmers B, Cub Cadet 102
I was in a very similar situation to Aaron about a year ago... Like old things, like tractors, so interested in buying an old tractor. Simple! I too was interested in a diesel, but like others said before, most tractors at the price/size are gasoline. I kept my eyes open and ended up with an AC B (1942). I agree with Cop, buy restored (or close to it). Mine was not recently restored tip-top, but it was gone over well not very long ago. Good enough to not give me any headaches yet! It came with an old rotary mower, so I've given it some work in addition to it's main purpose, joyrides with the kids. The seat is quite wide, so it works great for this!
I've used it to take wagon rides for 6-8 people without concern. I don't know how much weight everyone is thinking for a typical "hay ride" but I believe my usage has stayed within the abilities of the tractor. True, the brakes are not very effective, but the property is mostly flat and the tractor is heavy enough with the wheel weights for an enjoyable stroll.