Amen to that - it's location and the number of folks who want to live there. We've had a slow influx of Northeasterners into the Southeast, particularly the coastal areas. With the baby boomers beginning to consider a retirement location (one quarter to one third are planning to move from the Northeast to the Southeast per the Wall Street Journal), the price escalation is accelerating - in order of frequency, land on the ocean, land on flowing water (harbor inlet, large river, etc), land on static water (lake, pond), and land on golf courses - all with city/surburbia access factored in. Prices within 5 miles of Charleston, South Carolina, on the ocean, are at $500,000 plus per prime land acre, and similarly at the big ticket barrier islands of Kiawah, Isle of Palms, Hilton Head and the like. In fact there is very little land left to buy.
A while back I secured, for my future retirement, 7.5 acres on the Intracoastal Waterway about 20 miles north of Charleston - could not afford to buy it now - heck, even the taxes are steep. So, if you "need" to be on or near the coast in the Southeast (or California, etc) find the best land you can afford and wish to live on, and buy it before the "boomers" start to retire and further bid up all prime land.
Now, if your idea of a "little piece of heaven" is 10 plus acres of wooded country property with a stocked pond, a well, a septic tank and a power line - you're going to do just fine - god bless you - that's not what the urban/suburban boomers want, and they'll leave you alone, with low taxes, relatively inexpensive land, no traffic jams, and lots of peace and quiet.