what should I look for in a reliable yet relatively inexpensive model
Think long and hard about every possible use you can imagine and the size compressor you need for those uses (CFM), then buy one a little bigger because you'll find more uses for it after you get it.
Cheapest price will be the oilless compressors. They work well for many people, especially those who only use them for short periods of time. The bad news is that they last about one-tenth as long as an oiled compressor. The good news is that you can completely rebuild one in less than an hour and pretty cheap. As they wear, they get noisier.
One step up is the vertical, oiled compressors, single or twin cylinder, aluminum or cast iron. You'll find some of the aluminum (as well as the oilless) have a warning in the manual to not run for more than 10 minutes continuously; a fact that makes me lean toward the cast iron.
The really "good" compressors are V-twin cylinders of cast iron, but may be overkill for most folks.
Single stage compressors are adquate for most people and will go up to the neighborhood of 150 psi; two stage will go on up higher on pressure, usually about 175 psi, but can be even more, and the compressors get pretty expensive. Every air tool I know of is "rated" for 90 psi; more pressure will give it more power, but shorten it's life, sometimes drastically. So you only need a maximum of 90 psi for your air tools. The volume (CFM) is what varies and the higher the number you get there, the better.
Assuming you're going to get an oiled compressor, be sure to know how to add oil (very seldom needed) and how to tell when it's full. Some have a sight glass that makes it pretty simple, but some of the cheaper ones do not. And I've seen compressors destroyed, burned up, when the owner had removed the plug, looked in, saw oil just below the threads, and thought it had enough. On those kinds, the crankcase is so shallow that you need to fill them right up to overflowing.