For anything outside of a new residential development, it's hard to have a 'normal' house site built. Sure, custom, $750+, they won't be a problem, but a 1500-2500 sq ft, 3-4 bed, 2-3 bath house, not many builders do 'scatter lot' building. It's a big pain in the butt for them, compared to a development, where you move from house 84 to house 87, or whatever, and the price will reflect that. Mobile and modular are a good option. They aren't really cheaper, compared to a new development, and frankly, aren't That much faster, if you order it, but on the plus side, there are way less Unknowns, material increases, subcontractor delays, change orders, ect.
Wife and I have been casually looking for probably 3 or 4 years now; and although we like the idea of a site built on property we haven't found yet, every single step is more hassle. Need a land loan, with Large % down, then you need a builder and home plan to even talk about a construction loan. Lot of chicken-egg kind of problems; because you really want to have a pretty locked down loan, before you pick a plan/builder.
Now, Modular is kinda a big category. Many are no different then a manufactured home, other then local code vs HUD codes. Frame, skirting, ect; there of coarse also off frame modular, tilt up roofs, ones built with a dozen small "modules" and just a "doublewide". Some look no different than a "trailer"; others; you would never know its not site built.
Some cities don't allow manufactured, so first thing to do is see if modular are allowed; same with insurance; ect.
Will add, some manufactured homes, really its hard to tell if it's site built or not; take the Deer Valley Briar Ritz. It really looks site built, costs like one, and is also available as a manufactured or modular.