Well, that is less 'road' than I pictured. If you are going to bring that up to carrying log trucks, etc. and be able to get concrete delivered for your building projects, you have a challenge. Any chance there is a base of 'bank run' gravel under it? One of the wet spots looks like there might be some there.
If you put culverts in where needed and rock the wet spots, that lane will hold some traffic as long as you stay off it when it's soggy.
One question is, how long before you want to start building? If you have 2-3 years to work on the road, that would help.
I would recommend hiring or buying an excavator and make a first pass where you fix all the really ugly areas, get all your culverts in, etc. I would try to do it right the first time

, you will have plenty of maintenance without dealing with something that was done marginal to begin with.
You might find a contractor that does town roads in your area. They have the equipment and usually reasonable prices on large quantities of stone. If cost is an issue, be upfront and tell them you want to work on it over a period of several years where every year the road gets better until it's done.
You mentioned steel culverts, the smooth interior plastic culverts are good too. Easier to handle and don't rust. You need to bed and cover them 12" or so with fines before putting on the gnarly stuff. A mid to heavy front end loader can punch the big rocks into the culvert with down pressure on the bucket. If installed correctly, I think they are better than steel.
Dave.