Thank you for all your comments and help. I do have slope and before the gravel was brought in we had quite a few big rains. I would observe during and after the rain and there was no puddling and all water would drain off the drive very quick, almost instant. It was highway smooth before the gravel went down. When the full gravel trucks came they did not sink at all and it had rained the night before so I assumed the base was good.
There are parts of my back yard, (all clay fill), that I can, and do, drive trucks on, in the summer. The ground is very dry then, and has no problem supporting the weight. Those same areas, in the spring and fall, I cannot drive anything heavier than my cheapo Craftsman mower over them, without rutting it up. What is the difference? The ground has softened, because of an accumulation of moisture. Once it is saturated, it does not have any ability to support weight.
Each year, I add some drain tile to a different area. The tile keeps the water from saturating the ground, the problem is solved. Once the ground has dried out, it's able to accept large amounts of moisture, without turning in to muck.
If your ground became saturated because of the area being dug out for the gravel, and the gravel was able to hold water, it could have caused your problem.
I don't think that is the only problem, based on the photo. You simply do not have enough stone there, to hold enough water, to cause this much saturation.
I suspect that runoff, from the surrounding land, goes through this area, wetting it to the point of saturation. This can happen, even on a slope.
If you determine that is the case, you need to divert the water away from the drive area, with a swail, or collect it, with drain tile, or catch basins, or possibly even all three.
No matter what kind of drive you put there, it will not be stable, year round, if the saturation issue is not addressed.