Looking at your rock, it doesn't look like you have fines mixed in with the larger rocks. How thick is the rock? The fines will lock the rock together and make it look almost solid. Your road looks like loose rock. If it's not thick enough, it will never lock together.
Were I live in East Texas, 4 inches is the minimum for road base rock to lock together when compacted. When the County builds roads, they put down at least a foot of rock and roll it with vibratory compactors.
If the rock does not lock together, it will wash away. Once it's compacted, the worse thing you can do to a gravel road is to disturb the rock. Dragging it, grading it, or doing anything to it will weaken it. Potholes should be dug up, filled and compacted with a little damage to the surrounding rock as possible.
If you do not have enough rock in place, it will never stop eroding.