Jim,
I don't want to come across here as a smart --s. However, there is a reason "spoils" are re-piled somewhere and fresh fill brought in... Running the highway dept, if we moved large boulders from the any town dirt roads, which is done several times in any given year,, that spoil was brought back to the garage and fresh fill takes it's place. Replacing virgin dug dirt with the same, for some reason will never settle again.. Have some other gravel dropped next to this hole, dig out the original spoils and push the fresh stuff in. Put in a foot or so, and compact by rollng the hoe bucket, another foot, same thing, and keep going till your 6" above grade, Roll it with the tires several times and you should be 3" or so above grade. Within two weeks, it will be back to grade... I've tried several times over the years to replace fresh dug spoils and replace the same., EVERY SINGLE instance, I ended up returning, dig it out, and replace with fresh. Not good when several homes depend on a road and a "jelly hole" is permanent,