I'm a little late to this thread given the progress you've already made. I think you might get the pavers to set more even if your gravel sub-base is smooth and packed to -1" of your grade, then screed off about an inch of fines / gravel dust on top so you have a level, smooth surface with a bit of give. The typical build-up for setting brick or pavers in sand is 4-6" of sub base packed, then setting on 1" of sand/fines screeded off smooth, then tapping the brick in place and finally vibrating the whole shebang with sand on top, which fills in the cracks, locking the whole surface together.
I'm not sure how much a small vibrating plate tamper will move your mondo pavers. Maybe one of the larger plate tampers would work.
I'm not sure how much a small vibrating plate tamper will move your mondo pavers. Maybe one of the larger plate tampers would work.