Not to pescimistic, but your not going to get rid of them by messing with the hole. They have multiple escape holes or back doors. I've hunted them as a hobby for over 30 years and killed hundreds.
You've got to either killem' or move em'. Here in MD it's against the law to move a captured animal from one location to another. This leaves 1 solution, 22 shorts (CB caps are fine for squirrels but are rather impotent for hogs) out of a bolt action rifle.
Semi autos can be hard to feed by hand. Try for a head shot as that will keep them from running off wounded into your neighbors place. 22 shorts are not hypersonic and create no hypersonic crack and a bit less noise. Wait for a moment when no one is around, shoot the hog, put the rifle away, dump him in the trash and then fill in the hole with dirt. Chances are no one will notice the noise.
You can also dump them in a field for vulture chow.
If he gets the drop on you, wait about 20 minutes and he will likely show himself again.
I've always used a .22-250 Ruger and 50 grain bullets, A few years ago I had the ultimate ground hog rifle built to bench rest standards. It fires a 67 grain, 6mm frangible bullet at 3,500 fps, and under 250 yards it blows em' into hyperspace. My personal record is a lasered 432 yards. The rifle is capable of more that I am. It shoots tiny little 1 hole groups.
When I was in my teens my Grand Father had had it with them gettting them getting into his garden. We poured 4 milk jugs (the old fashioned metal kind with a lid) of water into the hole about 10 times with no affect.
If you really want them gone you gotta' kill em'.
Have you considered a second hand crossbow. Very effective at close ranges and no noise.