With almost 99% certainty and as a beekeeper, I can tell you these weren't bees.
It's probably not a good plan to mow at night either.
The problem with ground nests is that sprays may or may not kill them. Here's something that has worked for me.
1. In the middle of the day, quietly approach the area and observe where the little demons are flying in and out of.
2. Take a picture of the area or mark it by tossing a small object near it, it's going to be tough finding it at night and a flashlight will attract their attention.
3. That night, carefully approach the hole and dump about 4 ounces of Sevin (Carbaryl 5%) powder in the hole and place a small board over the hole.
4. Come back 3-4 days later and see if there's any activity. If they were still alive the would have tunneled around the board.
Problem solved.