I have regular wasps, yellow-jackets & bald-faced wasps. I eliminate those that build nests right in the entry way to the house. So far - 34+ years - none have found a way in to build nests in the house. About 25 years ago I noticed ants trudging along the exposed beams in the house. Tracing back to the source - I found a LARGE ant nest in the small attic I have. Threw two ant bombs in the space - we went to visit my son, in town, for the day. Came back - aired out the house and cleaned out the ant nest with my big shop vac.
Diazinon granulas applied liberally around the foundation annually has solved this problem and the ants now stay where they should be. I still check the attic space at least two time a year.