Unfortunately, been there, done that on our whole family having to get rabies shots due to bats and unvaccinated kitten about 2.5 years ago. No matter what we've done, we get a bat or two in the house every year or two. No rhyme or reason. Spring, Fall. The rabies one was on Christmas Eve.
So, best advice, keep pet vaccinations up to date. Always have kids sleep in their bedrooms with the doors closed. If you see a bat in your house and you have kids, catch the bat and have it tested. Statistically, rabies is 100% fatal by the time it's diagnosed. Nothing to mess with.
Why are your neighbors complaining about the bats? They're beneficial critters.
I think I've had a little success by putting LED lights in my attic that are on a revers photo cell, so they come on in the daytime and go out at night.
In the evening, bats look for light and head outside to feed. When the sun comes up, they go back to their roost.
That's why I'd suggest the bat excluders for your chimney, but only after you're sure there are no more babies, like in the fall.
DO NOT START A FIRE. Here's why...
First, if there's bats, there's bat poop. Guano. It dries and is flammable. So if you've had bats in your chimney for years, there's a good chance there's years of poop in there.
Second, during summer, when you see the bats leaving, that's the mommas going out to feed. They leave the babies behind. You start a fire and the babies will try and find a way to crawl out.... into your house. Or, they'll die of smoke and heat.
Third, besides being illegal to mess with bats during baby season, it's not necessary. Just wait until late summer and fall. Shine some lights up into the chimney at night to see if they're all gone, and put 1/4" wire mesh on top of your chimney and that's that. They won't be able to get back in. If they aren't all gone, you can make a long funnel and flap out of wire mesh and cloth mesh that acts as a one-way valve. Any bats inside can find their way out through the funnel and flap, but won't be able to get back in.
How to Remove Bats from the Chimney