Don't get me wrong. My grandparents made a decent living from being landlords. The bought a 2 story house, added external stairs to the 2nd floor, lived up there, and rented out the 1st floor. Grandpa had a barber shop in the garage behind the house. When it was paid off, they bought another 2 story and made 2 apts out of it. And a 3rd after that. They had a small cottage on an island in a lake north of us, and a trailer in Florida. The sold the cottage and built a small house next to my dad. Everything was paid for, so it worked out well for them.
However, when time came to sell them, they weren't worth diddly squat because of the neighborhoods they are in. They wouldn't sell and had to be sold on land contract. Those kept getting defaulted on and kept coming back. Grandpa died. My grandma spent a total of 17 years trying to get rid of those houses. Finally got it done in her mid 90s.
Looking back on the troubles they had with tenants, I'd never do that. He had his barber job and had to maintain 3 houses and a cottage and deal with the tenants. I have 1 job. 1 house. Few worries, and more income than he did. I'm OK with that.