In my area, most problems with people not following deed restrictions is almost completely economically driven. Folks who have a very low income are able to purchase property through the Texas Veterans Land Board and pay around $150 per month for 30 years. This is raw land with only a rural electrical company and telephone land line available. There are no water utilities nor municipal garbage/trash collection.
Within 1/4 mile of me, I have a fellow who tried to live in an old school bus until the neighbors (me actually) turned him in to the county because he had a child living in those conditions with no running water nor proper sewer. When he moved out, he left the school bus, an old pickup with camper, and eyesore trash. Before him, his father had decided to build a big steel framed teepee and got it about 5% started before he had to give it up. He also built a cattle-guard looking bridge to nowhere out of old trailer axles that he welded up. At that time, he was working as a welder and had a truck with welding equipment available to him. His solution for everything was to weld something up. It was a classic "when all you have is a hammer, everything looks like a nail" situation. When he lost his job, he went into a deep alcoholic depression where he thought all his actions were being watched by the CIA.
Next to him is Chainsaw Bob. Bob bought the property and set about building a small two-story kit house. He got all the raw framing materials delivered, but had no electric power. He used a chainsaw to cut all his framing materials and that gained him the nickname I created for him. Chainsaw Bob actually got his house framed, roofed, and tar-papered before he quit making progress. It took him two more years to get electrical power and four years to get a well drilled. At this time, he has two septic tanks sitting beside his driveway, but no sewer system. Around his house are no less than 5 inoperable vehicles. He has no job or hopes of getting one. His unemployment is sure to run out soon.
Next to me is Pow-pow! Larry. I call him Pow-pow! because for the first year he owned the property, all he did all day long was shoot guns. After a year, he bought a used mobile home and put in a well and proper sewer system. I remember him telling me that he had $14k to spend and that was all the money he had. He worked as a commercial electrician's helper and his significant other had one son. They set up housekeeping and soon were married. He started a welding business on the side and that was doing okay for awhile. His wife got a job at a local lumber company and things were looking up until he got fired from his electrician job. About two weeks after that, his wife hurt her back at the lumber yard and things really went downhill after that. Suddenly, junk started appearing all around their house. Larry would take anything metal he could get for free, but it all ended up in a pile on his property. While trying to support himself as a welder, he also tried to do several fabrication jobs like building barbecue smokers, choppers, VW dunebuggies, trailers, etc. Many of these projects ended up being half-done and just sat on his lot, contributing to the accumulating junk. Things continued to go downhill when his wife divorced him and moved out after the mobile home was repo'd. Life was not good for Larry and continues to be a challenge because he refuses to enter the mainstream and get a regular job. His life will continue to exist on the edge of poverty until he gets seriously hurt or some other catastrophic event causes his world to collapse.
At the risk of being redundant and long winded, I could go on and on with stories of at least three other neighbors whose lives parallel the ones above. All of them walk the thin line of poverty, and all are here because of the availability of easy credit and low payments. For these people, deed restrictions mean nothing and their lives would not be any worse even if you were to take them to court. Your options are to buy enough land that you can isolate yourself from their lifestyles and try to get along with them on a limited basis. I just think there have always been people who barely survive and there will continue to be others. I continue to love my lifestyle and try not to interfere with theirs unless they become too much of a danger to my family or property. I try to be neighborly without necessarily being friendly.