I suspect people that are towing heavier trailers without trailer brakes are towing with older rear antilock brake vehicles. Try putting that same no brake trailer behind one of the newer 4 wheel antilock vehicles, and see how it handles differently when approaching intersections with loose material.
Personal experience I had a 12 foot landscape trailer single axle no brakes that I towed behind a 96 Dakota, and worked fine, but could slide the truck tires on loose material, that truck was replaced with a 2011 Ram 1500 and was towing that trailer with two riders totaling around 2100 pounds, up to that same intersection, the truck hit that loose material, the antilocks kicked in and that small no brake trailer proceeded to push me all the way to the other side of the intersection.
Shortly after that incident that trailer was replaced with a 14 foot tandem axle with brakes on both axles. People don't realize how a light trailer can push a decent size vehicle around