The property where my RTV is kept right now in MI is all clay, hilly and slippery as **** anytime it rains or when the dirt is thawing in spring (wet clay on top and ice underneath). I agree with the ground clearance concerns compared to the "sporty" side by sides, but the suspension is hard for taking a load and having a load on board does not change it much. I personally avoid deep mud, given the kind of damage repeated exposure will do to any machine. With my backhoe, I usually find ways to improve drainage on the property and lay gravel and french drains such that I don't have to deal with mud all of the time. When riding trails on someone else's or state or Federal land, there is not much you can do about it, but that is usually for quads or the sport type vehicles anyway.
If snow depth gets to over a foot and there are no cleared trails, then yes it is over for the RTV, unless you fit tracks. Yes they are slow. I never drag downed trees or a sled with a deer in it at 30mph so limiting out at 20mph has never been a concern to me. The upside to the low speed is plenty of climbing power on steep slopes. It pulls a hay rake several months of the year and the front hitch is used for pushing 4 ton hay wagons into the barn. Much easier than trying to back it in...
More downsides is the lack of a true cab or doors for the RTV 500, but I do have a rigid roof and a glass windshield. You pay your money and make your choices..... Mine cost $6500 with 90 hours on it.