I have taken off my ROPS on occasion so that I can drive the tractor into a low barn while hauling in tobacco. I have little to no fear of rollover while operating the tractor. I have operated way more hours on a tractor without a ROPS than one with a ROPS. I have no problems with either one. Often, I forget to wear my seatbelt when operating a tractor with a ROPS. I also have no problem drilling, bolting, cutting, welding on my ROPS, content with the conclusion that if it fails it is my doing and that if I trade it in it will be subject to markdown because of such and if I sell it outright it will be without the ROPS.
There is also no doubt in my mind whatsoever that ROPS combined with seatbelts drastically reduce the risk of injury and fatality in the event of accident/rollover. The risk of rollover is probably one of the largest risks for a tractor operator. It could also offer some protection in the event of driving into a ditch/sinkhole etc by keeping your body on the tractor and in a relatively protecting environment (under the rops/between the fenders/in front of the steering wheel and out from beneath the tires/wheels of the tractor).
In short, I would say you face only a minor increase in the danger of operating a tractor by riding without the seatbelt or ROPS, but an increased danger all the same. Is it worth the risk....that is for you to decide. I would not personally lose a lot of sleep over it, because as mentioned earlier, you are a grown man (I guess) and can make your own decisions. ROPS or not, just take care during the operation of your equipment.