Leave it on, the pluses outweigh the minuses.
Take it from a custom mowing & haying contractor.
With the loader and a bucket, you can:
Push downed trees out of the way and break off limbs overhanging the field (my favorite use-has saved me a bunch of times).
A loader with bucket can pull you out if you get stuck.
Lowered to about 6” and curled back, You can use it as a “braille” method to feel for rocks & stumps in new, unfamiliar fields
You can also load the bucket with dirt to stabilize the front end and improve traction on hills.
If you encounter a rock or stump and can dig it out, you can use the bucket to carry out of the field.
If you have other chores to do (fencing, wood sawing) you can carry tools, fence posts, etc. in the bucket.
Yesterdays mowing.