Since you will be using the ballast with a loader there is no reason to keep it low to the ground, I raise mine up, at least high enough to clear any ground cover.
I keep mine as low as possible, not low enough to drag. but I keep it low to keep the center of gravity lower. should be more stable or atleast it worked that way in my head
PG
Keeping your center of gravity as low as possible on a tractor is a good idea. I keep ballast as low as possible - ie right over the height of things like curbs or rocks I might back into. This is much more important on hills obviously