If you don't want the bolt welded into the place, (or if it is a problem for it to become permanently part of the engine), I'd probably remove it.
I suppose you could put in a stud instead of a bolt, which might not be a bad idea. Then if it is welded in place, it is ok. If it comes out, it isn't bad either.
You can purchase "bottoming taps" for tapping into blind holes.