The first issue I would have with your arrangement is keeping the gear box aligned with the pulley shafts. The sleeve between the gearbox output shaft and the pulley shaft is not made to absorb this misalignment and will have a limited life. You could replace the sleeve with a universal joint but you are going to have to shorten the pulley shaft and matching a universal to your shaft's ends may be difficult.
Secondly, the idler pulley gives a measure of safety if you were to strike an object, stopping the flail rotor. The belt will slip, protecting the drive train. A very tight belt may not offer that protection.
I would think a welding shop could fab a U shaped bracket with a pipe pivot point on one end and the same pipe as a spacer on the idler end. Most farm stores carry idler pulleys and tension springs to finish the job. Good luck.