To check your theory on caster, make a sandwich of two flat sheets of metal with some light grease between them. (Called "grease plates" at the race track)). This will eliminate the tire effect from the steering load. Then jack up the rear of the tractor until the caster is zero and try it. Use a fish scale at the rim to pull on a spoke and measure the difference. There is also kingpin friction under load from the weight that could be a problem. With caster, the load will increase as you turn the wheel. With friction the load will be finite but constant.