If it was the brakes dragging it'd be the same with the PTO on vs off.
Do your tractor have an independent PTO, or live (two stage clutch) or transmission? Are you testing with an implement on the PTO?
"without the load on the PTO it seems to have good low end but as the RPMs climb the speed starts to struggle" could you explain that a little more?