Have you messed around with the brake rod length adjustment?
Sometimes disconnecting the pedal assembly and manipulating the brake actuator rod by hand can show if the "internals" are good but needing adjustment, or if there are parts worn broken or missing.
Brake linings suffer wear when driving off with the brakes set. On my Fordson, the brake drums rust from disuse, and first application is STOP RIGHT NOW! I've learned to feather the pedal while driving along to polish the drum friction surfaces.