THAT is a very good point - and a move that needs to be PRACTICED !
Whenever I hook up a trailer I check the manual override on the controller (called the "trolley brake" in various CDL literature and license exams).
I check again as I start to move at about walking pace - yes the trailer brakes ALONE do slow the truck and trailer so everything is connected and working - this check also reminds my subconscious where the manual override is.
I have only had to use it a couple of times to get me out of trailer sway, but was SO thankful that the reflex was there.
{I hate to have to admit it, especially HERE, but loading the tractor too far back has been the cause of insufficient tongue load that has lead to sway}
Once in a while I go through a test of the breakaway system, it goes about like this;
1) Get the rig on grass or dirt,
2) pull the trailer plug to simulate the trailer detaching from the truck
3) pull the breakaway pin on the trailer
4) drive forward a couple of feet and check that all trailer wheels have skidded, i.e. all trailer brakes were locked.
5) leave it like that for 10 or 15 minutes to check that the trailer's breakaway battery can hold enough charge to keep those brakes ON (legal requirement is 15 minutes).
6) pull forward again to see that the trailer drags its wheels.
7) Put the breakaway pin back in, reconnect 7 pin trailer plug, finish a normal circle check, etc., get on my way.
CHEAP ain't the way to go with trailers and trailer brakes, we have but one life, etc. and even ONE un-braked thousand pounds wiggling around on a little ball behind me is too much.