When I do work for people ... b2b is always a contract, b2c depends on the size of the job. Anything under a couple grand in labour I tend to do on word of mouth agreement and get paid after the fact. Bigger stuff (some of my jobs are 200k+ in labour), I get paid through escrow. Protects both of us. Ensures I get paid, and ensures they get their work done. It costs a bit to get a lawyer to set it up, but honestly it's worth it.
In your situation, I wouldn't move the trailer off the property, but to the rear of the yard and chain/lock it to a large tree etc. This will ensure it doesn't get stolen while you wait to settle the accounts (after all do you want to be responsible for his property over such a long time?). Next, talk to a lawyer. Make sure to have all supporting documentation available (records of calls, emails, paperwork, delivery receipts etc). Let the lawyer contact him and work on settling the account (either getting the work done or getting your money back, or having him submit the property to you for sale so you can recoup the costs)
If you go ahead on your own and sell his stuff, you can get in a lot of dodo (although it would feel good to turn the screw to him) ... and really, do you want to deal with that on top of trying to get the building built ?