I have heard lots of positives on the 5.4 for a work truck/commuter... complaints are working hard for heavier loads, and huge (50% or more) mileage drop for towing trailers, boats, etc.
Have heard a lot of positives on the 5.4 & 6.8 motors, except the mileage drop/transmission inefficiency. Folks that tow a lot really enjoy the extra power of the v-10... neither hit double digits, but one struggles.
I have '07 6.0 liter diesel, and have no problems towing or plowing, and weight doesn't seem to drop the mileage as much (30-40%) but it has a horrible & much discussed reputation. Its transformation from an efficient lower horsepower fleet engine to the mass market was not smooth. I think they tuned what was about a 200hp International local hauler up to a 325hp horsepower wars entrant.
The learning curve was public and on consumers backs, and by '06 once the (tuning) head studs, EGR, and casting debris issues were addressed, emissions changed, the reputation was established, and the 6.4 with new emissions came in '08.
You can likely get an '06-'07 diesel cheaper than the 5.4 or 6.8, and have more capability & better mileage, as well as a very dependable motor with less emissions stuff to worry about going wrong... you just have to get past the reputation 2003-2004, and somewhat 2005 established for the line. 
Sorry--the 5.4: good motor by all accounts, just a little weak for a medium duty truck.