If modern day diesel engines HAD to run at a constant speed to perform well, they wouldn't have variable speed throttles. There'd be a "light switch" style of controlling engine speeds. They perform BEST under maximum load at a given speed, USUALLY in close proximity to rated rpms to provide 540 rpm pto speeds, but variation from that rpm won't cause instantanious self destruction.
Best fuel economy is acheived by operating at the rpm's where power output and smooth operation cross paths with the rpms needed to do the job right. I've got a tractor that's been with me for 37 years. I NEVER run it any harder than t NEED to be run. I don't lug it, but I also don't wind it up to "rated rpms" and cruise around with half a load on it. On my newer tractors we use in the mowing business, my operators run at 75% to 80% of "rated speed" when conditions allow. Saves considerable amounts of fuel, in addition to prolonging the life of the tractor. We're talking engines that log upwards of 1500 hours PER YEAR.
I've got a pair of IDENTICAL Massey Ferguson 583's, bought on the same day, used almost side by side. Only difference is, one always has an operator who goes easy on the throttle, and has a light touch. The other has a regular operator who knows one engine speed. WFO. One has 1425 hours. The other (w/WFO operator) has 1410 hours as of last service. The latter has already started to get noticable blow-by, uses more oil, and uses CONSIDERABLY more fuel per hour.