I have looked and listened to the video a dozen times...even looked away, shut my eyes and can't hear any deviation in the RPM's. The puffs of smoke could be anything from steam to unburnt fuel...
Since you've done all of the difficult diagnosing and fuel injection pump testing, why not pull the head? Everyone seems to have an opinion, some similar and some very defferent...I believe if you pull the head and check for a broken swirl chamber, cracked head, porus valve face or seats or a bad headgasket you'll most probably solve the problem. Obviously, and since the head on any diesel tractor is the first thing to come up to temperature--or overheat, the problems seem to originate from there.
I'm not 100% sure, but if it were my Yanmar, pulling the head would be the next step. And if I were wrong...I would at least have eliminated the valve train, piston tops, rings and gap clearance, head gasket and with a straight edge and feeler gauges, the possibility of warped deck or head. I keep coming back to this, especially since you peformed the remote tank to fuel injection pump experiment which did NOT change anything.
Just MHO and good luck.
Mark