Here's an old thread with diagrams for setting up the tines to till flat, make a raised center bed, etc.
https://www.tractorbynet.com/forums/yanmar/214445-new-yanmar-tiller.html
i don't think any of the settings are for fineness. Rather, these Yanmar-Japan tillers were intended to be run behind a 3-speed pto. 540 rpm, then something like 750 and 1100 rpm. I presume the fastest speed was for turning the soil to fine flour, after earlier slow passes.
I have a RS-1400 Yanmar tiller. Here's a photo I made illustrating the 540 then 1000 rpm pto speeds I have on my Yanmar YM240, a tractor made for the US market. I started tilling at 540 then halfway down stopped and shifted to 1000 rpm. I think the second part at 1000 rpm is too fine and will turn to clay so I always use 540 to leave some air space in the soil.
I think top link adjustment is only to get the driveline straight to reduce stress when the tines are in the ground.
Do you have the special two-piece upper link shown in this photo? On many small tractors this is needed so the tiller doesn't contact the tires when raised. On your larger tractor an ordinary top link may be fine.