Well I did a little more mowing today and paid more attention to what the oil pressure is doing.
It idles right at 40 psi when warm. Then when the rpms start rising the needle will also start to rise to say 42-43 psi, then fall quickly down to 35 or lower. When up over 1500 rpms the pressure stays down between 30 and 35 psi. Bring it back down to idle and the pressure comes right back up to 40 psi. It acts pretty much the same way loaded or unloaded.
However, when cold the idle pressure is lower, and acts like you would think it should. By that I mean that the pressure comes up with RPMs or at least doesn't go down. This is what makes me think it isn't a wear or bearing clearance issue, because in that case the hot idle pressure would be lower than the cold idle pressure.
It almost seems like the oil pressure relief valve is sticking closed and then when it opens, it has a weak spring and opens all the way. This may be flawed logic, I'm not sure.
At any rate, it always has acceptable oil pressure. It just seems to me like it is something to do with the pressure relief valve, along the lines of flyerdan's post.