GG #1
I'm not sure I'm getting a clear picture of your situation.
So you're at it often enough for it to be routine?
Not sure what's going on there.
Might some shavings from the factory have gotten into the fluid circuit, and not actually indicate accelerated machinery suicide in your rig?
No doubt large-size particles can do great harm to engine tight tolerances.
But finer particles are not risk free.
Our submarines use centrifuges to remove such particles, which otherwise can remain suspended in the fluid.
It's YOUR coin-toss. Save some cash by not changing both filter & fluid. Or lose a bundle on expensive repairs due to having changed one, but not both.
Something else to consider:
You are right of course that physical solid foreign material is a danger to your equipment. But fluids reaching their life expectancy (no matter what the book says) is also risky. The obvious example, such fluids can develop corrosives which can accelerate wear.
You know the risks. 'tup to you.