Can you judge him on his past actions? (and who is giving the authority?)
Yes, of course past actions, but not past experience, upbringing etc. Yes, you can say"he had a bad start in life, poor role models, no father" as mitigating circumstances, but it is how the guy has dealt with life, allowing for such setbacks, that you have to make a judgement on.
Who gives the authority to judge- oneself. You have to make a judgement on things. I know it is an unfashionable word/concept, but if you are standing at the ringside, wanting to buy calves, do you not make a judgement on which ones to bid on? or do you just say "I'll bid up to £200 for each calf", regarless of age, weight, breed, health, lameness, horned, polled, steer, heiffer, bull.