Every stump will come out, it's just a matter of persistence on your part.
From the picture, I don't think you have taken out enough dirt around the sides. Keep digging. I've had them hold tight until I dug down as deep as I could, all the way around. What really kills me, is I'll fight and fight a stump while digging around it, but then change sides and it pops loose.
Those tap roots make them harder to get out, but it's not that bad. I think the suction of the ground on the bottom of the root ball is what keeps it there more then anything else. And that suction, for whatever reason, is very strong on most sides, but always seems to have a week side. The trick is finding that week side before digging any further then you have to.
Good luck, if you take out enough dirt, it will break free.
Eddie