I am a land surveyor, and I've read some ideas in this post that are both good and bad. My opinion is I wouldn't worry to much about adverse possession in this case. He saw the survey marker, you talked to him about it, and this puts him on notice that you know where the line is.
I wouldn't drive anything near the corner yourself. You might know roughly where the corner is, but not exactly. Don't offset the corner either. There is always a chance someone might think its the real corner.
Talk to your surveyor, he will probably reset the corner at a fair price since the original survey isn't that old. If I think there is a problem with someone removing one of my pins, I drive a corner down deeper, say about 12", then set another one on top of it. The person will pull the top pin, but not know the lower one is there.
Last of all, you are correct, don't mess with lawyers yet. You can't prove he pulled the pin, and if you get a lawyer involved, things have a way of escalating in a hurry.