The legal term is "adverse possession" and is very real and puts you in danger of losing the disputed property. Maybe not disputed now but someday...
And yes it is illegal to remove survey pins in any state. I would openly ask the neighbor if he removed the pin then mention the law and ramifications.
Adverse possession isn't an issue here and I'm aware of what the laws in CT state about it.
It appears popular opinion here so far is to approach the neighbor about the missing pin and the more I read the more I may do that in the near future