There aren't to many hunters that will pass on a good deer just cause he's standing on the wrong side of the fence. They will tell you "the deer jumped the fence after I shot it", unless the deer is head shot or you saw it get shot, you can't really dispute the claim.
40 acres isn't much room, it's enough for 1 or 2 hunters. You must have some decent deer crossing it or they wouldn't want to be on or close to it. If that's the case, the only way to protect your hunting interest is to be there and stand your ground. I doubt that these people are going to give up on it especially if they are not breaking any laws. You may be better off leasing it to them and hunting somewhere else.
Moth balls, coyote urine, your own urine, will deter deer. I would not broadcast this over a large area, just piss on the tree stand or tree he's hunting in every chance you get. This will keep the deer away from his stand but he's still going to shoot deer on your property if your not there.