Taking pictures from the street (or outer space in Google's case) is perfectly legal. Physically entering someone's private property and walking around taking pictures is not. I'd have asked them what they were doing, and then told them to leave. And I'd record any license numbers, get a description of the people, etc... and report it to the police just in case it turned into something, or there were other problems in the area. Most likely, it was just people taking pictures and not thinking about trespassing. But you never know.