JamesHW:
The NY property taxes people speak of here seem very high. Assuming they are based on market value, then keeping your house on the small side (sq ft, #bedrooms, #bathrooms) should help. Since you are effectively anchored in NY by the family property, then I would look to hold property taxes as low as I could.
The Germans have an expression: klein aber fein. It means "small but nice." I firmly believe that can be achieved in housing that you will enjoy living in, will be energy efficient, and taxed on its size, not its fineness. A small house can be very charming, warm and inviting if the design work is done to create that. Since you are planning a retirement home single floor living with easy access, wide doors, a bathroom laid out for possible disabled access, etc. would be high on my list too.
You could spruce up a couple of rooms in the old house to live in while you build that would later become a guest quarters/bunkhouse for visiting family and friends. Or, a barn house design lends itself well to an upper half-story that is used for guest bedrooms.
We downsized from 3000 sq ft to 1600 sq ft living space; two bedrooms, two baths, a study, open living/dining/kitchen area, separate laundry room, huge hallway pantry, walk-in closet and an attached garage. It's all we need and not a lot to take care of, heat or cool. I basically looked at what part of the 3000 sq ft we actually used and let that be my guide as to what is really needed. I didn't get it perfect, but no one does it seems.
The other space requirement--rural living toys, tractor and implements is also important to consider. Between the attached and detached garages I have as much vehicle and equipment space as living space, and could use a bit more.