I do not have backup heat unit in my house. It has gotten down below zero F and the house has stayed warm. Fairly well insulated house, thermopane windows, 1,100 sf daylight basement, 1,100 sf 9 foot ceiling first floor and 600 sf two bedrooms on second floor. Not the most energy efficient home construction from a floor space and design perspective. 3 ton unit with slinky ground loops.
I have 17 years of electric bills on a spreadsheet. I do know that my neighbors pay a lot more to heat or cool their houses.
My power co-op did the manual J calculation, I think it was manual J, off the blueprints. I believe they calculated 2.5 tons. Seat of the pants and rule of thumb is not a good way to size a unit. If you have a house with plaster walls and no insulation in the cavities is going to figure out different from a house with fiberglass insulation, or one with cellulose insulation, or one with sprayed closed cell insulation.
When you compare apples to apples remember that there are inexpensive air to air and expensive air to air. I do not know if there are inexpensive geothermal units. Two of my neighbors replaced their air to air units when they were less than 10 years old. My geothermal is 17 years old and is chugging along. Yes I had to have the coil replaced as it started leaking but I gather all coils did that back when my unit was made. If you shared the specs, brand and model of the air to air and the geothermal then someone might be able to make a more educated guess. It may be that your contractor is just tossing out ballpark numbers to a tire kicker.
As mentioned in a few articles, a geothermal tech will have another day or two of training over a regular tech. An air to air may take a half day to set up. A geothermal may take a day.
If you do some research you will see horror stories of loops not deep enough and freezing, leaks, improperly sized. I have been rather satisfied with my unit and installer. My house is not humid in the summer. It does not have hot or cold spots. My electric bill is not outrageous.
Geothermal Heat Pump Costs: Pros and Cons | QualitySmith
Cost of a Geothermal Heating System: Geothermal Heat vs. HVAC