What is your terrain like? I have a 2500 gallon cistern located so the bottom of the cistern is level with the window sills in the house. It give me about 8-10 feet of head at the sinks and toilets. It's great to be able to still flush the toilets when the power goes out. If arsenic is a problem, a small ion exchange filter will provide safe drinking water.
Most people go way too big with generators. I have a 1000 watt camping generator that will run for 4.5 hours on a gallon of gas. It needs to be run every other day to keep the freezer cold. It cost me $149 at a local farm store. Small inverter generators are also very fuel efficient, but more expensive. With my setup, a 5 gallon can of gasoline will last 10 days.
Don't get weird about food. Buy what you eat, except buy more of it until you get a supply built up, then eat the old stuff while replacing it. I have lots of canned chili, fruit, tomato sauce, beans, mushrooms, pickles, hot sauce, jellies and vegetables. Quite a bit of it I canned myself, in glass, which keeps much better than stuff canned in metal.
Dried foods keep well. Pasta, beans, rice, potato flakes, dried skim milk, egg powder, oatmeal, corn meal, flour, semolina (cream of wheat), buckwheat, wheat berries, pearl barley, etc. My family is getting potato flake mashed potatoes for Thanksgiving. Storing dried foods free of insects and rodents is a challenge. Don't trust plastic, a rodent can chew right through it. You can get glass gallon jars from fast food joints really cheap or free. Vacuum seal anything you want to last for a long time.
Don't forget dog food. Fido has to eat too. A couple 35 gallon metal trash cans full will last a while. For more independence, 55 gallon drums are good for both livestock and pet food.
Speaking of 55 gallon drums, check your local ice cream factory. The flavorings and syrups are shipped in food grade, teflon lined steel drums. One of the last batches I got still had 5 lbs. of solid chocolate in the bottom.
Soap. Lots of soap. Hand soap, dish soap, body soap, laundry detergent, degreasers, floor cleaner, window cleaner and ammonia.
Unfortunately, bleach has a very short shelf life, so you can't stock up. Instead, stock up on alcohol. Rubbing alcohol is cheap but poisonous. Ethyl Alcohol (ethanol) is just as good, plus you can use it to disinfect drinking water. Everclear (190 proof) is cheap. Iodine tablets have a reasonable shelf life, but still need to be rotated every year or two.
Yes, a major first aid kit. Don't buy one off the shelf, build your own, with a supply of 4x4s, 2x4s, adhesive tape, eye cup, tweezers, reading glasses (for finding tiny splinters), scissors, superglue (as good a sutures most times), antihistamines, NSAIDs and pain killers. Your doc will probably write a scrip for a supply of vicodin if you ask. In Canada you can buy codeine over the counter, but a surprising number of people can't metabolize the stuff.
Do yourself a favor and install a permanent wood stove in the basement, preferably with a fire clay lined masonry chimney. In that location, it will heat the whole house. You can cook on it. You can heat bath water on it. Build a wood shed to keep the wood dry and only burn dry wood, or you will waste a huge amount of heat boiling water up the chimney.
Yes, a .22 rifle should be your first gun. It's the world's finest survival gun. It will take small game without blowing it to bits. Anyone who has ever shot a grouse with a large bore rifle knows exactly what I mean. Learn how to kill with it. It's not humane to wound an animal, and you just end up hungrier.
Water, shelter, food, sanitation, in that order.