You have a problem in the hold-in winding circuit of the fuel shut-off solenoid. When you turn the key to start the pull-in winding of the solenoid gets power directly from the starter. When you let go of the key the solenoid loses its power. This is either a fuse issue, a wiring issue, or a solenoid issue. You will need to check for power at the solenoid. The pull-in power I believe comes from the white wire, then I think the red wire is for the hold-in circuit and then the black would be the ground. Do not hold me to any of these colors without a manual in front of me. But, you should get the idea. One of the 3 wires will be a ground, one will be the pull-in power coming from the starter, and the other will be the hold-in winding that gets key on power. Nobody can give a one answer solution to your problem without being there doing the diagnosis. But hopefully this will help.
Brian