Maybe the flat spring fingers are rusted to the filler neck, if the engine hasn't been run in a while. I'd turn the cap lightly clockwise to see if it tightens a screw/nut so you don't lose it down into the valve chamber. I don't know if these are riveted or have a stove bolt. I'd tap the thing down lightly with a hammer and block. THen try to pry straight up with something that will catch both sides of the cap at once - should break free.
If not, pry harder, after finding a replacement cap first. I think this should come straight off like you said, so I'm guessing it just stuck a little.
Jim