Looking at the illustration from my service manual it seems it wouldn't pull out at all unless it had already been rotated far enough to disengage the shifting shoe (4) from the sliding sleeve (6).
EDIT: I just noticed the shifting shoe (4) should be retained by a push nut (2) so maybe it is still in the lever (1). You may have rotated the shaft 180 degrees when you put the roll pin back in. You might be able to disconnect the lever, rotate it 180 degrees, then push the shaft all the way in and see if it will engage the slot when you rotate it back in place. Be gentle while feeling for the shaft engagement and you should be okay. Once you feel the shoe hitting the sleeve it might take a few tries to get it to turn right and go in the slot.