So, I went back and looked at the pictures a bit closer; but correct me if wrong.
So, the upstream end does have a cast in place headwall, the pipe is in the center of a stream, but the 30 or so feet left/right is flood plain of the creek? With that, the goal, im guessing with the pipe elevation is to have a year long pond across the flood plain, up stream of the pipe, and the pipe is meant as a control structure. Looking at the weeds/grass that appear pushed over, during heavy rain events, the entire flood plain is flowing.
So; if that is right, we mostly need to focus on 2 things. #1 is a Whole lot more fill: compacted and then the slopes stabalized with grass. #2 the pipe is serving as a bleed down, but we need a true over flow.
So, typically, you would have a small bleed down orifice, and then a much larger overflow. I would build the entire bankment up, probably 24-30", with well packed material; and pick one area, about 12 ft long, off set from the pipe, and leave it about 12" lower then the general embankment. That serves as the true over flow. It will need armored; with rip rap, soil cement bags, crushed ruble, geo rid, fabriform or similar, because that water, when it reaches over flow point, will have velocity.