it pulls straight off doesn't it?? I'd just pull real hard.
Mine used to fall off when i shifted, finnally a said screw that and bough a cheap aftermartket one at advanced or something. Cheapest one i could find. Its black and fake carbon fiber. Looks kinda nice actually. And it made my shift shorter becasue it didn't stick up as high. I have to tighten the hex screws once in a while though but whatever.
Anyway, they just clip in i think, pull it off. Maybe you should wait for someone to confirm this. I have a 1991 325i. thats what mine was.
by the way, what is an SE?