When asked why he was leaving American Idol, Simon Cowell coyly replied, “I’ve got other irons in the fire and it’s just time for me to go.” But sources close to the star claim that, like his fellow judge, Randy Jackson, he misses Paula Abdul and he can’t take another year of Ellen.

Cowell has reportedly said on more than one occasion that “DeGeneres is like the kid in school who would keep at it and keep at it until someone stuffed him in a locker for the rest of the day. I just wish to God we had a locker to stuff Ellen into.” And, after announcing his departure today, he was heard later stating “God, I’m so happy I made that decision. Can you imagine having to go through another season with that Mad Hattie?”