Do I have to pay my Employement and Support allowence (ESA)UK benefit Back?

It says I am no entitled to the who "ESA" from this time last year and I have failed my appeal. But I had doctors notes to cover it all. My question is basically do I have to pay it all back!!! I have been genuinely ill in the process aswell! but the !!!!!!s has stoped my appeal. And I don't think I can re-appeal. Where do I go from here.. Thanks peeps.