bringing powers back from the EU

In today's Telegraph William Hague says

“As a Conservative, I want to bring powers back from Europe, as we set out in our election manifesto. But a referendum on Britain’s membership of the EU, especially at this time of profound economic uncertainty, is not the answer.”

As I regularly admit I'm no fucking expert in this area but surely the simplest and quickest way to get back the ability for self governance is to leave the EU altogether? If we're not part of it we can make our own decisions about things like tax, border control, foreign policy ...... now whether that's a good thing or a bad thing is an entirely separate conversation but ......