As Deepwater Horizon enters into its seventh week, one can’t help having a bit of the ‘I’m stuck in a bad episode of the Twilight Zone’ feeling.

First, the problem seems unending. No high tech magic wands. No victory laps. Rather, we’ve witnessed a series of techno-flops. Top kill, robots, container domes, riser insertion tubes, and now back to more robots. A definitive remedy, we’re told, can’t be in place until August. Hmm let’s see 5,000 (or is it 100,000) barrels per day times another 45 (or is it 60 or 90) days. Only one thing seems for sure. We’re talking a mighty big number.

Deepwater highlights how far we have to go in terms of practicing innovation on the fly in relation to important and emerging agendas.

And that’s the second striking feature of Deepwater: the mushiness of the data and the conflicting positions that result from it. Leak volume is only one example. Opinions also vary on environmental impact, costs, and on policy remedies that are now needed. We are now in the realm of challenges that defy simple technological solutions or orderly chains of command. It’s business Rashomon now, since how we define the problem (profits, environmental and economic impact, academic truth, regulatory oversight) determines what we think should be done.

This takes us to the third aspect of Deepwater, the ‘could have been’s’ that emerge as we walk the cat back. If only BP had better safety procedures, or had questioned the well’s stability earlier or had used a more conservative remedy in the first place. If only the federal government had stepped in sooner. If only our policies with regard to regulation of oil companies and offshore drilling had been more strict. If only we had brought other expert opinion in at the outset to work the problem.

Welcome to the world of wicked problems.

I touched on this topic recently in relation to the UK elections, but it is worth expanding on in relation to Deepwater. Almost 40 years ago, the social scientist Horst Rittel proposed the term “wicked problems” as a way of trying to understand why some challenges were intractable and highly difficult to deal with. Wicked problems shared certain characteristics, in his view. First, they were hard to define. (Is Deepwater an issue of technology, safety, policy, or environmental policy? Yes.) Second, they involved many different and hard to reconcile perspectives? (For sure, in this case.) Third, solutions weren’t true or false, but better or worse, and hard to test in advance. (Yep.) And fourth, the resut is a brace of conflicting opinions. (Definitely.)

So there may be an important, teachable element to the Deepwater story. While we would have all preferred a straightforward technological success story (top kill worked and it’s Miller time), the failure goes much further. It’s about how we look at our preparedness to address complex challenges and how we might deal with them better in the future. Because we are in an era of wicked problems, folks, and something like this, sadly, is bound to happen again. And again.

How do you deal with wicked problems? The key for Rittel was bringing a highly creative process to bear and collecting all stakeholders and viewpoints under one roof to engage in the work. He felt it important not to give in to the temptation to simplify, but rather to examine the challenge in all its complexity. Some of this is reflected in our president’s recent statement, We will take ideas from anywhere. That’s crucially important, but success is not just about ideas – it’s what you do with them to make them happen and when, as well as how you use the convening power of government to create blended solutions and fast synchronization among divergent parties.

Where can we find some best or at least promising practices? Certainly the military is in the wicked problems business. And in that culture, you find situation rooms, sophisticated approaches to command and control, experience in crisis management, mental rehearsal and training to think the unthinkable, and investment in leadership development. But comparable capabilities still seem lacking for the kind of societal disaster that is Deepwater. And they are desperately needed to generate the kind of divergent thinking appropriate to wicked problems. In the Deepwater story, it took a long while for academic expertise to be allowed into the tent and for the full weight of government expertise to make itself known once we had resolved a few little problems in the Minerals Management Service.

So we need to know a lot more about how to do the work of innovation in order to address the kind of wicked problems that threaten the common good. In short, we need to design a new kind of innovative and fast capability in government that is also capable of blending in the perspectives of the private and NGO sectors as well as academic experts and civil society.

I have been saying for years that our government needs to get better at innovation. Deepwater highlights how far we have to go in terms of practicing innovation on the fly in relation to important and emerging agendas. We are still stuck in a government narrative that looks at innovation as being about cool, high-tech ideas, people in white lab coats or economic inputs within broad development policies. Well yes, but innovation is also about coming up with innovative solutions to wicked problems at the tempo of life. It’s about how senior people come together to figure out new solutions in a crisis to get us to where we want to go.

We need, I believe, a new kind of innovation rapid response capability that employs the state of the art in facilitation, communication and collaboration technology to generate speed and alignment. We need a new approach to cultivating facilitation skills that will enhance the work of stakeholder groups and allow the generation of rapid alignment among differing points of view. We need better tools for visualization and problem solving to create shared understanding. In short, we need a new kind of innovation SWAT capability with a seat at the table of power. I’ve even gone so far as to suggest during the last presidential campaign that every federal department have a senior facilitation team as part of a national facilitation corps that is prepared to deal with unexpected emergencies.

If we don’t do something like this, I fear that the history of our future will be written in terms of a series of wicked problems that turned out to be a lot more wicked than they could have been.

Dubbed "Mr. Creativity" by The Economist, John Kao is a contributing editor at The Daily Beast and an adviser to both public and private sector leaders. He is chairman of the Institute for Large Scale Innovation, whose i20 group is an association of national innovation "czars." He wrote Jamming: The Art and Discipline of Business Creativity, a BusinessWeek bestseller, and Innovation Nation. He is also a Tony-nominated producer of film and stage.