Each and every one of us has endured our share of pain over the last several years. In all my years in marketing, it's safe to say I've never experienced anything quite like the last five. It now appears that the worst of the economic calamity is behind us, but it would be foolish to think that the pain is over. In fact, in many ways, it is just beginning.

A recent IBM survey of 1,700 chief marketing officers across 64 countries spelled out the sources of our discomfort. It identified some of the biggest challenges facing chief marketing officers, each of which is a pervasive and universal game changer.

These include the explosion of data, and the stunning reality that we now create as much information every two days as we did from the dawn of civilization to 2003. The advent of social media, meanwhile, confronts us with a completely different mode of communication that most of us are just beginning to understand.

Mass markets, for all intents and purposes, don't exist anymore, having splintered into a broad spectrum of channels and devices. Demographics are shifting, with a new generation of consumers who have totally different needs and consumption habits.

Their expectations are rising: Shoppers want better products and services, more choices and greater value. They expect organizations to act with social and environmental responsibility -- and, thanks to emerging communications technologies, they have greater access to information, and know more than ever about corporate behavior.

Retailers are also more exacting than ever before. They are haggling harder on prices, demanding greater customization and, in many cases, are seeking to pass their risks onto their suppliers.

Finally, and most important, chief marketing officers must demonstrate accountability -- and not just on quarterly earnings, which is how we used to talk about it, or based on short-term decisions. They have to quantify their marketing spend, in total, every year, month, week and day. Their opportunity to innovate over the long-term depends on it.

These are all great challenges that are fraught with risks. But as we all know, reward is the flipside of risk. As we consider these challenges -- these points of pain -- it's critically important to approach them with the kind of positive outlook that tough times demand.

The explosion of data is daunting, but it is also potentially a huge advantage because the more information we have, the better off we can be -- if we know how to use that data. Social media certainly looks scary and chaotic at times, but it is also a powerful platform from which to gain insights, advance understanding and make connections in ways that weren't possible in the past.

The proliferation of channels and devices certainly do complicate matters, but they also enable us to reach shoppers more precisely, and in so doing, serve them with higher fidelity. As for shifting demographics, while it certainly can be hard to understand kids these days, only a complete and total curmudgeon would fail to marvel at the wired, wired world in which the younger generation lives and breathes.

What's especially comforting about all of these changes is that the core principles that guide us as marketers have not changed. When I was in graduate school, my dissertation was on whether businesses exist to make a profit or to serve the consumer. I took the position that they serve the consumer. This is the foundational principle of marketing, which many today call purpose-based marketing. It really is nothing new; it's nearly a law of nature: Serve the consumer and the profits will take care of themselves.

The concept of "integrated marketing" is another steadfast principle. Marketing, by definition, is the integration of multiple disciplines -- which is, of course, the absolute bedrock of our business.

However, the third -- and most challenging -- principle that refuses to budge is that marketing should deliver a measurable result, the return-on-investment. Of all the points of pain mentioned in the IBM study, quantifying accountability for our marketing activities is without question the most excruciating.

It's easy to blame the rise of mass media, in particular television advertising, which, for a time was so effective that it obviated the need to integrate much of anything. Those days are long gone, obviously, and now we'd all love a simple solution to our complex situation, and preferably one that doesn't involve any pain.

At the risk of oversimplifying things, I'd like to suggest that the solution is some combination of
a brand purpose and an integrated, holistic approach that delivers a total brand experience. If we can re-organize our marketing around the idea that the brand must be prepared to serve a variety of shoppers under a variety of circumstances, we've taken a huge step toward managing what is otherwise unmanageable. And, as the old saying goes: if you can manage it, you can measure it.

Accountability Through Analytics

For chief marketing officers, accountability is the whole game. The IBM study reports that 79 percent believe the level of complexity they face will increase over the next five years, while only 48 percent feel they are prepared to cope with it. Chris Zook and James Allen, in a paper published in Harvard Business Review, reinforce that companies will be forced to adapt faster than ever, writing: "Today, fewer than ten percent of companies achieve more than modest levels of profitable growth over a decade, and the odds of success are declining."

In the face of such dire predictions, according to the IBM study, most proactive chief marketing officers are now trying to understand individuals as well as markets. They are focusing on relationships and not just transactions, and are committed to developing a clear "corporate character."

However, they are still struggling with measuring the return-on-investment. They recognize that they need to get scientific about measuring the total brand experience. They need to find a way to synchronize their marketing investments to create that experience, grow their customer relationships and quantify their success every step of the way.

Today's chief marketing officers must develop analytics and technology to figure out how to sift through all the data from a brand, shopper and retailer standpoint, to mine insights to determine who the core customers are, what they need and how they would like to interact with the brand. Then they need to create customized offers and integrate them with their entire marketing program.

It's no longer good enough to do advertising and then promotion; both must be integrated into a total brand experience across paid, owned, shared and earned media -- which of course includes the retail environment itself. This is hugely different from the way marketing was done in the past, and the way it is still being done today. The era of specialization is ending. Marketers must take a broad-based, holistic view of the entire marketing experience because it's not one thing or the other -- it's everything.

The chief marketing officer needs to understand the channel, the retailer, the store and the shopper as much as the brand. You cannot win with a brand program without understanding retailers because they are calling the shots. You need to understand what their execution considerations are and include them up front as part of the brand planning process. Procter & Gamble calls this process "store back," but whatever you call it, it's the idea of understanding how your brand communication manifests itself in an experience at retail.

Marketing is not about a supply-chain approach. That's what operations does. Marketing always was intended to be about demand creation, so it has always been focused on topline growth. The way we get to that is to have a scenario-planning, predictive brand and ROI model that can analyze everything for a brand in a retailer-specific environment.

Agencies Must Innovate

The implications of all of this for agencies could not be more clear: You can carry on as before ... and die! As points of pain go, death is right at the top of the list. Or, you can seize the day and leverage your competencies by responding to the reality that delivering value to shoppers on their terms is critical to success. The pressure is on for chief marketing officers to be accountable for both brand equity and sales return-on-investment. This requires new approaches, tools and skills.

Agencies need to develop analytics and technology to understand how to sift through both retailer and brand data to mine the consumer and shopper insights that identify who the core consumers are, what they need, how they would like to interact with the brand, and then create customized offers that integrate paid, owned, shared and earned media.

They need to move at Mach I speed to create client-obsessed centers of excellence that embrace both brand and shopper marketing, competencies that create a more consistent brand experience across the customer life cycle.

Agencies need to integrate shopper marketing into the brand marketing world while delivering a unique combination of superior marketing process and programs across the four P's (not just promotion), spending optimization and in-market/in-store execution that delivers the customer's experience with the product at shelf. They need to deliver retailer, store and shopper-specific integrated brand campaigns that begin with the endpoint in mind by incorporating in-store execution considerations upfront in the brand planning process.

Above all, agencies need to deliver top-line sales growth by delivering a world-class scenario-building ROI and brand-equity model that can analyze all major marketing programs in a retailer-specific environment and predict results within 90-plus percent accuracy.

If this sounds like a tall order, that's because it is. But the fact is that there are now models that enable us to identify the return-on-investment within a channel, a retailer or market and the performance of each touchpoint. These models can simulate different amounts of spending in each pot and predict brand equity and sales return-on-investment.

We can now see the relationship between all of a brand's earned, owned, paid and shared media and their influence on each other and their effect on brand objectives. We can also predict a competitor's performance. We can answer all the tough questions facing today's chief marketing officer. We just have to do one thing: We just have to do the math.

It's All Marketing

We have to remember that, ultimately, it's all marketing. Shopper marketing is expanding beyond the retailer's brick-and-mortar world to include a consumer-based life cycle, empowered by technologies that enable shoppers to make smart decisions and then tell 100 other people about them.

The notion of the path-to-purchase, as popularly understood, is not a panacea. It is far too linear and disconnected from the realties of how people live their lives and make decisions to offer a total solution. To make sense of -- and bring order to -- today's chaotic marketplace, we need to understand that it's not just about locking into a rational, orderly path; it's more about understanding an increasingly empowered and restless consumer.

So, after all my years in marketing, I am still committed to integrated marketing. I am still committed to serving the consumer and understanding how to communicate, connect and interact with that consumer, and provide them with differentiated products on their terms, not the brand's. I'm still committed to innovation.

What's different is that we now have the benefit of scenario planning with predictive results so we can truly satisfy a consumer with a brand that has a purpose in their lives. We can find better ways to distribute brands that truly solve consumer and shopper problems. We are getting that done because the world is changing and forcing us to get it right. We really have no choice.

We must embrace both brand and retailer issues and focus on the brand experience across the customer's life cycle. Above all, we must do the math to make sure we are delivering on our promises to our consumers.

It's the creative plus the mathematics, not just creative based on insights. That's where real innovation is coming from.