Several months ago I contacted a certain big company. One that's been using social media very well, and that's often used as a case study for how businesses can successfully use social media. I wanted to know what was their direct bottom line impact from social media? How had sales increased or decreased? What was the ROI of their efforts in terms of dollars?
Their answer surprised me. "We have no idea."
They further clarified "My boss has never asked for the ROI, because he sees the value in our efforts."
If you'll notice, I've been asking companies when I interview them in the Social Media Mavens series about ROI.
Best Buy's Gary Koelling had this to say about how they track the return on Blue Shirt Nation:
"I could correlate half a hundred metrics we watch as indicators of how we’re doing as a business with half a hundred metrics we watch as indicators of how we’re doing as a social network and probably correlate half a billion dollars in growth or savings as a result. One reason I don’t is because I don’t care, it doesn’t interest me. In fact it makes me sleepy. The other reason is that my sponsors have never asked for it. Not only do they see the value but they understand what happens when you measure things you don’t fully understand. A machine, any machine, will tend to become its own criteria. If I measure how effective BSN was at reducing turnover, it would become a machine that reduces turnover. I’m sure it does or doesn’t have an effect on turnover. The bottom line is it’s a network of humans, a true and thriving and vibrant network and every time someone uses it, it’s more valuable to the next person that uses it."
Kodak's Tom Hoehn said this is what the company measures the effectiveness of its A Thousand Words blog:
"
- Reverb – That comment we noticed multiplied by the number of followers, fans, friends, BFFs, etc. the person making it has.
- We are after hearts and minds instead of eyeballs. We feel we have a good story to tell on many levels and it seems to be resonating with people.
- We of course look at table stakes measures like page views, refers, search ranking, etc. If you don’t do those things by now your head is probably swimming from all of the other information I have shared ;-)
- ROI is the holy grail we are all chasing but we like to think that return on ignoring (thanks @DavidAlston) is even worse.
PepsiCo's Bonin Bough said this about how Pepsi tracks the ROI of its social media efforts:
"All our efforts in the social media space are totally focused on ROI, and demonstrating the value of engaging with consumers on social media platforms. Measuring a program against key metrics and relationships is really important as we scale up our social media work across brands. For Trop50 specifically we are working with Ogilvy and their Conversation Impact model which tracks key metrics that ladder up to Reach, Preference and Action."
Graco's Lindsay Lebresco said this about how the company measures the effectiveness of its company blog:
"We use a variety of methods, some of them fairly standard (how many comments, how many links, how many visitors, how many clicked through to Gracobaby.com, etc) and we also look at the engagement of our readers. Are readers coming back over time?, how often are they commenting?, is the relationship with our blog readers extending to other SM platforms of ours?, etc) SEO is certainly another measure we look at."
These are some of the largest and most successful corporate social media case studies on the planet. And in re-reading these quotes, I noticed two things:
1 - None of them are mentioning the direct ROI (in terms of dollars) on their social media efforts.
2 - All of them can see the VALUE in their efforts, and apparently, so can their companies.
Some people will argue that if you mention the ROI of social media, that you have to speak in terms of dollars in, and dollars out. I'm not going to argue that point, but based on what I'm hearing from BIG companies that are having success with social media, they are getting C-suite buy-in and continued support from their efforts because they are creating something that the c-suite sees the value of for their companies. I especially thought Gary's comments about Blue Shirt Nation were interesting and not wanting to turn BSN into a 'machine' for a particular metric.
All of the above companies would likely be hard-pressed to find an ROI for their efforts that fits a pure business definition of the term. Does that mean their efforts are a failure? Or does it mean that the end goal should be that our social media efforts should generate something of value? Are we too hung up on finding the ROI of social media, or should we be focusing on value creation?
At the end of the day, should companies engaging or considering using social media ask 'What's the ROI of our efforts?' or 'What value is being created for our business via social media?'
I'm not sure, that's why I'm throwing this out to get your thoughts.
35 comments:
Great post Mack. I agree that we have to help our clients understand that it is not always about finding the "dollar" ROI but the the "influence" and "ignoring" ROI. I think this is another clear example of how social media has to be an integral part of a company's culture not just something they are doing because it sounds good.
This is the first honest discussion on the ROI of Social Media that I have ever heard.
And I agree with your position. I'm a heavy user of social media in my business, and I, too, have seen non-monetized results from my efforts. Results that are substantial enough for me to continue to use it.
Perhaps this will end the discussions of all the ethereal non-quantitative, and therefore meaningless, measures of ROI that so many "social media experts" are using to justify their existence.
Well we know what @thebrandbuilder is going to say, and for some companies I'm sure "you have to tie it to dollar outcomes" is the right answer. For others, it is enough to "see the value" and I think that is great. In many ways, I think it's "better" in the long run for the more prevalent mindset to be that it is enough to know that the business is relating to it's people. Arguably, this is more in keeping with the social spirit. Though, yes, I understand that's not going to be enough for everyone.
Like so many of the things we debate every day, there is no ONE right answer that applies to every situation.
Sue that's the main point I was trying to make. For some companies, they want to see the direct ROI in terms of dollars, of their social media efforts. Other companies will get buy-in from the c-suite much easier because the CEO sees the VALUE in adopting social media.
There's no one-size-fits-all method that works here. BTW at the end of the day, all companies need to prove the worth of their social media programs, but how they do that is up to them.
This is so tricky! I did a presentation recently, a panel discussion about social media and search. I presented a Twitter and video, and blog project I worked on that was themed around a product line of ours. After my presentation, the first question asked is “what was the percentage of increased sales?” My answer sounded kind of namby-bamby to many, I’m sure. But, that answer was: “I have no hard numbers. But, I can name a number of people that we met and now have a relationship with because of the campaign.”
The campaign also established us as a voice in our industry for everyone who became involved in the campaign, and to his or her friends too. That is of long-term value, and who knows how much it will benefit the company in terms of brand awareness, networking possibilities, andof future sales that may or may not be related to the product we featured.
The way I look at it is this. When I go to a party, I go to meet people, have interesting conversations, and to contribute to the good time everyone is having in some way while enjoying those same contributions from others. I have never gone to a party, and then come away trying to figure the percentage of cool points I gained by attending, or what the gain was in terms of my friendships with others balanced against the same metric before I attended. Who does that? No one does.
But is there long-term value in going out with friends, and meeting new people? Most times there is, if you’ve got good friends. Lasting value from a single event is not guaranteed, but you do it because the chances are pretty high that there will be a gain of some kind, because spending time with people is the way to build and maintain trust.
And there is an assessment made about whether or not it’s worth going to another party at that one guy’s house that usually turns out to be a bore or a waste of time. And those types of assessments are important for time management reasons, and because there is only so much time in a day. But, it doesn’t matter if you can’t measure it in that precise way in terms of percentages. That’s not the point.
Excellent post! This might sound too ethereal, but sometimes I think people need to 'feel' the value even if they can't always 'see' the value. In other words, just because there isn't always a handy metric available to point to doesn't mean there isn't value that can influence the bottom line. For instance, brand involvement in social media opens up new channels of communication with customers, employees, vendors, media, etc. When the CEO goes out into the field and talks with people, he or she might ‘feel’ a difference in attitude, perception, willingness to engage, etc. with all of those groups. Can you point to that in an annual report – not always. But can the CEO feel the impact and communicate that to constituents – probably so.
Best Buy’s approach is excellent – they are not focusing on trying to justify it – they know it’s working. Instead of spending time and effort into quantifying it, they are using those resources to engage their customers. Love this approach!
This may be a step in the direction of understanding other indirect ROI functions-- like PR.
Interesting
The type of product being sold and the sophistication of the marketer may be an added component here. I would expect Pepsi to have a more granular understanding than Best Buy simply because of their marketing engine. ROI is absolutely fuzzy at many levels when you try to pry out various efforts even when direct response is involved.
For marketers with less sophisticated mechanisms for measurement, demand for ROI is similar to saying "I don't like your idea."
Mack, you raise a key point. These companies are leading the change and realize that the old metrics don't cut to the heart of what is being created/achieved. I am all for ensuring efforts are directed in the right way but the old values don't always fit. Companies that are willing to look beyond traditional metrics to deeply engaging their user base, in my opinion are completely on track. I love the "return on ignorance" comment, right on target!
Love the anecdotes, Mack. Value isn't always quantitative, as Gray, Tom, et al. explain.
Small and mid-sized business owners still seem pretty devoted to a dollars-and-cents ROI. It's funny how ROI only comes up BEFORE you try a new investment -- after a new marketing initiative passes the short-term "I have a hunch it's working" test, beancounting tends to melt away.
Fantastic quotes and a great summary of the questions that have been rolling around in my head for the last year.
I think it's also important to remember that the value or return sought is a moving target. As well it should be. Every company or brand should be defining it's goals according to its own industry, audience, etc. So measuring the impact of social media on those specific goals will require different tactics and case-by-case rethink.
Where do all the people quoted report in the organization? I would suspect Marketing. So we have Marketers claiming that there is no reason to measure the ROI of Marketing. "Half the money I spend on advertising is wasted; the trouble is I don't know which half" has been the screen that Marketers have hidden behind for 80 years for not justifying their spend. That has changed, a lot, as Marketing is increasingly required to monetize their spend.
I think it's a bad precedent in the early years of social media to play the same old "we don't need to measure this" card. As soon as budgets are cut, non-monetized social media will be the first to go. Better to put a framework in place to be able to justify this spend with hard dollars.
Anonymous, first, please sign your name to your comment. Everyone else did and you should be willing to as well.
Second, I can't speak for anyone else, but I am DEFINITELY in favor of companies having to prove the worth of their social media initiatives. You HAVE to do this.
The question I have is, since it's so difficult to tie social media efforts directly back to $$$ in many cases, should we be focusing so much on that? Some of the most successful social media case studies on the planet have no direct tie from their efforts to $$$. If we demanded that from all efforts, none of these successful efforts would have ever happened.
MY big problem is when a company says that the return on their blogging strategy is that they get 4 comments a post. So freakin' what?!? But if that company knows that more comments = more traffic = more links = more sales, then fine. I think instead of worrying so much about the pure business definition of ROI, that we should focus on finding the VALUE from our efforts.
IMO saying you have 1,000 followers or 4 comments a post is NOT value, unless you can draw a line from those metrics to an impact on your business. Or at least to another metric that you know DOES impact your business.
Mack, I had to chime back in as the anonymous comment struck a chord with me. As a Marketer, I must disagree with the assertion that we do not want to deliver bottom line results. My mission was the same when I managed Marketing in Corporate America, to create value and help the company (and now the client) exceed their objectives. However, there is danger in measuring the WRONG metrics. I believe that is what is being said at the heart of this discussion, by all means let's measure but let's determine the right objectives and measure the right things.
Mack, thanks for calling out 'Anonymous' - that's not going to fly here. You're totally spot on with the assertion that posts, followers, comments, etc. don't mean anything without looking at them in context of the big picture. Lest we forget, SM should be looked at as a long burn, not a 'campaign' that can be tied up nicely with a bow on top. If done well, the ROI will be realized over time. It's a slippery slope when we start talking about tracing the efforts to dollars right away. Just as investing in good customer service and PR don't always have an immediate path to the bottom line, social media won't always either. And to Karen's point, if we measure the wrong things just for the sake of having numbers for a report, we haven't done ourselves or our clients any favors either. Great discussion everyone!
I think it's best to focus on the value you create by using social media rather than the actual dollar number. You will see greater results focusing on people and relationships rather than focusing on money and numbers.
I agree with your thoughts on ROI but how does a smaller team get convinced you aren't selling them ocean front property in Az?
I still think you need to demonstrate some value. You need to know how you are going to benefit the organization.
In the end, even in the job market, the bottom line is how you can add value to the organization or save them money.
Some of us seem to be missing the central point - which is that it is always about the bottom line (or the top line). The "value" of social media activities that everyone is talking about is ultimately "dollars." Whether measured precisely or intuitively, companies engage in SM to drive sales or efficiencies, both measured in currency.
Remember these are corporate entities with budgets/targets to meet and shareholders to satisfy. They don't (knowingly) engage in any activity that doesn't purport to advance those goals.
As many of us know (and wrestle with constantly) measuring the direct, precise monetary impact of social media efforts is difficult. And that is exactly the reason people say that they don't care about measuring it, that it makes them "sleepy". If it was easy to measure, they would most definitely do it!
So what many do is measure as far as they can; they measure the intermediary steps (clicks, comments, mentions, etc) and understand/believe that those results lead to sales or savings.
But make no mistake, the execs that support social media efforts but don't expect to measure those efforts in monetary terms still believe that those efforts are delivering monetary returns... or else they wouldn't support them.
"Value" in the (real) business world is about the bottom line.
I agree that there can be non-dollar value in the use of social media for marketing; that's a no-brainer. The question is how do you optimize your efforts and become more effective? To be effective at optimization, there must be some objective and quantitative measures. The closer they align with other business measures, the better, which is why dollars are generally the ideal ultimate measure. But it doesn't have to be dollars. If one of your business's key metrics is, say, number of subscriptions, and you can discern an increase in subscriptions from the social media channels you're working in, then you have a useful measure.
I think for someone to say that they "see the value" in social media is a little bit intellectually lazy; it's stopping short of fully articulating what that value is. Once you articulate it, you can maximize it.
(By the way, it was great to meet you in Birmingham, Mack!)
Let's assume that it a good thing to derive value, however value is define, with social media efforts.
Now it is the end of the fiscal year, and it is time to hand out salary increases and promotions.
Your CMO looks at the Pay-Per-Click Manager, and sees that the PPC Manager increased profit by $475,000. The beancounters love this story, it is easy to quantify the cause and effect generated by the PPC Manager's efforts. The PPC Manager gets a 10% salary increase.
Now your CMO looks at what you accomplished during the past twelve months. You strongly believe that your efforts created value, however value is defined. What argument would you use to defend the concept that you deserve the same salary increase as the pay-per-click manager? Do you believe you could argue your case effectively? What metrics/facts would you use to support the hypothesis that your salary increase should be equal to or better than the pay-per-click manager?
Answering that question takes an ROI discussion to a different level. Now it is about your salary --- what metrics/facts do you need to make sure your efforts are recognized and appropriately compensated for?
Kevin thanks for your comment, because you just nailed the KEY issue I was trying to tackle. I can tell by many of the comments, that I didn't do a good job of making my key point.
I wasn't saying that you shouldn't justify your social media efforts by attempting to find the bottom line impact. I was saying that if you CAN'T find that bottom line impact, then that shouldn't necessarily mean that you shouldn't use social media. Let's be honest, for a company that's never done social media before, it's very difficult if not impossible for them to find a bottom line impact BEFORE they launch their efforts.
But if they have a blog (for example), I don't think it's enough to say that you averaged 4 comments per post, and gained 50 links this quarter and sent 3,000 visitors to the company's main website. All those stats are great, but back to Kevin's point, you have to find the VALUE that was created by your blog's efforts.
For example, what if the blog sent 3,000 visitors to the company's main site. You could then drill down and see that of those 3,000 visitors, 20 ended up placing an order after arriving on your website. You can then find the dollar figure of those 20 orders, and that is a bottom line impact of your blogging efforts.
I think at the end of the day you have to either find a direct bottom-line impact of your social media efforts OR you need to have your social media efforts affect something that your boss sees the value in. For example, I assume Graco can't directly tie a change in sales from their blogging efforts (since I've never seen anything that suggests they can). But they CAN track a change in traffic, in links. 18 months after launching their blogging strategy, online mentioned had DOUBLED, and the percentage of POSITIVE mentions had increased from 68% to 83%.
Graco's upper management can look at these numbers and tell that the blog is creating value. They don't need to see a direct bottom line impact (although I am sure they are trying to find it) in order to tell that the blog is working.
I too find it quite interesting. It's one thing not to measure ROI in dollars--but quite another not to measure it ALL. It seems to me that non-monetary results should be quantifiable nonetheless. I would think going "commando" with a total lack of measurement would be a bad choice for most.
The only time it's relatively safe to go with your gut is when a)you're VERY highly experienced and b) the campaign/program in question is a runaway success. In that case, it's hard to demand accountability if all you'll turn up is slight variations on just how insanely great everything is. But that's rare. Most campaigns are flirting with success or only slightly over the line. That's when measurement's really valuable.
If social media marketers drift into the gut check approach to measurement, I worry they may morph into the present day equivalent of ad creative directors of old, who made "art," didn't care what the client thought or what the results were, and lived on 3 G&T lunches. I think it's possible to have the self-discipline to measure without being a slave to the numbers.
Years ago marketers started to shift budgets from traditional advertising to integrated marketing discipline and tactics because those effort could be effectively and meaningfully measured.
At some point marketers will want to measure the effectiveness of their social media marketing efforts. But what and how they measure can be as unique to their businesses as their brands are to their consumers.
I believe what marketers have to understand (and many do) is that with social media marketing still in it's infancy - trial and error and learning as you go is the price of admission to this new game. And it's going to take some time to figure it out.
"SM ROI" is the buzz word of the moment, isn't it?
I recently collaborated with Mark Schaefer ({grow}) on a series of posts on this very topic. It was eye-opening how quickly the two-post project we'd planned mutated into an 8-post long series on the qualitative and quantitative benefits of SM marketing. There's so much to learn and consider.
At the end of that exercise, my "bottom line" was about finding ways to leverage both types of measurement, collaboratively. I think that there is too much emphasis placed on the competition - "soft influence" vs "hard $$ ROI." SM is a new venue that requires attention on both sides. I think that the future holds great promise for marketers who focus more on creating great conversations that create greater opportunities - whether they tally up on the hard-line ROI board or on the softer "return on engagement" board.
Defining what we're accountable for has to come before assigning accountability.
Cool post, Mack. It's great to actually see different big brand perspectives side by side like this. Very interesting.
Two things:
1. You use R.O.I., effectiveness and value as if they meant the same thing. (Maybe not you, but the post does.) ;) I would caution that R.O.I., effectiveness and value are not necessarily the same thing. Gary at Best Buy understands ROI and actually talks about ROI here. However, Bonin at PepsiCo thinks he is talking about ROI but is really talking about engagement. When everyone has a different definition of R.O.I. and starts referring to web traffic, impressions and sentiment as ROI, the conversation turns into a mess. So it's crucial to distinguish R.O.I. from other types of impact.
2. (See my first point.) Too many companies today look for R.O.I. measurement in all the wrong places. When I hear an otherwise bright and talented guy like Bonin say that his SM program is completely R.O.I. focused but in the same breath describes the engagement (not R.O.I.) metrics his ad agency has in place, it makes me wonder why in 2009, so many big brands are still having trouble with this. Every first-year MBA in the world learns what R.O.I. is. What's the disconnect?
Great post, Mack, but some of the respondents' answers really baked my noodle. I'm torn between working even harder to help companies get this once an for all, and just giving up altogether and doing something else.
I'll add one more thing: It is very refreshing to hear that some companies understand the value of real engagement without having to tie it to metrics (financial or otherwise). I wish more company cultures were that evolved.
;)
Not quite sure what would apply for a big company would apply for a small business. For example, a large company can afford to experiment by hiring interns to tool around on social media all day.
However, even a limited amount of time on social media with no ROI would be foolish.
best,
Chris o.
Referral Key
"Your Trusted Referral Network"
Information, information, information, that is the key. If you are waiting for someone to ask you what your ROI is, then you are missing the boat and behind the ball. You need to beat these people to the punch and be ready to show you what you are gaining out of your community/social media efforts. Distribute reports and analyze the information that you gather (whether it is sentiment, traffic, trends....). $ ROI can be difficult to directly measure, but informing business units as to what people are saying about various topics is priceless. This direct and immediate feedback can have a very positive effect on any aspect of your business.
Are we too worried?....I think that some are, but it is up to "us" to vocalize what it is "we" are measuring, and what "we" will be reporting. We need to educate those that are new within our "world". Whether it can be directly tied to revenue for your business or not, make sure that you are not caught off-guard - always be ready for the question and make sure that you have an answer.
Mike
Lithium Technologies is organising a Live Webinar on Sept 2 at 19:00 GMT entitled "The New Customer Service Imperative : Driving ROI with Peer-to-Peer Social Networks" http://bit.ly/1HJM7S
Their approach is to take measurable benefits such as reduced Support costs by encouraging peer-to-peer support (measuring indicators such as deflected Customer Support calls) and such to get buy-in from the C-Levels. Once the solution is in place, it will be easier to be more creative and derive more indirect value, such as engaging customer communities to drive product and services innovation.
Although it will be a more difficult job to measure it accurately, improved C2B and especially C2C communication will ultimately boost Brand Loyalty leading to the much sought-after increase in Sales...
I have long been confused as to why companies cite lack of firm ROI data as a reason not to embrace social media. I worked in advertising for a long time, which was invariably measured simply using pre and post attitudinal studies. Long term brand awareness and the communication of the proposition was key.
In any case, even for tactical campaigns, advertising would often be integrated with other elements of the communication mix: PR, promotions, sponsorship etc. and ROI would have been applied to the collective effort.
Why therefore wouldn't Social Media simply be viewed in this way and used and measured as part of a holistic approach to communications?
I think the thing that often gets missed in these discussions is that you don't just measure ROI to prove that something is paying for itself, you do it to determine which activities you should focus on. Resources are scarce and companies have to decide between competing projects, investments, etc. Measuring ROI helps make this decision.
That said, I think your comment that it's enough to know there's a link between comments-links-leads-sales is right on target in many cases. I'd argue that this is still a form of measuring ROI though (it's just a thumb-to-the-wind approach to doing it). It's not really an issue of whether or not you need to measure, it's HOW MUCH you need to measure. If you know intuitively that the project is paying for itself, the activities aren't competing for significant resources, and the company is learning about something that will provide long-term value, ad hoc methods are probably good enough.
Seems like people hear the word ROI and immediately picture an excel file with 26 worksheets, the inclusion of every single cash flow and calculations down to the sixth decimal point. At the end of the day, you should put exactly the amount of energy into measurement that's required to make the right decision, and not an ounce more.
Mack,
Thanks for initiating a great dialogue on this topic. I spend a fair amount of time researching this topic and this is one of the best conversations I've seen.
I'm so jealous that private sector has the luxury of experimenting with social media however they want! We in government are having a harder time selling the value of social media to C-levels and often look to the private sector for translatable examples.
Many of use meaningful storytelling as a way of defining the value (or potential value) of social media so these conversations are extremely interesting to someone like me whos job is to demonstrate value to a different type of shareholder - citizens.
For those semantic types on the thread, we define "value" in terms of efficiency (money spent) and effectiveness (impact/desired change), and also measure satisfaction as a gauge of our success.
Mack, great discussion! The "Return" companies look for is based on their objectives. The key is to understand the objectives and measure against that. If it can be tracked to money, even better but it may not be possible to measure a direct relationship, yet. The key is to measure something that is of importance to the organization that will ensure the support needed.
Also, most of the comments are focused on marketing type applications. But, we also need to consider other applications for Social Media that DO have distinctly measureable value propositions. These "returns" can often times be used to fund the intangible values discussed here. Quite often a visionary "C" intuitively knows that something is important but may not be able to directly justify it in light of other more pressing issues. They will then look for any avenue to justify the effort to gain the intangible value (knowing full well that eventually the proof will come). By ignoring these other opportunities, sometimes I think we leave many opportunities on the table by not broadening the scope of the potential Social Media delivers.
I agree with Kevin above. The industry will not move forward with out understanding key metrics. So the answer to the title...are we too worried with finding the ROI of social media? Absolutely not.
For me to explain in full would take a full on ebook...which I am working on by the way. In the meantime check out Olivier Blanchards blog.
Great post. These companies have the branding approach in mind vs. lead generation. And while there is certainly room for both, I think we will see more return if we have approach social media as brand custodians rather than direct marketers; at least in today's world. As soon as once market research begins to figure out how to effectively analyze the data there will be even broader executive buy in.
Post a Comment