Posts Tagged: Measurement


23
Jan 12

Doing Measurement Right From The Start!

This post got its inspiration from  Building Data Into The Stuff That Agencies Make written by @kevinrothermel where he spoke about integrating data into the work that agencies do. I thought this was an important idea and perhaps something that is not often thought about enough when awesome and creative works of advertising, public relations or marketing are put in front of consumers.

In considering how to integrate data to effectively measure a campaign or initiative (as well as to ensure that measurement is most effective overall!)  it is in my honest opinion necessary that it is put in place from the very start! I don’t mean the first day of the campaign or when all the pieces of the initiative are starting to come together but starting when the very first utterance of the initiative is said long before it is even decided what is going to be done.

Measurement At The First Meeting:

All marketing and even more generally communications efforts have a goal of what they are to achieve. This is the basis of what the measurement initiative should be. In the end the firm wants to measure how effectively their efforts work to achieve the end goal. Having this goal set out clearly and  understanding of how measurement can be integrated will spark everything to come.

Reviewing Past Data:

Remember all of those reports you had from previous campaigns? They aren’t something that should be put on the shelf once you have reviewed it at campaign end. They should be categorized and brought out every time you start to plan an initiative. Why? Because they hold the key learnings that you can form your strategy around. From this data you can see what worked, what didn’t and what can be reused in your work going forward. Without consulting with what you previously did you doom yourself to repeat the same mistakes.

Realizing Your deficiencies:

With your goals and your plan nearly in place  you can begin to realize if measurement can be done thus allowing the project to move forward. Simply put, if you can’t measure an initiative based on its goals then it should be highly questionable if it should go forward. If you don’t know whether something did great or completely flopped there is no way you can go back to your stakeholders and prove that you didn’t waste their money.  Some limitations in measuring a campaign can be technical (does the technology exist to measure this?) or can be as a result of  a strategic or tactical issue where the campaign is improperly aligned or perhaps the campaign goal set at the start needs to be revaluated as its either not defined enough or just unrealistic.

Improving The Visibility Of Measurement:

The point in all this that I’m trying to make plain and simple is that measurement needs to be given as much attention as everything else. Effectively measuring your work cannot be done when measurement is brought in as an afterthought because then its effectiveness is compromised. When its brought in at the start of the process it is then kept at top of mind and as a result allows any opportunities or potential difficulties to be considered. In a continually competitive landscape measurement isn’t a nice to have but a must have to prove your work.


20
Sep 10

Do you want Foursquare with that?

The most important concept in social media marketing that I’ve learned is that for efforts to be effective they should be connected to how the firm earns money. Having fans and creating engagements is great but when it’s not driving firm revenue in some way, shape or form it’s not useful.

That’s why when I read Mashable’s recent article about the McDonald’s head of Social Media Rick Wion celebrating a 33% increase in Foursquare check ins with a pilot program they ran I was disappointed that there wasn’t more to it. While the goal was apparently not to increase sales immediately this campaign could have been much cooler in seeing if McDonalds actions actually created an increase in sales( or not…).

How McDonalds could have measured the campaign (if they had wanted to):

McDonalds could have easily done one of two tests to see if this campaign was effective. First and probably most easy is that they could have just compared sales and check-in results from previous days for an extended time period to see if this 33% increase in check-ins actually led to an increase in money into stores. If revenue saw a significant jump when foursquare activity spiked (barring other variables) Mcdonalds would have at least some proof to try the campaign again.

The second test is running the campaign on a smaller scale comparing it between two similar markets. One would run the Foursquare campaign and the other would have no promotions. In the end, sales and check-in figures would be compared between the two. If the market with the Foursquare campaign saw an increase in check-ins along with revenues in comparison to the control market McDonald’s could further test this campaign and have it rolled out on a larger basis.

While I don’t know the full goal of the McDonalds Foursquare campaign I feel that despite the celebration within the Mashable article it’s still important to clarify that social media in business is not successful by engagements alone. If I would go on twitter or another network and tried giving away something for an audience action people would have jumped on it as well. In the end social media campaigns need to be attached to financial drivers because most companies can’t operate on foursquare check-ins alone. ;)

Do you have a question or comment ? Feel free to reach me at @kevrichard or kevin@kevrichard.com


29
Apr 10

A Framework for Social Marketing Analytics

Like with everything that is done in business, it has to be done for a reason or else it’s just wasted effort. Measuring simply anything from your social media initiative without a reason behind it is just that – a wasted effort. In reading “Social Marketing Analytics” by John Lovett and Jeremiah Owyang this point is further proven. In measuring social media what you measure needs to be attached to specific business goals, not measured just because the number is available.


The paper by Lovett and Owyang starts off with a good definition of what Social Marketing Analytics is (what can I say I like when things are plainly defined):

Social Marketing Analytics is the discipline that helps companies measure, assess and explain the performance of social media initiatives in the context of specific business objectives.

Pay particular attention to the last part – “…in the context of specific business objectives.” What did your social media initiatives achieve for your company? Social media measurement is not just looking at the numbers of fans go up or down (or whatever shiny number there is out there) but looking at the end result of what your work achieved for your organization. Very simple but great stuff.

Lovett and Owyang write about a three layered approach to Social Media Measurement, which I feel is effective in further demonstrating the need for measurement:

1. Strategy: The reason behind the initiative.

2. KPI (Key Performance Indicator): What the measure of success or failure is with this initiative.

3. Granular Metrics: The information or inputs that need to be gathered to allow you to formulate the KPI.

Social Marketing Analytics Framework

In keeping the paper accessible to general business practitioners Lovett and Owyang present four major strategies (although they mention that there are more) along with suggested corresponding KPIs. I’ll leave the KPIs for more in depth reading in the paper but I thought I’d present the strategies as these apply to many businesses:

1. Foster Dialog: Developing conversation around your company or brand. This is the development of content to create interest in what you’re trying to achieve.

2. Promote Advocacy: The development of ambassadors for your organization. Encouraging people to actively discuss and promote your brand online.

3. Facilitate Support: Provide a channel where customers are able to get their problems/concerns addressed in a timely manner ( Ex: @Rogershelps or @Toronto311).

4. Spur Innovation: The development of active discussion around your brand or initiatives focusing on the topic of improvement. Essentially putting like minds together online in developing ideas to make things better.

Overall in reading this paper it really puts it out there that:

A: Knowing what you want to achieve with social media is important and,

B: Measuring your progress (and not wasting your time) needs to be followed right after.

I really encourage you to check out the paper as it’s a great first step in developing a measurement program. I would also add that while it’s a helpful framework, it’s not a complete instruction guide. Measurement, like any other business initiative, needs to be scaled and customized to fit with the specific organization. Measurement and tools that apply for a large multinational corporation don’t have the same effectiveness for a smaller, local business.


22
Mar 10

Are you learning anything from Social Media?

So I’ve been procrastinating with posting this for a bit , but now I think I have a good intro…. So at my new job at Syncapse, I do a lot of measuring and spending my day looking at client data.  I’m finding it  really exciting because essentially I get paid to do the learning for companies so they can consistently improve their Social Media strategy. Reflecting on the work I do though it seems for a good portion of the ‘social media world’ that metrics and measurement and the whole idea of ‘learning’ from what you’re doing doesn’t get talked about a whole lot. With that, I thought I’d put down a few of my thoughts of why Social Media measurement or the idea of ‘learning’ within Social Media strategy is so important.

You Get To See if Things Work: With Social Media measurement you get to gauge how effective online strategies really are.  Are there no conversations surrounding your content? Is  there conversation but not the kind you want to hear? Then there is obviously something that needs to be fixed. Perhaps you haven’t told your audience about your initiatives or maybe your content isn’t relevant to them. By implementing strategies and comparing them to what you’ve done in the past  you are able to see what you’re doing right or wrong and how to make things even better moving forward.

You get to see if customers love/hate you (and how to improve it!): As a part of online measurement you are able to look at  what people are saying about a company or organization online .  This is looking at what users are saying directly AT you or what they are talking indirectly ABOUT you with everyone else.  Are they having problems with a particular product of yours? Is there something that people want to see more or less of from you? The insights you get from conversations are way different than what you can get from a survey  or traditional market research. In a survey people have time to consider and alter their answers to give you just half truths. Within online communications you’re getting the raw knee jerk reactions regarding what they think about you and how they use your product in everyday life.  ( CAVEAT: Watch out for Trolls! No matter what you do there will be someone looking to stir things up…. all I can say is DONT FEED THE TROLLS )

You Find the Unexpected: Often times its important to stick your head outside of what you’re doing to get a different perspective of what is actually happening with your company. This is especially true with Social Media. By looking at what occurs outside of your organizations’ initiatives every once and a while you may  see surprising things happening with your customers that you haven’t heard of. Perhaps they are re-purposing your product in a way that you didn’t think was important (ex: Coke and Mentos blasters) or maybe some previously unreleased news about your company has become public and is gaining an unexpected amount of traction with your customers. By learning what your customers and the public is doing ‘out there’ you’re able to adapt to things as they come up rather than let events plow you over!

In closing while a lot of discussions regarding social media surround  customer engagement and use of the tools, I think measuring your success and learning from what you’re doing online is just as important. Without knowing what you’ve achieved and what is occurring out in the wilderness that is the ‘internet’ you’re only going to continue making the same mistakes and not improve upon yourself.  With my new position in  Measurement Science I hope to be able to share some  brief tidbits  about online measurement and why its a useful part of the campaign process. Stay tuned!


13
Mar 10

Geurilla Analytics Simplified

I think now is a good time to bring up this follow up post to the recent Geurilla White Paper posting. I’m sure for some, looking at the paper was an immediate shock of words and numbers! I hope in this small version there’s a bit more understanding of what the paper tried to get across.

So whats this paper about? Social Media Measurement and the common use of machine sentiment engines as a way of measuring effectiveness….. or how un-effective this is based on the research we’re presenting. Getting right down to the point, language is COMPLEX! Thats to say that the words and messages we share with our friends are full of double meanings, slang and one of my personal frustrations lack full context. I’m sure most people have had online conversations where you’ve had to defend what you’ve said as it was read in the wrong way.

Knowing that online messaging is really complex, often times in measuring the success/failure of social media initiatives   firms use a sentiment engine to rate mentions of their brand on a scale of Negative, Neutral and Positive. With this in mind, we the Syncapse Measurement Science team looked to see if this was an a reliable measure.

What did we find? That again language is complex and no two people will perceive a set of  messages the same way.

This research was based on an earlier survey we released on twitter  where we asked  people to rate a set of 20 messages ( all taken from a twitter search for ‘books’).  The most significant finding of this survey is the lack of agreement on the survey as a whole. With such a simple answer set  we did not find  a single pair of respondents (out of 102) who could agree on the sentiment of these messages.

To put a further test in our survey we also put a wrench in the works with a trick question. We decided to repeat a question. While a majority of people rated sentiment of this question the same on each response,  19% didn’t!  So in a span of a few minutes a person’s feelings about a message can change [ meaning you have to be particularly careful in how you time your emails! ;) ]

What in the end do the findings of this paper mean? Well to quote directly  from the paper:

If a group of 102 humans could not agree, how can a single machine output a single score that everybody would agree with?

The current sentiment score system that’s in use isn’t netting  information that should be depended upon to make real business decisions. Language is just too complicated for a human programed machine to define the sentiment of social media messages.

But WAIT there’s more! If current  sentiment scores should not be used in decision making then what should be put in its place? Well I know a team that does some mighty fine work in social media measurement ;) …..  Ok,  so there’s ALOT to measure in regards to social media measurement and no one has yet created the gold standard. As an emerging industry, measurement is still playing catchup to everything else currently happening (which keeps things interesting). What we could agree on with things as they currently stand  is a more pragmatic approach, looking at the measures and feedback that is important to a  brand( or ‘Brand Health’). This means looking at the overall online communications about a  firm and determining what aspects of these activities are most important to have knowledge of.

Have any questions/comments ? Contact me at kevin@kevrichard.com or send me a twitter message .

PS: I know I’ve been on a massive blog hiatus, I’m looking to change this ASAP!


8
Mar 10

A Guerilla Experiment In Sentiment Analysis

As tweeted a few weeks ago the Syncapse Measurement Science team put together an experiment on sentiment analysis. Here is the final result of this research:

Syncapse Sentiment Analysis White Paper

and the data set for those interested in looking at the data collected

Geurila Analytics Data Set

Feel free to leave a comment or give me a tweet @kevrichard .