One of the problems with PR evaluation has been that each organization that measures does so in its own way. For example, if you are a client organization and have two agencies doing programs for you, and each agency does its own evaluation, it is a virtual certainty that you will not be able to effectively compare to each other the results of the two programs. At a very basic level, the two agencies might be counting impressions differently. If the agencies place stories in publications, one might use paid subscribers to indicate how many might have read the article, while the other might use "pass along" readers, which means paid subscribers and all the people they supposedly pass the publication along to. As you go up the measurement complexity chain to content analysis, things can become even more squirrelly. The two agencies likely will have different definitions of positive and negative and different methods for coding, so any tone measurements will be inconsistent between the two.
When we move on to measuring the effect of a program on a target stakeholder group through surveys, the different agencies will likely choose their sample set differently, phrase questions intended to get at attitude and intention differently, groom the data differently and finally interpret the results differently.
For many years the Institute for Public Relations (IPR) Commission on Measurement and Evaluation, of which I am a founding member (full disclosure here) has discussed the need for standards in PR measurement. At our last meeting, June 7, we agreed on a set of standards for evaluating results in traditional media. While this may seem behind the times in this day and age of social media, it is a start. Eventually, we will have a paper on the Commission website discussing this at length. Until then, Katie Paine has done an excellent summary in her Blog, so I direct you there.
With regards to standards for survey research, David Michaelson and Don Stacks have made a start on that in their paper "Standardization in Public Relations Measurement and Evaluation" in the Public Relations Journal, Vol. 5, No. 2, Spring 2011. If you are a member of PRSA, you can access the paper here.
IPR, through Tim Marklein of WCG and Katie Paine, is participating in a coalition of organizations intent on developing standards for social media measurement. Other members of this coalition include the Public Relations Society of America (PRSA), the International Association for the Measurement and Evaluation of Communication (AMEC) and the Council of Public Relations Firms (CPRF). I will let you know how they're doing as I receive updates.
David Geddes, the current Chairman of the IPR Commission on PR Measurement and Evaluation wrote a great Blog outlining an overall framework for what I've discussed briefly here.
Post a Comment