Observing the saga of Klout, a venture designed to measure the on-line influence of an individual or organization, has been nothing short of fascinating. A recent change to the Klout algorithm, and its impact upon scores, sent the world of social media into a frenzy. To put it mildly, open anger and rampant confusion concerning Klout have not been in short supply.
A difficult task
I have a bit of a different perspective on Klout’s struggle to develop into a meaningful measure. I find the struggle to be quite predictable. Probably because where I come from, when a new construct and its measurement are proposed, it often takes a very long time to determine true value and identify prudent uses in the real world.
On one hand, the outcry that resonated after the scoring revision occurred was a good sign. It let them know that Klout, at the very least, was actively being considered as one measure of influence. On the other hand, it was obvious that the scope of the development phase may have needed to be more controlled to allow for necessary iterations key to its development.
Measuring a new construct
In psychology, the development of a new construct is an important and often tedious process. But when you consider the importance of measuring key ideas, such as intelligence and motivation, the development of that construct – and its valid measurement are paramount. Yes, it is often useful to differentiate among individuals in a meaningful way – and it seems that a measure of on-line influence was a necessary development in the world of social media. But you have to tread very, very carefully.
A framework to apply
It may be useful to view Klout in reference to a few traditional elements of measurement:
- Does Klout have Face Validity? In other words, does the idea and its components seem to make sense to most people.
- Does the measure demonstrate reliability? In other words, does the measure show consistency.
- Does Klout possess Content Validity? Do the components that make up the measure seem representative of the elements of influence.
- Does the measure have Construct Validity? When you look at individual scores on the measure, the scores should jive with other key markers of influence.
There is another point to briefly consider here. Klout may be measuring “Potential to Influence”, and not influence itself. We simply do not know. As with other constructs, such as job satisfaction and its relationship with turnover – Klout scores may have a relationship with action, yet the relationship may not be a causal one.
Time to mature
In the end, Klout has to be allowed the time to develop fully. The algorithm should be subject to changes and iterations, as the organization sees fit, to adequately develop the measure.
Dr. Marla Gottschalk is a Workplace Psychologist. Contact her practice at marlagottschalk@comcast.net. You can also find her on Twitter and Linkedin.
