Stop Measuring ROI on Soft Skills Training
Let’s tackle a garden variety corporate orthodoxy: the one that says your company shouldn’t do training without a measurable return on your training investment.
Variations on the theme: if you can’t measure it, you can’t manage it; all training must be defined in terms of behavioral objectives; each objective must link to behavioral milestones, each quantifiable and financially ratable.
Let me speak plainly: Subjecting soft-skills training to pure skills-mastery financial analytics is intellectually dishonest, foolish, wrong-headed, useless at best and counter-productive at worst.
There, I said it.
Now let me explain—and offer an alternative.
There are are sprinklings of truth in the rush to measure soft-skills ROI—but they are surrounding a germ of crap, like a Bizarro oyster and anti-pearl. Worse yet, the ones who buy and propagate this dogma are those who buy training, and those who sell and deliver it.
The ROI-behavioral view of training is fine for pure cognitive or pure behavioral skills. If your focus is on teaching Mandarin to oil company execs, mastering the report generation functions of CRM systems, or teaching XML programming, you can stop reading this now.
But if you’re talking about communications skills, trust, customer relationships, listening, negotiation, speaking, giving and receiving feedback, consultative thinking, influencing, persuasion, team-building and collaboration, then read on. There are at least four problems with measuring "return" on these kinds of programs.
First problem: definitions. We evaluate golf coaching by lowered golf scores—neat, clean, unarguable. But try defining “good communication.” Or trust. Or negotiation. You might as well define the taste of water, or the quality of love. To accept behavioral indicators (“she smiles, she touches me”) is to miss an essence.
Second: causality. All causality is unprovable, though we know when to accept it anyway. “I had 3 lessons with a golf coach, and cut my score by 8 strokes. It was the coaching—you can quote me!”
But what if I take one course in trust, and another in listening. Suppose my sales go up next year by 50%. Which course did it? Or did my company’s 70% growth have something to do with it? Or my happy new marriage? Too many variables.
Third: the Hawthorne effect. (Or, the Heisenberg Principle in physics). Sometimes the act of measuring alters the measurement of the thing being measured. If I know I’m being graded on listening, I’ll do whatever it is I think that you think makes me look like I’m listening. Which destroys real listening.
If you hype net-promoter scores, many will game the scoring—thus reducing the genuineness that underlay the original idea.
Fourth: the perversion of individual measurement. Most soft skills deal with our relationships to others. The drive to individually behavioralize, then metricize, has the effect of killing relationships—an ironic outcome for relationship-targeting training.
Suppose a course teaches focusing more on the customer, listening, helping others achieve their goals, helping teammates grow—worthy objectives, found in many programs.
The only reason to define those results financially is to evaluate them financially. Thus someone—somewhere between the CEO and the person getting trained—is responsible for deciding to do more, or less, relationship-building programs—by using short-term individual measurements, usually with short-term incentives.
Hence the perversity: training people to focus on relationships, by measuring and rewarding them individually.
“The more unselfish you are, the more money we’ll give you for being unselfish.
“The more you get rated as providing ‘excellent customer service,’ the more we’ll pay you” (which leads to pathetic begging by CSRs)
“The more you focus on others, the more we’ll pay you.
“Quick, get over here, I want to genuinely listen to you so I can raise my quarterly bonus and get promoted.”
Raise this perversity to the level of an industry over decades, and you can understand why pharmaceutical and brokerage companies have accrued such low ratings on trust.
So what’s the answer? Simple. And you don’t even have to give up your addiction to metrics.
Just measure subjective rankings.
Ask people these simple questions, over time:
1. Would you do that training again?
2. Would you recommend others attend?
3. Would you include it in your budget?
4. How do you rate that training compared to these other five programs?
You can run regressions, chi-squares and segmentations on that data to your heart’s content—as long as it’s measuring subjective data in ranking terms. Just stop trying to monetize interpersonal relationships by measuring ROI on soft skills training.
While I agree with the inappropriateness of measuring soft skills I am caught between the logic you present and Tom Peters truism, "What gets measured gets done." Check out the October 2008 edition of Tom Peters Times. I suspect that there is a very crooked line out there separating good measurement and bad. I would be very interested in your comments.
Bruce,
Great question. I have idly wondered about how reconcile my notion with that phrase by Peters; now you’re forcing me to tighten it up! I’d appreciate your reaction.
I think Peters’ dictum is an accurate sociological description of what usually happens. That doesn’t mean it’s a useful pre-scription, however, either for what has to happen, or for what should happen.
The problem is that not only "what gets measured gets done," it’s that "the thing that gets done is aimed at affecting the measure." Which is no longer about the thing being measured, but often just about the measure.
The best short-term performance comes from operating within a long-term strategy. But the tyrannous ubiquity of short-term measurement too often obscures that, and you get short-termism of quarterly earnings, monthly sales quotas, CSR time-per-call limits, and the like.
If too many people take too literally the "measurement drives action" dictum, it turns into a Frankesteinian parody of the original. The means turns into the end. Salesmen sell just to get the money, rather than helping customers and earning money as a byproduct. Product sales companies engage in channel loading. GE reportedly used their financial services businesses to smooth quarterly earnings. It’s as old as Tom Sawyer cornering the market in sunday school tickets by trading for more earthly delights.
Measures are supposed to be indicators of something meaningful. They are supposed to be representations, not the thing itself. But when we all act by "what gets measured gets done," we create a self-fulfilling prophecy. We start maximizing the measures, not the real goal.
What can we do about it? I remember a senior Intel exec, a good man, who said "a great sales manager tells people what to do despite the measures, not because of them. And I have heard the CEO of two major global professional services firms say "I don’t want to hear about conflicting incentives–do the right thing, and we’ll fix the measures later."
More recently, Don Peppers and Martha Rogers have written about the delusion that short-term measurement-maximizers are really effective; they point out that the market knows the difference between manufactured earnings gained by short-term measure-tweaking and high quality earnings gotten from the consistent application of basic customer-focused principles. That’s stock prices they’re talking about. That’s hard stuff.
So that’s my best reaction: Peters is an accurate describer of reality on this issue. But–and this is unusual for him–in this one instance, I don’t think he’s at his best as a prescriber.
What do you think?
Brilliant. This should be essential reading especially for asinine training managers besotted with quantifying and metricizing soft sciences and falling prey to what Richard Feynman called the “Cargo Cult Sciences”. Some people need to wake up and smell the coffee.
Horit’s comment above intrigued me; I went to find the Feynman reference on “Cargo Cult Sciences,” and it’s fascinating. It’s about a much broader topic than just this one, but it’s extremely applicable to the issues of today.
Have a look at:
http://www.lhup.edu/~DSIMANEK/cargocul.htm
Enjoy! And thanks to Horit.