Return on Expectations: 7 Critical Flaws that Undermine Training Effectiveness
So, here we go again with another problematic methodology called “return on expectations” (ROE). Just when training professionals are in damage control with the disappointment of “training ROI,” out of the woodwork, comes another “quick-fix” and repackaged methodology trying to demonstrate training impact on business objectives. ROE, however, is a more elusive and misleading approach compared to others that came before.
For what it’s worth, we are in favor for any (credible) approach providing training professionals an ability to tangibly demonstrate training effectiveness. What we are against is how many training experts try to repurpose obvious business approaches through disguised methodologies that further undermine training’s credibility. As formally educated business and accounting professionals, those that follow our work know our position on training ROI, but ROE is something that concerns us more.
To be objective and fair, this article assesses the ROE methodology solely through the perspective of senior management and management accounting rules. We take this approach to respect the ROE methodology since it states that training must meet key business stakeholder expectations. This article intends to determine the ROE’s credibility to evaluate training and not to mislead training professionals into believing that it is repackaged common business sense (or what we refer to as common BS).
The ROE concept claims that it is a foolproof way to show the “value of training” in the terms desired by key stakeholders. Regretfully, the concept fails to recognize that “training” is already considered a business activity and is always expected to demonstrate value like every other business activity. What key stakeholders want is simply common BS.
From a business professional’s perspective, ROE/ROI methodologies have holes like Swiss cheese. They do not build substantial credibility for training professionals and will not gain senior management acceptance. Below are seven critical flaws (there could be more) that undermine the ROE’s validity and analyzed from the perspective of a senior-level stakeholder.
First Concern: Inappropriate Use of Jargon
Like “training ROI,” ROE is just another attempt to throw around what we to refer to as false jargon. At first glance, ROE and training ROI appear valid and possibly credible. It is when you begin to scratch the surface that it becomes apparent both, and especially ROE, lack substance.
Using jargon and believing that ROE is something innovative is incorrect. ROE is actually repackaged common business sense (BS). In short, it is only common sense that every business activity, including training, must satisfy key stakeholders expectations all of the time or the activity is irrelevant to achieving the organization’s ultimate goal, its mission.
Another aspect of poor judgment and false jargon are the chosen acronyms and names. ROE is already a well-established financial term: return on equity. The poor use of acronyms extends to training “ROI” and one called ROA (return on attendance). Not that we support it, but at least training ROI actually uses an actual return on investment calculation even though the results do not make sense.
Using the ROE acronym with senior stakeholders, or any formally educated business professional, is more misleading to them than ROI itself (even though training ROI is misleading for many other reasons). The only and universally accepted use of ROE in any business and financial context is “return on equity.”
Some say that these experts create “similar” jargon on purpose in an attempt to sounding relevant, but their intentions simply further mislead innocent training professionals. Proponents will then say, “Change the acronym.” The problem is partly with the acronym, but the theory, as you will read, is still weak and common BS. Suffice it to say, improperly using any type of jargon and misleading acronyms in business only further undermines your credibility and make you look uninformed about business concerns.
Second Concern: ROE Superseding Needs Assessments
This is not as much a flaw, but more as an obvious point and common sense for trainers. The ROE concept believes that the value of training begins before a program even starts. Trainers recognize the ROE statement as a needs assessment or skills gap analysis (the first step in the ADDIE approach). Every training designer knows that before even trying to suggest a training solution you must recognize and identify the business gaps to determine if training is the answer.
Training professionals are not stupid. They know that their involvement is to respond, contribute or resolve a business need. To do this requires an effective needs assessment to identify the specific skills requirements and ensure that any proposed learning initiative is effective. So naturally, meeting business expectations is part of the training function.
Third Concern: Taking Credit for Established Practices
Proponents claim that ROE is the ultimate indicator of value. ROE fails to disclose that this “miracle” methodology is simply a 25-year-old formal business practice called performance management. ROE simply took credit and repackaged existing performance management business practices. If you don’t believe us, do a quick search for “performance management framework,” “Balanced Scorecard” or “Kaplan and Norton” and you will quickly discover ROE’s repackaged claims.
ROE proponents will say that our argument is a matter of semantics. Using “semantics” as a defense is a very big stretch to make and somewhat simplistic. Formally business-educated professionals recognize that performance and change management is a highly complex process that requires a holistic approach in executing strategy. Experienced performance strategists view this as a balanced performance approach incorporating lagging (financial) and leading (non-financial) indicators within all business activities to achieve specific strategic goals (its mission and vision).
ROE’s basic claim is that as long as you clarify how training contributes to the highest organizational results through leading indicators then ROE meets stakeholder satisfaction. For performance strategists, skilled employees driving improved performance is only one (very important) component to ensure the organization meets its goals. Senior stakeholders, however, are only satisfied when performance improves all operational activities in the organizational value chain. This is common BS left out of the ROE process.
Fourth Concern: Believing Training Has Influence in Performance
ROE claims that the development of leading performance measures is a negotiated process implying that training professionals have input. This isn’t true. Simply put, after significant competitive research and close analysis of internal operational activities, senior management develops a mission (the strategic objective) and an accompanying strategy to achieve it. Senior management’s concern is to operationalize strategy to meet expected performance objectives and targets.
From a “training” perspective there is little, if any, negotiation since management sets forth key performance objectives. The only possible negotiations are for the skills development costs (not investments) - and even then, budgets are usually set in advance for the year. Even the T&D department plans their performance objectives and financial allocations in advance, according to organizational requirements.
So, the notion of “negotiation” is also common BS. You don’t agree? Then go to an operational unit manager and suggest negotiating any performance metric with them…but you were warned.
Fifth Concern: Asking Questions Already Answered
Another ROE claim is that learning professionals often need to convert the typically broad, un-quantified expectations into observable, measurable leading indicators by asking the question “What will success look like to you?” The only correct part to this statement is the need for leading indicators. The question about “what success looks like” closely resembles what the founders of performance management (Dr. Robert Kaplan and Dr. David Norton) state in the granddaddy of frameworks, the Balanced Scorecard. They ask very several similar questions for each operational activity, including learning.
Yes, it’s true that senior management desire tangible leading performance indicators. This is because there are sufficient lagging indicators (usually financially based) available. Non-financial leading indicators are the only performance measures that provide management with competitive intelligence. These leading measures, however, are already defined and targets set within your organization’s performance management process. Training’s role is to ensure that employees within each operational activity possess the skills to attain the targets.
Sixth Concern: Not Recognizing Partnerships in the Value Chain
This is more of a common BS moment for ROE advocates. Yes, ROE correctly endorses the need to develop “business partnerships.” But again, this is something clearly stated in all management theories. Unlike the ROE theory, this is not a bias, but only what is expected from every performance management process.
Business school educated managers know that “partnerships” (or the more relevant management term, strategic alignment) must exist throughout every organizational process. Harvard economist and strategist, Michael E. Porter, developed this widely accepted theory in the 1980s. The theory is called the organizational value-chain and it clearly illustrates alignment (partnerships) among business processes.
Until recently T&D didn’t recognize this concept. The concept of “learning” gained attention only to the credit of the performance management founders Kaplan & Norton and KPMG (1989-1990) in the first performance framework called the Balanced Scorecard. This performance framework is structured around the three core operational business activities and, to the surprise of many in the business, a fourth activity called “Learning & Growth.” This is when the thought of aligning human and knowledge capital with core operational activities arose, but here’s the issue: It is not to the credit of the ROE movement.
For senior managers, business alignment is all about what we refer to as Kirkpatrick’s Level 3 and 4. This has always been the case. With respect to T&D, business managers don’t care about whether the training went well (Level 1) or if employees retained any of the learning (Level 2)…they never did. This is largely why T&D suffers from a lack of business credibility.
The one thing ROE gets right is to have training align with existing operational business objectives and targets and not to create independent metrics. But once again this is common business sense in a performance management process. To build any type of business credibility training must prove value demonstrating tangible results around core operational activities.
Seventh Concern: Reinventing the “Wheel”
ROE states that to create value at Level 3 and 4 training should develop a dashboard to monitor effectiveness. If you haven’t discovered it yet, your organization already has a performance dashboard in place. If you haven’t seen or used it then you are not aligning to core operational needs (see the sixth concern). If you think you are “aligned,” then you probably are getting lucky, meeting performance needs or not addressing anything relevant at all - even though you may think you are.
Don’t re-invent the wheel creating a T&D dashboard. Use the performance tool in place and discover where “learning” fits. Trust me on this; “learning” is incorporated into your organization’s current performance framework. It may not be obvious since management does not view “training” as a significant business activity. Be sure to align with this performance dashboard.
Final Truth: Be Critical, Analytical and Question What You Hear
We are very passionate about helping T&D gain true credibility. But it troubles us that training professionals buy into the concept of manipulating accepted business principles to build credibility. The issue, however, is the misinformation disseminated by the next “latest and greatest” marketed training evaluation methodology. ROE is another example of repackaged common BS and existing performance management practices.
These types of theories simply further undermine training’s credibility and benefit only those that preach it. Do yourself a favor: Focus your professional development around business skills and financial literacy. You don’t have to become a business expert, but you do need to learn the fundamental business concepts to facilitate communication with your business leaders.
Read as many performance management and business books available by proven and qualified authors including performance experts Kaplan & Norton and Paul Niven, along with business publications by authors Michael Porter, Prahalad & Hamel, and Treacy & Wiersema, to name only a few.
Lastly, be leery about misleading theories. Become a critical learner and question what you read and hear, including this article. Experts are either genuine or opportunistic. You are a learning professional so be informed and knowledgeable. Challenge what is presented to resolve your training challenges and ensure substantiated results for all of your training initiatives. Beware and be cautious.
Ajay M. Pangarkar and Teresa Kirkwood are founders of CentralKnowledge.com and LearningSourceonline.com. They are renowned performance management experts and three-time authors, most recently publishing the leading performance book, “The Trainer’s Balanced Scorecard: A Complete Resource for Linking Learning to Organizational Strategy” (Wiley 2009), and award-wining assessment specialist with Training Magazine. Read their blog, “Workplace Revolution” or contact: email@example.com.
Written for TrainingIndustry.com