Demonstration dashboard. Every name, region and number here is fictional sample data, built to show how the field ride calibration reads end to end. Nothing on this page reflects a real client.
The project
How Well Calibrated Are ASM Ratings?
A calibration check on how competency ratings are going. Independent Echelon observers joined fifteen full-day field rides, five in each region, and rated every call on the same competencies and scale ASMs use in their coaching system. Comparing the two sets of ratings shows where ASM ratings are well calibrated and where they drift. It looks at the ratings themselves, not at the individuals on the rides.
The big picture
ASM ratings are well calibrated on Preparation and Planning, and run high on Closes for Action.
Where the evidence shows up in the plan and the tools, ASM and observer ratings largely agree. On Engages the Customer and especially Closes for Action, where the rating depends on what happens in front of the provider, ASM ratings run ahead of what observers recorded. The opportunity is less about rating more and more about rating consistently.
Field
What sits behind the rating gaps
Themes seen across observers and regions, each followed by what observers saw in the field.
Organization
Across the organization
Each competency below is shown three ways, all on the same Developing, Demonstrating and Exceeding scale.
1ASMs rated, whole organization: every competency rating ASMs recorded over the review period.
2ASMs rated, the 15 observed rides: how those same ASMs rated the fifteen people on the observed rides. This is the apples-to-apples comparison.
3Observers rated, the same 15 rides: how Echelon observers rated every call on those rides.
Region
By region: the Closes for Action gap is everywhere, but rating habits differ
The same comparison, split three ways, with five rides observed in each region.
Individual
Ride by ride: Closes for Action often goes unrated
▸Ride-by-ride view: ASM and observer ratings for all 15 ridesClick to open a grid comparing the ASM’s rating with the observer’s for each ride.Open
One row per ride. Each cell shows the ASM’s rating first and the observer’s second. Shaded cells are where the two disagree.
How to read this
- Scope. Fifteen full-day rides, five in each region, nominated by each RBD. No day was curated.
- The two ratings are built differently, and that is the point. An ASM files one rating per visit covering a day or more. Observers rated every call, so a single visit-level rating cannot hold the weaker calls alongside the stronger ones.
- Same scale, same competencies. Both sides use Developing, Demonstrating and Exceeding. Competencies that can’t be observed on a single ride are rated in the coaching system but excluded here.
- Sample. Fifteen rides show the pattern at organization and region level. They are not enough to characterize an individual ASM.
- Demonstration data. Every figure and name on this page is fictional.
© 2026 Echelon Performance. All rights reserved. Confidential and proprietary. This dashboard, the Evidence-Based Coaching™ program, and the field observation method and rating approach are the intellectual property of Echelon Performance. No part of this material may be copied, reproduced, distributed, republished, or used to create derivative works without the prior written permission of Echelon Performance.
Demonstration only. Every organization, region, manager, team member and result shown here is fictional sample data created to illustrate the analysis.