Evaluations

Webinar 33 in the "I Didn't Know ClearPoint Could Do That!?!" webinar series

Range evaluations, weighted objectives, and custom statuses, oh my! Evaluations are without a doubt an important part of tracking your strategy – but are you taking advantage of all the features ClearPoint has to offer? Take your RAG statuses to the next level as we dive into ClearPoint’s evaluation capabilities on this episode of the “I Didn’t Know ClearPoint Could Do That!?!” webinar series.

Happy reporting!

Webinar 33 – Evaluations

Introduction (0:00)

  • Hello everyone and welcome to the “I didn’t know ClearPoint could do that!?!” webinar series
  • Today, we are tackling an important part of keeping your strategy on track, and that is evaluations.
  • Before we get started, here are your smiling hosts, I’m Catherine
  • Brendan: and I’m Brendan
  • Catherine: And as a few housekeeping announcements…
    • We will be recording this session and will make it available within a week
    • Webinar will last about 25 minutes with time for questions
    • Submit questions to [email protected]
    • Any that we don’t get to at the end, we’ll follow up with via email
  • On our last installment, we walked through a plethora of calculations, and I’d highly recommend checking out the recording on our blog, vimeo channel, or in the support center – just search ‘Webinar’
  • Today, we are exploring the world of evaluations. Chances are that you use some form of red-amber-green, or RAG, status to quickly check on the progress and health of your objectives, measures, and initiatives.
  • Even Red statuses can be good, as you can redirect focus and resources to the areas your organization needs support to improve. But how do you determine which status is assigned to which elements in your strategic plan?
  • Are you spending a ton of time manually updating them each month? Are your evaluation criteria as consistent as possible across the board? We’ll give you a few tips today that will hopefully save you some time.
  • And for those of you who are a little on edge after the last episode, I can assure you we aren’t going to rhyme anything and everything with ‘evaluation’ today. Been there, done that.

Agenda (1:36)

 

  • We’ll show you how to customize your statuses
  • Set up various types of series and measure evaluations
  • Give you some ideas for effectively displaying evaluation summaries
  • And, walk through setting up automatically evaluated objectives

So, let’s get right to it! We are flying the friendly skies at upward air.

 

Adding Custom Status Indicators (1:55)

  • I’m going to start off here by going into the Initiatives menu, and clicking on an initiative.
  • And, as many of you are surely aware, you can manually select the status of any element by double clicking on the status indicator next to its name at the top of the detail page.
    • So, let’s click into our statuses here.
    • By default, we have Above Plan, Caution, Below Plan, and No Information, as well as the default grey bar, Not Defined.
      • Brendan: As a quick note here, you might be wondering how No Information differs from the default Not Defined status. Various organizations use them differently, but one approach is to use these two statuses to differentiate between elements that still need to be addressed, and elements where it was intentionally noted that not enough information is present to assign a RAG status – for example, an initiative that just started, or a new measure that doesn’t have a target yet.
    • Catherine: That’s a good point. And there’s one other scenario that’s missing here – what if our initiative is marked complete?
      • You might choose to just show whichever status the initiative held when it was completed,
      • But it is conceivable that you might want to have another status that quickly indicates the initiative is complete.
      • And that’s where ClearPoint’s status customizations can come in handy!
  • Let’s click over to System Settings > Admin Options
    • We’ll see a panel in the upper right for Status Indicators, and I’ll click on that
    • Here we can see our existing statuses,
    • And a place to add a new status. Let’s click the add icon,
    • And first thing’s first, I’ll name it ‘Completed’.
    • We’ll come back to this Score field when we talk about objectives…
    • And then we could choose to use one of the default status icons here, but for this example we’re going to want to check the box next to ‘Use Custom Status Indicator’.
    • Then, I can click on the panel that appears, or I can drag my icon in from my files.
      • As far as icon specs, you can use a PNG or a JPEG image – with one key difference being that PNG image can have a transparent background.
      • Then for image size, 24×24 pixels is best.
      • So, let’s click to add our file, and we can see what it’s going to look like here!
    • Then under Elements, we can actually restrict this status icon to particular element types.
    • Brendan: That’s Right, you might not want to have “Completed” confusing your users when it comes to objectives or measures for example, because they don’t really have end dates.
      • Brendan: So, we can just check the boxes next to Initiatives and Milestones to use the completed status on those elements.
    • Catherine: exactly! And then if you’re using any custom fields that display a status icon, you can decide whether to display the icon as an option there too.
  • So now, let’s click save, and go back to our initiative.
    • Let’s say we’ve completed this initiative today. High five!
    • I’ve got a check box here in the Completed field, and I’ll mark it Complete.
      • And the completed date is going to automatically show today’s date. We could change that if we wanted to, of course.
    • So now, I’ll go ahead and click to click to change the initiative status.
      • Double click next to the title… and there’s our new completed status. We’ll choose that one, and click Save.
    • Brendan: One thing to point out here – when we marked our initiative complete, you might have noticed a change to the initiative’s current period, which you can see in the upper right corner of the detail page.
      • Since we marked the initiative complete for a date in Jan-18, any month after January is still going to display the status updates for Jan-18!
      • So that’s where we’ve marked the completed status, and it’s going to carry forward for all future periods
      • Catherine: Right, so if we click forward to March, we’ll still see our completed status.
      • Brendan: By contrast, if we had been looking at Nov-17 in the period selector and assigned the completed status, but the completed date isn’t until Jan-18, then that completed status won’t carry forward.
      • It is important to make sure that you apply the completed status to the period in which the completed date falls.
  • Catherine: Right on! And with that set, let’s switch gears to talk about measure evaluations.

Setting Automatic Series Evaluations (6:27)

  • Thanks Catherine for showing us how to create custom status indicators. Now, let’s set up an automatic evaluation for our ‘Revenue’ measure. I am sure many of you are familiar with manually changing the status of a measure. But if there are many measures in your account, this can become tedious and take up a lot of your time. Automatic evaluations save you this time by automatically selecting a status indicator based on the data that is entered.
  • To set an automatic evaluation up, let’s navigate to our ‘Revenue’ detail page.
  • For revenue, we can see that we want our ‘Actual’ series to be greater than our ‘Target’ series. Since the ‘Actual’ series will determine the status for each reporting period, we’ll double click on its column header.
  • Let’s then navigate to the ‘Evaluation’ tab.
  • On this tab, we can check each box next the status that we want to include in our evaluation.
  • After selecting each status indicator to include, we will now choose how we want to set up the comparison.
  • In the second column, I will leave the ‘Target’ series as our comparison because we want to compare our ‘Actual’ data against our ‘Target’ data.
  • For ‘Above Plan’, I will make the inequality greater than or equal to the ‘Target’. To double check that I entered this inequality correctly, I will read the evaluation: For my status indicator to be ‘Above Plan’, the ‘Actual’ data will be greater than or equal to the ‘Target’ data
  • For the ‘Caution’ status indicator, I want the ‘Actual’ values between 90% and 100% of the Target to have this status – for when we’re almost there, but not quite. I will make the inequality greater than or equal to the ‘Target’. Then, in the third column, I will enter *.90.
  • Catherine: So this is where things can get kind of confusing when you’re first seeing ClearPoint’s automatic evaluations. It seems like that would indicate that values above the target would be evaluated as Caution now – right?
  • Brendan: I see what you mean, but don’t worry. ClearPoint evaluates top to bottom, meaning that it is going to assign the first status indicator in the list that fits that particular value, and then it will not pay attention to the other statuses further down.
  • Catherine: That’s helpful to know! So in this case, since everything above the target value is already going to be green, our Caution range will ONLY occur when the Actual value is between 90 to 100% of the target.
  • Brendan: Correct! For the ‘Below Plan’ status indicator, I will make the inequality less than the ‘Target’.
  • Finally, for the ‘No Information’ status, I will keep ‘Any’ in the first column. This means that it is going to cover Any values that don’t fit our other statuses – basically, only when the Target and the Actual series are null.
  • Again, the evaluation is going to read from top to bottom. If our ‘Actual’ data is greater than or equal to our ‘Target’ data, than the ‘Above Plan’ status indicator will be selected. If our ‘Actual’ data is not greater than or equal to the ‘Target’ data, but is still within 90% of the ‘Target’ data, then the ‘Caution’ status indicator will be selected. And finally, if the data does not meet the ‘Above Plan’ and ‘Caution’ criteria, then the ‘Below Plan’ status will be selected.
  • One other important idea is to keep the evaluations consistent across your account. This way, you will know exactly what ‘Above Plan’ actually means when seeing the status indicator.
  • Now that I have my evaluation set up, I will save the window. However, I still don’t see my evaluations!
  • This is because we still need to complete one more step. We need to tell the measure to actually use the automatic evaluation we just created!
  • So, I am going to edit the measure, and navigate to the ‘Measure Series’ tab.
  • From this tab, I can select the ‘Evaluation Option’ at the top. We can see that by default, it’s still set to ‘Manual Evaluation’. I will change this to use my actual evaluation, and save out of the window.
  • Now looking back at my measure detail page, I now see the status indicators in my data table!

Locking Series (10:23)

  • Awesome! Not only does this save time selecting a status indicator, but it also assures that you have the appropriate status assigned for every period.
  • Now, we’re sure your updaters are not particularly devious, but as they say, there are two ways to meet your target – you can perform well, or you can lower your target numbers.
  • So if “minor tweaks” to the target are something you’re worried about, we wanted to point out a small safeguard that ClearPoint has in place – if I double click into the target series,
  • I’ll see an option at the bottom that says “Lock Series for Updaters”. Let’s check that box and save.
  • With this option selected, users with editor permissions and above can still assign target values to this measure, but updaters can’t change them when they’re entering data month to month.
  • Now you can be really really sure the right statuses are assigned.

Range Evaluations (11:20)

  • Thanks Catherine for showing us how to lock series! I’d been a little nervous that Jenna was fudging her target numbers.
  • Catherine: Typical.
  • For the ‘Revenue’ measure, we are comparing whether the ‘Actual’ series is higher or lower than the ‘Target’ series.
  • However, there are certain instances where you might be tracking proximity to a target, without caring whether it is higher or lower, as long as it falls close to that number.
  • Catherine: Right, that might be something like employee turnover in a large organization, or maintaining your budget.
  • Brendan: Exactly! So we can compare the ‘Actual’ series on the proximity to the ‘Target’ series through a ‘Range’ evaluation.
  • Let’s go into our ‘Budget’ measure to create a ‘Range’ evaluation.
  •  By the following the same steps as our previous evaluation, let’s double click on the column header of the ‘Actual’ series, and go to the ‘Evaluations tab.
  • First, I will select the status indicators that I would you like to use.
  • I am then going to select ‘Range’ in the first column for Above Plan, Caution, and Below Plan. For No Information, I will keep ‘Any’ in the first column, which is saying if the first three status indicators are not selected, then the No Information status will be chosen.
  • I am going to leave the ‘Target’ series in the second column because I still want to compare my ‘Actual’ series against my ‘Target’ series.
  • In the third column, I am going to enter the upper and lower boundaries of the range I want to compare to my ‘Target’ series. I will enter a *.9 and *1.1.  So, this evaluation is saying if the Actual data is between 90% and 110% of the target point, then the ‘Above Plan’ status indicator will be selected.
  • For Caution, I will enter a *.8 and *1.2, meaning if the Actual data is between 80% and 120% of the target point, that the status will be Caution.
  • For Below Plan, I will enter a very large range, like *.0 and *100. This evaluation is saying if the Actual data is any farther away than 80% or 120% of the target data, then the status will be Below Plan.
  • Catherine: And again here, ClearPoint’s evaluations work top to bottom. So that caution range is going to be from 80-90%, and 110-120%, since everything in between will already be evaluated as Above Plan.
  • I will save out of the evaluation, and let’s not forget our final step in creating automatic evaluations! We need to edit the measure, go the ‘Measure Series’ tab, and change the ‘Evaluation Option’ from ‘Manual Evaluation’ to the ‘Actual Evaluation’.
  • Now, after we save out of the edit window, we can see that our status indicators have now changed!

Static Value Evaluations (14:32)

  • Catherine: That’s really useful! But, so far, we’ve always compared to another series. What if we don’t want to compare the actual series against the target series, or for that matter, any series?
  • Brendan: Well, there is a way that we can compare the ‘Actual’ series against a static value.
  • Let’s navigate to our ‘Total Amount of Time at Gate’ measure, and create a static value evaluation. For this measure, the criteria will be opposite of the ‘Revenue’ measure. While we want our revenue to be above our target, we want the ‘Total Amount of Time spent at gate’ to definitely be below the target.
  • By the following the same steps as before, I will double click on the column header of the ‘Actual’ series, and go to the ‘Evaluations tab.
  • I will select the status indicators that I want to use in the evaluation.
  • And then, in the first column, I will choose my inequalities. I will make Above Plan less than or equal to, Caution will less than or equal to, and Below Plan will be greater than. For No Information, I will select Any.
  • But, instead of comparing my ‘Actual’ series to the ‘Target’ Series, I am going to select Static Value for the second column
  • Now, in the third column, I can enter in a static value to compare against.
  • For Above Plan, I will enter in the value 50. So, my evaluation reads if my ‘Actual’ data is less than or equal to 50 minutes, than the ‘Above Plan’ status indicator will be selected.
  • For Caution, I will enter in 60.
  • For Below Plan, I will enter in 60 as well. So for any ‘Actual’ value higher than 60, the status will be Below Plan.
  • Let’s save out of the evaluation. And now the final step: Edit the measure, go to the ‘Measure Series’ tab, and switch the ‘Evaluation Option’ to the ‘Actual’ evaluation.
  • Once we save the edit window, we can see that the actual series is now evaluating against the static value I entered, not the target series!
  • Catherine: So a static evaluation is a good option for instances where know you are comparing to a constant that is not going to change – something like 100%, and this is especially handy if you have a really complex measure with a large number of series in place – you don’t have to add in an extra series just to display 100 every month. And in our case with Time At Gate, there are so many variables like the airport and the passengers, that we just want to keep it at a consistent 50-60 minutes.
  • Brendan: Just remember, when you’re comparing against a target series and your target changes, your evaluation adjusts with it. With our static evaluation, you’re comparing against 50 no matter what.

 

Multiple Series Evaluations in One Measure (17:28)

  • Catherine: Great, so we’ve now got a few ways to automatically show whether or not we’re on track.
  • I want to pop back over to our Revenue measure to show you one more helpful way to leverage the evaluations you have in place.
  • As you can see, we have our Actual and Target, and we’ve also got a YTD actual and target series here, calculating our sum for the year…
  • What happens to our status if we’re on track for the month, but not doing so hot for the year overall?
  • Well, currently nothing. But we can change that to incorporate the YTD actual series into our evaluation.
    • So let’s go back into the Edit Measure window > and over to the Series tab.
    • These little blue labels show us that both the Actual AND YTD series have automatic evaluations set.
    • And if we click to edit the YTD series > Evaluation tab, we can see that we’re comparing to the YTD Target rather than the target series, which makes sense given the numbers in each column.
    • In the Evaluation Options field, you’ll notice that the YTD Actual status is also an option for evaluating the measure overall.
      • So, you could decide that, rather than use the Actual evaluation, you want to track how you’re doing for the Year To Date.
      • But there’s also a couple of OTHER options that are valuable when you have more than one series evaluation set.
    • And that is the Best and Worst series options.
    • If i select Best Series from this list, then the measure is going to take into account the status displayed in both series, and select whichever is higher to display for the month.
      • So if we’re Below Plan for the month in June, but On Target for the Year To Date, then June is going to show as green for the measure.
    • And similarly, we can opt to choose the Worst Series instead – so in that same example, because the monthly value is performing worse than the year overall, the Worst Series would show red in June.
  • So you’ve got the option to show the glass half empty or the glass half full, but as a best practice we’d recommend on keeping this choice consistent where possible – that way you’ll spend less time wondering or double checking which measure is using which option.

Series Status and Series Evaluation Fields (19:45)

  • Now, in any case, you’re only showing one series evaluation as your measure status – so you might be wondering how you can easily keep track of and compare the others.
  • There are two really helpful fields that you can pull out onto your measure detail page to get a sense of how all these series evaluations are doing
    • Not to mention remind yourself of how they are evaluated!
    • And those are the Series Status and Series Evaluation fields
  • So, to add those to your layout, we’ll click into the edit menu,
    • We can search for ‘Series’ in the left-hand bar,
    • And drag these both out onto the screen. Let’s give them some real estate right above our data table
    • And click save!
  • So what you have in the Series Evaluation field is each series with an automatic evaluation, followed by the criteria that defines their status, and the current status highlighted.
  • And then in the Series Status field, you just have the simplified view of the status for each of those series that the evaluations produce.
  • Great! You might be wondering where else you can show all your series statuses at once… which is where heat map or matrix reports come into play.

Creating a Matrix Summary Report (21:10)

  • Brendan: Yup, you heard us right. You take the blue pill, you can keep using your current reports; or, you can take the red pill, and see how awesome your data can truly look.  
  • Automated evaluations not only help us save time on the measure detail page, but also allow us to create really cool summary reports.
  • A Matrix report is one where you can show multiple periods of data across measures, using status colors or icons as a quick look at how you’re doing overall.
  • For the sake of time, I have already created automatic evaluations for 5 different measures.
  • To create a matrix report, I will go to the ‘Manage Measure Reports’ page, and create a new report.
  • Let’s call it ‘Upward Airlines Matrix Report’.
  • Now, we want to display the last 6 periods of data in the summary report. We can do this by selecting ‘Last N Periods’ for the Period field, and then 6 for the ‘N Periods’ field.
  • Next, let’s click on the ‘Columns’ tab, and include the ‘Measure’ and ‘Series Status’ columns. The ‘Series Status’ column will show the data for all series in each measure. If we have automatic evaluations built, which we do, it will also show the status color or icon for each series.
  • Another important step to remember is that we want to show multiple periods of data. We need to click on the dropdown menu for the ‘Series Status’ column, and check the option to ‘Display multiple periods of data for this column’.
  • I will then go to the ‘Measures’ tab, and select the 5 measures that I want to include in this report.
  • Now, let’s save this report, and go into it.
  • Here you can see the 6 periods of data displaying, and the status color for each period. It is important to remember that the status color is only showing because we have created automatic evaluations for these 5 measures.

Gauge Charts (23:20)

  • Cool, I think this basically makes us Keanu Reaves-slash-Neo.
  • So what else can we do to show off how awesome and automated we are?
  • Well, there’s one chart type that allows us to see very clearly where we fall in the range of red, yellow, and green values
  • And that is our Gauge charts.
  • Let’s take a look. We’ve already got a chart in this measure, but we can always add another one by clicking to Edit the Measure and going to the Chart Tab.
  • And we’ll click to Add chart, calling this one our ‘Revenue Gauge’.
  • So, let’s change the chart type here to our 9th default chart, the Gauge chart.
  • And normally, you’d think to show an actual series and a target series on a chart, but we’re going to switch it up for this one,
    • So we’ll want to go over to the chart series tab.
    • And it’s going to prompt us to save our chart, so we’ll just click Save
  • Cool, look at that! Since our Actual Series has an automatic evaluation set, the gauge chart can pick up the cutoffs between red, yellow, and green, and displays those ranges of numbers on the chart accordingly.
  • We can see it doesn’t look quite right with those two tick marks. So let’s un-check our Target series.
  • And one more step, we’re going to want to go to the Chart Periods tab to make sure that we’re just displaying the current period, since there’s only one value being displayed.
    • Now, let’s click Save.
  • So for our last step here, let’s just add it to the measure detail page.
  • I’ll go into Edit Layout > and search for our gauge chart, and give it a good spot on the page.
  • And there you have it! Now if we toggle over to our Expenses measure, you can see a gauge chart here too.
    • To contrast this with the gauge in our revenue chart, the TOP of the chart is red, rather than the bottom, because it’s bad when our expenses get too high.
    • So this is a great visual indication of our automatic evaluations.

Automatically Evaluated Objectives (25:40)

  • Great! Now, we have been speaking a lot about measure evaluations, but did you know that objectives can be automatically evaluated as well?
  • Objectives can be evaluated based on the elements that they are linked too. Creating an automatic evaluation for an objective is a three part process.
    • First, we need to assign scores to each status indicator.
    • Second, we can assign a weight to each element that is linked to the objective.
    • Using these weights and scores, the objective is given a overall score based on a weighted average.
    • And, for the third part, we can choose our evaluation criteria for the objective!
  • To set one up, let’s first navigate to Admin Options, and then to Status Indicators.
  • By editing each status indicator, we can assign each one a score.
  • For Above Plan, I will give it a score of 3.
  • For Caution, I will give it a score of 2.
  • For Below Plan, I will give it a score of 1.
  • Catherine: As long as the scores make sense relative to each other, you can use whatever scale you see fit – some organizations use 1,2, and 3, others use 1,3, and 5, or even 25, 50, and 75.
  • Brendan: That’s a good point Catherine! Now I will save out of the Status Indicator window, and will navigate to the ‘Increase Shareholder Value’ objective which I want to create the automatic evaluation on.
  • Now to set up our weights, I am going to edit the objective, and navigate to the ‘Links’ tab.
  • When we get to the ‘Links’ tab, let’s first ‘Enable the Automatic Evaluation option’. Now we can see more information appear.
  • I can also see that my objective is linked to four different measures. The status of these four measures will determine the status of the objective.
  • I will give each linked measure a weight of 25. This means that each measure will have an equal share in determining the status of the objective. These weights should always add up to 100.
  • Catherine: And just because there are four linked measures, that doesn’t mean they all have to be equally weighted, or even included in the score. You could just use two of these and weight them 60-40, for example, if it made sense for your evaluation.
  • Brendan: In the next column over, we can see the score that I assigned each status in Admin Options. For the ‘Expenses’ measure that is ‘Above Plan’, we can see that a 3 is showing.
  • Looking at the bottom of the page, we can see the total weight we have given the measures, which adds up to 100, and the total score.
  • The total score is the weighted average score, given the scores you applied to your status indicators and the weights you have given to your linked elements.
    • And here you can also see how much each measure contributes to that score, given its weight.
  • For the third part of building automatic objective evaluations, let’s move to the ‘Evaluation’ tab to set evaluation criteria.
  • I am first going to select all the status indicators, and then in the first column, I can see the total score for my measures. In the third column, we are going to want to choose scores we can compare against our weighted score, which is 2.
  • So for ‘Above Plan’ status indicator, I will make the inequality greater or equal to 2.5.
  • For the ‘Caution’ status, I will make the inequality greater or equal to 1.5.
  • For the ‘Below Plan’ status, I will make the inequality greater than 0.
  • For the ‘No Information’ status, I will choose Any.
    • These evaluation criteria should make sense with the scores you’ve selected for your status indicators. To go back to our earlier example, if you’re using 25, 50, and 75 rather than 1,2,3, these evaluations wouldn’t make sense.
    • But with the way our scores are set up, we’ll land in the caution range when we have a mix of red, yellow, and green measures, whereas we’ll be ‘Above Plan’ if we’ve got three green measures and one yellow.
    • Catherine: Right, so it just depends on how you want your objective statuses to be evaluated – but just like with measures, it’s best to keep the criteria consistent across your account!
  • Brendan: Also, you may notice some indications here that your objective does or does not meet the criteria you specified, which are the ‘x’ and ‘check marks’ we see to the right.
  • While there may be more than one check mark, objective and measure evaluations work the same way – that is, from the top down.
  • So since the objective doesn’t meet the Above Plan criteria, and the first check mark is shown next to Caution, we can expect this objective to be yellow.
  • Once I have completed my evaluation criteria, I will ‘Save’ out of the edit window, and my objective is now automatically evaluated!

 

Wrap up and Questions (30:37)

Awesome, and it’s showing the Caution status as expected! So we’re reaching the end of our time here today, we’ve covered a lot of material and we want to make sure to leave time for your questions.

  • Can I use other elements besides measures to weight my objective?

 

Brendan: That is a great question! And yes, as long as they’re linked to the objective, you can use any element in your objective evaluation. So besides measures, you can use initiatives, action items, and risks. You can even use parent and child objectives. All of these different element types will follow the same process as measures did, when you build out your automatic evaluations.

  • You mentioned icons and colors in the matrix report. How would I switch between those displays?

 

Catherine: So, if we go back to our Matrix report, the choice to use color is actually being set by a default that administrators can determine in Admin Options. But if we want to just edit the display of this report, I can just click to Edit the Report, and expand Report Options. Then Subtable Indicators is where we’ll find this – we can swap in Color for the organization default if we always want it to display this way, None if we want all the rows to be uniform regardless of evaluations, and to display our status icons, I can just click Icon. Now when I save, I’ll see the up and down arrows and caution squares here in each cell.

 

That’s all we have time for today, thank you so much for joining us for this episode of the IDK webinar series, we hope to see you next time, and Happy Reporting!

New Call-to-action

EXPLORE MORE VIDEOS