Process Behavior Chart (Beta)

Labs Feature (Early Access)
This is a lab feature (read more about our lab). After incorporating early adopter feedback we will make this chart available in our standard ActionableAgile Analytics offering.

Process behavior charts are for characterizing a process as predictable or unpredictable. It does this by identifying:

points that represent exceptional variation (aka signals),
the amount of routine variation (aka noise) to expect from a predictable process in the future.

There are 2 sub-charts within this Process Behavior Chart, both reflecting information about your Cycle Time data:

Individuals Chart - a running record of the Cycle Times of individual work items
Moving Range Chart - a running record of the variation in Cycle Times

Here’s what we cover in this document

The Individuals Chart (Finding Process Signals)

Here you can see the Cycle Times of individual items plotted over time

The horizontal axis represents a cumulative number of items ordered by the date they are finished. Items finished on the same day listed consecutively, ordered by item ID, alphanumerically. The vertical axis represents the Cycle Time of the items.

Process Signals Chart Control

The Signal Detectors controls allow you to separate signal from noise and find patterns within your noise to get early indicators that process changes are afoot (for better or worse). These detectors are listed in order of signal strength and should be used in that order.

Process Signal (in decreasing strength)	What it shows you	How it is defined
Large change	Highlights individual items that are out-of-control. These out-of-control items are the only true signals in your data. The rest is noise.	Any single point outside of the natural process limits.
Moderate change	Highlights patterns inside the noise of your data that indicate moderate process changes.	A run of 3 items where 2 out of any 3 consecutive points within the process limits are above the 2-sigma line.
Moderate, sustained shift	Highlights patterns inside the noise of your data that indicate sustained, moderate process changes.	A run of 5 items where 4 out of any 5 consecutive points within the process limits are above the 1-sigma line.
Small, sustained shifts	Highlights patterns inside the noise of your data that indicate small, sustained shifts in your process.	A run of at least 8 successive values within the process limits on the same side of the average line (above or below)

In practice, most people who use process behavior charts effectively find that they have plenty of signals to tackle. However, if greater sensitivity is required, you can use the other signal detectors.

The Moving Range Chart (Validating your data)

The moving range chart is also sometimes called the mR chart or the XmR chart.

While the Cycle Time Run chart shows the actual cycle time values for the work items, the moving range chart shows the differences between successive Cycle Time values. In other words, it is a running record of the Cycle Time variation generated from your process.

The XmR chart is your first, best, and probably last chart for characterizing the variation in your system. Nowhere else are you going to find such a clear, concise visualization of the probable noise and possible signal contained within your data. – Dan Vacanti

The vertical axis represents the variation between the two Cycle Times being compared.

The horizontal axis represents a cumulative number of comparisons made. There will always be one less point on the Moving Range chart than there is on the Cycle Time Run Chart. That’s because it takes two Cycle Time Run Chart points to plot one Moving Range chart point.

Chart Controls

Summary Statistics

Process Signals

Basis Lines

These are lines that are used to analyze your data in the charts. The first 4 are in the Individuals Chart (top) and the last 2 are in the Moving Range chart (bottom).

The Process Signal detectors in the Process Signal chart control use these lines to find signals. When you read the description of each signal you can see that the calculations correlate to a specific basis line.

Note: the sigma lines have nothing to do with standard deviation. They just unfortunately use the same nomenclature

Basis Line	How they are calculated
Natural Process Limits (aka 3 Sigma lines) INDIVIDUALS CHART	The upper and lower boundaries of the natural variation in your process. These lines are also known as 3 Sigma. Limits = Average ± 2.66 Average Moving Range*
2 Sigma INDIVIDUALS CHART	The 2-sigma limits are two-thirds of the Natural Process Limits (aka 3-sigma) 2-sigma = Average ± (2.66 * 2/3) * Average Moving Range
1 Sigma INDIVIDUALS CHART	The 1-sigma limits are one-third of the Natural Process Limits (aka 3-sigma) . 1-sigma = Average ± (2.66 * 1/3) * Average Moving Range
Average Cycle Time INDIVIDUALS CHART	The arithmetic mean of the dots represented in the selected data on the Individuals chart. In other words, the average of the cycle time values.
Average mR MOVING RANGE CHART	The arithmetic mean of the dots represented in the selected data on the mR chart. In other words, the average of the difference between successive cycle time value.
URL MOVING RANGE CHART	The upper limit of natural variation of the variation itself in your process. Upper Range Limit = 3.27 Average Moving Range*

Layout

Item Filter

You can filter down the dots shown on this chart by choosing one or more available filters.

If you want to clear your filters so that all dots show up again, you click the Reset button.

Workflow Stages

FAQ

How do I tell if my process is stable?

A process with data that falls completely within the Natural Process Limits is stable and predictable.

How do I know if my data is good enough to use for forecasting?

If your process is largely stable then your data should be reasonably reliable for forecasting. You can tell if your system is stable by seeing how often items fall outside of the Natural Process Limits. You can choose the Large Change option from the Process Signals chart control to identify items that are “out of control”.

Which data is signal and which is noise?

Items that are above the Natural Process Limits are signals. The rest of the data points are noise. To identify your signals, use the Process Signals Chart control and select the first option. This will highlight every item outside of your natural process limits.

In order to use this chart effectively, investigate each out-of-limits point in search of a cause. This will allow you to address them and bring your system back into control.

Why is it important to separate signal from noise?

When we don’t have a clear differentiation between signal and noise we can overreact to data points and cause more problems than we fix. When we can correctly identify true signals, we can take timely and appropriate actions. In cases where the data signals improvement, you can endeavor to sustain it. In cases where the data signals a turn for the worse, you know to look for causes so you can work to improve the situation.

How can I get earlier signals that we are becoming less stable, less predictable?

Please note that, in practice, most users of this chart will find plenty of signals using just the Large Change Process Signal detector

However, you can use the more sensitive signal detection rules to find early indicators within your data’s noise. These rules should be investigated in order as you investigate smaller and smaller shifts in your process.

Moderate change: when 2 out of any 3 consecutive points within the process limits are above the 2-sigma line.
Moderate, sustained change: when 4 out of any 5 consecutive points within the process limits are above the 1-sigma line.
Small, sustained shifts: This signal is the weakest and is present when you see at least 8 successive values within the process limits on the same side of the average line (above or below)

If there are no items “out of control” does that mean everything is great?

First, congratulations, you have a predictable system! Now, you might have a system that is technically predictable but with too wide of a range of Cycle Time data. In other words, you could be predictably unpredictable.

How can you tell if this is the case? Look at the distance between the Natural Process Limits lines on the chart. (If no lower line is shown then use 0 as the lower limit.) If this is a problem, work on reducing work item age and see if signals start appearing. If they do, work to mitigate the causes as you continue to reduce your Cycle Time range through Work Item Age management

How trustworthy are the process signal controls?

The goal of these detection rules is to eliminate over 99% of noise. According to Dr. Wheeler, the use of three-sigma limits strikes a balance between the economic consequences of the dual mistakes of missing signals and getting false alarms. Using two-sigma limits on its own has a false alarm risk more than 4 times larger than with our signal detectors.

So, if you’re seeing a signal for any of these rules then the cause is not likely due to chance. The cause of these signals can reasonably be interpreted as an assignable cause.

Where to learn more

Daniel Vacanti - Actionable Agile Metrics Volume 2 https://leanpub.com/actionableagilemetricsii

Donald J Wheeler - Contra Two Sigma: The consequences of using the wrong limits, Quality Digest Daily, May 1, 2013 http://www.spcpress.com/pdf/DJW255.pdf