Outliers

Introduction: Outliers (Statistical Deviations)

Outliers are data points that significantly differ from the rest of a dataset. They may be unusually high or low values that lie far outside the expected range of variation. In statistics and Lean Six Sigma, identifying outliers is essential for understanding process stability, detecting measurement errors, and improving data quality.

Background

The concept of outliers dates back to early statistical analysis, with Sir Francis Galton and Karl Pearson developing methods to detect abnormal values. Outliers can arise from random variation, special causes, or data recording errors.
In Lean Six Sigma, detecting outliers helps distinguish between common cause variation (normal process fluctuation) and special cause variation (specific, correctable issues). Recognising this distinction supports effective problem-solving in the Analyse phase of DMAIC.

Key Elements / Features

• Definition: Data points that fall outside the typical pattern of a dataset.
• Causes: Measurement errors, process changes, environmental effects, or rare events.
• Detection Methods:
  • Statistical tests: Using z-scores or interquartile ranges (IQR).
  • Visual tools: Boxplots, scatter plots, or control charts.

Impact: Outliers can distort averages, standard deviations, and regression models if not properly handled.

Rule of Thumb (Normal Distribution):

Where:

• x = observed value
• xˉ = mean of the dataset
• s = standard deviation

A value with a z-score greater than 3 (or less than -3) is often considered an outlier.

Applications / Examples

• Quality Control: Detecting unusually high defect rates or extreme measurement readings.
• Finance: Identifying abnormal price movements or fraud indicators.
• Healthcare: Recognising patients with outlier recovery times or lab results.
• Manufacturing: Flagging production cycles with exceptional scrap or rework rates.

Example:
If the average part weight is 50 g with a standard deviation of 2 g, any measurement above 56 g or below 44 g may be treated as an outlier and investigated for root cause.

Relevance / Impact

Identifying and managing outliers is essential for maintaining data accuracy and process control. In Lean Six Sigma, understanding whether an outlier reflects random noise or a true process issue ensures that improvement efforts are focused and effective.
Removing invalid outliers improves the reliability of statistical analyses, while investigating valid ones often leads to breakthrough insights.

See also

• Control Chart
• Common Cause variation
• Special Cause variation
• Control Limits