Anomaly detection identifies cases that differ materially from an established pattern. This can reveal early signs of failure, fraud, error or operational change in the large event streams businesses now generate. A sensitive model can also flood investigators with false alarms. Marketways calibrates the reference pattern, alert threshold and response workflow together so scarce attention reaches unusual cases that merit action.
The decision this method supports
We use Anomaly & Novelty Detection to help clients answer: Which observations require investigation because they are unusual for the relevant context?
How the method works
Anomaly detection identifies observations that differ from an expected pattern. Novelty detection focuses on new patterns not represented in the original data. An unusual record is a signal for investigation rather than proof of fraud, failure or error.
A business example
A payment system may flag a transaction because its timing, location and amount differ from the customer's history. The workflow must decide whether to block, request verification or send the case for review, taking account of the cost of false alarms.
How the client uses the result
Anomaly detection concentrates limited investigative attention on behaviour that differs from the expected pattern. It can help a business find emerging failure, fraud, error or operational change earlier than routine monitoring.
What we deliver
We produce predictions, scores, groups, alerts or extracted information together with validation evidence. A manager needs operating thresholds, error consequences, escalation rules and a plan for monitoring change after deployment.
Limits and complementary methods
Sensitive detection finds more unusual cases but also creates more false alarms. Thresholds should reflect investigation capacity and the cost of missing a real event.
Selected methods and techniques
We select from these established methods according to the decision, evidence and operating conditions.
- Anomaly detection: Identify observations or patterns that depart materially from a defined reference using statistical, rule-based or machine-learning techniques. State the reference population, features, threshold and expected error trade-offs, then route flags for interpretation. This is the general method family; unusualness alone does not establish harm, fraud, error or cause.
- Change-point detection: Estimate when the statistical behaviour of an ordered series changes, such as its level, variability, trend or relationship with other variables. Specify the reference period, minimum persistence and uncertainty around the change time. A detected change identifies a structural break for investigation; it does not explain the cause.
- Concept-drift detection: Check whether the relationship between a model's inputs and the outcome it predicts has changed. This differs from data drift, where the inputs themselves change in frequency or pattern.
- Data-drift detection: Check whether the data a model receives have changed compared with the data used as its reference. Data drift concerns the inputs; concept drift concerns changes in how those inputs relate to outcomes.
- Event detection: Identify occurrences of defined events in data streams by applying explicit event rules, extraction models or patterns. State what constitutes the event, its time window, source and confidence, and distinguish multiple reports of one occurrence from separate events. Detection establishes that the defined pattern was observed, not its cause or consequence.
- Exception analysis: Identify and examine cases that breach a business rule, control, tolerance or expected process path. Record the governing rule, evidence, materiality, owner and disposition, and distinguish authorised exceptions from errors. An exception is defined by a rule or requirement; it is not the same as a statistical anomaly.
- Leading-indicator analysis: Evaluate whether a measure changes before a defined outcome, consistently enough and with sufficient lead time to support action. Test timing, stability, false alarms, missed events and whether the indicator adds information beyond current conditions. Precedence is necessary for warning but does not by itself establish cause.
- Multivariate anomaly detection: Model the joint reference pattern across several variables and flag combinations that are unusually distant, sparse or unlikely even when each value appears ordinary by itself. Scaling, dependence and the reference population materially affect the result. It differs from univariate outlier detection and does not explain why the unusual combination occurred.
- Outlier detection: Identify individual observations that are unusually extreme, distant or sparse relative to a stated reference, using a statistical, distance or density rule. Scaling, population choice and rare but valid cases affect the result. Outlier detection focuses on individual cases; broader anomaly detection can also examine sequences, relationships and process changes. A flag is not proof of error or harm.
- Population stability analysis: Compare distributions of a current population or its measured attributes with a versioned reference using interpretable measures, bins and uncertainty. Identify which groups or features shifted and whether seasonality or data handling explains the change. Population shift is data drift; separate outcome evidence is needed to establish performance or concept drift.
- Regime / change-point analysis: Locate points where a series or relationship appears to change in level, trend, variability or dependence, then characterise the operating periods or regimes between those points. Test whether the evidence for a break is strong enough and whether models or controls should differ by regime. A detected change point does not by itself explain the cause, prove permanence or predict the next regime.
- Residual analysis: Examine residuals, the differences between observed values and model predictions, for patterns across time, fitted values, inputs, locations or groups. Structure, dependence, changing spread or extreme residuals can reveal missing relationships, unstable variance, leakage or unusual cases that average error hides. A pattern diagnoses model inadequacy to investigate; it does not identify its cause by itself.
Parent method family
Machine Learning & Predictive Analytics explains how this method connects to adjacent methods and relevant services.
Related service families
These service families contain business questions supported by this method. Service pages link to the wider method family so readers can understand the complete analytical approach.