Sampling selects observations from a wider population so the business can learn without measuring everyone. A large response count does not correct a sample that excludes or overrepresents important groups. Marketways examines coverage, selection, non-response and the precision required for key subgroups. Clients know which population the result represents, how uncertain the estimate is and where evidence remains weak.
The decision this method supports
We use Sampling & Representation to help clients answer: Who or what must be represented, and which conclusions can the resulting sample support?
How the method works
Sampling selects people, events or records from a wider population. Representation concerns whether the selected observations support conclusions about that population. The method must consider coverage, selection, non-response and the precision required for important subgroups.
A business example
A citywide survey collected mainly through one digital channel may underrepresent older residents or people with limited access. Weighting may help only when the missing groups and relevant population information are known.
How the client uses the result
Good sampling allows leaders to draw a defensible conclusion about a wider population without measuring everyone. It also shows which groups remain uncertain, preventing a convenient response pool from being mistaken for the market or workforce. A larger sample does not remove selection bias when the wrong people had the opportunity or incentive to respond.
What we deliver
We produce structured responses, coded qualitative evidence, observation records or a secondary-evidence base. The result include the population, collection conditions and limits needed for responsible interpretation.
Limits and complementary methods
Greater precision and subgroup coverage require more time and cost. The design should match the decision rather than seek a needlessly large sample.
Selected methods and techniques
We select from these established methods according to the decision, evidence and operating conditions.
- Sampling / representation analysis: Assess whether development and evaluation data cover the people, cases, conditions and outcomes of the intended use, including small and intersecting groups. Examine selection, missingness, label availability, time and source mechanisms and quantify gaps where possible. Numerical balance alone does not establish representation if the sample excludes relevant situations.
- Sampling design: Plan which people or items to include in a study, how many to select and how their results will represent the wider population. If some groups are sampled more heavily, specify how results will be weighted to avoid distorting the overall picture.
- 360-degree feedback: Collect structured assessments of a person's observable workplace behaviour from several perspectives, commonly self, manager, peers and direct reports. Define the behaviours, ensure raters had a reasonable opportunity to observe them, keep source groups and disagreement visible, and apply confidentiality and minimum-group rules. The ratings are perceptions shaped by role and context; agreement is useful development evidence but is not proof of demonstrated competence or the cause of a performance result.
- Behaviourally anchored scorecards: Turn service or performance standards into a scoring procedure whose levels are tied to specific observable behaviours. Define when each behaviour has an opportunity to occur, distinguish failure from not observed or not applicable, train assessors and check whether they apply the anchors consistently. Behaviourally Anchored Rating Scales provide the reusable scale format; this method builds and applies a scorecard for a particular setting. Clear anchors improve interpretation but do not by themselves establish validity or reliability.
- Confirmatory factor analysis: Test whether a survey is actually measuring the underlying dimensions it was designed to measure, such as engagement, trust or satisfaction. Instead of letting the data decide the structure, start with a structure already expected and check whether the responses support it. Exploratory factor analysis asks what dimensions seem to exist; confirmatory factor analysis asks whether the dimensions already specified fit the data.
- CSAT measurement: Measure customer satisfaction from a stated question or set of questions for a defined transaction, interaction, product or relationship. Record the exact wording, response scale, timing, eligible population, sample, collection mode and scoring rule, including whether the result is a mean, top-box percentage or another calculation. State the valid-response denominator, treatment of missing answers, weights, response distribution and uncertainty so comparisons can be checked. CSAT describes satisfaction in the measured context; different questions or scoring rules are not automatically comparable, and the score is not a universal measure of experience, loyalty, retention or business performance.
- Customer Effort Score: Measure customers' perceived effort in completing a defined task or interaction using a specified question and response scale. State the task, timing, eligible population, sample, exact wording, scale anchors, direction and scoring rule because common Customer Effort Score instruments differ and a high number can mean either more or less effort. Report the denominator, missing responses, weights, distribution and uncertainty. The reusable CES convention is represented separately as a framework; this method is the measurement procedure, and Customer Effort Score is the populated result. Perceived effort is distinct from counted process steps, satisfaction, loyalty and the causal effect of making a process easier.
- Customer survey design: Design a customer survey so its responses can answer a defined research question for a stated population and period. Translate the objective into clear constructs and questions; choose the sampling frame and method; specify scales, recall period, order, routing, language, collection mode and timing; pilot the instrument; and plan the scoring and analysis before fieldwork. Check coverage, non-response, accessibility, response burden, leading wording, common-method effects, missing data and any weighting needed. Good design reduces avoidable error but does not make a convenience sample representative or turn self-reported attitudes and intentions into observed behaviour.
- Employee survey design: Design the complete employee-survey process around a stated research question. Define the employee population and constructs, write and order questions, choose response scales and routing, plan invitations and sampling, specify anonymity and confidentiality protections, and set response monitoring, analysis and reporting rules before fieldwork. Survey design creates the instrument and collection plan. Questionnaire validation separately tests whether the questions and scores work as intended, while sampling determines which employees provide evidence about the target population.
- Exploratory factor analysis: Look for underlying dimensions that could explain why survey answers vary together, without fixing their structure in advance. The dimensions need interpretation and checking. Confirmatory factor analysis instead tests a structure specified before examining the results.
- Factor analysis / latent-variable modelling: Study how observed questions, tasks or ratings relate to underlying qualities that cannot be measured directly. Exploratory factor analysis looks for a possible dimension structure, confirmatory factor analysis tests a structure specified in advance, and broader latent-variable models can relate those dimensions to other measures. State the sample, estimator, scale, fit evidence, competing structures and uncertainty. A model that fits the response patterns does not by itself prove that the intended quality is valid, causal or suitable for an employment decision.
- Inter-rater reliability testing: Have multiple assessors independently rate the same work, response or case under a common rubric, then measure their agreement or consistency with a statistic suited to the rating scale and design. Inspect where disagreement occurs and whether assessors used the same evidence and anchors. High agreement means the ratings are being applied consistently; assessors can still agree on an invalid interpretation, so reliability does not establish that the assessment measures the right capability.
Parent method family
Research & Evidence Collection explains how this method connects to adjacent methods and relevant services.
Related service families
These service families contain business questions supported by this method. Service pages link to the wider method family so readers can understand the complete analytical approach.