Psychometric analysis tests whether a questionnaire measures the concept it claims to measure. A large response count cannot repair ambiguous questions, unreliable scales or items that mean different things to different groups. Marketways examines the structure, consistency and fairness of the instrument before its scores guide decisions. The client can then distinguish a dependable measure from a questionnaire that merely produces numbers.
The decision this method supports
We use Survey & Psychometric Analysis to help clients answer: Does the questionnaire measure the intended concept consistently across the relevant groups?
How the method works
Psychometric analysis examines whether a questionnaire measures an intended concept consistently and fairly. The work considers reliability, factor structure, item behaviour and comparability between groups. High response volume cannot compensate for a measure that changes meaning across respondents.
A business example
A leadership assessment may produce one overall score. Psychometric analysis could show that the questions actually measure two different capabilities and that several items behave differently across job levels. The business can then improve the instrument before using it for consequential decisions.
How the client uses the result
Psychometric analysis makes a questionnaire safer to use for management decisions. It shows whether a score is consistent, whether the questions measure the intended idea and whether comparisons between groups are fair.
What we deliver
We produce estimates, comparisons, intervals, model results and sensitivity tests. A manager should be able to see the size of the result, the uncertainty around it, the assumptions required and which interpretation the evidence does not support.
Limits and complementary methods
A reliable scale measures a concept consistently; it does not prove that the concept causes a business outcome or that the questionnaire represents the population.
Selected methods and techniques
We select from these established methods according to the decision, evidence and operating conditions.
- Confirmatory factor analysis: Test whether a survey is actually measuring the underlying dimensions it was designed to measure, such as engagement, trust or satisfaction. Instead of letting the data decide the structure, start with a structure already expected and check whether the responses support it. Exploratory factor analysis asks what dimensions seem to exist; confirmatory factor analysis asks whether the dimensions already specified fit the data.
- CSAT measurement: Measure customer satisfaction from a stated question or set of questions for a defined transaction, interaction, product or relationship. Record the exact wording, response scale, timing, eligible population, sample, collection mode and scoring rule, including whether the result is a mean, top-box percentage or another calculation. State the valid-response denominator, treatment of missing answers, weights, response distribution and uncertainty so comparisons can be checked. CSAT describes satisfaction in the measured context; different questions or scoring rules are not automatically comparable, and the score is not a universal measure of experience, loyalty, retention or business performance.
- Customer survey design: Design a customer survey so its responses can answer a defined research question for a stated population and period. Translate the objective into clear constructs and questions; choose the sampling frame and method; specify scales, recall period, order, routing, language, collection mode and timing; pilot the instrument; and plan the scoring and analysis before fieldwork. Check coverage, non-response, accessibility, response burden, leading wording, common-method effects, missing data and any weighting needed. Good design reduces avoidable error but does not make a convenience sample representative or turn self-reported attitudes and intentions into observed behaviour.
- Employee survey design: Design the complete employee-survey process around a stated research question. Define the employee population and constructs, write and order questions, choose response scales and routing, plan invitations and sampling, specify anonymity and confidentiality protections, and set response monitoring, analysis and reporting rules before fieldwork. Survey design creates the instrument and collection plan. Questionnaire validation separately tests whether the questions and scores work as intended, while sampling determines which employees provide evidence about the target population.
- Exploratory factor analysis: Look for underlying dimensions that could explain why survey answers vary together, without fixing their structure in advance. The dimensions need interpretation and checking. Confirmatory factor analysis instead tests a structure specified before examining the results.
- Factor analysis / latent-variable modelling: Study how observed questions, tasks or ratings relate to underlying qualities that cannot be measured directly. Exploratory factor analysis looks for a possible dimension structure, confirmatory factor analysis tests a structure specified in advance, and broader latent-variable models can relate those dimensions to other measures. State the sample, estimator, scale, fit evidence, competing structures and uncertainty. A model that fits the response patterns does not by itself prove that the intended quality is valid, causal or suitable for an employment decision.
- Human-factors analysis: Examine how the design of tasks, interfaces, workload, environment and procedures affects human performance, error and wellbeing within a system. It identifies conditions that make errors more or less likely. It improves the fit between people and the system rather than attributing failures to individual carelessness.
- Inter-rater reliability testing: Have multiple assessors independently rate the same work, response or case under a common rubric, then measure their agreement or consistency with a statistic suited to the rating scale and design. Inspect where disagreement occurs and whether assessors used the same evidence and anchors. High agreement means the ratings are being applied consistently; assessors can still agree on an invalid interpretation, so reliability does not establish that the assessment measures the right capability.
- Longitudinal / repeated measurement: Collect comparable measurements at more than one time to understand change. A panel follows the same people or units, while repeated cross-sectional measurement samples the same defined population at each wave. State which design is used, keep the construct and scoring comparable, and account for repeated observations, attrition, changes in respondent mix and changes to the instrument or organisation. A pulse survey is one short recurring collection format; longitudinal or repeated measurement is the research design that makes time comparisons interpretable.
- NPS measurement: Measure stated recommendation likelihood using the Net Promoter Score convention when that question is relevant to the customer relationship. Ask the defined 0-to-10 recommendation question and specify the population, timing, mode, valid-response denominator and weights. Classify 9-10 as promoters, 7-8 as passives and 0-6 as detractors; calculate the percentage of promoters minus the percentage of detractors, keeping passives in the denominator; and report the full distribution, sample size and uncertainty. NPS is expressed in score points from -100 to +100. It measures stated recommendation, and differences in wording, sampling or context can limit comparison; it does not directly measure actual recommendation, retention or growth.
- Point-factor job evaluation: Evaluate jobs by scoring them against defined factors, such as knowledge, problem-solving, responsibility and working conditions, each with weighted levels, and sum the scores to rank job size. It produces a consistent, documented basis for comparing jobs. It evaluates the job's demands, not the incumbent's performance or market value.
- Psychometric reliability testing: Check whether a questionnaire or assessment gives sufficiently consistent measurements for its intended use. Depending on the question, check agreement between its items, stability over time or agreement between assessors. Consistency alone does not prove it measures the right thing.
Parent method family
Statistics & Econometrics explains how this method connects to adjacent methods and relevant services.
Related service families
These service families contain business questions supported by this method. Service pages link to the wider method family so readers can understand the complete analytical approach.