By the time a supervisor sees an output table, the decisive errors have usually already happened — in the instrument, the sampling, or the coding scheme. Analysis is where problems become visible, not where they originate.
Measuring something adjacent to what you claim
A questionnaire item asks how satisfied respondents are with a service. The paper concludes something about service quality. These are different constructs, and the gap between them is not closed by a good regression.
Before collecting anything, write the sentence you hope to put in your conclusion, then ask whether the instrument can actually support it. If it cannot, change the instrument, not the sentence.
Running the analysis the software makes easy
SPSS makes a t-test easy. That is not a reason for your data to be analysed by one. Assumptions — independence, distribution, homogeneity of variance, adequate cell sizes — are conditions, not formalities, and the software will happily produce a p-value when none of them hold.
Choose the test from the question and the data structure, then check its assumptions explicitly and report the check. An examiner who sees the assumption test reported stops worrying about the rest.
Treating missing data as absent rather than informative
Deleting incomplete cases is a decision with consequences, not a cleanup step. If responses are missing because a question was sensitive, the people who skipped it differ systematically from those who did not, and listwise deletion has just biased your sample.
Report how much data was missing, where, and what you did about it. This is a paragraph, not an appendix, and reviewers look for it.
Coding qualitative data without an audit trail
Thematic analysis is not less rigorous than statistics; it is rigorous differently. The rigour lives in the trail: an initial codebook, documented revisions, examples of coded extracts, and — where the design calls for it — a second coder with reported agreement.
Themes that appear in the results without any account of how they were derived read as impressions. The same themes, with the derivation shown, read as findings.
Fishing, then reporting the catch
Twenty comparisons at p < .05 will produce roughly one significant result from pure noise. Running many and reporting the interesting ones is how a field fills with results that do not replicate.
Decide the primary analysis in advance. Exploratory work is legitimate and worth doing — but label it exploratory, correct for multiplicity where appropriate, and do not let a discovered pattern quietly become the hypothesis the study was designed to test.
Confusing statistical and practical significance
With a large enough sample, trivial differences become significant. A two-point difference on a hundred-point scale may be real and still mean nothing to anyone.
Report effect sizes and confidence intervals alongside p-values, and say plainly what the magnitude means in the context of your field. "Significant" answers whether an effect is likely present; only effect size answers whether anyone should care.
The habit that prevents most of this
Write the methodology chapter before you collect the data. Not the final version — the working version, in full sentences, describing exactly what you will do and why.
Almost every pitfall above becomes obvious while writing that chapter, and every one of them is cheaper to fix before the data exists than after.
We do this work every week — for MPhil and PhD candidates, faculty, and whole departments.
Start a conversation
