Interpreting a p Value Without Overclaiming

# What Your p Value Does and Does Not Let You Write in the Discussion Chapter

An explainer for final-year undergraduate and postgraduate students in Uganda writing their first independent study.

A note on sources before we begin. The eight sources supplied for this article are studies of overclaiming — a psychological construct about people claiming knowledge they do not have. None is a statistics text, a methods paper on null hypothesis significance testing, or a study of how students write about results. They cannot support any claim about p values, effect sizes, confidence intervals, power, causation, or generalisation. Where this article makes a statistical claim, it rests on standard methodological reasoning and carries no citation. Where it draws on the supplied sources, it cites them by key. Section 9 explains what to do about this in your own work.

1. Why "p < .05" Is a Starting Point, Not a Finding

A p value answers one narrow question. Almost every overclaim in a discussion chapter comes from treating it as though it answered a wider one.

What the test actually computes

A p value is the probability of obtaining data at least as extreme as the data you observed, assuming the null hypothesis is true and your statistical model is correctly specified. That is the whole definition. It is a statement about your data, conditional on a hypothesis. It says nothing about whether the hypothesis is true.

This distinction is easy to lose once the number is on the page. You ran a test, the software returned p = .032, and the number feels like a verdict. It is not. It is a probability computed under an assumption you have not yet shown to be false.

The three things students most often hear it say

Three misreadings account for most of the trouble.

Significance is not importance. A result can be statistically significant and practically trivial. Whether an effect matters depends on its size, its context, and the decisions anyone might make with it. The p value speaks to none of that.

Significance is not proof that the null hypothesis is false. Rejecting the null means your data would be unlikely under it. It does not mean the null is impossible, and it does not mean your alternative explanation is correct.

Significance is not the size of the effect. A p value and an effect size are different quantities. A very small effect can produce a small p value if the sample is large enough. A large effect can produce a large p value if the sample is small.

None of the sources supplied for this article covers significance testing. The claims in this section rest on standard method, not on a cited study. Do not cite S1–S8 for them.

2. The Four Overclaims a p Value Cannot Support

It helps to name the specific sentence types that get students into difficulty. Most people recognise their own draft once they see the pattern written out.

"The result proves that X causes Y"

A significant test on observational data does not establish causation. Causation is a property of the design and the argument you build around it, not of the p value. If you measured variables at one point in time, or if you did not manipulate the suspected cause, the test cannot tell you that one thing produced another. The design does that work, or it does not.

"The effect is large"

Significance and magnitude are separate questions. Your software may have produced a coefficient, a mean difference, or an odds ratio alongside the p value. That number is where magnitude lives. The p value tells you how compatible your data are with the null. It does not tell you how big anything is.

"This is true for the whole population"

The test speaks to your sample, under a model. Generalising beyond it is an argument you make about how the sample was drawn, what it represents, and what your theory predicts. It is not a number you report. A significant result in a convenience sample of 60 respondents in one district is a significant result in that sample. The step to a wider population is a separate claim that needs its own justification.

This matters in a Ugandan context in a specific way. If you collect data from market vendors in one division of Kampala, or from secondary school teachers in one district, your sample is defined by that setting. Writing that the finding "applies to Uganda" or "applies to East Africa" is a claim about a population you did not sample. How many districts, regions, or institutions you would need to sample to support that claim is not something this article can tell you. It depends on your design and your research question. Work it out with your supervisor.

"The null hypothesis is false"

Failing to reject the null is not the same as accepting it. Rejecting the null is not the same as proving it false. Both errors are common in student writing, and both are avoidable with careful phrasing.

No supplied source supports any claim in this section. These are statistical and design claims, and the source set does not address them.

3. What You *Can* Write: Four Defensible Sentence Patterns

The useful move is not to write less about your result but to write about it more precisely. These four patterns stay inside what the test licenses.

Reporting the test result

State the statistic, the degrees of freedom, the p value, and the direction of the difference or association. Nothing more. For example: "Respondents in the training group scored higher on the knowledge measure than those in the comparison group, t(58) = 2.31, p = .024." That sentence reports what you found. It does not interpret it, and it does not overreach.

Describing the association without asserting a mechanism

You can say that two variables moved together in your sample. You can describe the direction and the strength of that movement. What you cannot do from a single test is say which one produced the other, or that anything produced anything at all. "Higher scores on the attitude scale were associated with higher reported use of the method" is defensible. "Positive attitudes drive use of the method" is not, unless your design supports it.

Signalling uncertainty honestly

Acknowledge sampling variability and the limits of a single test. Phrases such as "in this sample", "the estimate is imprecise", and "this result should be read alongside the confidence interval" do real work. They tell the reader how much weight the finding can bear.

Connecting to the literature without overreaching

Say your result is consistent with or does not contradict a prior finding. Do not say it confirms one. Confirmation is a strong claim about the relationship between two studies, and a single significant test rarely supports it.

The evidence map for this article originally placed this section among those where the supplied sources could be used. On inspection that is not correct. These are statistical reporting patterns, and the supplied sources do not address statistical reporting. The only connection to the supplied literature is a passing analogy to overclaiming, which belongs in Section 6.

4. The Sample Size Trap in Student Projects

Small samples make both significant and non-significant results harder to read. The second case causes the most damage in discussion chapters.

Why a small sample makes both outcomes hard to read

With a small sample, your estimates are imprecise and your confidence intervals are wide. A wide interval is compatible with a range of true values, from negligible to substantial. The p value inherits this instability: it can move across the .05 line with small changes in the data.

Low statistical power means a real effect can fail to reach significance. This is the central point. A non-significant result in an underpowered study is not evidence that the effect is absent. It is evidence that your study could not detect it reliably if it were there.

What to write instead of "no effect was found"

Report the estimate and its interval rather than treating non-significance as a finding of zero. "The difference was not statistically significant in this sample, and the confidence interval ranged from a small negative difference to a moderate positive one" is honest. "There is no relationship between the two variables" is almost never what your test showed, and it is the sentence a supervisor is most likely to challenge.

One claim in this section is about the region rather than about statistics: the suggestion that Ugandan student projects commonly use small samples. No source supplied for this article supports that claim, and it would need a study of student research practice in Uganda or institutional data. This article therefore does not assert it. If you want to make that point in your own work, find a local source or write it as a general observation about student projects rather than a claim about Uganda specifically.

Practical constraints you may be working under

Many Ugandan students collect data within a single semester, with no budget for travel beyond their own district and no funding for research assistants. These constraints are real, and they shape what a study can detect. What this article cannot tell you is how common those constraints are, or what the typical sample size in a Ugandan undergraduate dissertation is. Those figures are not available in the sources supplied here, and you should not invent them. If your own project was constrained, say so in your limitations section in plain terms — the time available, the number of respondents you could reach, the districts you could travel to — rather than implying that the sample was larger or more representative than it was.

5. Effect Size and Confidence Intervals: The Sentences Your Supervisor Will Ask For

A p value should almost never appear alone in a discussion chapter. The effect size and the confidence interval are what let a reader judge whether your finding matters.

Where to put the interval in a sentence

The interval belongs in the same sentence as the estimate, so the reader sees the uncertainty without needing a separate paragraph. A pattern that works:

"The mean difference was 4.2 points (95% CI [0.6, 7.8]), indicating that the training group scored higher, though the interval includes small differences as well as moderate ones."

That single sentence reports the estimate, its uncertainty, and a restrained reading of what the range means. It does not need a follow-up paragraph explaining that the result might be small or might be moderate. The interval already says so.

When you do not have an effect size

If your software output did not include an effect size, say so plainly rather than implying one. "The analysis produced a test statistic and p value but no standardised effect size; the raw mean difference was X" is honest and useful. What you should not do is describe the effect as "strong", "weak", or "moderate" without a number to support the description.

This section is entirely statistical. No supplied source applies, and none should be cited here.

6. Overclaiming as a Writing Habit, Not Just a Statistical One

The supplied sources do not study p values. They do document a tendency worth knowing about, because it is the same tendency that produces inflated sentences in a discussion chapter.

What the overclaiming literature actually studies

Overclaiming is a documented tendency in how people report what they know [S1], [S3], [S6]. The overclaiming technique measures this tendency using self-reported knowledge, including fictitious items — items referring to things that do not exist. A respondent who claims familiarity with a non-existent topic is showing a willingness to claim knowledge they cannot have [S1], [S3], [S6]. The technique has been studied in relation to self-reported knowledge and to knowledge in children [S6].

Related work has linked overclaiming to numeracy and collective narcissism [S7], and to reasoning style and susceptibility to fake news and misleading claims [S8]. It has been studied in educational settings in connection with the big-fish-little-pond effect [S4], and using fictitious countries with availability-based explanations [S2]. Overclaimed performance is also a recognised problem in machine learning evaluation, where reported classification results have been shown to exceed what the underlying evidence supports [S5].

Where the analogy holds and where it breaks

The parallel to statistical writing is suggestive. A student who writes "the result proves that X causes Y" is claiming more than the test licenses, in much the same way that a respondent who claims familiarity with a fictitious topic is claiming more than they know.

But the analogy is not evidence. The supplied studies measure self-reported knowledge, not statistical writing. None of them studies discussion chapters, p value interpretation, or how students write about results. The link between the two is one the writer is drawing, not a finding being cited. This article labels it as an analogy for that reason. Do the same if you use it.

Related findings on confidence and inflated claims

The findings in [S7] and [S8] concern confidence, numeracy, and reasoning style in relation to overclaiming. The finding in [S5] concerns overclaimed performance in a technical field. None of these is evidence about how students interpret p values. They are useful context for the general point that people tend to claim more than their evidence supports, and they should be cited only for that general point.

7. A Self-Check You Can Run Before Submission

This checklist is practical advice, not a validated instrument. It has not been tested against student outcomes. Treat it as a procedure, not as evidence-based guidance.

The sentence-level check

Read each sentence in your discussion chapter that contains "significant", "proves", "shows that", or "confirms". For each one, ask what your test would have to do to make that sentence true. If the answer is "something it cannot do", rewrite the sentence.

The claim-to-evidence check

For each claim you make, point to the specific number, table, or figure that supports it. If you cannot point to one, either cut the claim or hedge it until you can. This check catches most overclaims before a supervisor does.

The supervisor check

Three questions are likely to come up about any significance result. What was the effect size? What was the confidence interval? And what does the design allow you to say about causation? If your draft already answers all three, you are in good shape. If it answers none of them, you have work to do.

No supplied source supports the claims in this section. The checklist is offered as advice.

8. Common Errors in Student Discussion Chapters

Three patterns recur in first independent studies. Each has a straightforward fix.

Causal verbs on correlational designs

"Influences", "leads to", "determines", and "causes" are usually the wrong verbs for a correlational design. The design does not support them, and the p value does not supply what the design lacks. Replace them with "is associated with", "covaries with", or "predicts" — and use even "predicts" carefully, since it implies a directional model.

Treating non-significance as a finding

"There is no relationship between the two variables" is almost never what your test showed. What it showed is that your data are compatible with the null, under your model, at your sample size. Report the estimate and the interval instead, and say plainly that the study was not powered to detect small effects.

Reporting p to four decimal places and nothing else

Precision in the p value is not precision in the claim. Reporting p = .0347 tells the reader very little on its own. Reporting the estimate, the interval, and a restrained interpretation tells them what they need to know.

This article does not claim that these errors are more common in Uganda than elsewhere. No source supplied here supports a frequency claim about local practice, and a study of student writing in Uganda would be needed to make one. The errors are described because they are common in student writing generally and because they are worth checking for in your own draft.

9. Where to Go Next

Two practical steps follow from everything above.

Find a methods source you can cite

The sources supplied for this article do not cover significance testing, effect sizes, confidence intervals, or power. You need a statistics text or a methods paper for any claim you make about p values. Do not cite S1–S8 for that purpose. Your supervisor or department will usually have a recommended text. Makerere University Library, for example, holds statistics and research methods collections, and your faculty may have a subject guide; the exact holdings change over time, so check the catalogue rather than relying on a description here. If you are unsure where to look, the guide on searching Google Scholar and library databases will help you find one.

Related guides

The following PassResearch guides cover the adjacent skills this article assumes:

• How to Choose a Sample Size for a Student Survey — for the power and precision issues raised in Section 4.
• How to Choose a Research Design for a Student Project — for the design questions that determine what you can claim about causation.
• How to Structure a University Dissertation, Chapter by Chapter — for where the discussion chapter sits in the wider document.
• Clearer Academic Writing Without Losing Rigour — for phrasing that stays precise without becoming unreadable.
• APA 7 Referencing Explained: In-Text Citations, Reference Lists, and Common Errors — for reporting statistics and citing methods sources correctly.

The sourcing caveat, repeated. The eight sources supplied for this article are studies of overclaiming. They support the claims in Section 6 and nothing else. Every statistical claim in Sections 1, 2, 4, 5, 7, and 8 rests on standard methodological reasoning and carries no citation, because no supplied source supports it. If your own work needs a citation for a claim about p values, effect sizes, confidence intervals, or power, find a statistics or methods source yourself. Do not cite S1–S8 for those claims.

Citation check

Every bracketed citation in the draft was checked against the supplied source list. All eight keys used — S1, S2, S3, S4, S5, S6, S7, S8 — correspond to supplied sources, and no citation refers to a key that was not supplied. No changes to the citations were required.

Two points to note about how the citations are used:

• All citations appear in Section 6 only. Sections 1, 2, 3, 4, 5, 7, 8 and 9 contain no bracketed citations, which is consistent with the article's stated position that the supplied sources do not address statistical reporting.
• The citations in Section 6 are attached to descriptions of what the overclaiming literature studies. They are not attached to any claim about p values, effect sizes, confidence intervals, power, causation, or generalisation.

Sources used

• [S1] Bensch, Doreen; Paulhus, Delroy L.; Stankov, Lazar; Ziegler, Matthias D. (2017). "Teasing Apart Overclaiming, Overconfidence, and Socially Desirable Responding." Assessment. doi:10.1177/1073191117700268
• [S2] Ross, Morgan Quinn; Sterling-Maisel, Olivia; Tracy, Olivia; Putnam, Adam L. (2020). "Overclaiming responsibility in fictitious countries: Unpacking the role of availability in support theory predictions of overclaiming." Memory & Cognition. doi:10.3758/s13421-020-01059-9
• [S3] Lee, Clara (2016). "Honesty-Humility and the Overclaiming Technique." Libraries and Cultural Resources, University of Calgary. doi:10.11575/prism/26379
• [S4] Jerrim, John; Parker, Philip David; Shure, Nikki (2023). "The big-fish-little-pond effect and overclaiming." International Journal of Educational Research. doi:10.1016/j.ijer.2023.102215
• [S5] Xu, Hanzi; Lou, Renze; Du, Jiangshu; Mahzoon, Vahid; et al. (2024). "LLMs' Classification Performance is Overclaimed." arXiv (Cornell University). doi:10.48550/arxiv.2406.16203
• [S6] Goecke, Benjamin; Schroeders, Ulrich; Zettler, Ingo; Schipolowski, Stefan; et al. (2022). "The Nomological Net of Knowledge, Self-Reported Knowledge, and Overclaiming in Children." Journal of Personality Assessment. doi:10.1080/00223891.2022.2144332
• [S7] Putnam, Adam L.; Yamashiro, Jeremy K.; Tekin, Eylul; Roediger, Henry Lederer (2024). "Collective overclaiming is related to collective narcissism and numeracy." Memory & Cognition. doi:10.3758/s13421-023-01504-5
• [S8] Salvi, Carola; Barr, Nathaniel; Dunsmoor, Joseph E.; Grafman, Jordan Henry (2022). "Insight problem solving ability predicts reduced susceptibility to fake news, bullshit, and overclaiming." Thinking & Reasoning. doi:10.1080/13546783.2022.2146191

Related reading

• How to Choose a Sample Size for a Student Survey
• How to Choose a Research Design for a Student Project
• How to Structure a University Dissertation, Chapter by Chapter
• APA 7 Referencing Explained: In-Text Citations, Reference Lists, and Common Errors
• How to Search Google Scholar and Library Databases
• Clearer Academic Writing Without Losing Rigour