Hypothesis Testing and P-Values

A Hypothesis Test is a way of testing data collected for research against the status quo.

Let’s say you’ve done a research paper, based on a hypothesis. You’ve done your research and collected your data points, and now it’s time to test your hypothesis.

Understanding Hypotheses in Research

We usually formulate a Null and Alternative Hypothesis.

The Alternative Hypothesis (Ha or H1)

• This is your “what if” idea.
• It’s what you think might be happening.
• You collect data to see if this idea is correct.
• Scientists write it as “Ha” or “H1”.

The Null Hypothesis (H0)

• This is the “nothing special is happening” idea.
• It assumes your “what if” idea is wrong.
• Scientists write it as “H0” (H-zero).
• It’s the opposite of your Ha or H1.

Example:

Let’s assume you want to know whether there is a difference in the extremism score between men and women.

• Ha (or H1): There is a significant difference between the extremism score of men and women.
• H0: There is no significant difference between the extremism score of men and women.

Your research would then try to prove Ha is true (or show that H0 is false).

Remember: We usually try to disprove H0 rather than directly prove Ha.

One or two-tailed hypothesis

Put simply, a hypothesis test compares the distributions of your variables to check whether there is overlap or not. Distributions have what we call two tails.

© University of Lincoln

This image visualises what we mean by one and two-tail tests.

One Tailed

A one-tailed test is used when you hypothesise that what you are testing is either less than or greater than what is stated in the Null Hypothesis.

Re-rewriting the example above we might say the following:

• Ha: Women have a lower extremism score than men.
• H0: There is no significant difference between the extremism score of men and women.

A Right-Tailed Test is used if you hypothesise that what you are testing is greater than that of the null hypothesis. In this scenario, you are testing the top end of the distribution.

A Left-Tailed Test is used if you hypothesise that what you are testing is less than that of the null hypothesis. In this scenario, you are testing the lower end of the distribution.

In any instance, you should test against the significance level converted to a decimal. For example, if working at the 5% level, you should test against the value of 0.05 for a one-tailed test.

Two Tailed

A Two-Tailed test is used when you hypothesise that the null hypothesis is completely incorrect. In other words, in this situation, you wish to see if the status quo is untrue.

In this scenario, you have to test both extremities of the distribution you are working with. As a result, in the Two-Tailed Test, the significance level should be halved before being converted to a decimal. For example, at the 5% level, you should test against the value of 0.025 in a two-tailed test.

Don’t worry too much about the Significance level (sometimes also called Alpha). Most statistics programs will allow you to just select whether you want to do a one- or two-tailed test and the direction you want to test for at whatever level you set it at (e.g. 95%, 99%, 99.9%).

Interpreting and understanding Significant Levels and P-values

What is a Significance Level?

• A significance level is like a “cut-off” point.
• Commonly, we use 5% (or 0.05).
• It means we’re okay with being wrong 5% of the time.

What is a P-Value?

A P-value tells us how likely it is to see our results if the null hypothesis (H0) is true. It’s a number between 0 and 1.

How to Interpret P-Values:

1. P-Value Higher than 0.05 (Insignificant)
   • If the P-value is more than 0.05, it means there’s more than a 5% chance that our results could happen by random chance.
   • We say the results are “insignificant.”
   • We stick with the null hypothesis (H0), meaning no change to the status quo.
2. P-Value 0.05 or Lower (Significant)
   • If the P-value is 0.05 or less, it means there’s a 5% or lower chance that our results are just random.
   • We say the results are “significant.”
   • We reject the null hypothesis (H0) and accept our research hypothesis (Ha or H1), meaning there is a change to the status quo.

Example:

Imagine you’re testing if a new drug works better than the old one.

• Null Hypothesis (H0): The new drug does not work better.
• Research Hypothesis (Ha): The new drug works better.

If your P-value is 0.08 (higher than 0.05), you say, “There’s no strong evidence that the new drug is better.”

If your P-value is 0.03 (lower than 0.05), you say, “There’s strong evidence that the new drug is better.”

In summary, a P-value helps you decide whether your results are significant or not, based on the 5% significance level.