What Is Hypothesis Testing in Statistics? Types and Examples

TL;DR: Hypothesis testing starts with a population claim and asks whether sample data look unusual under H0. Write H1, choose alpha and the test, then calculate the statistic and p-value. Reject or fail to reject H0 according to the threshold. Statistical significance does not establish certainty or practical importance.

Average order value rises after a website redesign. The team would like to credit the new page, but one unusually strong sample could show the same pattern. That gap between what the data show and what the team can safely claim is where hypothesis testing becomes useful.

Start with the less exciting possibility: “the average has not changed.” A statistical test works out how surprising the sample would be under that claim. An unusual result may justify rejecting it. Study design and sample size still matter, as do the assumptions behind the selected test.

What Is Hypothesis Testing in Statistics?

Hypothesis testing compares sample evidence with a reference claim about the population. That reference is the null hypothesis. The alternative hypothesis describes the competing possibility that the data may support.

Take a delivery company that reports an average delivery time of 30 minutes. The two-sided question can be written as:

• H0: population mean delivery time, mu = 30
• H1: population mean delivery time, mu != 30

The test ends with one of two statements. Either the evidence is strong enough to reject H0, or it is not. In the second case, the correct wording is “fail to reject H0.” That wording matters because the analysis has not proved H0 true.

Null and Alternative Hypotheses

The statistical question has to land on something measurable in the population. Depending on the study, the chosen parameter may be a mean, a proportion, a variance, an association, or a difference between groups. The test must be able to evaluate that parameter.

Your Data Analytics Career is Around The Corner!Data Analyst CourseExplore Program

Types of Hypothesis Testing

“Types of hypothesis testing” covers two separate choices. One concerns direction and places the rejection region in one tail or both tails; the other concerns the procedure, which has to match the research question and the form of the data.

One-Tailed vs. Two-Tailed Hypothesis Testing

Direction should be stated in the research question before anyone examines the data. A one-tailed test follows a stated direction; a two-tailed test leaves room for a difference on either side.

All of alpha sits in one tail for a one-tailed test, increasing power in the chosen direction. The trade-off is that the opposite direction is outside the test. Pick the tail after seeing the result, and the stated error rate can no longer be trusted.

Common Statistical Tests

ANOVA answers an initial group question: do all the means look compatible with the same population value? A significant finding says at least one mean differs. A post hoc comparison usually identifies the difference, which makes ANOVA the initial test in a longer analysis.

Z-Test vs. T-Test in Hypothesis Testing

Look first at the standard deviation available for the population. A known sigma supports a one-sample z-test against the reference mean. In most real samples, only s is available. The one-sample t-test uses that sample estimate, and its t distribution accounts for the uncertainty introduced by estimating the spread.

A large sample size does not automatically make the analysis a z-test. With an unknown sigma, a t-test is normally the better fit. Its distribution approaches normality as the sample size grows, which is why the two results often look very similar at large sample sizes. The full z-test vs. t-test comparison covers their assumptions and variants.

Become an AI-Powered Data Analytics ExpertWith Our PCP in Data AnalyticsExplore Program

Hypothesis Testing Formula

Different questions call for different formulas. For tests built around a mean, the z and t calculations share the same basic shape:

test statistic = (sample estimate - value stated by H0) / standard error

Two common versions are:

• One-sample z-test: z = (x_bar - mu0) / (sigma / sqrt(n))
• One-sample t-test: t = (x_bar - mu0) / (s / sqrt(n))

The sample size appears as n. The observed mean is x_bar, while mu0 records the mean claimed by H0. Spread comes from sigma when the population standard deviation is known, and from s when it must be estimated from the sample.

Not every departure from H0 concerns a mean. A chi-square test works from observed and expected counts through chi-square = sum((observed - expected)^2 / expected). ANOVA instead builds its F statistic from the contrast between within-group and between-group variation.

The calculated statistic is not the decision by itself. Its location in the reference distribution supplies a p-value or critical value, after which one of these equivalent rules can be used:

• P-value method: the condition p <= alpha calls for rejecting H0
• Critical-value method: rejection of H0 follows when the test statistic enters the rejection region
• Confidence-interval method: in a two-sided test, reject H0 when the matching interval excludes the hypothesized value

Under identical assumptions and the same alpha level, these are three routes to a single statistical decision.

Strengthen your data analytics portfolio through hands-on projects involving customer churn, marketing performance, sales intelligence, forecasting, and AI-driven workflow automation. Explore Simplilearn’s Professional Certificate in Data Analytics and Generative AI.

Steps of Hypothesis Testing

1. Frame the Question and State the Hypotheses

A usable research question identifies the population parameter first. From there, write H0 and H1 and settle whether the alternative points in one direction or allows either.

2. Set the Significance Level

Alpha is a commitment made before the result is visible. Although 0.05 is common, a false positive with serious consequences may call for a stricter cutoff.

3. Choose the Test and Check Its Assumptions

The data and study design narrow the choice of test. Check the outcome type, number of groups, and sampling structure, then examine the relevant assumptions: independence, distribution shape, equal variance, or adequate expected counts.

4. Collect and Prepare the Sample

The sampling or experimental plan governs what happens next. Rules for missing values, outliers, and exclusions should already be in place. Writing them after the result appears gives the outcome too much influence over data handling.

5. Calculate the Test Statistic and P-Value

Whether the work is done by formula or statistical software, keep the full output needed for review. That record includes the statistic, the p-value, a confidence interval, and, when relevant, degrees of freedom.

6. Make the Statistical Decision

Crossing the prespecified threshold calls for rejecting H0. On the other side, the decision is to fail to reject it. Both outcomes carry error risk; neither amounts to proof.

7. Explain the Result in Context

Statistical output must answer the question that initiated the analysis. Explain the observed effect’s direction and size, then give readers the uncertainty, assumptions, and practical relevance needed to interpret it.

Hypothesis Testing Examples

Example 1: One-Sample T-Test

Ten scores from one class have a mean of 78.6 and a sample standard deviation of 4.06. The school wants to know whether that class result is compatible with a population mean of 75.

• H0: population mean score, mu = 75
• H1: population mean score, mu != 75
• alpha = 0.05, selected in advance
• n = 10

No population standard deviation is available here. That makes a two-sided one-sample t-test the appropriate choice.

t = (78.6 - 75) / (4.06 / sqrt(10)) = 2.80

With 9 degrees of freedom, the p-value is about 0.0206. Because 0.0206 falls below 0.05, the school rejects H0. The sample mean sits 3.6 points above 75. The 95% confidence interval places the population mean at approximately 75.7 to 81.5. Together, those results support a difference from the reference mean.

Calling the class “better” would go beyond this calculation. It only compares the mean with 75. The test cannot identify a cause or decide whether a 3.6-point difference matters educationally.

Example 2: Chi-Square Test

For an online retailer, device type and checkout completion form a table of observed session counts. A chi-square calculation asks how far those counts sit from what independence would predict. Independence is H0 in this setup, while H1 represents an association between the categorical variables.

A p-value below alpha gives the retailer grounds to reject H0. The finding supports an association, not a claim that device type caused the difference. Cramer’s V may then be used to describe the strength of that association.

P-Value and Significance Level

Alpha answers a procedural question, not the probability that H0 is true. With alpha at 0.05, a valid procedure repeated while H0 holds would mistakenly reject it in about 5% of those runs. That long-run false-positive limit is why alpha belongs in the plan, before the analysis begins.

Now consider the observed result under H0 and the chosen statistical model. The p-value reports how often a result at least this extreme would occur under those conditions. A small value makes the sample difficult to reconcile with H0; it says nothing about how large or important the effect is.

• Reject H0 when p <= alpha.
• Continue with H0 unrejected when p > alpha; the formal decision is to fail to reject it.

The cutoff separates 0.049 from 0.051 even though they carry nearly the same evidence against H0. One receives a significant label, and the other does not. Neither value makes much sense on its own, so interpretation should also consider effect size, confidence interval, study quality, and prior evidence.

Move From Analyzing Data to Building AI SystemsWith our PCP in Agentic AI & Multi-Agent SystemsExplore Program

Type I and Type II Errors

Sampling uncertainty leaves room for two different wrong decisions. One rejects a true null hypothesis; the other leaves a false null hypothesis unrejected.

Alpha tracks the false-positive risk; beta tracks the false-negative risk. Power is written as 1 - beta. Although a larger sample often raises power by reducing standard error, effect size and study design still shape the result.

Hypothesis Testing in Python

Once the test choice is settled, SciPy can handle the reference distribution and p-value. Here is the school-score calculation in Python:

from scipy import stats

scores = [72, 75, 76, 78, 79, 80, 81, 84, 85, 76]

result = stats.ttest_1samp(scores, popmean=75, alternative="two-sided")

print(result.statistic)  # 2.8035

print(result.pvalue)     # 0.0206

The printed p-value is below 0.05, consistent with the manual decision. Current SciPy versions also return the test’s 9 degrees of freedom.

Hypothesis Testing in R

R can run the identical score test through its base stats package:

scores <- c(72, 75, 76, 78, 79, 80, 81, 84, 85, 76)

result <- t.test(scores, mu = 75, alternative = "two.sided")

result$statistic  # t = 2.8035

result$p.value    # 0.0206

An online hypothesis-testing calculator still requires a sound setup. First establish whether the input consists of raw observations or summary statistics, since the wrong format yields the wrong calculation. Tail direction, test choice, and assumptions must also match the problem before the resulting p-value means anything.

Hypothesis Testing in Research and Machine Learning

Research methodology sets the boundaries before a statistical test is run. Researchers define the research question and variables, decide how the sample will be obtained, and document an analysis plan that includes the hypotheses. These early choices reduce the freedom to keep searching for significant later results.

A recommendation team may replace a machine learning model and then watch click-through rate. An A/B test can examine whether the new model improved that outcome. If both models are evaluated on the same folds or observations, the measurements are paired, and the analysis must reflect that dependence. Several metrics or model variants may also require multiple-comparison controls. Model monitoring raises similar questions.

The same reasoning appears in routine operational work:

• Campaigns, pricing, and product changes provide business teams with testable questions, often measured by conversion rates.
• Treatment comparisons in healthcare sit alongside ongoing checks for safety problems.
• Claims about policy, demand, or market behavior can be examined against economic data.
• After a quality change, manufacturers can compare defect rates or process means.

Statistical reasoning is a core part of modern AI. From testing hypotheses to evaluating AI outputs, strong analytical thinking remains valuable as AI systems become more autonomous. Simplilearn’s Agentic AI Professional Certificate Course helps you build on that foundation by designing intelligent agents, RAG systems, and multi-agent workflows.

Common Mistakes to Avoid

• With p > alpha, the null hypothesis is not rejected; this outcome does not prove H0 true.
• Decide on a one-tailed direction in the analysis plan, not after the sample reveals its movement.
• Dependence between observations can invalidate a test whose assumptions require independence.
• Effect size and confidence interval are needed alongside statistical significance.
• Every additional outcome creates another false-positive opportunity. Reporting only significant results conceals how many were tested.
• Convenience sampling does not guarantee that a sample resembles its population.

Key Takeaways

• Hypothesis testing checks a population claim against evidence found in a sample.
• A prespecified rule determines whether H0 is rejected or left unrejected; certainty is not one of the outcomes.
• The test choice follows from the research question, data type, and the design and its assumptions.
• Under a statistical model, the p-value describes compatibility with H0 rather than practical importance.
• Effect size and confidence interval give the final decision the context it needs.

This step-by-step Data Analyst roadmap is designed for professionals seeking to understand the full scope of the profession. Explore the skills, tools, salary potential, and career roadmap needed to build a successful career as a Data Analyst.

Conclusion

The calculation is often the easiest part of hypothesis testing. The harder work comes before it: framing the right question, checking whether the sample fits it, choosing an appropriate test, setting the alpha level, and deciding between a one-tailed and two-tailed test. When reporting the result, include the effect size and confidence interval. A p-value can guide the decision, but it should not carry the entire interpretation.

To practice statistical analysis with real datasets, explore Simplilearn’s Professional Certificate in Data Analytics and Generative AI. The program covers Python-based statistical analysis, regression, correlation, multivariate analysis, predictive modeling, and hands-on projects based on business problems.

FAQs

1. What is a hypothesis in simple words?

Put simply, a hypothesis states what you expect the data to show. Statistical work requires sufficient precision to test that expectation.

2. What is another name for hypothesis testing?

Significance testing and test of statistical significance are other names for it. Null hypothesis significance testing, often abbreviated as NHST, is the familiar framework that uses H0 and a test statistic with a p-value.

3. What are the seven types of hypotheses?

No universal classification contains exactly seven. A common research methodology version lists simple, complex, null, alternative, directional, non-directional, and associative hypotheses, but textbooks may organize or label these categories differently.

4. What sample size is needed for hypothesis testing?

There is no fixed sample size that works for every test. Calculate it before collecting data through a power analysis that considers the expected effect, alpha level, desired power, variability, and study design.

5. What is the difference between statistical significance and practical significance?

Statistical significance indicates whether a result is difficult to explain under H0. Practical significance asks whether the observed effect is large enough to matter, which is why the p-value should be reported with an effect size and confidence interval.