A **statistical hypothesis test** (also called as a **significance test**) is a procedure designed to answer the question of whether or not a null hypothesis provides a plausible explanation of the data.

The **null hypothesis** is typically the hypothesis that the data being gathered is just the result of random chance (one example being a model assuming no treatment effects). A significance test examines whether the **null hypothesis** provides a reasonable explanation of the data.

The **alternative hypothesis** is typically the research hypothesis that the data being gathered are influenced by some non-random effect (one example being a model assuming that a treatment will on average work better than a control).

A **one-sided alternative** is a research hypothesis that favors deviations of the parameter in only one direction away from the null value. For instance, if the null hypothesis is H0: p = 0.25, then an example of a one-sided alternative hypothesis would be Ha: p > 0.25 (another example would be Ha: p < 0.25).

A **two-sided alternative** is a research hypothesis that favors deviations of the parameter in both directions away from the null value. For example, if the null hypothesis is H0: p = 0.25, then the two-sided alternative hypothesis would be Ha: p ≠ 0.25.

A **population mean** is the numerical average of a variable in the entire population of interest. The null and alternative hypotheses in a significance test might be statements about a **population mean** or some other population value.

A **population proportion** is the proportion of times something occurs in the population of interest. The null and alternative hypotheses in a significance test might be statements about a **population proportion** or some other population value.

A **population percent** is the percent of times something occurs in the population of interest. The null and alternative hypotheses in a significance test might be statements about a **population percent** or some other population value.

A **sample estimate** is a statistic based on sample data being used to estimate a corresponding population value (parameter). For example, a sample proportion is a **sample estimate** of the true population proportion. In significance testing, the hypotheses are statements about population parameters and the **sample estimates** are thus used to form the basis of test statistics.

A **test statistic** is a measure of the difference between the data and what is expected when the null hypothesis is true.

The **p-value** represents the likelihood of getting a test statistic at least as extreme as the one we observe if, in fact, the null hypothesis is true (where “extreme” is in the direction specified by the alternative). A small **p-value** indicates that the null hypothesis is a poor explanation of the data. A large **p-value** indicates that the null hypothesis is a reasonable explanation of the data.

A **type 1 error** is the type of error you would commit if you decide that the null hypothesis is a poor explanation of the data even though it was really true.

A **type 2 error** is the type of error you would commit if you decide that the null hypothesis is a reasonable explanation of the data even though it is really false.

Data are **statistically significant** when a null hypothesis is rejected (i.e. when the null hypothesis appears to be a poor explanation of the data). It is important to know that **statistical significance** is not the same as practical significance. For example, with a large sample size even small deviations from the null hypothesis would be statistically significant. In the other direction, differences that might be important cannot be detected with a small sample size (and would not be **statistically significant**).