Statistical testing of a hypothesis
A Significance test is used to reject or accept claims or hypotheses about a population. Statistical analysis starts with the collection of sample data through an experiment or measurment of a population. Claims or hypothesis are then made about the population. Finally tests are made to arrive at a conclusion.
Every test of significance is associated with a basic concept known as a hypothesis. The hypothesis is basically a statement about the population parameters. It can be classified into two categories: the null hypothesis and the alternative hypothesis. In the null hypothesis, it is assumed that the two groups differ by a population parameter such as mean, standard deviation and variance. In an alternative hypothesis, it is assumed that there is no chance factor involved in the difference between the parameters. Once the hypotheses have been established for the test, a suitable statistical test is carried out. Based on the conclusion of the test, we either reject the null hypothesis or accept it.
Any statistical test outputs a number, known as probability or p-value. This value lies between 0 and 1, with 0 implying that the two groups are not related at all and 1 implying that there exists perfect relation between the groups. In real life, the p-value always lies between 0 and 1. Normally, a confidence interval should be specified for a given study. 95% and 99% confidence intervals are commonly used standards for establishing significant difference. So, if the p-value is less than the 0.05, then, we conclude that there is high significant difference between the two groups. Otherwise, it is assumed that there is no significant difference.
The p-value can be one-tailed or two-tailed. A one-tailed p-value establishes the fact that a parameter is either larger or smaller than the value given by the null hypothesis. A two-tailed p-value signifies that the parameter is not equal to the value given by the null hypothesis. In the later one, direction is not important.
Consider a scenario where we tend to evaluate the performance of girls and boys in mathematics class. In this case, the null hypothesis will be that mean performance score of both the boys and the girls is the same. Whereas, the alternative hypothesis will state that the mean performance scores are not equal. Either the one way Anova test or the two sample t test can be used. Yet, another form of the null hypothesis is where we have to compare the mean score of either group with a specific value. So, the null hypothesis can be stated as: the mean performance score of girls is less than 70, out of the scale of 1-100.
Statistical significance testing is the backbone of any statistics theory. A significance test revolves around the concept of accepting or rejecting null hypothesis, which is further decided by observing the confidence interval. To decide which significance test should be used, the number of groups being compared must be known.
The links below are specific questions and answers about statistics and how to use them.