Posted on by

Statistics and the “Scientific Method”

The “Scientific Method”

Most secondary school students are taught about the “Scientific Method” during their general science classes. From these early classes, students are encouraged to believe that all scientists follow an objective and almost linear process in their research. The process is generally described as something like this:

  1. an unusual observation is made
  2. an explanation is proposed for this observation
  3. experiments are performed to test the explanation
  4. after repeated experiments a theory is formed
  5. experiments are performed to validate the theory
  6. once validated the theory becomes a new scientific law
  7. the scientific laws are used to predict new observations

Of course, an unusual observation can be made at any time, particularly when conducting experiments. Equally, some explanations or theories can arise without observation. So the “Scientific Method” is really an idealised description of how scientific laws are determined. The “Scientific Method” does, however, provide insight into how behave scientifically, namely:

  1. use existing knowledge or theories to make predictions about what will happen before you make an observation
  2. a new or unusual observation is made when that observation doesn’t fit the current knowledge or theories
  3. propose an explanation for this new or unusual observation (this explanation could simply be “we observed it wrong”)
  4. test the explanation through controlled experimentation
  5. do not make firm conclusions until they have be verified through replication (preferably by independent researchers)
  6. once verified, update the current knowledge or theories to incorporate these new or unusual observations

Statistics and Science

Many experiments deal with complex or difficult to measure phenomena. Equally these experiments use sample data and seek to generalise the results beyond the observations made with the sample data. As such, determining what a sample set of observations tells us about a proposed explanation in general requires us to make an inference, or as we statisticians call it, to “Reason With Uncertainty”. Reasoning with uncertainty is the core of statistical inference and is typically done using a method called Null Hypothesis Significance Testing (NHST). NHST, in essence, follows these steps:

  1. state what you expect to happen if we assume nothing has changed
  2. select the amount evidence required to be obtained in order to reject that assumption (i.e. that nothing has changed)
  3. using the sample data and an appropriate statistical technique, compute a decision statistic
  4. if the decision statistic provides enough evidence, then we reject the assumption that nothing has changed and conclude that we have a set of sample data that suggests that something has changed

It is important to recognise that when conducting NHST that it is not a proof or definitive statement that something or nothing has changed, but rather that it lends weight to one conclusion or the other. As such, replication IS required in order to make a firm conclusion. Just like the great Aristole once said, “One swallow does not a summer make” so too, “One hypothesis test does not a firm conclusion make”.

From the above descriptions, you should be able to see that Statistics and the “Scientific Method” are rather similar. This should not be surprising since statistics itself is a science, the science of uncertainty, and therefore follows good scientific principles.

 

CITE THIS AS:  
Ovens, Matthew. “Statistics and the “Scientific Method”” Retrieved from YourStatsGuru.

First published Feb 2012 | Last updated: 2 November 2024