The validity of your conclusions

In this piece I want to examine the sort of conclusions that can be drawn from your research. How much evidence you need, how it is analysed and how the arguments proceed, will all depend on the habits and customs of your academic discipline. But inevitably you will gather some evidence and draw some conclusions.

For the conclusions to be interesting, they should be applicable in situations that replicate some aspects of your evidence. If you studied the use of sarcasm in Shakespeare's plays, people could well find it interesting to take your conclusions and see how they might apply to those of Kyd or Marlowe. If you studied the suspension of integrity constraints in enterprise applications, you might hope that your conclusions could also apply to some other enterprise applications than the particular ones you studied. If you were studying the economic effects of the current crisis in banking, despite the unusual and possibly unique aspects of the current difficulties, academics will attempt to find ways of drawing some general lessons.

After all, if the evidence you examined was so unusual, so conditioned by circumstance, or your generalisation so feeble, the conclusions might end up being too weak to be of interest. The common practice of the discipline will be your guide. If your evidence about fluctuations in the earth's magnetic field was gathered in 2006, it would be unusual to limit your conclusions to that year. But on the other hand, at several stages in the history of the earth, the magnetic field has changed direction, so that there is no logical reason to suppose it might not do so again. Many recipes in computing are validated by analysing their performance on a few well-known data sets: but how can you be sure that similar performance will be observed for other data? More generally, sources of evidence need to be carefully chosen not to have specific traits that limit the generality of the discussion.

Niels Bohr, one of the founders of quantum mechanics, was fascinated by the Heisenberg principle of indeterminacy, and viewed it as complementarity: position and momentum at the quantum level are complementary properties, and the more precisely one of these quantities in known, the greater the uncertainty in the other. In explaining the principle, he drew on other examples of complementarity, famously of truth and clarity. He observed that the more precise we are in describing something completely as it is, the more complex and unclear our description becomes, while a clear statement, such as a soundbite or slogan, will never precisely specify anything. Similarly, the more careful we are in owning up to the limits of the evidence we have gathered, the more we constrain the claimed area of validity of our conclusions.

In his famous discussions with Einstein (1949), he addresses this very aspect: 'simply that by the word "experiment" we refer to a situation where we can tell others what we have done and what we have learned and that, therefore, the account of the experimental arrangement and of the results of the observations must be expressed in unambiguous language'.

This is why research methodology sets out to propose hypotheses or research questions based on previous research results, carefully setting out the parameters of an investigation and proposing in advance the limits on the applicability of the conclusions that may be reached. In practice, authors of PhD theses or published papers always cheat a bit, and refine some of these details during the investigation. Any surprises in, or reservations about, these conclusions can be discussed in the section of the thesis entitled "Suggestions for further work".

This is also what was wrong with Karl Popper's famous metatheory of falsifiability. Today, we say instead that negative results do not falsify a theory: usually they merely qualify it, defining its inapplicaibility to the particular circumstances of the results, and instigating further research into its limits of applicability, or a new theory that somehow applies better or more generally.

References

Bohr, N: Discussions with Einstein on Epistemological Problems in Atomic Physics, Albert Einstein: Philosopher-Scientist (1949), publ. Cambridge University Press, 1949.

Popper, K: Conjectures and Refutations (1963) London: Routledge and Kegan Paul, 1963, pp. 33-39