Saturday, March 7, 2009

Software and Research

It can be disappointing for people with good programming skills that software development per se is not regarded as research at all.

To count as research, work must (a) be based on an existing body of knowledge in an academic discipline (b) investigate some question along a path others can follow (c) form conclusions based on public evidence (d) result in a contribution to knowledge according to the standards of the academic community. As I argued in my chapter in "Interdisciplinary Research" (Atkinson et al 2006) this has been the plan for research set out since the days of the founding of the Royal Scociety and other national academies.

Naturally, the identities and expectations of academic communities change over time. The verifiability requirements imposed by different communities are very diverse: the rules for experimental design in medical research are very different from the rules for experimental design in marine biology or the criteria for evidence used in social economics. In the computing community we no longer see articles with titles such as "XYZZY: An Interactive Whizzywig" in our respected journals (and maybe we never did, despite examples such as Stallman 1981).

As a result, research applicants who approach us with ideas such as "I want to use JavaFX on the Semantic Web" pose us a particular problem. It can be easy to see the kind of innovative software they plan to produce, but it fails the research criteria given above. For example, we fail immediately to find examples of research literature that they can use as starting points for their research: the manifest starting point of JavaFX is not really found anywhere in the research literature. At the time of writing (7 March 2009) it can be found in the title of just one MSc thesis and no journal articles. The Semantic Web is of course a different story, and so patiently we try to persuade the student to use start from one of the lines of research in the literature, and maybe find a way to bring JavaFX into the work at some stage.

If software itself is not seen as a contribution to knowledge, it is more obviously useful for other purposes within the research paradigm. It can demonstrate the viability of a mathematical or analytical approach. Programs can be used to conduct experiments on some other research idea. Most such experiments, however, are carried out using throw-away software. Some new way of visualizing something, or a new framework or pattern might be of interest, but the research would be about the impact or usefulness of the approach rather than the actual code.

So, what are the options for computing research? We can
  • Develop some new theoretical approach or algorithm for problems of some type

  • Investigate the best way of doing something (reasonably general)

  • (Re)evaluate some suggestions by other researchers in some slightly different context

  • Examine the demand for some kind of computing approach

We should try to avoid

  • Discussing or comparing solutions from different vendors
  • Improving the efficiency of existing solutions

Comments on these ideas are welcome.

References:

Atkinson, J., Crowe, M. (eds)(2006) Interdisciplinary Research: Diverse Approaches in Science, Technology, Health and Society, Wiley, Chichester, 978-1-86156-47-2: also Chapter 1: Research Today

Stallman, R.: EMACS: The Extensible, Customizable Display Editor. ACM Conference on
Text Processing, 1981

Update 11 April 2009: Interestingly the recent Research Assessment Exercise RAE2007 in the UK did accept some published software as high-quality research outputs. But I believe the comments above contain good advice for PhD students.

No comments:

Post a Comment