The seventh International Congress on Peer Review and Biomedical Publication was held in Chicago a couple of weeks ago.
Participants tend to be editors of medical journals. They meet every four years. They are not happy about the current situation. Neither am I.
Scientific American was there. I quote Hilda Bastian:
Most findings are false
– “John Ioannidis pointed to the very low rate of successful replication of genome-wide association studies (not much over 1%) as an example of very deep-seated problems in discovery science. Half or more of replication studies are done by the authors of the original research:
“It’s just the same authors trying to promote their own work.”
Industry, he says, is becoming more concerned with replicability of research than most scientists are.
I first read John Law’s Organizing modernity ten or fifteen years ago.
I thought it was original, well-written and exciting. Law spent a year observing the activities in a large UK laboratory in the late eighties. His approached the organization like an anthropologist – and combined observations, personal experiences and theoretical questioning in an attractive way.
Picture: Daresbury laboratory
Last week I read it once again. The book is still engaging. But now I am more conscious of its weaknesses.
Twenty years ago deconstruction was a big thing. John Law had learnt from Foucault, Baudrillard and Derrida. He questions himself as well as his informants. He is much more accessible than the French nouveaux philosophers, though. British common-sense shines through.
I accept that all orders are social constructs. I am all for the sober analysis of dogmas, myths and ideologies. Deconstruction is a useful tool. But the moment we generalize the tool, we lose more than we gain. Systematic deconstruction is a dead-end. It can be done once or twice, to shake things up. Provoking the established order is always fun. But deconstruction is necessarily parasitic. It feeds on existing social orders. There must be something to take apart.
More importantly: deconstruction is a purely verbal game. It separates theory from practice. In the midst of furious discussions, life goes on. Experiments continue. Salaries are paid. People go home for dinner. Even Derrida gets hungry.
A major new study of social classes in the UK has proposed seven groups:
- Elite – this is the most privileged group in the UK. They are set apart from the other six classes, especially because of their wealth, and they have the highest levels of all three capitals.
- Established middle class – this is the second wealthiest class group and it scores highly on all three capitals. It is the largest and highly gregarious class group and scores second highest for cultural capital.
- Technical middle class – this is a small, distinctive new class group that is prosperous but scores low for social and cultural capital. It is distinguished by its social isolation and cultural apathy.
- New affluent workers – this young class group is socially and culturally active, with middling levels of economic capital.
- Traditional working class – this class scores low on all forms of capital, but is not completely deprived. Its members have a reasonably high house values, which is explained by this group having the oldest average age (66 years).
- Emergent service workers – this new, young, urban group is relatively poor but has high social and cultural capital.
- Precariat (The precarious proletariat) – this is the poorest, most deprived class and scores low for social and cultural capital.
Bourdieu must be smiling from his cloud.
Oslo and Akershus University College of Applied Sciences publishes several open access e-journals.
One of them is called Reconceptualizing Educational Research Methodology (RERM)As a teacher of statistics and social science methods I am interested in new approaches to research. But several of the articles read more like streams-of-consciousness than scientific prose. Even the titles are incomprehensible.
Last year the author wrote a doctoral thesis on the potential of the mobile phone as a pedagogic tool in a senior secondary technical school. But he felt that some philosophical issues had been sidelined in the process.
- An articulation and elaboration of this underlying tension between the philosophical and the practical only became possible after the work was completed.
The rhapsodic paper ends with an (In) Conclusion:
- At best, the hope for both my dissertation and these reflections is that they stimulate the glands and lubricate the tracts to produce an appetite for some tasty methodological morsels that might otherwise lie buried, or hidden, in the familiar fields of qualitative empirical research in education.
- … our re-search stands to be emboldened and enriched by a process that pays more attention to “the images in the ideas, the fantasies in the facts, the dreams in the reasons, the myths in the meanings, the archetypes in the arguments, and the complexes in the concepts” (Romanyshyn, 2007, p.12).
- This might not, in the short term, help to secure more research dollars … [TH: that is probably true]
JISC has published a metastudy – summarizing twelve separate investigations – about user behavior in academic libraries. I quote:
Among the central findings
- Disciplinary differences do exist in researcher behaviours, both professional researchers and students.
- E-journals are increasingly very important to the process of research at all levels.
- The evidence provided by the results of the studies supports the centrality of Google and other search engines.
- Google is often used to locate and access e-journal content.
- At the same time, the entire Discovery-to-Delivery process needs to be supported by information systems, including increased access to resources.
- Journal backfiles are particularly problematic in terms of access
Statistical testing is tricky – as a newspaper article from yesterday shows.
Both publics, journalists and researchers tend to overemphasize the value of single studies. Publics want simple answers, journalists want spectacular news and researchers want results that will further their careers.
Science does not work that way. Knowledge accumulates slowly, through trial and error, missteps and detours. Established truths are never fully established. Answers are valid until further notice. I quote the crucial parts:
How can two studies of the same topic reach opposite conclusions?
Stanford University epidemiologist John Ioannidis has famously estimated that 90 per cent of published medical research is wrong, thanks to factors such as sloppy statistics, inadequate study size and duration, and bias – both conscious and unconscious.
Three common research failings.
Nice blog post by Barbara Fister
She writes about three new articles in College & Research Libraries
The first, by Brett Bodemer … is about how we help undergraduates conceptualize the research process.
- We should stop thinking of search as a relatively simple step that happens before reading and writing.
- These activities are recursive and connected processes:
- Search involves reading (because you have to do at least some reading to make choices and refine terms)
- Writing should ideally drive a search, not be saved for the final act, when it’s too late to pursue a thought that bubbled up from the pages of your draft.
- This seems obvious, yet the influential ACRL standards for information literacy that so many librarians draw on implicitly separate searching from doing something with what you’ve found.
In Great Britain, the “major stakeholders in library, archive and information science (LAIS) research” have just established a coalition to do something about the fragmented state of the field. As Elkin (2007) says:
- …there is research going on, much of it ICT focussed, but it is piecemeal, of varying quality, with no co-ordination or coherence and poorly disseminated
- there are no real think pieces being produced or encouraged
- we have lost the locus / the space for research, within a co-ordinating framework
- we have lost the continuity and the context, as well as the culture and the capacity
I hope we can learn from this initiative.
The quotes below comes from a brave and engaged librarian.
I see many of the same weaknesses in my professional environment. But US librarianship is a more vulnerable since it has a longer research tradition than Europe. The gap between pretense and reality is more obvious.
Norway and most of Europe is still in a start-up phase when it comes to professional research. But the lack of open discussion about validity, reliability, bias, data and methods is dangerous. Aspiring researchers are not exposed to the heat of real academic debate.
For debate in action, see the discussion pages in Historisk tidsskrift.
In her recent editorial, Denise Koufogiannakis discusses the concept of evidence.
Koufogiannakis is Editor-in-Chief of the EBLIP journal, or Evidence Based Library and Information Practice. Like many other professions, librarians have constructed a rather conservative culture. Melvil Dewey was a great reformer, but he is safely dead and buried. It is his system rather than his spirit that lives on.
I support EBLIP as an effort to change traditional library and information practices by applying scientific or sceptical reasoning rather than value-based or politically correct arguments. EBLIP is inspired by evidence-based medicine (EBM). But the world of librarianship is weaker and more blurry than the world of medicine. Medicine walks with life and death. Libraries support culture and learning.