Tuesday, July 5, 2011

PL 23/11: Soft research

Filed under: 1bib, debate, research — plinius @ 8:24 pm

The quotes below comes from a brave and engaged librarian.

I see many of the same weaknesses in my professional environment. But US librarianship is a more vulnerable since it has a longer research tradition than Europe. The gap between pretense and reality is more obvious.

Norway and most of Europe is still in a start-up phase when it comes to professional  research. But the lack of open discussion about validity, reliability, bias, data and methods is dangerous. Aspiring researchers are not exposed to the heat of real academic debate.

For debate in action, see the discussion pages in Historisk tidsskrift.


As a research-embedded health librarian, I often go to non-LIS-type conferences that are aimed at health researchers.

We need to toughen up.

When I am listening to a health librarian give a research presentation, it is all too common for me to end up cringing at what is either weak research methodology or weak defense of methods (or both).

  • The speaker in this example is a very well-respected tenured LIS faculty member in an ALA-accredited institution, who has a long and established track record of important research and advocacy for libraries, and thus I think fair game for public critique of research presentations….
  • In the question period after the talk, rather than defending the reliability of her research, or explaining why reliability was perhaps not the appropriate question to ask about the qualitative portion of her investigation, the speaker demurred, basically saying that well, nothing’s perfect and we all do our best.
  • As a leader in our field, she should be modelling rigourous research and the ability to explain and defend it for us.

 Policy-based evidence

  • Librarians do a lot of surveys.
  • Especially Masters-level academic librarians, who are supposed to do some research and base policy decisions on some sort of evidence.
  • User surveys are pretty common, and this is reinforced by our love of LIBQUAL+.
  • We also have this idea that survey research is “simple”
  • Thus a masters-level professional can do it just fine with no methodological problems. This I would dispute.
  • Surveys *can* be simple, just as many other research methods can be simple.
  • But surveys are also really easy to do badly.
  • At pretty much every library conference .., I see presentations of surveys with conclusions that do not follow from the actual results
  • and/or surveys that were … designed to justify a particular policy move.

When I see librarians doing things like:

  1. presenting surveys with extremely low response rates
  2. no demographic information to assume this small sample is representative…
  3. basing conclusions on the responses of the majority of a tiny minority of the whole,,
  4. with no discussion of response bias,

I am frankly appalled.

  • ML/IS research methods classes tend to be generic and weak.
  • That we …  continue to reward shoddy research methods with conference presentations …  is horrible.
  • Not only are we probably making poor decisions based on lousy research
  • We are completely undermining our own efforts to position librarians as professionals with research expertise.


Most LIS graduates – who enter the academic stream within universities – are vastly under-prepared to conduct reliable and valid research in this setting.

  • They are entering institutions where ‘some’ other faculty have vast knowledge and training in research methods. Most other faculty posses both a masters degree in their respective field AND A PhD!
  • This extra 4-6 or more years of education, with some departments specifically focusing their curriculum on qant. and qualit. research methodologies, provides these faculty with the research with the tools to conduct well designed research projects.


 Questionable research methods are certainly not limited to health librarianship or to conference presentations.

  • Our journals are replete with them.
  • Reports and studies on the information needs of this or that population make ridiculous and embarrassing claims based on incomplete and unreliable data.
  • we need to learn more about reliable data gathering AND about analysing this data.
  • If we don’t know how to do it, then why not collaborate with those who do?
  • I do not mean LIS students taking a research methods course.
  • Doing crappy research just discredits our profession.
  • we need to stop being so nice and start being more critical.
  • I think we can take it, if we can’t, then we need to toughen up.

Leave a Comment »

No comments yet.

RSS feed for comments on this post. TrackBack URI

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Create a free website or blog at

%d bloggers like this: