What do they do in the World Wide Open? Can activities be measured, services planned, benefits evaluated, dead-ends avoided?
This is what e-metrics is all about. If you can’t count it, it doesn’t count. The e-metrics SIG had a lively round-table session yesterday, ably chaired by Colleen Cook.
For new readers: Colleen is the outgoing chair of the Statistics and Evaluation Section. Ulla Wimmer is incoming. Plinius is replacing Ulla as secretary.
The round-table was actually a slapdash contraption made from four round tables. But it worked. Real talk with engaged and knowledgeable professionals is one of the joys of IFLA.
The discussion went round and round. The whole area is slippery. Librarians have been struggling with these metrics (measurements, indicators) for more than a decade.
As I see it, we have to start by separating two big concerns (or communities of interest):
Academic and special libraries want to measure the traffic to paid database and full-text vendors, in order to assist their “collection” development. Increasingly, they also want to document the use of institutional repositories and locally produced learning resources, in order to evaluate their investments in production. The big COUNTER project was a cooperative project in that direction. But uptake was limited.
Public and national libraries, on the other hand, mainly want to measure traffic to their websites, their catalogues and other open resources. They need to justify their investments to owners/funders and their internal allocations to their staff.
I know very little about the first topic. I have written one paper on the second topic, however – How much is much? – and try to promote the introduction of a few standardized traffic indicators in Norwegian public libraries.
The problem both communities face, is the lack of structure in the discussion itself. When a new phenomenon – in this case web traffic – appears, we try to manage it verbally, through definitions of concepts and descriotions of indicators. But the digital reality is too varied, too changeable and too full of struggle to be controlled that way.
When we make concepts and indicators, we do not describe a given territory. We create the teritory by mapping it.
The Middle Ages is not the name of a historical period. It was not discovered “out there”. The Middle Ages is the construction or invention of a new historical object belonging to the period class (genus). The class itself is a construct.
The Web Visit is not out there waiting to be discovered. The Web Visit is a conceptual tool (constructed object). We don’t know whether this particular tool will be useful. The only way to find ut, is to try it out in library practice.
Intelligence and literacy
Take intelligence. As a psychological construct, it is about a century old. It has been used extensively over the years. Its operationalization has been revised and refined continuously. The word has entered ordinary language. A successful new concept, in other words.
Success breeds imitators. Recently, the concept of emotional intelligence has ben popular. It is unlikely to establish itself in the same way. Many other “intelligences” also clamor for attention. I don’t think they will succed. I suspect the efforts to establish lots of literacies (information, digital, cultural, artistic, visual ..) on top of good, old-fashioned verbal literacy will face similar problems. Concepts do not succeed through “correct definitions”, but through their value as tools in relation to regular social practices.
Concepts are tools. They must prove their usefulness through use. In the long run, concepts can not be forced on people. People must experience them as useful in their ongoing practices.
All definitions are tentative. In the case of e-metrics, I think the most fruitful approach would be for interested library people (researchers, developers and practitioners) to carry out empirical case studies of web traffic – with a strong awareness of the experimental nature of the concepts itself. Do the concepts and the indicators produce relevant and interesting results? Will there be a demand (market) for such results in the future? From whom? Why?
- PL 40/11: Hard work for OCLC. With Norway as a case we show how hard it is to get reliable and current statistics from many countries.
- PL 38/11: School library statistics. Good research with practical results in California
- PL 36/11: Sharing global statistics. A realistic approach to statistical information.
- PL 35/11: Explore and process. About Plinius Data.
- PL 28/11: A German quality index. BIX for academic libraries
- PL 27/11: An American data hero. Robert E. Molyneux sets up archive
- PL 26/11: A necessary evil. Singers and statisticians.
- PL 25/11: Good statistics advice. How to deal with numbers.
- PL 21/11: Learning from practice. EBLIP editor discusses the concept of evidence.