Plinius

Thursday, February 25, 2010

PL 7/10: Performance and impact

Filed under: IFLA, statistics — plinius @ 11:33 am

Statistical indicators are deeply integrated in the Global Libraries program.

Below I reproduce some of the main points made by Sandra Fried, Maciej Kochanowicz and Marcel Chiranov in their excellent paper  Planning for impact, assessing for sustainability.

The basic GL approach, which relates project grants to social outcomes, requires :

  1. GL grants (“vibrant libraries that meet user needs”)
  2. Measures of increased access (performance metrics)
  3. Measures of improvements to people’s lives (impact indicators)

The data captured in the performance metrics are intended to help monitor progress of individual programs (for local learning and course correction) but also for GL’s purposes, to track performance of our grant portfolio as a whole.

The performance metrics are few in number but are perceived as mainstream and desirable by managers, funders and policy makers. Where possible, they draw on existing international standards.

Performance metrics

Required

  1. No. of libraries providing public access computing
  2. No. of workstations available in country/target region
  3. No. of physical visits to public libraries
  4. Workstation use rate
  5. No. of library staff trained under the GL grant
  6. No. of library users trained by GL program in information seeking / use of ICT
  7. Spending on public library services (GL and other)

Recommended

  1. Users’ activities in the library / on computers
  2. State of training of public library staff
  3. No. of unique users of public access computing in public libraries
  4. No. of virtual visits
  5. Repeat patron usage
  6. User demographics

The second category of data collection is focused around country-specific impact indicators, as a necessary component of the totality of evidence needed to describe a country grant program’s contributions to local and national goals.

Impact indicators

We have found that library programs can aspire to contributing to changes in people’s lives in one of the following 6 areas:

Education

  • Libraries offer on-line reading programs
  • People acquire skills at library

Health

  • Libraries refer people to relevant on-line information and services
  • Users have improved health choices

Culture and Leisure

  • Library become social center of community
  • Events and activities lead to community re-vitalization

Economic Development

  • Librarians direct business people to relevant information
  • Small businesses are more productive

Communication

  • Librarians teach people to use e-mail
  • Isolation of marginalized communities decreases

E-government

  • Librarians teach people to access e-government services
  • Improved citizen uptake of government services

When designing the set of metrics and collection methodology for each program/system, close attention should be paid to collecting useful evidence – that which can be used for outreach and advocacy purposes.

Causes and contributions

GL does NOT require our grantees to measure whether programs have caused – in attributable ways – ultimate change in people’s lives.

For example, GL does not expect our library programs to prove, conclusively, that installing computers in libraries has led to an X% increase in local incomes. GL considers that this level of research rigor would require an inappropriate usage of resources in the public library context.

  1. It is important for libraries to design programs and activities destined to contribute to changes in peoples’ lives.
  2. This is achieved by deeply engaging stakeholders in program design, in order to plan activities that contribute to meeting users’ needs.
  3. What follows are activities to collect evidence describing this contribution – whether through stakeholder perceptions, surveys, case studies, relevant statistics compiled by a third-party or government entity (e.g. labor and health statistics), or other methods. Any valid method of social science research could be acceptable.
  4. Collecting this kind of evidence can be demanding of people and time; therefore, evidence should be collected based on what will be most convincing in local outreach and advocacy efforts to demonstrate to local, regional and national governments how libraries contribute to their priorities.
  5. For example, programs could ask librarians to report stories of users who have found new/better jobs as a result of library programs.
  6. Alternatively, programs could seek a higher level of proof by asking a random sample of users whether the library has assisted them in finding new/better jobs.
  7. Both would beacceptable, and the method should be selected to meet local capacities, and advocacy and outreach needs.
  8. GL urges our grantees to consider the 6 key impact areas contained in GL’s areas of impact (shown above) as they decide on the focus areas of their program’s impact. …
  9. Programs should invest time and effort in creating locally relevant metrics that are appropriate to the focus area.
  10. Programs should carefully consider what evidence they will need to show their contribution toward the desired impact, as well as how often measurement is required, in order to avoid measuring more than is necessary.
  11. Impact Planning and Assessment is by definition an iterative process.
  12. Library programs should continually gauge users’ and stakeholders’ needs and interests, refine services appropriately, collect data to show contribution toward meeting these needs, and use that evidence for advocacy purposes.

Measurement and learning

Measurement and learning are critically important to Global Libraries and to our grantees.

Measurement helps grantees make decisions about necessary program improvements and, perhaps more importantly, provides the evidence needed successful advocacy efforts. This forms the backbone of libraries’ strategy for ensuring long-term program sustainability: making the case for ongoing, local financial support is built on creating an understanding in
public and government supporters for what public library programs accomplish and the benefits they bring
.

Resources

Advertisements

1 Comment »

  1. I see you are interested in statistical research. I have put one of the most comprehensive link lists for hundreds of thousands of statistical sources and indicators on my blog: Statistics Reference List. And what I find most fascinating is how data can be visualised nowadays with the graphical computing power of modern PCs, as in many of the dozens of examples in these Data Visualisation References. If you miss anything that I might be able to find for you or you yourself want to share a resource, please leave a comment.

    Comment by crisismaven — Thursday, February 25, 2010 @ 7:24 pm


RSS feed for comments on this post. TrackBack URI

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Blog at WordPress.com.

%d bloggers like this: