Thursday, April 12, 2012

How to Measure the Success of a Collection

Accountability is a bit of a buzzword in higher education. Everywhere I turn, it seems to be in the air. As my job starts to include some responsibility for the library collection, the question of how to assess it has come to mind. So far I haven't found a single standard that would apply, because so much depends on local factors and needs.

But one obvious variable is usage. An indication of needs met is a collection that experiences heavy use. Books and other materials that circulate, online resources that get a lot of hits -- these numbers are all easy to generate, and we do so regularly. A collection that experiences 100% use is unrealistic, so what precise proportion of the collection would have to be used for it to be considered "successful"? 10 percent of the collection? On an individual item level, if one particular thing is accessed or circulated 27 times during the first year while it's hot, 2 times the year after that, and then 0 times for the next 30 years, was it a successful purchase? If a comic book circulates and a philosophy tract does not, should we purchase more comic books and fewer philosophy tracts?

Another, almost contradictory, variable is comprehensiveness. If the library has a vast, endless collection -- resources beyond resources -- intuitively that seems like it would be a good thing. Any topic that someone could think of to research would be covered. But inevitably, the bigger the collection, the smaller the percentage of it that could be humanly used in a given amount of time. So while comprehensiveness might at first seem like an obvious indication of success, if much of it goes untouched over the course of a year, thus lowering the percentage of the overall collection accessed, it might not be a useful measurement in the end.

Other variables include quality and cost. We could create some metric, similar to impact factor for journals, to assess the entire collection. We could use RCL, standard reviews, and other resources to gauge how good the collection is from independent perspectives. But then these wouldn't necessarily be addressing the needs of our local patron population. As for expense, I haven't figured out how we can incorporate cost as a factor of success when the prices of library materials continue to rise while the library budget remains flat.     

An experiment I would enjoy is this: Give me some money for a certain subject area. I'll build a collection to the best of my ability, so that quality and comprehensiveness would be taken care of. Then I would track usage, adjust accordingly, and track every decision. Doesn't this sound remarkably like what collections librarians already do?

And perhaps the elephant in the room is the potential for an electronic collection that could shift and reform every year, depending on the previous year's feedback and usage. True, a certain amount of continuity would be lost, but so would a lot of irrelevance. Usage would be high, with potentially no loss to comprehensiveness and quality. I don't think anyone is there yet, but could this be where academic libraries are heading?


  1. I'd LOVE LOVE LOVE to be able to assess my collection. The problem, really, is that librarians come and go, so you are usually inheriting a collection that someone else has started building, that a third librarian had begun long ago... so nobody exists who really has a good sense of its strengths and weaknesses. I try to compensate by using the collection, but of course, that only works on a very limited scale!

    I liked this article: .. but it relies on a version of WorldCat that my institution doesn't subscribe to. There was also some product you can buy that assesses the collection within certain call ranges, but, again, my institution didn't end up getting it. So it's all just guesswork where I am.

    I'm skeptical of the shifting electronic collection, though; current electronic collections do indeed change their holdings, but they do it based on publisher agreements and without input from librarians. Of course, librarians also don't choose what goes into them, so. Maybe it doesn't have to be like that, but at the moment, I'm really frustrated with ebrary, which is the largest e-book collection I have contact with. There are a lot of practical difficulties, though--how would you pay for books under a model such as you describe? And wouldn't it be prohibitively expensive in terms of staff time if someone had to choose an entire collection every year, even with the assistance of data?

  2. Thanks for the input, and the link to the article! I can never tell if I'm the only one thinking about these things :-)

    I deliberately avoided bringing up the potential cost of such an electronic collection, with the thought that the appeal of automation would mitigate concerns, but yes if something like this came out tomorrow I can imagine only a few would be able to afford and manage it.