• How many libraries hold the book in their collections? Check WorldCat to find how many libraries have a record for the book in their collections.
• Are there any book reviews?  To find book reviews, check bibliographic databases that index book reviews, Google Books, Amazon, Barnes and Noble, publisher websites, to name a few. Some reviews are available in video format.
• Are there any discussions about the book in the Amazon Customer Discussion forum?
• Is the book available in an e-reader format as well as print?
• Has the book been publicly recommended on Google+?
• Has the book been tweeted on Twitter?
• Is the book noted on a publisher’s best-seller list?
• What are the sales figures for the book?
• Has the book been included in bibliographies?
• Has the book or book chapter been cited in subsequent publications? Some bibliographic databases provide citation data to book or book chapters from scholarly literature. One example of a database is the Thomson Reuters Book Citation Index.
• Has the author been invited to present on the topic related to the book or book chapter?
• Has the author been invited to book signings?
• Are there subsequent editions to the book?

Any others?

“Citation analysis is not a substitute or shortcut for critical thinking; it is, instead, a point of departure for those willing to explore the avenues to thorough evaluation . . . citations tell us nothing about a researcher’s teaching ability, administrative talent, or other non-scholarly contributions. And they do not necessarily reflect the usefulness of research for curing disease, finding new drugs, and so on.”  [Source: Current Contents, (43): 3-9, October 28, 1985.]

Examples of “points of departure” for review of research impact:

Economic Impact Reporting Framework, Medical Research Council

Excellence in Research for Australia (ERA) Initiative, Australian Research Council

Logic Model for Medical Research, Anthony Weiss

Mixed-Method Approach, Trochim, William M., et al.

Payback Framework, Health Economics Research Group (HERG)

Research Impact Framework, Kuruvilla, Shyama, et al.

Can Tweets Predict Citations? Metrics of Social Impact Based on Twitter and Correlation with Traditional Metrics of Scientific Impact. Gunther Eysenbach. J Med Internet Res 2011 (Dec 16); 13(4):e123.

“Tweets can predict highly cited articles within the first 3 days of article publication. Social media activity either increases citations or reflects the underlying qualities of the article that also predict citations, but the true use of these metrics is to measure the distinct concept of social impact. Social impact measures based on tweets are proposed to complement traditional citation metrics. The proposed twimpact factor may be a useful and timely metric to measure uptake of research findings and to filter research findings resonating with the public in real time.”

HTML (open access)

PDF (members only)

“We may not be able to demonstrate or to measure significant re-use of the outputs of the research that we fund now for many years. But what we can do is focus on optimizing the capacity and the potential for future exploitation. Where we can’t demonstrate re-use and impact we should demand that researchers demonstrate that they have optimized their outputs to enable future re-use and future impact. Where research doesn’t have near term applications we should apply a higher standard of curation and archival, and above all require researchers to ensure that their work remains discoverable over the next decades when its possible applications may surface.“

[Source: Mabile, Laurence, Dalgleish, Raymond, Thorisson, Gudmundur , Deschênes, Mylène, Hewitt, Robert, Carpenter, Jane, Bravo, Elena, Gourraud, Pierre-Antoine, Hofman, Paul, Harris, Jennifer, Munoz-Fernandes, Maria Angeles, and Cambon-Thomsen, Anne. The BRIF (Bioresource Research Impact Factor) as a tool for improving bioresource sharing in biomedical research. Available from Nature Precedings <http://hdl.handle.net/10101/npre.2011.6568.1> (2011)]

Read the article in Nature: The Athletes of Science by Maria Pavlou and Eleftherios P. Diamandis. Nature 478, 419 (2011). doi:10.1038/nj7369-419a.

How is impact of innovation in practice and changes to policy measured? Calford points out several examples of innovative research projects.

“The Kids Safe Sleeping Campaign, developed by an international group of public health researchers, has reduced the incidence of sudden infant death syndrome by 85 per cent since its inception in the 1990s.”

“The Jameson cell and the reflux classifier, for example. Invented and developed by Graeme Jameson and Kevin Galvin respectively, these innovations add more than $4 billion in value to the mining industry each year.”

According to the author, these  significant innovations are “virtually anonymous” using the current Excellence in Research for Australia framework.

A recent commentary published in Science Translational Medicine, “Quality Versus Quantity: Assessing Individual Research Performance” by José-Alain Sahel (DOI: 10.1126/scitranslmed.3002249), reviews issues related to evaluating scientific performance.

Among the sections included are:

• Current Thinking in Assessing Impact
• Innovative Methods for Impact and Engagement
• A ‘How To’ Guide to Measuring Your Own Academic Impact
• Improving Academic Communication

To learn more about the effect of the length of a title and punctuation, read the entire article by Sarah Huggett in Research Trends, September 2011.