Bibliometric Cloud

Bibliometric 2017
Word cloud created by The Voyant Tool with the article by Cox, A. M., Gadd, E., Petersohn, S., & Sbaffi, L. (2017). Competencies for bibliometrics. Retrieved from http://journals.sagepub.com/doi/pdf/10.1177/0961000617728111

The Voyant Tool does make lovely t-shirt designs and this one is for the ‘bibliometricians’[1]. The chromatic word cloud is a resultant of ingesting the article, ‘Competencies for bibliometrics’ by Cox, A. M., Gadd, E., Petersohn, S., & Sbaffi, L., published in2017[2]. While it appears to shout ‘bibliometrics’, ‘research’, ‘use’, ‘tasks’, ‘professionals’, ‘tools’……, in a meeting at work recently, I heard a class of semantically relating words echoing among the discussions, ‘REF (research excellent framework)’, ‘research support’, ‘impact’, ‘Altmetrics’, ‘embedding’…. Evidentially, bibliometric supporting work has been duly assigned to library professionals in academic settings. Nonetheless, the paper by Cox et al. raised a key question to this seemingly perfect role for librarians: what skills and knowledge are needed in order to provide bibliometric and altmetric support to their stakeholders? This question corresponds to the core tasks of “raising academics’ bibliometric literacy” and explaining ‘responsible use’ of bibliometrics, as laid out in the tri-level hierarchical ‘Competency model for bibliometric work’ drawn up with the study results. But what exactly are ‘bibliometric literacy’ and responsible use of bibliometrics?

competency-model-frame-work-design-page-001

competency-model-frame-work-design-page-002
Competency Model (Cox et al. 2017) Downloaded from https://thebibliomagician.files.wordpress.com/2017/10/competency-model-frame-work-design.pdf

Bibliometric literacy and Responsible Use of Bibliometrics

In the article, the authors explained that ‘Bibliometric literacy’ is an aspect of information literacy – it empowers users to improve their own publication performance and impact. The ‘an aspect’ bit is helpful to determine what ‘Bibliometric literacy’ is. Information literacy as an empowerment has been endorsed in a lecture of the very subject in Week 8, whereas I would like to borrow a quote on information literacy from the IFLA website, in order to 1. Have a revision on the concept of information literacy, 2. To single out any elements applicable to bibliometric literacy (information well fitted to information needs), 3. To illuminate the idea on the manner of information usage (wise and ethical):

“Information literacy is the adoption of appropriate information behaviour to identify, through whatever channel or medium, information well fitted to information needs, leading to wise and ethical use of information in society.”[3]

Information well fitted to information needs – Given bibliometric indicators are built on the raw facts or observable feature of Scientific outputs, references, and citations[4] of published documentation, it represents a more confined group of audience, for instance research group leaders, individual academics and PhD students, as illustrated in the article. They all use the bibliometric indicators differently to achieve their needs. Being literate to bibliometrics also means being able to identify/combine the right tools to cater their needs. For example, being familiar with and the ability to differentiate indicators on article, author, journal and institutional level. The writers mentioned also altmetrics in the Competency Model for bibliometric support work. Thus, ‘bibliometric work’ in this context has a wider meaning spanning over alternative-metrics as well as traditional metrics.

Indeed altmetrics has surpassed the stage of being a word exiting in urban dictionary or as a hashtag. Fuelled by new publication formats, open access movement and influence of social media, it gains wider acceptance as a force to enrich traditional impact indicators. As explained by Amy Reese at Almetric.com in her presentation, altmetrics ‘capture more diverse “flavours” of impacts than citation-based metrics’. This implies that bibliometrics literacy entails the knowledge of understanding a broad range of resources where the altmetric scores have drawn from, for instance, various social media, Slideshare, Figshare, institution repository statistics, etc.

Wise and Ethical use– Whether it is termed as ‘Wise and Ethnic use’ or ‘responsible use’, there must be unwise, unethical or irresponsible use and hence the emphasis of applying better practises relating to bibliometrics. It sounds to me almost like abusing of alcohol and leading to dangerous driving. I acknowledge this is hardly a good parody, but given the fact that a number of documents have been produced to counteract misuse of bibliometrics, the risks are hard to ignore. Below are three prominent ones that are worth paying attention to:

I. The San Francisco Declaration On Research Assessment (DORA)

DORA was initiated by the American Society for Cell Biology in 2012. In short, it calls for abolishing the use journal-based metrics, such as Journal Impact Factors, as a surrogate measure of research output and to assess an individual scientist’s contributions, or in hiring, promotion, or funding decisions.

II. Appropriate use of bibliometric indicators for the assessment of journals, research proposals, and individuals

The strong worded document was issued by the prestigious Institute of Electrical and Electronics Engineers (IEEE) in 2013. It criticises research output being boiled down to a number (namely journal impact factor) is a misguided and biased indicator of ‘scientific quality’, in particular using it to make hiring, promotion or funding decisions. It offers three tenets in conducting ‘proper assessment’:

  1. The use of multiple complementary bibliometric indicators;
  2. Any journal-based metric must not be used to assess single-article quality or to evaluate individual scientists .
  3. Peer review remains as the primary assessment of scientific quality of a research project or of an individual scientist. Bibliometrics may be used as an additional source of information.

III. Leiden manifesto for research metrics

Published in 2015, the manifesto serves as a roadmap to guide researchers and evaluators towards the best practises in research assessment with ten principles:

  1. Quantitative evaluation should support (not substitute) qualitative, expert assessment.
  2. Measure performance against the research missions of the institution, group or researcher.
  3. Keep data collection and analytical processes open, transparent and simple.
  4. Allow those evaluated to verify data and analysis.
  5. Account for variation by field in publication and citation practices.
  6. Protect excellence in locally relevant research.
  7. Base assessment of individual researchers on a qualitative judgement of their portfolio.
  8. Avoid misplaced concreteness and false precision.
  9. Recognize the systemic effects of assessment and indicators.
  10. Scrutinize indicators regularly and update them.

Thoughts and Conclusions

The Competency Model can in fact be an useful reference to any aspiring LIS newbie like myself. It provides some insights in what possibly considered to be skillsets on various level for the purpose of supporting bibliometric and altmetric work. Reading the Competency Model gave me mixed feelings. On the one hand, I am glad that we were given an overview on bibliometrics (as part of the topics of information search and infometrics) at CityLIS, not least the authors revealed that most of workforce involving in bibliometrics work did not have any formal training during their LIS studies. On the other hand, even if we receive some sort of training on bibiometrics and altmetrics, there is much CPD to be done to keep up with the pace of developments. Whether these CPD are initialled at a personal level or institutional level, they are important aids to strengthen skills, to get to know up-to-date developments, or to cater institutions’ specific requirements.

A related literature published in 2013 by Åström & Hansson[5], expounded the role of academic libraries in Sweden with the implementation of bibliometric work, had presented it as an opportunity of broadening the responsibilities for librarians. This gives a sense of hope out of a poignant outlook of library profession projected by the constant reminders of AI are replacing librarians. However, as pointed out by Cox et al., ‘librarianship has traditionally attracted people trained in humanities and relatively few library roles involve data manipulation and analysis’. This might be seen as one of the weakest link for librarians. Such skills are highlighted across different sessions of DITA (module INM348). Albeit it takes time, practice, imaginations and experiences to be able to apply these knowledge in various scenarios on a practical level, proficiency in data handling and analysis is without doubt an advantage in an increasingly digital landscape. It could open more doors for aspired new professionals and take librarians to a more centre stage.

Footnotes and references:

[1] In the paper “How implementation of bibliometric practice affects the role of academic libraries”, by  Åström, F., & Hansson, J. (2013). Journal of Librarianship and Information Science, 45(4), 316-322. doi:10.1177/0961000612456867, the author mentioned that academic libraries have created a job title called ‘bibliometrician’. It explicitly exhibited the trend of bibliometric support being integrated into wider academic library services.

[2] Cox, A. M., Gadd, E., Petersohn, S., & Sbaffi, L. (2017). Competencies for bibliometrics. Retrieved from http://journals.sagepub.com/doi/pdf/10.1177/0961000617728111

[3] Todeschini, R. & Baccini, A. 2016, Handbook of bibliometric indicators: quantitative tools for studying and evaluating research, Wiley-VCH, Weinheim, Germany.

[4] The quote was taken from IFLA website available at https://www.ifla.org/about-information-literacy

[5] Åström, F., & Hansson, J. (2013). How implementation of bibliometric practice affects the role of academic libraries. Journal of Librarianship and Information Science, 45(4), 316-322. doi:10.1177/0961000612456867

One thought on “Bibliometric Cloud

  1. An insightful overview of the (sometimes difficult) issues around metrics, and a good point about the need for LIS folk to gain at least a basic insight into these issues to avoid the weakest link. Nice word cloud:)

    Like

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s