Increasingly, individual researchers and academic institutions are being required to rate and rank publications as a metric of both individual researcher and organisational performance [
The term “bibliometrics” describes a mathematical method for counting the number of academic publications and related citations and is based on authorship. Measures such as citations, impact factors (IFs),
Bibliometrics are one way used to measure the impact of the research although the notion of impact is often difficult to measure. Another common criticism is that historical approaches to bibliometrics, such as impact factors and citations, disadvantage some disciplines. Reasons for this are complex including the often competing demands of a practice-based discipline, whereby published research findings may be measured in changed and improved clinical practice rather than citations [
The electronic databases CINAHL, Medline, and Scopus were interrogated using the search terms including
Journal impact factors (IFs) are the measure of the frequency in which an “average article” in a journal has been cited over a defined period [
However, impact factors have been subject to ongoing criticism by academics and scholars for both methodological and procedural imperfections. There is also debate about how IFs should be used. Whilst a higher impact factor may indicate journals that are considered to be more prestigious, it does not necessarily reflect the quality or impact of an individual article or researcher. Other metrics have therefore been developed to provide alternative measures to impact factors, such as the Journal Evaluation Tool [
Originally developed in 2005, the (Hirsch)
Papers can be cited for many reasons, such as proposing contentious positions; other than being of high quality and with the
Similar to the Immediacy Index for journals, the
A number of studies have been undertaken tracking the increased performance of both nursing journals and individual researchers. Wilkes and Jackson analysed a total of 530 articles from five Australian and five USA and UK journals and found an increase in output from the period of prior analyses in 2000 [
The notion of impact is also not easy to measure, a case for many disciplines. In health care, measures of impact can be constructed as being of scholarly impact (where citation measures are very useful) or impact on clinical practice [
In order to deal with the complexity of citation impact analysis, a range of approaches have been introduced including percentile rank scores, as indicators of relative performance [
Globally, the use of the Internet is increasing exponentially [
The World Wide Web has not only revolutionized how information is gathered, stored, and shared but also provided a mechanism of measuring access to information. The current debate and discussion of the online publishing forum and the importance of access to information and challenging traditional gatekeepers to knowledge are a critical consideration [
Webometrics refers to the quantitative analysis of activity on the World Wide Web, such as downloads, which draws on infometric methods [
Thus it can be seen that measurement of scholarship and impact can occur using a range of metrics. Within both traditional and evolving approaches it is useful to review the performance of nursing and midwifery according to established measures.
Criticisms of traditional approaches to bibliometrics, such as impact factors and citations, included a perceived disadvantage for certain disciplines. However, conversely it can be said that citations in nursing and midwifery, like other areas of health, can accumulate relatively quickly. This is attributable to the large number of journals, the volume of research being conducted, and also the rapidly changing nature of the field and the increasing representation of nurses and midwives in research. This is particularly so when compared to disciplines with fewer journals or disciplines in which change or evidence of impact is achieved rather more slowly, such as in mathematics, for example.
Sponsored by the Council of Deans of Nursing and Midwifery in Australia and New Zealand, the Journal Evaluation Tool (JET) rates journals according to four quality band scores [
Twitter is a microblogging platform that allows users to “tweet” text of up to 140 characters to users and is publically available to anyone with online access. Twitter is commonly used as an online communication platform for personal communications; however it is rapidly becoming used for work related purposes, particularly scholarly communication, as a method of sharing and disseminating information which is central to the work of an academic [
A study conducted by Eysenbach [
The twimpact (tw
The twindex is a metric ranging from 0 to 100 indicating the relative standing of an article compared to other articles. The twindex7 of specific articles is the rank percentile of an article when all articles are ranked by the twimpact factor tw7. For example, if an article has the highest twimpact factor tw7 among its comparator articles, it has a twindex of 100. In Eysenbach’s seminal work on the ability of tweets to predict citations, twindex articles with >75 often turned out to be the most cited [
Whilst the study identified that the buzz of the blogosphere is measurable, many limitations are also noted including the fundamental observation that the number of hits is a metric of success. The authors also identified that correlation is not causation, and it is difficult to decide whether additional citations are a results of the social media buzz or whether it is the underlying quality of the article or news trustworthiness that drives both the buzz and the citations—it is most likely a combination of both [
A preliminary study conducted by Yan and Kaziunas identified that merely measuring the dominance of an academic institution in Twitter is not a comprehensive measure of the true worth of a tweet. Additionally, users in academic institutions are more likely to derive value from the quality of the content. Results of this study are limited due to the small sample size [
A range of online services, such as Klout!, PeerIndex, and Kred, attempt to measure influence in social media using various (undisclosed) algorithms and metrics; all are available free of charge. Klout! (
Similarly, PeerIndex (
These are a taste of the tools available to measure and examine impact in the social media and online world. Others exist including Twitter Grader and Social Bro. The main disadvantage with such tools is that they merely measure activity and engagement. However, central to an academic’s work are credibility and peer review.
In reducing impact to a quantitative, numerical score, it could be argued that bibliometrics are highly reductionistic and, when viewed in isolation, are not representative of a researcher’s performance or capacity. In taking this view, one would view bibliometric measures as only one aspect of performance upon which academic/scientific standing can be judged. However, bibliometrics have a high utility, and this is likely to continue because in pragmatic terms they represent a relatively simple, notwithstanding any weaknesses, and accurate data source.
As we have suggested earlier in this paper, there are various sources of bibliometric data, each with their strengths and limitations. What is needed is broad agreement on the most useful indices. Though bibliometric measures are best applied in a combination of methods of impact and esteem, other measures of these are far more difficult to quantify. Measures of esteem are defined as the recognition of researchers by peers, for their achievements, leadership, and contribution to a field of research [
Expanding access to research findings is paramount for scientific progress. Debate continues concerning the public’s right to access taxpayer, publically funded research findings. The National Institute of Health (NIH) in the Unites States now mandates that published results of all NIH funded research are archived in the National Library of Medicine’s PubMed Central, to be available to the public no later than 12 months after publication [
Open access journals are becoming increasingly in evidence, and their presence presents new options for scholars seeking to disseminate their work. Their open access status is a major advantage. The fact that these papers are widely (and freely) available should assist in ensuring optimal citations. A limitation of open access for readers however means that many of these journals charge authors a publishing fee. This fee is applied in addition to the usual stringent peer review process. A recent survey of peer-reviewed, English-language open access nursing journals (
Ranking universities as a single entity may not be the most appropriate way to identify where the best discipline-based research is performed, and it is unlikely that any single university will excel in all disciplinary areas. Therefore the ranking of disciplines (as independent entities) may have some broader utility, although an unintended consequence of this may be the stifling of interdisciplinary research, clearly an important goal within constrained funding environments.
Nurse and midwife researchers can no longer choose to avoid the process and politics of bibliometrics or measure of impact. The productivity and quality of research produced by individual researchers, research groups, and universities are an important metric of their success and contribution to the productivity of the economy. Despite the criticism and acknowledged weaknesses of bibliometric measures they form a vital function of this equation. Like most measures these indices should be scrutinised for validity and fitness for purpose. This will require ongoing development and evaluation on a regular basis as new opportunities emerge, particularly though online media.
The authors declare that they do not have any financial conflict of interests.