Full Citation
Title: Why summing up any bibliometric indicators does not justify a composite indicator
Citation Type: Miscellaneous
Publication Year: 2023
ISBN:
ISSN:
DOI: 10.31234/OSF.IO/UHD7M
NSFID:
PMCID:
PMID:
Abstract: Composites of bibliometric indicators received strong critique in the literature thus far. In this work, we thoroughly reinvestigate a composite proposed by Ioannidis et al. (2020). We rely on the congeneric measurement model (and related models) rooted in classical test theory from the psychometrics literature. We found that one of the proposed indicators clearly violated unidimensionality as a fundamental assumption of the congeneric model. In addition, two more indicators were excluded for redundancy. The newly proposed composite based on only three bibliometric indicators was found to display excellent reliability. Importantly, reliability was on par with reliability for a composite based on five indicators and clearly better than the originally proposed six-indicator composite. We further found rather homogeneous effective weights (i.e., relative contributions of each indicator to composite variance) for simple sum scores, which were pretty close to weights based on an algorithm for equally effective weights. Finally, an algorithm for reliability maximizing effective weights resulted in a composite that correlated perfectly with factor scores based on a congeneric measurement model. Strong measurement invariance of the congeneric model was found across sexes. Loadings and intercepts of the congeneric measurement model, however, were not found to be measurement invariant across scientific fields and academic age groups. Finally, various derived composites were found to correlate positively with academic age which hints at a lack of fairness of the composites. We discuss our findings towards practical applications and a more constructive discussion of bibliometric composites in the scientometric literature.
Url: https://psyarxiv.com/uhd7m/
User Submitted?: No
Authors: Forthmann, Boris; Doebler, Philipp; Mutz, Rüdiger
Publisher: PsyArXiv
Data Collections: IPUMS USA
Topics: Methodology and Data Collection
Countries: