Google Scholar vs. Scopus & Web of Science

A couple of interesting correspondences (here and here) just appeared in Nature on the legitimacy of Google Scholar for tracking citations. Interesting because I’ve recently been pondering the same issue but came up with the opposite conclusion, namely that Google Scholar is actually a better tool for tracking citations than either Scopus or Web of Science (and not only because it’s a pain to access the latter two).

The main concern by the authors is that citations for a given article are higher in Google Scholar than Scopus or Web of Science (well, one mainly focusses on the fact that they’re different). Higher because they include theses, patents, websites etc. Maybe also false positives.

I started thinking about this issue when I began applying for jobs. I didn’t have any fancy papers and so, for people outside of my field, noting the number of citations in my CV seemed like a decent way to make the point that some of these articles have had an impact. As I sent my applications I’d go back to Web of Science and check for the latest numbers and, this past year, I noticed that citations of some of my articles suddenly increased by up to 20%. Yay! But then I’d check the next day and they were low again (perhaps relating to changes in how citations were detected). So there was some glitch and I certainly didn’t want to appear to be inflating the numbers so I turned to Google Scholar. The numbers were higher, but consistent. For the most part the citations seemed completely legitimate as well.

So why were Google Scholar’s citation counts higher? Looking at my most cited paper, which has been cited 367 times (Google Scholar) or 267 times (Web of Science) or 287 times (Scopus) I found that Google Scholar included 11 Chinese articles, 10 book chapters, 15 theses, 4 patents, 1 blog (yours truly), 1 grant application, and 6 mysteries. Eliminating these 48 still leaves 319. Quite a bit higher than Web of Science and Scopus, probably because Google counts citations from articles that are still in press (my Neurobiology of Aging paper was published online but “in press” for 23 months, during which citations could be tracked in Scholar but not Web of Science). This is probably also why Google Scholar counts 17 citations (16 “normal”) of my most recent paper whereas Web of Science only counts 9 – many of these citing articles were recently published.

So should Chinese articles be excluded? Are book chapters irrelevant? Theses, well, no one reads theses so maybe there’s a bit of inflation there. I do think it’s a sign of impact when a blog, grant, or patent refers to your paper and believe that these things should be included in the citation counts (Google still isn’t great in this regard – I know of several blog posts on my papers that haven’t been detected by Google).

These are the reasons I specifically decided to use Google Scholar when adding citation counts to my CV. And so it’s funny to read about the “limitations of Google Scholar’s personalized citation reports” and how “I would not recommend using the reports for decisions that could affect careers”.

(for what it’s worth I did get a job, though clearly it’s all downhill from here)