Why you should look suspiciously at Google Scholar citation rankings: an ongoing saga

I previously discussed the oddities of Google Scholar in 2011 and 2015.  Another four years have passed, and yet...

1) Something there is that doesn't love a (moving) wall.  As I noted in a Twitter conversation a few weeks ago, it has become clear that GS scrapes a lot of its data from JSTOR.  But JSTOR's moving wall means that there is at least a three-year delay on journal issues becoming accessible in its database.  And, as it turns out, the knock-on effect for our purposes is that it can take a similar three years (or more) for citations to appear in your profile.  (One reference to Book Two from 2015, for example, cropped up only this year.) 

2) When Springer and Palgrave Macmillan attack! Books published by Palgrave Macmillan and available through Springer show up in GS as both individual titles and individual chapters.  As a result, GS inadvertently generates duplicate citations--an article with six citations may actually only have one.  

3) GoogleBooks blues.  Not surprisingly, one of GS' primary sources for book citations is GoogleBooks.  But this means that GS replicates all of the notorious metadata errors in GB.  Moreover, if, as is sometimes the case, there are multiple versions of the same book in the GB archive in slightly different formats, GS will simply pick them up as different citations.

4) Book review aggregation.  GS seems to have a hard time with the publication format of book reviews, as it is often the case that one ends and another begins on the same page.  It thus may count a review twice--once individually and once as part of a larger set of reviews treated as a single article.  (Moreover, it will do this both with reviews you have written and with reviews of your work.)

5) Amazingly, there are academics on this planet who do not write in English.  GS' access to non-Anglophone scholarship seems to be...scattershot.

My advice remains as it has been for the last eight years: anyone using the numbers for purposes of hiring, tenure, and promotion (or polemic, for that matter) needs to treat them as approximate, at best, and proceed with due caution.