How to Make a Bad Statistic

Thanks to sKIP for bringing to our attention the amazing claim that both Aarons, Judge and Hicks, and Giancarlo Stanton are all on the top ten of the "Least Clutch Superstars of 2018":

https://bleacherreport.com/articles/2792929-exposing-the-least-clutch-mlb-superstars-of-2018?utm_source=newsletter&utm_medium=newsletter&utm_campaign=mlb 

If you think this is in the realm of all those idiotic statistical analyses you used to read about how "Derek Jeter is the worst-fielding shortstop, ever!", well, you'd be wrong.

This is even more stupid.

But it does provide us with a great example of how truly bad statistics get made.  Let's take a look.

Start with nomenclature, often the hidden key to bad statistics.  For instance, Aaron Hicks is a superstar?  Since when?

But never mind.  Go to the bleacher report link that sKIP thoughtfully provided, and you find an article by one Zachary D. Rhymer that does, indeed, list those three Yankees amongst the 10 worst clutch hitters of 2018.  While Judge and Hicks rank No. 7 & 8, respectively, Stanton is No. 1—supposedly the very worst clutch hitter in the major leagues this season.

Surprised?  Just take a look on some of the other names on this list, who include the likes of Freddie Freeman, Jose Ramirez, and Mike Trout, who is No. 2.

Amazed that Mike Trout is the second-worst clutch player in the majors this year?  Join the club.

(Other names include the like of Mallex Smith.  Again:  "superstar"?  To quote Inigo Montoya: "I don't think that word means what you think it means.")

But wait!  Mr. Rhymer's article contains an even more shocking "fact":  Aaron Judge was "the worst clutch hitter ever" in 2017.

Say what?

I mean, I have to say, I have found our Jolly Giant to be rather disappointing in the clutch.  But...THE WORST CLUTCH HITTER EVER???

What about the fabled bo-bos of my youth?  Has Zachary Rhymer ever seen Charlie Smith play?  Or Jerry Kenney?  Aaron Judge in 2017 had a worse clutch season than Matty Alou, who hit .296 over 538 plate appearance in the no. 3 hole for the 1973 Yankees...and drove in just 28 runs?  Worse than that??

Again, I think we're talking nomenclature.  Just how does Mr. Rhymer and bleacher report define "clutch"?

He's good enough to tell us right up front that it comes from a statistic labeled, "Clutch" and created by the good folks at FanGraphs.  To quote Rhymer's description of this definition:

"...it uses the leverage index (which measures the pressure of various moments) to find "how much better or worse a player does in high leverage situations than he would have done in a context neutral environment." The higher the number, the better. The lower the number, the worse..."

Sound of needle scratching across vinyl.  Stop right there.

There is no such thing in baseball as a "context neutral environment."  All situations are different.  If they mean "all statistics combined, " they should say that.

Second, did you pick up how this is a comparative number?  There are certain obvious problems with that.

For instance, by this FanGraphs definition, if, say, Shane Robinson hits .156 overall and .136 in "high leverage situations," he is a better clutch hitter than someone who, let's say, hits .360 and "just" .300 in high leverage situations.

Also, what this means is, by definition, the MORE hits you get in NON-high leverage situations, the WORSE clutch hitter you become.

In other words, if Aaron Judge comes up with the Yanks leading, 10-5, in the 8th inning, and hits a two-run homer to make it 12-5—and the other team rallies to cut the final score to 12-11—Judge has not hit what turned out to be a badly needed home run to win the game.  He has actually LOWERED his "Clutch" by doing this in a non-clutch situation.

There are several other problems with "Clutch," including, according to Rhymer, how it treats walks. For instance, Rhymer notes that in 2018, Judge has been getting more walks in "high leverage situations":

"But while walks are nice, it would be better for the Yankees if Judge was able to consistently tap into his peerless power when the pressure's on." 

Sorry, Zach.  Walks are more than nice.  Walks are inevitable if the pitcher won't throw the ball to you  And walks won't get that pitcher out of trouble.

The same argument was made against Ted Williams, one of the very greatest hitters who ever lived:  he wasn't clutch, because he drew too many walks.  The whole idea was studied when teams were even walking Barry Bonds with the bases loaded, at the peak of his juiced-up home run spree, and the statistical analysis said otherwise:  you were still better off pitching to him.

Walks are not marginal.  According to the likes of Bill James and Allen Barra, the extra walks made Mickey Mantle a slightly better ballplayer than Willie Mays, at both players' peaks.

But let's get back to that whole, "Aaron Judge was the worst clutch player EVER" in 2017, the sort of pernicious nonsense that drives too much statistical analysis today.

Rhymer gives us his link to this FanGraphs article, from one Travis Sawchik (Not to be confused with Terry Sawchuk, the old Rangers goaltender, who despite his tragic early death some hockey statistician is probably "proving" right now was the worse clutch goalie ever.).

Once again: nomenclature.  Scroll down this piece a bit, and you get a serious caveat about "ever" from Mr. Saltlick:  "dating back to 1974, at least, the first year for which WPA data is available."

So, all that leaves out is the first 103 years of professional baseball, a fact I intend to take up in my forthcoming book, A Complete History of the United States of America.  From 1892.

After this, Mr. Sartorial starts to list a whole bunch of unusual suspects as some of the worst, EVER:  Jose Ramirez, again, but also Kris Bryant, Gary Carter, Bill Mueller, Dwight Evans, Dave Henderson, Sammy Sosa, Cecil Fielder, Ryne Sandberg, and A-Rod.

Well, okay, A-Rod.  But if you'll note, this list includes players who have compiled some of the biggest clutch hits in baseball history—Mueller, Henderson, Carter, etc.  That should give us pause right there.

But what about that "Judge worst clutch hitter ever ever ever in 2017"?  (I mean, 1994-2917.)

According to Mr. Sawmill River Expressway, that year Judge was "slightly below league average" in those high leverage situations.

Say what?  How exactly do you get from "slightly below league average" to "worst (1994-on-ish, etc.) ever?" 

Well, it gets back to the basic flaw in the statistic, which Mr. Salpeter admits: "when you set a really high overall performance standard and your performance is lagging in high-leverage situations, your “clutch” rating will take a hit."  He adds:

"Why are there so many 2017 players — and, really, 21st century players — on the list? It could be that they’re facing better relievers, being more effectively targeted and neutralized by opponents. It could also just be random. And high-leverage appearances often represent smaller samples of work, lending the results to randomness."

So, to paraphrase Joseph Welch, this statistic is a useful depiction of exactly nothing.