Saturday, September 1, 2018

How to Make a Bad Statistic

Thanks to sKIP for bringing to our attention the amazing claim that both Aarons, Judge and Hicks, and Giancarlo Stanton are all on the top ten of the "Least Clutch Superstars of 2018":

https://bleacherreport.com/articles/2792929-exposing-the-least-clutch-mlb-superstars-of-2018?utm_source=newsletter&utm_medium=newsletter&utm_campaign=mlb 

If you think this is in the realm of all those idiotic statistical analyses you used to read about how "Derek Jeter is the worst-fielding shortstop, ever!", well, you'd be wrong.

This is even more stupid.

But it does provide us with a great example of how truly bad statistics get made.  Let's take a look.

Start with nomenclature, often the hidden key to bad statistics.  For instance, Aaron Hicks is a superstar?  Since when?

But never mind.  Go to the bleacher report link that sKIP thoughtfully provided, and you find an article by one Zachary D. Rhymer that does, indeed, list those three Yankees amongst the 10 worst clutch hitters of 2018.  While Judge and Hicks rank No. 7 & 8, respectively, Stanton is No. 1—supposedly the very worst clutch hitter in the major leagues this season.

Surprised?  Just take a look on some of the other names on this list, who include the likes of Freddie Freeman, Jose Ramirez, and Mike Trout, who is No. 2.

Amazed that Mike Trout is the second-worst clutch player in the majors this year?  Join the club.

(Other names include the like of Mallex Smith.  Again:  "superstar"?  To quote Inigo Montoya: "I don't think that word means what you think it means.")

But wait!  Mr. Rhymer's article contains an even more shocking "fact":  Aaron Judge was "the worst clutch hitter ever" in 2017.

Say what?

I mean, I have to say, I have found our Jolly Giant to be rather disappointing in the clutch.  But...THE WORST CLUTCH HITTER EVER???

What about the fabled bo-bos of my youth?  Has Zachary Rhymer ever seen Charlie Smith play?  Or Jerry Kenney?  Aaron Judge in 2017 had a worse clutch season than Matty Alou, who hit .296 over 538 plate appearance in the no. 3 hole for the 1973 Yankees...and drove in just 28 runs?  Worse than that??

Again, I think we're talking nomenclature.  Just how does Mr. Rhymer and bleacher report define "clutch"?

He's good enough to tell us right up front that it comes from a statistic labeled, "Clutch" and created by the good folks at FanGraphs.  To quote Rhymer's description of this definition:

"...it uses the leverage index (which measures the pressure of various moments) to find "how much better or worse a player does in high leverage situations than he would have done in a context neutral environment." The higher the number, the better. The lower the number, the worse..."

Sound of needle scratching across vinyl.  Stop right there.

There is no such thing in baseball as a "context neutral environment."  All situations are different.  If they mean "all statistics combined, " they should say that.

Second, did you pick up how this is a comparative number?  There are certain obvious problems with that.

For instance, by this FanGraphs definition, if, say, Shane Robinson hits .156 overall and .136 in "high leverage situations," he is a better clutch hitter than someone who, let's say, hits .360 and "just" .300 in high leverage situations.

Also, what this means is, by definition, the MORE hits you get in NON-high leverage situations, the WORSE clutch hitter you become.

In other words, if Aaron Judge comes up with the Yanks leading, 10-5, in the 8th inning, and hits a two-run homer to make it 12-5—and the other team rallies to cut the final score to 12-11—Judge has not hit what turned out to be a badly needed home run to win the game.  He has actually LOWERED his "Clutch" by doing this in a non-clutch situation.

There are several other problems with "Clutch," including, according to Rhymer, how it treats walks. For instance, Rhymer notes that in 2018, Judge has been getting more walks in "high leverage situations":

"But while walks are nice, it would be better for the Yankees if Judge was able to consistently tap into his peerless power when the pressure's on." 

Sorry, Zach.  Walks are more than nice.  Walks are inevitable if the pitcher won't throw the ball to you  And walks won't get that pitcher out of trouble.

The same argument was made against Ted Williams, one of the very greatest hitters who ever lived:  he wasn't clutch, because he drew too many walks.  The whole idea was studied when teams were even walking Barry Bonds with the bases loaded, at the peak of his juiced-up home run spree, and the statistical analysis said otherwise:  you were still better off pitching to him.

Walks are not marginal.  According to the likes of Bill James and Allen Barra, the extra walks made Mickey Mantle a slightly better ballplayer than Willie Mays, at both players' peaks.

But let's get back to that whole, "Aaron Judge was the worst clutch player EVER" in 2017, the sort of pernicious nonsense that drives too much statistical analysis today.

Rhymer gives us his link to this FanGraphs article, from one Travis Sawchik (Not to be confused with Terry Sawchuk, the old Rangers goaltender, who despite his tragic early death some hockey statistician is probably "proving" right now was the worse clutch goalie ever.).

Once again: nomenclature.  Scroll down this piece a bit, and you get a serious caveat about "ever" from Mr. Saltlick:  "dating back to 1974, at least, the first year for which WPA data is available."

So, all that leaves out is the first 103 years of professional baseball, a fact I intend to take up in my forthcoming book, A Complete History of the United States of America.  From 1892.

After this, Mr. Sartorial starts to list a whole bunch of unusual suspects as some of the worst, EVER:  Jose Ramirez, again, but also Kris Bryant, Gary Carter, Bill Mueller, Dwight Evans, Dave Henderson, Sammy Sosa, Cecil Fielder, Ryne Sandberg, and A-Rod.

Well, okay, A-Rod.  But if you'll note, this list includes players who have compiled some of the biggest clutch hits in baseball history—Mueller, Henderson, Carter, etc.  That should give us pause right there.

But what about that "Judge worst clutch hitter ever ever ever in 2017"?  (I mean, 1994-2917.)

According to Mr. Sawmill River Expressway, that year Judge was "slightly below league average" in those high leverage situations.

Say what?  How exactly do you get from "slightly below league average" to "worst (1994-on-ish, etc.) ever?" 

Well, it gets back to the basic flaw in the statistic, which Mr. Salpeter admits: "when you set a really high overall performance standard and your performance is lagging in high-leverage situations, your “clutch” rating will take a hit."  He adds:

"Why are there so many 2017 players — and, really, 21st century players — on the list? It could be that they’re facing better relievers, being more effectively targeted and neutralized by opponents. It could also just be random. And high-leverage appearances often represent smaller samples of work, lending the results to randomness."

So, to paraphrase Joseph Welch, this statistic is a useful depiction of exactly nothing.










15 comments:

  1. What a pile of bullshit. Do these guys have to fill space that badly?

    Great takedown, Hoss. Terrific work.

    ReplyDelete
  2. Thanks, Hoss.

    You should be the researcher for Cooperstown.

    I make a lot of outrageous comments, based on uncontrolled emotions. Your facts can correct them all.

    Gulp.

    ReplyDelete
  3. Two outs, bases loaded. First pitch pop up to first by Stanton. Maybe that clutch stuff isn't that far off.

    ReplyDelete
  4. Thanks, guys. I'm just spitballing out here like everyone else.

    And yeah, all that said...our clutch hitting still sucks!!!

    ReplyDelete
  5. The Professor be clutch! That was some fine pitching today, Tanaka-san.

    ReplyDelete
  6. He's had his issues, but I think Tanaka is the best starter we have. It's not even close now. One game playoff, I sure as hell hope Boone has the brains to start him.

    ReplyDelete
  7. Masahiro is making my bosom heave with delight!

    ReplyDelete

  8. I don't know if anyone else is listening to the game on the radio but I'll note for the historical record that John and Suzyn are really on their game today/tonight. Their spirits are up, their commentary is loose and flowing, and their insights are spot on.

    A true listening pleasure. I wish there was a way to tell them.


    ReplyDelete
  9. Wait, why would Bleacher Report publish something stupid?

    ReplyDelete
  10. Mustang,

    Can I have a hit of whatever you're smoking? I mean, it's legal here now...

    ReplyDelete
  11. AHH! AHHH!!! Aha-ah-haaaaa!!
    AAaAhhaaaa-haha-haaa-hhhhhHHHHHHHHaaaaAAHHHHHHHHHHHaaaa-ah-ah-ah-aaaaaaaaAaAaHaHaHaaaAAAAAAhhhhhhhhhhhh !!!!!

    ReplyDelete
  12. Snuck off with one today. What, two hits? But we put it in the win column.

    Been awhile since we did that.

    ReplyDelete

  13. I really want the world to know about this great man who brought back happiness into my life again after my husband left me and the kids 3 years ago for another women online when i contacted Dr Believe he cast a love spell for me within 48 hours my ex husband start calling me and begging for forgiveness for everything that have happened between us. I was so happy to have my family back together with love again here is the email of Dr Believe via believelovespelltemple@gmail.com a man with the great powers you can also call him or add him on Whats-app: +2348156148821
    God bless you
    I am very grateful for your help in my marriage.

    ReplyDelete

Members of the blog can comment. To receive an e-mailed invitation, write to johnandsuzyn@gmail.com. And check spam if it doesn't show up. (Google account required.)

Note: Only a member of this blog may post a comment.