{"id":208,"date":"2015-05-26T12:38:39","date_gmt":"2015-05-26T19:38:39","guid":{"rendered":"http:\/\/www.umpirebible.com\/ubBlog\/?p=208"},"modified":"2016-12-24T15:36:18","modified_gmt":"2016-12-24T23:36:18","slug":"calling-balls-strikes-the-matthew-effect","status":"publish","type":"post","link":"https:\/\/www.umpirebible.com\/ubBlog\/archives\/208","title":{"rendered":"Calling Balls & Strikes: The Matthew Effect"},"content":{"rendered":"
Jerry Kim<\/figcaption><\/figure>\n

There was quite a bit of chatter in the press last year (Spring of \u201914) about an academic study published in the journal Management Science<\/a><\/strong> entitled \u201cSeeing Stars: Matthew Effects and Status Bias in Major League Baseball<\/a>,\u201d by Jerry Kim and Brayden King. The study analyzes Pitch f\/x<\/a><\/strong> data for every pitch thrown in all of the 2008 and 2009 Major League games in which the batter did not swing. The data covers 4,914 games, 313,774 at-bats, and 756,848 pitches (non-swinging pitches only). That\u2019s over three-quarters of a million pitches.<\/p>\n

Braydon King<\/figcaption><\/figure>\n

Kim & King\u2019s study was not about calling balls and strikes. It focused instead on our\u00a0unconscious biases that come into play when we make judgments. They simply used three-quarters of a million ball\/strike decisions to test their hypothesis that subjective factors can affect these otherwise objective decisions.<\/p>\n

So they looked at Pitch f\/x data to see how often umpires mistakenly called a ball a strike and vice-versa. The error rate, according to their analysis, is 14.7%. So on nearly one in every eight pitches, MLB pitchers incorrectly undervalue a pitcher\u2019s performance (call a strike a ball) or overvalue his performance (call a ball a strike). When you consider that the average Major League game has close to 300 pitches, and if roughly half of those are called pitches, then we\u2019re talking about roughly 22 mistakes each game.<\/p>\n

But even that number becomes somewhat skewed, because (as we learn) baseball superstars (pitchers like Greg Maddux and Felix Hernandez, and hitters like Ted Williams and Pete Rose), what they call \u201chigh status players,\u201d are frequently the beneficiaries of these mistakes.<\/p>\n

These numbers \u2013 this error rate \u2013 seems awfully high. Umpire instructors typically say that if you miss four to six pitches then you\u2019ve had a pretty good game. These numbers, then, don\u2019t line up well with the impression among umpires of their own fallibility. That said, it\u2019s hard to argue with the data. Nevertheless, Kim and King are not the first to come up with this error rate for MLB umpires. A number of studies (I\u2019ll talk about some of them in future posts) have also arrived at an error rate between 14 and 15%, or roughly one in every seven or eight pitches.<\/p>\n

But Kim & King were interested in performance biases, not umpire error rates. So what they did next was to evaluate these roughly 216,000 umpire \u201cerrors\u201d and analyze how they correlate with various factors like right\/left-handedness, the race of the pitcher, whether home team or visitor, the current count, the stage of the game, and so forth. And this is where their study gets really interesting. Because the umpire error rate changes pretty significantly when some of these factors (but not all) come into play.<\/p>\n

What is the Matthew Effect?<\/h2>\n

Before going farther, let\u2019s ask, just what is the Matthew Effect and what\u2019s it got to do with crappy calls?<\/p>\n

In a nutshell, the Matthew Effect states that performers for whom we expect superior performance (an all-star pitcher, for example, or a league-leading batter) that they tend to be judged more favorably than performers for whom there is no such expectation. In short, we have an unconscious bias in favor of those who\u2019ve performed well historically. Sociologists refer to the Matthew Effect as enabling the sense that\u00a0\u201cthe rich get richer and the poor get poorer.\u201d Academics lament that the Matthew effect leads to judgments that good readers read well and poor readers struggle.<\/p>\n

While the Matthew Effect plays a large role in the fields of education and sociology, it has interesting applications in sports as well. Who, for example, hasn\u2019t complained that basketball superstars like Michael Jordan, Larry Bird, and Magic Johnson receive more favorable treatment than others on foul calls? Or that Greg Maddux and Randy Johnson got more strikes on the corner than, say, Danny Darwin or Bill Singer? (Remember them? Didn\u2019t think so.)<\/p>\n

Well, King and Kim set out to measure this Matthew Effect (what they call \u201cstatus bias\u201d) by analyzing the decision making of MLB umpires calling balls and strikes. In their own words, their goal was \u201cto observe the difference in a pitch\u2019s objective quality and in its perceived quality as judged by the umpire.\u201d Three-quarters of a million pitches later, they have some pretty interesting results.<\/p>\n

What did the study reveal?<\/h2>\n

The study revealed a lot. It revealed that there is an expectation that high status pitchers will throw more strikes and that high status hitters are better at evaluating the quality of a pitch as it approaches the plate. Of course, having that expectation is just common sense. We all have it, more or less. What\u2019s not common sense is what happens to our judgments (unconsciously) while swimming in the stew of our expectations. So here are a few of the study’s\u00a0more interesting outcomes.<\/p>\n

    \n
  1. They showed that the home team pitcher experiences a nearly eight percent advantage over visiting pitchers with respect to the \u201cover-recognition\u201d mistakes (that is, mistakenly calling a true ball a strike).<\/li>\n
  2. There are nearly five percent more over-recognition mistakes in the ninth inning than in the first. That\u2019s odd, isn\u2019t it? You would expect the opposite \u2013 that is, that accuracy would increase (and the error rate decrease) over the life of the game. But I suppose that’s why we collect the data.<\/li>\n
  3. Here\u2019s my favorite: Umpires are more accurate with right-handed batters than lefties. In fact, left-handed batters have a 41 percent greater likelihood of getting a mistaken call than right-handed batters. Of course, this high frequency of mistakes include both over-recognition errors (balls called strikes) as well as under-recognition errors (strikes called balls), so many of these (they don\u2019t say how many) cancel each other out. To me, though, it\u2019s incredibly interesting that there is such a vast difference here. We know from experience that the view for lefties is different from our view for righties, but I\u2019m surprised that the difference in the viewpoint results in such a large\u00a0difference in the error rate.<\/li>\n
  4. No surprise here: The closer the pitch to the edge of the zone, the greater the likelihood of a mistake. We all know that. And I\u2019ll have more to say about this in the next section of this post.<\/li>\n
  5. Here\u2019s where it gets good (and this shouldn\u2019t be surprising, though many of us will deny it): \u00a0The count (balls and strikes) at the time of a given pitch has a huge influence on the error rate. The odds of mistakenly calling a strike is 62 percent lower with an 0-2 count; with a 3-0 count, on the other hand, the likelihood of getting a mistaken strike call is 49 percent higher. This certainly validates that an 0-2 zone shrinks, while a 3-0 zone expands.<\/li>\n
  6. Several pitcher-related factors influence the error rate, including the number of years a player has been in the major leagues and the reputation for control (or wildness) for a given pitcher. (For control\/wildness, the researchers used a pitcher’s base-on-balls percentage.) A pitcher\u2019s ethnicity, it turned out, has no effect on the error rate, but pitcher\u2019s status (measured arbitrarily by the number of All-Star game appearances) had a very definite impact. Pitcher\u2019s with five All-Star appearances enjoyed a 15 percent greater chance of getting mistaken strike calls (over the baseline), and a 16 percent advantage over non-All-Star pitchers in getting such favorable calls.<\/li>\n
  7. The \u201cunder-recognition\u201d errors (that is, a true strike called a ball) follows a similar pattern. High-status pitchers are less likely to experience these errors. While the baseline across all pitchers showed the under-recognition error rate of about 19 percent. For pitchers with five All-Star appearances, this drops two full percentage points. That\u2019s not a huge amount \u2013 that is, until you count each pitcher\u2019s total number of pitches in a season; then do the math.<\/li>\n
  8. The Matthew Effect works for batters, too. Again, using the number of All-Star appearances as a proxy for status, high-status batters get a 1.3% bump for each All-Star appearance in both types of error. The two error types combine for a nearly three percent advantage for each of a batter\u2019s All-Star appearances. So a five-time All-Star has (statistically) a nearly 15% advantage in combined under- and over-recognition errors. Of course, a high-status batter facing a high-status pitcher (who is getting his own favorable errors) is going to have some of this effect nullified.<\/li>\n
  9. Finally, Kim and King analyzed error rates for 81 MLB umpires that called more than 1,500 pitches over the two seasons and (as you\u2019d expect) detected patterns for given umpires. The image below is taken directly from the article and is, unfortunately, difficult to read. But it nevertheless shows that MLB umpires vary noticeably one from another. The predominant tendency (shown in the upper left quadrant) is to over-recognize high status while at the same time under-recognizing low status. That\u2019s tough on the rookies. Only a handful of MLB umpires over-recognize low status.<\/li>\n<\/ol>\n

    So what does this all mean?<\/h2>\n

    Your initial reaction to all of this information might be a rather jaw-dropping recognition that umpires are really bad at calling balls and strikes. That was my first reaction \u2026 and that reaction was the impetus for my deciding to write this post: To basically cop to our fallibility behind the plate. Because if the pros are missing one in eight pitches, what does that suggest about the rest of us? I\u2019m suddenly very grateful that Pitch f\/x is not installed at my fields.<\/p>\n

    But in the roughly two weeks that I\u2019ve been working on this post, I\u2019ve softened. And my softening has to do with item #4, above: That the closer the pitch is to the strike zone, the higher the likelihood of an error. In other words, it\u2019s all about the corners.<\/p>\n

    In their study, the researchers used a variable they called \u201cdistance,\u201d which is measure of the number of inches from the border of the strike zone of a given pitch. They go on to report that \u201c\u2026the relationship between distance and over-recognition exhibits a non-linear relationship with a rapid decline in the odds of mistake as distance increases\u201d (p. 27). What they mean by \u201cnon-linear relationship\u201d is that the error rate does not decline gradually as the distance from the edge of the zone increases. The relationship is not proportional. Rather, the error rate changes very rapidly as the distance begins to increase. This means that there is a measurable error rate within an inch or two right at the corners, but then the errors decline very significantly as you get to three and four inches from the corner.<\/p>\n

    And what does that<\/strong><\/em> mean? Well, it means what we all know already \u2013 that the corners are soft. Pitches off the plate are pretty easy to call. Pitches in the meat of the zone are pretty easy to call. And all of us get almost all of those right almost all\u00a0of the time. But the corners are different, especially the pitches that hit two corners (down and in, down and up, up and away, low and away). In fact, I would guess (the data doesn\u2019t tell us) that most of the under-recognition errors (true strikes called balls) are at the four corners, where the strike zone gets rounded just a tiny bit.<\/p>\n

    Here\u2019s another look at Pitch f\/x data on MLB batters. This is from The Hardball Times<\/a><\/strong> and this article (written by Jon Roegele) focuses on the expanding strike zone (particularly at the bottom). But for our purposes the point is the obvious rounding of the zone at the four corners. That, as you can plainly see, is where a great many (probably most) of the under-recognition errors occur. Even the \u201cexpanded\u201d 2014 strike zone shows the rounding effect.<\/p>\n

    So what about the over-recognition errors? Where are those errors coming from? Well, I can\u2019t say for certain, but I suspect they\u2019re on the outside of the zone, a ball to a ball-and-a-half out, roughly\u00a0six to eight inches above and below the vertical center of the zone. In other words, right around the belt, but just off the plate.<\/p>\n

    So the MLB umpires we watch and model on aren\u2019t so damn bad after all \u2014 or so I say.<\/p>\n

     <\/p>\n

    \"\"<\/a><\/p>\n","protected":false},"excerpt":{"rendered":"

    There was quite a bit of chatter in the press last year (Spring of \u201914) about an academic study published in the journal Management Science entitled \u201cSeeing Stars: Matthew Effects and Status Bias in Major League Baseball,\u201d by Jerry Kim and Brayden King. The study analyzes Pitch f\/x data for every pitch thrown in all […]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"jetpack_post_was_ever_published":false,"_jetpack_newsletter_access":"","_jetpack_dont_email_post_to_subs":true,"_jetpack_newsletter_tier_id":0,"_jetpack_memberships_contains_paywalled_content":false,"_jetpack_memberships_contains_paid_content":false,"footnotes":"","jetpack_publicize_message":"","jetpack_publicize_feature_enabled":true,"jetpack_social_post_already_shared":false,"jetpack_social_options":{"image_generator_settings":{"template":"highway","default_image_id":0,"font":"","enabled":false},"version":2}},"categories":[3,1],"tags":[],"class_list":["post-208","post","type-post","status-publish","format-standard","hentry","category-balls-and-strikes","category-uncategorized"],"jetpack_publicize_connections":[],"jetpack_featured_media_url":"","jetpack_sharing_enabled":true,"jetpack_shortlink":"https:\/\/wp.me\/p8epXH-3m","jetpack-related-posts":[],"_links":{"self":[{"href":"https:\/\/www.umpirebible.com\/ubBlog\/wp-json\/wp\/v2\/posts\/208","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.umpirebible.com\/ubBlog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.umpirebible.com\/ubBlog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.umpirebible.com\/ubBlog\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.umpirebible.com\/ubBlog\/wp-json\/wp\/v2\/comments?post=208"}],"version-history":[{"count":8,"href":"https:\/\/www.umpirebible.com\/ubBlog\/wp-json\/wp\/v2\/posts\/208\/revisions"}],"predecessor-version":[{"id":220,"href":"https:\/\/www.umpirebible.com\/ubBlog\/wp-json\/wp\/v2\/posts\/208\/revisions\/220"}],"wp:attachment":[{"href":"https:\/\/www.umpirebible.com\/ubBlog\/wp-json\/wp\/v2\/media?parent=208"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.umpirebible.com\/ubBlog\/wp-json\/wp\/v2\/categories?post=208"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.umpirebible.com\/ubBlog\/wp-json\/wp\/v2\/tags?post=208"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}