August 1, 2014

Statistically Speaking: Bullpen Efficiency

The Washington Nationals bullpen as a unit are having a fantastic season in support of their more acclaimed starting rotation brethren. While the actual ranks differ by which all-encompssing statistic you prefer to use—the bullpen ranks tied for second in MLB with 2.8 wins above replacement (WAR) and fourth in RE24 at 27.53—the overall sentiment that the team’s relief corps is among the best in the business is not lost without the statistical confirmation.

It hasn’t been a smooth ride throughout the course of the season overall, with the likes of ever-dependable setup man Tyler Clippard and immensely talented former starter Ross Detwiler taking their lumps in the form of blown leads and inherited runners scoring. Yet, these shaky outings have been countered and exceeded by the efforts of Drew Storen, Rafael Soriano, and rookie Aaron Barrett, among others, and has kept the bullpen ledger in the black and the team in whispering distance of first place in the NL East.

Looking further at the polarizing outings of Clippard led me to come to this particular stat last week:

With the polarizing outings of Clippard to go along with the some similar clean outings by polarizing personality of Soriano, the Nats have a pair of relievers that face the minimum number of hitters half of their outings, which goes a long way to accruing the WAR and RE24 values the bullpen has thus far. It also speaks to how efficient the guys in the ‘pen are in getting hitters out and preventing the big inning for the opposing team. Do the rest of the Nats relievers follow suit and could this ability to keep additional runners (and potential runs) at bay be a reason for the success of 2014 from a group that hasn’t changed much in terms of roster from last year’s staff that finished 18th and 20th in MLB in WAR and RE24, respectively?

First, let’s outline what bullpen efficiency means. Efficiency is essentially how many batters a pitcher faces over the number that was expected from an outing. From there, we will also look at ‘clean outings’, where a pitcher faces the minimum number of batters for a given outing, with game situation considered. The fewer batters faced over the minimum, the better, as this obviously keeps runners off the base paths.

Let’s look at some data.

Name G IP xIP IP, Diff TBF xBF BF, Diff Efficiency(%) AppClean/Pct. RE24
Aaron Barrett 28 25.2 26.2 1 108 77 31 59.74 13/46.4% 2.77
Craig Stammen 22 38.1 39 0.2 152 115 37 67.83 6/27.3% 6.72
Drew Storen 29 24.1 26.2 2.1 93 73 20 72.80 16/55.2% 5.94
Jerry Blevins 33 27 29.1 2.1 116 81 35 56.80 16/48.5% 2.82
Rafael Soriano 31 31 31 0 114 93 21 77.42 17/54.8% 9.83
Ross Detwiler 20 29 31.2 2.2 137 87 50 42.53 4/20% -7.08
Tyler Clippard 37 34 36.1 2.1 137 102 35 65.70 19/51.4% 2.11


The table above is a little busy, but the explanations of the various columns are very straightforward and on the lighter side, mathematically. Aside from the standard games, innings pitched, and RE24 values, we also have a couple of variables that were calculated to help capture efficiency.

The first of these is expected innings pitched (xIP), which is the number of inning pitched that were expected from a pitcher, with game and outing specific information included. For example, if a pitcher has an outing where he pitched 0.2 IP, he could have an xIP of 0.2 if he came in relief with one out in the inning—he was only expected to get the other two outs to complete the inning.

Conversely, he could have a xIP of 1, but failed to get the third out of the inning before being pulled. Calculating xIP and confirming game situations was dine using game log data from Baseball Reference. Total batters faced (TBF) is simply that and expected batters faced (xBF) is calculated similar to xIP, with game situation taken into account. With xIP and xBF, care was taken with the Nats bullpen members who are more situational relivers, in particular, Jerry Blevins, to account for how they were pulled.

If they left an outing due to poor performance with runs scored or runners put in scoring position, then they were allotted the full inning of work expected and the batters faced. If they were pulled due to situation—bringing in Blevins to face a tough lefty, for example—then a full inning pitched was not assumed. Differences between actual performance and expected data re capture with the ‘Diff’ categories. From the game log data also comes the clean outing data (AppClean/Pct.), where the number of clean outings specific to game situation were tallied, with percentages also provided for comparison.

With the variables exhaustively described, let’s talk results. Not surprisingly, the Big Three of the Nats bullpen—Clippard, Soriano, and Storen—lead the way in clean outings, with Soriano and Storen also showing the most efficiency in terms of batters faced over the minimum (BF, Diff.).  Percent efficiency was calculated by taking the percentage difference between xBF and TBF and then subtracting this value from 100 and again shows how well both Soriano and Storen have been, not only in terms of performance, but in terms of being economical.

Not to be forgotten are the performances of Barrett and long man Craig Stammen, who both show a high rate of efficiency, despite subpar clean appearance numbers. Despite some encouraging recent outings, a very rough start to the season skews Ross Detwiler’s numbers greatly and shows a propensity for big innings and difficulties in keeping hitters off of the base paths.

Does this idea of efficiency trend with performance?

Screen Shot 2014-06-24 at 10.45.04 PM

In our very small sample, it indeed does, as the above graph of RE24 by number of batter faced of the minimum (BF, Diff in our table above) shows. As the number of extra hitters faced rises, RE24 drops, which makes this a negative correlation with a very strong R-squared of 0.72, providing us confirmation of good fit of the data. However, with seven data points, it would be very unwise to make any grand inferences out of these results. Despite this, we do see an interesting aspect of the bullpen’s success that doesn’t necessarily show up in the box score or in the formulas of the numerous advanced metrics available—not only are they keeping runs off of the scoreboard, they’re doing so in tidy fashion.

Data courtesy of FanGraphs and Baseball-Reference and current through 6/24/2014.

Stuart Wallace is a Contributor to District Sports Page. A neuroscientist by day, the Nevada native also moonlights as an Associate Managing Editor for Beyond the Box Score, stats intern at Baseball Prospectus, and a contributor at Camden Depot. A former pitcher, his brief career is sadly highlighted by giving up a lot of home runs to former National Johnny Estrada. You can follow him on Twitter @TClippardsSpecs.

%d bloggers like this: