Training Camp - What I'm Watching For

[HardRightEdge]

Two things. One, pressures are a better indicator than sacks of performance in pass rush. If you don't believe that then you must believe that Matthews had his worst season in 2011 because he only had 6 sacks (had 43 hurries and 22 hits though). Two, Raji graded out positively in pass rush in 2010 (he was negative in run defense) and 2012, his 2011 was terrible. Should be noted that Raji's pass rush grade was less than half of Mike Neal's in 2012.

As for using PFF stats and grades; I don't know of any other site that grades every player on every snap in an objective fashion. The stats they provide are valuable because they actually give context to a player's performance. I watch the Packers A LOT...I don't watch other teams that often. I can't say how good a safety Burnett is because I don't watch every other safety in the NFL. PFF allows me to compare Burnett to other safeties. Any problem that they incorporate should also be occurring with other players in the league since their system is pretty much uniform. It may not be perfect but everyone player should receive the same problems, so they should wash out (not perfectly but enough to minimize issues).

I understand that's why such stats are popular. But that doesn't respond to the questions I raised, specifically the value of "consequential pressure" vs. the other kind. As noted, I don't limit value to sacks. I like hits a lot, rushed passes that result in interceptions even more, and QBs throwing off balance incomplete passes are another happy event. But if all you're doing is pressuring the QB to step up in the pocket, there are several QBs in this league who will eat you alive.

PFF may allow you to compare players, but how reliable is it, really? You assume their "system is pretty much uniform", but how do you know that? Many of those stats involve subjective judgements by, as I noted, unpaid volunteers. How consistent are the judgement calls by hundreds(?) of volunteers? They ask for volunteer commitments of about 10 hours per week...that would suggest several people each breaking down some limited aspects of each game. That's a lot of cats to herd in a largely subjective endeavor.

And when we follow these stats, to what degree are they black boxes? What is PFF's definition of a "pressure", for example? If you find one I'd like to see it. Maybe you need to be a paid subscriber to get full access to methodology and interpretive guidelines...I couldn't say...maybe not even then?

Team grades are a lot more valuable...they're developed by professionals who know what the player's assignment is on each play. That might not be so clear to PFF volunteers. Who blew that coverage? Who missed that blitz pick-up? Etc., etc. Unfortunately, teams don't regularly or fully disclose that info...we get a few glimpses here and there. But if PFF data is what's available, that doesn't mean it is valuable if it happens to be inconsistent or otherwise flawed.

This is heresy in the fantasy football age, but no stat is better than an inconsistent, inaccurate, misleading or un-vetted stat.

This is not a blanket indictment of PFF. I like the stuff that is measurable. To take one example, the time-in-pocket data recently discussed in this forum raises some interesting questions. But that requires only a stop watch and a definition, such as time in the tackle box. But even then, what is THEIR definition of "time in the pocket"? What do they tell their herd of cats to look for? That's relevant.

 

[Sunshinepacker]

Not sure you have an accurate idea of what PFF does. You keep saying volunteers, implying that anybody can run stats for them. In fact, the analysts are paid positions. Granted, they aren't all full time analysts, but they are paid for their work (and one would assme that only the good ones get full-time jobs). PFF is very upfront that they don't evaluate scheme, they assume that the players know the plays and they simply evaluate the outcome.

As for your comment on a QB steeping up to avoid a rush, why should the player who dlineman who beat his guy not get credit for a pressure just because the DT got knocked on his butt, leaving room for the QB to step up? Pressure is pressure. It doesn't have to adversely affect the QB because other players have to be involved as well. If Neal pressures Stafford but Raji is five-yards downfield after getiing mauled by the center, why shouldn't Neal get a postiive grade? It's not his fault Raji stunk on the play (disclaimer, this is a made up situation, so don't ask me when it happened).

Finally, I think you should give more credit to the "herd of cats" as you call them. There are millions of football fans in the world. that means that even with a few hundred volnteers (which, btw, i think is a massive overestimate) PFF can still afford to be very picky. The site is refernced often by ESPN, Peter King and many others, including players. They must be doing something right.

 

[FrankRizzo]

Only problem with the Starks/Neal comparison is that Neal actually brings something valuable relative to the other dlinemen. Starks, even when healthy, has never really done anything special outside of the playoff game against the Eagles. Neal on the other hand is actually a pretty good pass rusher, and that's extremely valuable from a 3-4 guy.

Neal was pretty good last year when on the field, definitely our best DL pass rusher.
Datone should surpass him there this year..... can we keep both of them healthy, get 28-30 games outta the 2 of them?

 

[HardRightEdge]

Not sure you have an accurate idea of what PFF does. You keep saying volunteers, implying that anybody can run stats for them. In fact, the analysts are paid positions. Granted, they aren't all full time analysts, but they are paid for their work (and one would assme that only the good ones get full-time jobs). PFF is very upfront that they don't evaluate scheme, they assume that the players know the plays and they simply evaluate the outcome.

As for your comment on a QB steeping up to avoid a rush, why should the player who dlineman who beat his guy not get credit for a pressure just because the DT got knocked on his butt, leaving room for the QB to step up? Pressure is pressure. It doesn't have to adversely affect the QB because other players have to be involved as well. If Neal pressures Stafford but Raji is five-yards downfield after getiing mauled by the center, why shouldn't Neal get a postiive grade? It's not his fault Raji stunk on the play (disclaimer, this is a made up situation, so don't ask me when it happened).

Finally, I think you should give more credit to the "herd of cats" as you call them. There are millions of football fans in the world. that means that even with a few hundred volnteers (which, btw, i think is a massive overestimate) PFF can still afford to be very picky. The site is refernced often by ESPN, Peter King and many others, including players. They must be doing something right.

Last year PFF had an open call for volunteer applications on their web site, referencing the 10 hour commitment. I've not checked to see if it's still there. These are part time jobs for hobbiests, presumably with some kind of credentials. Maybe there's a path to full time work; I can't say. Even if all of the "analysts" are highly credentialed, there can be wide disparities in interpretation when it comes to judgement calls.

Here's an analogy. The baseball strike zone is clearly defined by rule. MLB umpires are highly trained, working years in the minors before making the bigs. Yet, there remains some wide disparities in how umpires call balls and strikes. It's not a matter of experience, pay or guidelines...it's a problem of variability in human judgement. MLB periodically tries to enforce consistency, but the practitioners always stray. MLB umpires are also a herd of cats.

Does PFF have the guys (or gals) work the same teams every week? We don't know, do we? That could lead to some wide inconsistencies in season long aggregate numbers for obvious reasons.

As far as PFF analyzing outcome, not scheme, I endorse that. But answer me this, Batman...when you see a CB expecting S help get burned over the top, followed by the CB and S jibber jabbering about it in the end zone (we saw several Packer instances in 2011), who gets charged for the TD? The coaches judge who should have been where and when, even if there are instances where it's a matter of some debate. Discussion with the players, exploring what they were thinking, plays into it. Part time analysts are in the dark.

Yes, lots of media people quote PFF stats. People pay for the expanded data. But who's audited them? I've yet to see anybody analyze their process. Maybe that's not even possible; PFF may maintain proprietary non-disclosure to even paid subscribers. I can't say, but if that's the case we're looking at black box modeling.

As for your example, maybe teams let Neal ride around the pocket, if he ever does that, because they know Raji sucks, if he in fact does. Maybe they plan to have the QB step up because they expect little resistance up the middle, and have no problem yielding a pressure. I think if one is going to wade into these murky waters one might as well look at results...does the pressure make the QB do something negative.

Like I said, I like PFF's work when it comes to things that are measurable and require little judgement. What troubles me is the elevation of these judgemental stats to the level of sabermetrics. I read several of Bill James early Baseball Abstracts. The formulas were spelled out using measurable data. Rationales and conclusions were thoroughly explored. This is not to say James was infallible...but you knew what you were looking at.

So, I go back to some simple questions...how does PFF define a "pressure"; what are the instructions handed to analysts to guide judgements; what measures are taken to assure consistency of interpretation? And in then end, you still have that MLB umpire problem that you just cannot get around.

 

[Sunshinepacker]

We'll simply agree to disagree. The issues that you have with PFF stem from the fact that multiple people must make judgement calls on subjective stats. That's not going to change. I can't think of a way that one person could audit every player of every game, so there will always be differences. It's the best available data that we have. When something better comes along, we'll use that. In the mean time, PFF provides the best, most accurate way for the normal fan to understand how a player they're interested in is doing compared to the rest of the NFL.

 

[Oshkoshpackfan]

BTW, a group of cats is not called a herd, they are called a "clowder of cats". lol......just saying.

 

[13 Times Champs]

BTW, a group of cats is not called a herd, they are called a "clowder of cats". lol......just saying.

So it is.

You must be logged in to see this image or video!

 

[Oshkoshpackfan]

Damn 13, you are Johnny on the spot with pics lol

 

[HardRightEdge]

BTW, a group of cats is not called a herd, they are called a "clowder of cats". lol......just saying.

Or a "glaring" of cats. You can glare at them, but they just won't clouder for you. Of course if they're lions, I'm proud to say that's a pride. Just a little more off topic amusement for you "blush" of boys:

http://oxforddictionaries.com/us/words/what-do-you-call-a-group-of

 

[El Guapo]

Starks adds absolutely nothing unique to our stable of running backs.

http://espn.go.com/blog/nfcnorth/post/_/id/58592/little-sorted-out-yet-in-packers-backfield

Van Pelt called Starks’ camp so far “outstanding” and said the 6-foot-2, 218-pound fourth-year pro is running “extremely hard, violent.” He noted that Starks is making better reads and has been more patient on the outside zone runs, where the back needs to stretch the defense before making a cut. It’s a play coach Mike McCarthy often calls.

I'm still looking for a similar report regarding Neal

 

[Oshkoshpackfan]

http://espn.go.com/blog/nfcnorth/post/_/id/58592/little-sorted-out-yet-in-packers-backfield
I'm still looking for a similar report regarding Neal

All that may be true.....so far..... in camp.....just wait till he gets turf toe and has to sit on the pine and get paid. He has been a virtual grab bag assortment of injuries and he needs to take the dirt road IMO.

 

[13 Times Champs]

Damn 13, you are Johnny on the spot with pics lol

You must be logged in to see this image or video!

 

[HyponGrey]

http://espn.go.com/blog/nfcnorth/post/_/id/58592/little-sorted-out-yet-in-packers-backfield
I'm still looking for a similar report regarding Neal

You'll have to wait til he gets over that ab injury.

 

[easyk83]

I understand that's why such stats are popular. But that doesn't respond to the questions I raised, specifically the value of "consequential pressure" vs. the other kind. As noted, I don't limit value to sacks. I like hits a lot, rushed passes that result in interceptions even more, and QBs throwing off balance incomplete passes are another happy event. But if all you're doing is pressuring the QB to step up in the pocket, there are several QBs in this league who will eat you alive.

PFF may allow you to compare players, but how reliable is it, really? You assume their "system is pretty much uniform", but how do you know that? Many of those stats involve subjective judgements by, as I noted, unpaid volunteers. How consistent are the judgement calls by hundreds(?) of volunteers? They ask for volunteer commitments of about 10 hours per week...that would suggest several people each breaking down some limited aspects of each game. That's a lot of cats to herd in a largely subjective endeavor.

And when we follow these stats, to what degree are they black boxes? What is PFF's definition of a "pressure", for example? If you find one I'd like to see it. Maybe you need to be a paid subscriber to get full access to methodology and interpretive guidelines...I couldn't say...maybe not even then?

Team grades are a lot more valuable...they're developed by professionals who know what the player's assignment is on each play. That might not be so clear to PFF volunteers. Who blew that coverage? Who missed that blitz pick-up? Etc., etc. Unfortunately, teams don't regularly or fully disclose that info...we get a few glimpses here and there. But if PFF data is what's available, that doesn't mean it is valuable if it happens to be inconsistent or otherwise flawed.

This is heresy in the fantasy football age, but no stat is better than an inconsistent, inaccurate, misleading or un-vetted stat.

This is not a blanket indictment of PFF. I like the stuff that is measurable. To take one example, the time-in-pocket data recently discussed in this forum raises some interesting questions. But that requires only a stop watch and a definition, such as time in the tackle box. But even then, what is THEIR definition of "time in the pocket"? What do they tell their herd of cats to look for? That's relevant.

Outstanding post, I wish I could have provided additional positive ratings.

 

[HardRightEdge]

We'll simply agree to disagree. The issues that you have with PFF stem from the fact that multiple people must make judgement calls on subjective stats. That's not going to change. I can't think of a way that one person could audit every player of every game, so there will always be differences. It's the best available data that we have. When something better comes along, we'll use that. In the mean time, PFF provides the best, most accurate way for the normal fan to understand how a player they're interested in is doing compared to the rest of the NFL.

As I suggested to another poster, which he did not get, telling somebody what they agree to is a false authority. The only way you could enforce that "agreement" is if you followed that statement with silence. You didn't do that.

I can only repeat, no data is better than bad data. Unvetted data is a crap shoot. Oh, I don't doubt the proliferation of unvetted subjective stats will continue apace on the path to fantasizing every aspect of the game. That doesn't mean they are related to winning and losing football games.

I could give you some details about how the Wall Street "fantasy finance" risk models came to be so consistent across the industry, born out of group think and woefully flawed judgements, but it would be either boring or unbelievable, besides being off topic.

 

[HardRightEdge]

Outstanding post, I wish I could have provided additional positive ratings.

Thanks, man. I appreciate that.

 

[jaybadger82]

Got it- the pressures statistics on PFF are not adequately vetted for your taste and, based on certain assumptions, you think the numbers are misleading. (I found the "consequential pressures" suggestion annoying since it presents the same vetting and judgment problems as the statistic you criticize in the first place.)

As a general observation, deconstructing a statistic is pretty easy. I could sit here and lecture everyone about the problems with the way the Bureau of Economic Analysis calculates GDP or the Bureau of Labor Statistics calculates unemployment, but that doesn't make these statistics valueless as a tool for measurement.

Stats in football will always be flawed. There's relatively little data to collect over a sixteen-game season and it's a team sport with enough moving parts that it's difficult to isolate individual players (outside of film). Many of the numbers contain subjective elements. But that doesn't make a statistic like pressures worthless, either. IMO, organizations wouldn't track pressures if they weren't useful for evaluating disruptive impact on the defensive front (granted, their criteria may differ from that of PFF).

I'm not particularly interested in defending PFF. I don't know what procedures they use to ensure the quality of their data. But if the information they're selling is really worthless (re: crap), it seems like the market would've rooted that out some time ago.

 

[HardRightEdge]

Got it- the pressures statistics on PFF are not adequately vetted for your taste and, based on certain assumptions, you think the numbers are misleading. (I found the "consequential pressures" suggestion annoying since it presents the same vetting and judgment problems as the statistic you criticize in the first place.)

As a general observation, deconstructing a statistic is pretty easy. I could sit here and lecture everyone about the problems with the way the Bureau of Economic Analysis calculates GDP or the Bureau of Labor Statistics calculates unemployment, but that doesn't make these statistics valueless as a tool for measurement.

Stats in football will always be flawed. There's relatively little data to collect over a sixteen-game season and it's a team sport with enough moving parts that it's difficult to isolate individual players (outside of film). Many of the numbers contain subjective elements. But that doesn't make a statistic like pressures worthless, either. IMO, organizations wouldn't track pressures if they weren't useful for evaluating disruptive impact on the defensive front (granted, their criteria may differ from that of PFF).

I'm not particularly interested in defending PFF.

It's not a matter of "taste" and I've made no assumptions on that matter. If you can show me how PFF determines what a "pressure" is, the guidelines used making these judgement calls, and the quality controls assuring the guidelines are followed, I'd be happy to see it.

If there's any assumption being made it's these subjective stats purveyors MUST be doing it right otherwise Peter King or agents or whoever would not be quoting them. Big echo chambers built on sand are not that uncommon. Should I talk about Wall Street risk modeling after all? A key risk modeling tool widely used on Wall Street before the calamity was developed by Morgan Stanley and subsequent shared with fellow firms...oh, never mind.

I never said "pressures" as presented by PFF is a "worthless" stat. I said it is suspect. You don't really know what you're looking at. If you want to take it on faith that's your business. I would point out that the only way you could identify flaws in a government statistic is by knowing how that statistic is calculated, which you can know because it is publicly available. See the point here?

I mentioned "consequential" pressures to make the point that if you're going to all that trouble to stat-ify (to coin a term) subjective things requiring judgments, you might as well measure things that matter. I was making a multi-level argument, despite your annoyance. Meaningful football stats are based on outcomes. If that were not the case, we might as well add dropped balls into the QB's completion total with an estimate of forgone yards gained.

Yes, STATS tracks pressures just as PPF does. They both think it's important but they cannot agree on what a on the numbers which suggests there are disparities in the definition or applicaiton. You see this particularly in something as simple as "drop" stats where there are meaningful disparities between the two.

 

[Sunshinepacker]

As I suggested to another poster, which he did not get, telling somebody what they agree to is a false authority. The only way you could enforce that "agreement" is if you followed that statement with silence. You didn't do that.

I can only repeat, no data is better than bad data. Unvetted data is a crap shoot. Oh, I don't doubt the proliferation of unvetted subjective stats will continue apace on the path to fantasizing every aspect of the game. That doesn't mean they are related to winning and losing football games.

I could give you some details about how the Wall Street "fantasy finance" risk models came to be so consistent across the industry, born out of group think and woefully flawed judgements, but it would be either boring or unbelievable, besides being off topic.

Huh. "Agree to disagree" is a polite fiction commonly used to allow two parties to walk away from a discussion that is obviously not going to be solved. It's not meant to be analyzed. The alternative would be me simply stating that I think I'm right and you're wrong; a statement which is both confrontational and somewhat redundant considering the discussion we were having.

As for no data being better than bad data. I completely agree. I simply disagree that data which is imperfect is automatically "bad". Just because the data isn't perfect doesn't make it worthless. You feel that some data is misleading, the example that you give being pressures which are not always recorded in a uniform fashion. The data would only be bad if it was unrelated to the phenomenon being observed. That's not the case. Unless the analysts are being willfully misleading the number of subjective calls that you question won't be that significant. I would venture that a substantial majority of hurries are pretty cut-and-dry. I would also venture that the number of blown coverages which result in a player getting a positive/negative grade that they don't deserve is relatively small; the reason being that if players blow coverage that much, the coaches will stop playing them. Coaches like players that know their assigments so, it would follow, that plays are being run as the coach wants the majority of the time.

Now, having said all that, would it have been so bad to simply accept that we disagree on the point? I mean, it was obvious from our discussion that we disagreed. So unless you contend that we don't disagree, simply pointing out the obvious is not false authority.

 

[jaybadger82]

This isn't Wall Street modeling or GDP formulation. Pressures are observable, fairly discrete occurrences on the football field that are simply being tallied. While I might disagree with the assignment of a pressure here and there (whatever the criteria), by and large I think they provide a useful tool for evaluating a player's disruptive impact on passing downs relative to his peers.

I can appreciate that the criteria for pressures likely varies between databases like Stats and PFF and this probably makes comparisons between these resources problematic. However, comparisons within the same statistical ecosystem should not pose a problem since- theoretically- players are being evaluated using the same guidelines. Unless there's specific reason to doubt the accuracy of the data being compiled on these sites, I don't see reason for "suspicion."

I mentioned "consequential" pressures to make the point that if you're going to all that trouble to stat-ify (to coin a term) subjective things requiring judgments, you might as well measure things that matter.

There's already an objective measure for what matters called wins and losses. Stats like pressures just help us understand how we got there and, IMO, it's best to keep the criteria for these sort of metrics as simple and discrete as possible in order to curb subjectivity.

By the way, this thread:

You must be logged in to see this image or video!

 

[HardRightEdge]

[quote="jaybadger82, post: 505866, member: 6211"Pressures are observable, fairly discrete occurrences on the football field that are simply being tallied


By the way, this thread:

You must be logged in to see this image or video!

[/quote]

Suit yourself. Now state the definition of a "pressure".

 

[HardRightEdge]

Huh. "Agree to disagree" is a polite fiction commonly used to allow two parties to walk away from a discussion...

Right. And you didn't. There was also the problem of using "must agree".

We agree on one thing...it's a fiction.

 

[13 Times Champs]

This has boiled down to disagree to agree.

 

[I asked LT to delete my acct]

BuggyBill would be proud of your spelling of "Offence" and "Defence".

Bite me !!

 

[DevilDon]

I was all over drafting a big fat guy for the D=Line and didn't think DatOne was big enough. Since the draft and all the analysis of scouts I'm most excited by his progression and development and possible impact on the D-line.
Obviously the RB group will be fun to watch and is probably the best positional battle. I have the idea that whoever they keep this is going to be the year the Packers add a running game to one of the best offenses in the NFL. That's exciting!
I can only see a weakness in the MLB spot but if the DL improves that should help there. The Packers sure seem loaded across the board.