Over at Baseball Prospectus writer Sam Miller took on the meaning behind top 101 prospects dropping in their rankings in Pebble Hunting: The Importance of Top Prospect Trajectories ($). There were a few noted sample size issues, but the idea was interesting. The concept is especially interesting to the Padres who, as mentioned in the article, had 2 prospects that fell off their top 101 prospects rankings from 2013 to 2014. Casey Kelly and Rymer Liriano both went from being ranking in their top 101 to no longer being ranked at all.
As an initial way of trying to illustrate what a drop like that could mean, Miller went back to the first two top prospect rankings the site ever did (2007 and 2008). Between those two years the following players fell off the rankings just like Kelly and Liriano:
Jeff Niemann, Philip Humber, Brandon Erbe, Chuck Lofgren, Donnie Veal, Billy Rowell, Troy Patton, Trevor Crowe, Sean West, Humberto Sanchez, Ryan Tucker, Ryan Sweeney, Will Inman, Matt Harrison, Glen Perkins, Sean Gallagher, Brad Lincoln, Sean Rodriguez, Cesar Carrillo, Chris Parmelee, Dellin Betances, Brett Sinkbeil, Cedric Hunter, Pedro Beato, Javier Herrera, Kyle Drabek.
I think even the casual fan would be able to tell you that is not a list you want to be on. There is one current closer (Perkins), a former opening day starter (Harrison) and a host of others ranging from complete bust to damaged goods to guys with uninspiring major league trials to sometimes useful utility players. It also includes 3 of the Padres' own prospects that never panned out (Inman, Carrillo and Hunter). As I mentioned earlier this is a small sample size, but it is one that makes you wonder if there is a trend here.
Since guys that fall off the rankings are, by definition, no longer ranked Miller took issue with trying to assess how much value is lost by falling off the rankings. Some tumblers may have simply become the 102nd best prospect and others the 1500th. So what he went on to analyze were players that dropped significantly in the rankings from one year to the next, but still were ranked in the top 101. He compared these prospects eventual value (using their player value stat WARP) to prospects that ranked similarly, but did not also tumble that year.
The results showed that the prospects that dropped probably should have been dropped even farther. Their value by WARP ended up lower than the prospects that they were now supposed to be similarly ranked. A similar finding held true when the author went back even further than 2007-2008 and used Baseball America's rankings from a few different season pairs (2003-2004, 2000-2001 and 1995-1996).
This does not say anything conclusive about Kelly or Liriano, and perhaps it bodes well that their drops were due to injury instead of production, but it does make you wonder about their futures. Judging by this data, however it would still seem that if you were a betting man you would not wager on either or especially both having long successful big league careers. However, if you were to write off every such player you would miss out on gems like Adam Wainwright, Carlos Gomez and Michael Cuddyer and solid contributors like Dexter Fowler, Danys Baez, Dustin Hermanson and LaTroy Hawkins.