I’ll take that as an admission.
Take it however you want. You were smart enough to see that the sample was too small but not smart enough to identify that it applies to any other analysis.
Brian Smith's overall finals record is 15 wins from 28 games, a reasonable win rate in a reasonable sized sample. But that is close to the smallest sample that can tell us anything meaningful. Here's some mathematical evidence, shown by breaking Smith's win rate down into smaller (i.e. less meaningful samples):
| Brian Smith finals weeks | Games | Wins | Win rate |
|---|
| All | 28 | 15 | 54% |
| Weeks 1-2 | 17 | 12 | 71% |
| Weeks 3-4 | 11 | 3 | 27% |
The dopey dumb merkin assessment to draw from this is that Brian Smith was a week 3-4 choker but a week 1-2 specialist. It's f**king asinine. The better assessment is that 11 games is too small a sample, as it could easily be skewed by playing stronger opponents. In this case (and why I chose to split Smith's finals record this way) we know the opponents in week 3 and 4 are (in general) stronger than the week 1 and 2 opponents. So we know the explanation up front. But even if we didn't know the reason for supposedly significant results in this small sample of 11 games, the reason would still be there. It's not because Brian Smith is shit at coaching finals footy. If he was, he wouldn't have won half his 28 finals games. The 11 games in week 3-4 is skewed toward stronger opponents. Likewise Brad Arthur's 12 game finals record is skewed toward stronger opponents from week 1 through to week 4:
| Year | Finals week | Opponent | Eels differential | Opponent diff. | Relative diff. | Result |
|---|
| 2017 | 1 | Storm | +39 | +297 | -258 | L by 2 |
| 2017 | 2 | Cowboys | +39 | +24 | +15 | L by 8 |
| 2019 | 1 | Broncos | +60 | -57 | +117 | W by 58 |
| 2019 | 2 | Storm | +60 | +331 | -271 | L by 32 |
| 2020 | 1 | Storm | +104 | +258 | -154 | L by 12 |
| 2020 | 2 | Rabbitohs | +104 | +169 | -65 | L by 14 |
| 2021 | 1 | Knights | +109 | -143 | +252 | W by 8 |
| 2021 | 2 | Panthers | +109 | +390 | -281 | L by 2 |
| 2022 | 1 | Panthers | +119 | +306 | -187 | L by 19 |
| 2022 | 2 | Raiders | +119 | +63 | +56 | W by 36 |
| 2022 | 3 | Cowboys | +119 | +272 | -153 | W by 4 |
| 2022 | 4 | Penrith | +119 | +306 | -187 | L by 16 |
This shows, with a more granular data than just W/L, that we actually underperformed in one of our finals wins (vs Knights) while overperforming in a number of close losses. We also faced clearly superior opponents in 7 of 12 finals matches (for a 1 from 7 record). We were superior in only 2 of the 12 (both wins). Against relatively evenly matched opponents we were 1 from 3. However the main point is that even a sample of 12 games is too small to be meaningful. Brian Smith's finals record is proof of this.
This will remain the case for future Eels coaches. Ryles' first half season (coincidentally also 12 games) will tell us nothing about his coaching ability. Neither will his first 12 finals matches.
