I saw this video shared in other forum topics and I thought it might help here more than anything:
This video shows a lot of data presentation and interpretation that might be helpful for discussion. It undoubtedly has some really good data presented, but I am going to suggest in my opinion some glaring points we need to be on the lookout for when we are presented with such things.
Two glaring things in particular really,
the first is this 'noise' thing that the presenter suggests and shows this graph:
This is a good graph but not for what he says it is good for, for example, any football fan/manager knows that we cannot really figure out what the team is like until we are like 10-12 games into the season and if we could predict positions with such accuracy, we'd have no shirt sponsor. And when he says that the prediction gets better with data as the season progresses, it might be true but not based on the presented data. Like, you have 38 games, and if you have to predict the final position at the 37th vs 15th, that is going to be wildddd. You can't say on the last day of the season, 'Hey, with the data I have, I am now able to 'predict' the table with 100% accuracy.'. This would also be very different for 'relegation fights', 'title race', and 'midtable battles'.
the second is the way 'overperformance' is portrayed;
This essentially says, that if your target is 35 points and you have 1 point, you need 34 points to reach your target in different ways. Essentially it is their version of 'we need 4 out of the next 9 points' argument, but they present instantaneous measures as something that stretched over the season to inflate the actual impact of the statistic. A thumb rule should be to not blindly trust statisticians who use more numbers to explain something fewer numbers can explain.
They claim that their model can predict the league winners every year, which just means they will predict Man City will win every year and if Arsenal or Liverpool win, the next year they will go we can now 'predict' why they won the title last year, which is boonnnnnkkeerrrrrrrssss.
Here is a very good example of how data presentation can be simple but really impactful->
https://projects.fivethirtyeight.com/epl-consistency-2023/