Lies, lies, damn lies and statistics

[jonkool]

Sending offs distort the stats over a short number of games - the sending off of Novak so early in week 1 resulted in us being far less attacking which reduced our chances and increased the opposition's.

The effect will be diluted after 10 games rovidibg we avoid further red cards of course. If we continue to play as we have started our chances stats are going to go through the roof!!

[seriouslyred]

Sept 3, 2017 14:33:15 GMT jonkool said:

Sending offs distort the stats over a short number of games - the sending off of Novak so early in week 1 resulted in us being far less attacking which reduced our chances and increased the opposition's.

The effect will be diluted after 10 games rovidibg we avoid further red cards of course. If we continue to play as we have started our chances stats are going to go through the roof!!

Totally agree, and the sending off yesterday changed the game in our favour, thus changing the stats. So ten or 11 games is a good shout to limit the influence of outliers. The main point about these stats and and the model and scatter graphs which the author generates is that they are based upon the quality of the chances created / concede rather than actual goals.

And this shapes an envelope of expected outcomes which rules out the extremes such as relegation or how the club might spend £3M on a couple of players to "guarantee" promotion. And secondly it shows just how much improvement we have compared to last season without the randomness of referee decisions and other factors. And how much further we have to go.

Nothing about sofas, ex players etc. Just 100% football data showing us where we are.

Perhaps the owner wants to see real improvement BEFORE he invests in more players. That is somewhat counter intuitive to the typical fan view who always wants money spent last month. But perhaps the management team has to deliver first? Just a thought - hope it's not too revolutionary?!

[jonkool]

No it's not - the owner and SMT are clearly invested into Robbo and his decision to bring in LB & JJ as coaches. Heis even more invested into our Academy set up that cranks out more and more first team squad potential.

At the same time they are correct in putting a cap on the budget and it's a bit rich when the anti regime fraternity slaughter the owner for not funding a £1M striker when he is funding what we believe to be a £1M per month deficit.

Then it's more than a bit rich when the owner is accused of running a player farm business model when Daniel Levy has been doing this over many years and has developed and sold on quality young players which has resulted in the Spuds coming close to winning the Prem and building a 60000+ stadia.

We are on the right road and so long as we keep creating the chances that we have so far then the owner is correct in holding back in order to stake stock in January

[Mundell]

Thought I'd update this thread following yesterday's impressive victory against Southend. Usual caveats apply. My own personal perspective is that we shouldn't ignore data of this type because it might be telling us something, but nor should we see it as 'the truth, the whole truth and nothing but the truth'. In any event, it's very early days yet.

Anyway, here are Experimental 3-6-1's timelines from yesterday's League One fixtures. Based on this analysis our game against Southend was another very close match which might have gone either way. This has been a familiar pattern in each of our six games to date.

That conclusion is confirmed by this scatter plot (You need to scroll down a little to the charts and click on the one top right) which shows 'expected goals' for and against for all teams in Division One. This chart tells us that we are currently ranked 11th on 'expected goal difference' with attack being our relative strength (8th) and defence a relative weakness (14th). This is, potentially, a cause for concern and, at a minimum, suggests that there is no room for complacency.

We may well need to improve if we are retain a top six position, i.e. it may not be enough to simply keep playing as we are, somewhat counterintuitive though that may seem. My own view, for what that's worth, is that we are probably capable of improving upfront though that may require some better decision making by the front four, perhaps from Holmes and Fosu in particular, but that the defence is more of a concern.

Lies, lies and all that, but we haven't had a single game where our expected goals against has been as low as the season averages for Doncaster, Fleetwood and Wigan. There appear to be several sides in League One this season that are currently much harder to create good chances against than we are. Tuesday night against Wigan will be very interesting.

 ... and finally, evidence that we are still creating chances for Josh Magennis and that his conversion rate is at least as good as average. 

[seriouslyred]

Thanks for the link and comment Mundell

What that appears to show is that we have improved our expected goals scored over last season and yet our defence is about the same at 1.4 goals per game or right in the middle of the division. So to ensure success, we need more improvement on the chances created AND to make a difference at the back.

In our defence, we have had Magennis missing for two games due to injury and call ups so perhaps the stats will improve of their own accord? And we do have Marshall and Reeves in the wings trying to gain match fitness. These stats will develop over the next five to ten games and we will soon see if we can stay in the top six.

Given that this side has really only played together since April is it reasonable to suggest that there should be a distinct improvement as they play more games and the coaches get stuck into more training around positioning and precision? We have seen the development of Holmes and Magennis as Charlton players over the last 14 months - perhaps the same applies to the more recent arrivals and thus the performances improve.

One strange factor around yesterday's game is that Southend had just one chance on target and yet their stats suggest that they created more? So maybe we need to see more about how effective the attacking players are in front of goal to give us a better view of how competitive Charlton really are.

[seriouslyred]

Whilst we await the findings for yesterday by experimental 361, some might be interested in this piece from a site called stats bomb:
statsbomb.com/2017/09/inferior-tactical-strategies-the-bizarro-world-of-brentford/
In it the author is trying to understand why Brentford are so poor this season. Low and behold, loadsa chances but taken from way outside the box so the expected goals (xG) is way down compared to the past.
I can't get my mitts on the relevant data for Charlton but suspect that many of our 122 chances this season have had a very low possibility of ending in the back of the net.

[reamsofverse]

Because we have one striker and midfielders who are not natural goalscorers.

[Mundell]

Sept 17, 2017 9:45:59 GMT seriouslyred said:

Whilst we await the findings for yesterday by experimental 361, some might be interested in this piece from a site called stats bomb:
statsbomb.com/2017/09/inferior-tactical-strategies-the-bizarro-world-of-brentford/
In it the author is trying to understand why Brentford are so poor this season. Low and behold, loadsa chances but taken from way outside the box so the expected goals (xG) is way down compared to the past.
I can't get my mitts on the relevant data for Charlton but suspect that many of our 122 chances this season have had a very low possibility of ending in the back of the net.

Thanks for posting seriouslyred , that's very interesting. You have regularly commented on the number of shots we've had, both on target and off target, suggesting that this is an improvement on last season and, hence, a positive sign of improvement. However, we know from the Experimental 3-6-1 data that our 'expected goals' so far this season is not much better than average and, perhaps surprisingly, not that different to last season. Here is a quote from a comment I posted earlier on this thread;

Based on this analysis our game against Southend was another very close match which might have gone either way. This has been a familiar pattern in each of our six games to date.

That conclusion is confirmed by this scatter plot which shows 'expected goals' for and against for all teams in Division One. This chart tells us that we are currently ranked 11th on 'expected goal difference' with attack being our relative strength (8th) and defence a relative weakness (14th). This is, potentially, a cause for concern and, at a minimum, suggests that there is no room for complacency.

We may well need to improve if we are retain a top six position, i.e. it may not be enough to simply keep playing as we are, somewhat counterintuitive though that may seem.

This was posted before the game against Wigan Athletic. It might be argued that we have now been 'found out', i.e. by Wigan and then Gillingham, though I'd like to reserve judgement on the Gillingham game since I wasn't there, but an equally plausible explanation is that we weren't that good to start with, i.e. that we had won more points than our performances warranted.

Returning to the apparent disconnect between the number of shots and expected goals, there were only ever two possible explanations. Either Experimental 3-6-1's data and model for expected goals were misleading (i.e. wrong) or, alternatively, many of our shots had a below average expected goal value, i.e. we have been shooting from too far out or from wide positions where the likelihood of scoring is low. There has never been any doubt in my mind which was the more likely explanation. It does indeed appear that we have the 'Brentford problem', though I agree it would be interesting to get the data to prove it.

If, like me, you are interested in this stuff, the content on the 'statsbomb' website is both impressive and fascinating, but I think there is a risk that the author is oversimplifying. It's all very well to say 'let's get into better positions before we shoot', but the challenge is to work those better openings and that's obviously not easy. Our problem might well be that the front four aren't sufficiently capable of opening up defences to create lots of good chances (more expected goals) so that, as result, we are taking shots from distance instead. There is a clear trade off here and perhaps what we are doing makes sense given the players we've got, but if so its going to mean that focusing simply on the number of shots we are taking will flatter any assessment of how we are doing.

Its now clear that we need to get Mark Marshall fit and then to hope that he can be a bit more incisive. When both Marshall and Ben Reeves are available Karl Robinson is going to have a dilemma. Who to leave out? I suspect that most people would expect the choice to be between Reeves and Billy Clarke with Fosu missing out completely, but for me Ricky Holmes is a conundrum. He works tirelessly and often pulls off the spectacular, but he plays with his head down, rarely 'sees a pass' and doesn't help to get other people into the game. In my view, for what that's worth, it is by no means certain that as a team we'd be any less effective without him. I'm sure many would disagree with that assessment though. 

Either way, if we are going to make the playoffs we need to improve. Despite our five wins in eight, what we've produced so far probably won't be good enough, not least since, as I noted above, our expected goals against is even more of a concern. Karl Robinson has a lot to think about. Perhaps the most important thing is that he is brave enough to recognise there is a problem and doesn't simply believe we've been unlucky. 

[Mundell]

Funny old game. Here are Experimental's timelines for yesterday's League One fixtures. This would suggest that we should have won our game at Gillingham comfortably. It appears that we were the strongest side from start to finish. Paradoxically, our ability to create 'good chances' in this match is probably quite encouraging. 

[seriouslyred]

Sept 17, 2017 13:34:38 GMT Mundell said:

Funny old game. Here are Experimental's timelines for yesterday's League One fixtures. This would suggest that we should have won our game at Gillingham comfortably. It appears that we were the strongest side from start to finish. Paradoxically, our ability to create 'good chances' in this match is probably quite encouraging. 

Thanks for the link. Some points to note on those graphs - Wigan look like they will walk this league for they scored three again and deserved three. Conversely Shrewsbury appear to be jammy and may slip up soon whereas Southend continue to outplay sides but fail to win very often. They are a decent side.

When the author of that website has a few more games captured he will probably start sharing graphics on the number of chances and quality of finishing. And the percentage chances of how sides might finish the season.

In relation to your earlier post we are centainly up on chances created this season with an average of 15 as opposed to 11 or 12 last time. And we appear to be in violent agreement that the finishing needs to improve. Marshall and Reeves should help that process as they are certainly top six players from past seasons.

But is is our ability to contain the oppo which remains in question. More coaching plus the side playing more together is one possible answer but an additional defensive coach and some more quality in the January window maybe the way to enhance our chances of a top six finish. So again we agree that continuing as we are is not the solution.

What I won't do is criticise the squad nor the last window for it has taken us some of the way from where we were 12 months ago towards a really competitive side. As long as the coaching and the next window continue that trend then the journey will be relatively smooth with the occasional bump.

For what it's worth, I think we will most probably win next week but that changes nothing about the fact that we are not yet one of the six best sides in the division. In two weeks we start a run playing the better sides - the performances and stats will tell us all how we match up and whether this might be a good season.

[Mundell]

Sept 17, 2017 14:02:58 GMT seriouslyred said:

Sept 17, 2017 13:34:38 GMT Mundell said:

Funny old game. Here are Experimental's timelines for yesterday's League One fixtures. This would suggest that we should have won our game at Gillingham comfortably. It appears that we were the strongest side from start to finish. Paradoxically, our ability to create 'good chances' in this match is probably quite encouraging. 

In relation to your earlier post we are certainly up on chances created this season with an average of 15 as opposed to 11 or 12 last time. And we appear to be in violent agreement that the finishing needs to improve. Marshall and Reeves should help that process as they are certainly top six players from past seasons.

Don't want to appear pedantic seriouslyred , but I think the point of the @statsbomb piece you quoted earlier is that it is important to distinguish between shots and chances. We may be having more shots this season than last, but that's not quite the same thing as saying we are creating more chances. I'm not sure that our finishing is the problem, at least not prior to yesterday.   

This post from Experimental 3-6-1, which he calls 'Form tables', covers our last six matches up to, but not including yesterday's match against Gillingham, and gives an indication of how we and other clubs are faring (you need to scroll down to League One). As you say, Wigan's form appears to be outstanding and a cut above the rest of the division. Rotherham and Oxford also look strong. Blackburn less so, encouragingly. Most of our games could have gone either way, with the exception of Tuesday's defeat against Wigan.  

[seriouslyred]

Taking the two points separately, in the distribution there is often one or two clubs way out in front. Last season it was Sheffield United. And they are followed by another four or five.

We are in the top six in the league table but the stats indicate that we will drop out sooner or later unless we improve. For our stats do not match the leading clubs.

On your other point, as per "statsbomb", never mind the quantity, feel the quality! We are increasing the number of chances and the expected goals by about 20% each. So that implies the quality of chances is staying the same, just more of them.

Which makes sense when one considers the personnel: Holmes and Magennis are the same as last season while we have second choices Clarke and Fosu.

Therefore if the club and KR have made the right decisions over the summer then we might expect further improvements in expected goals scored.

Conceding chances is another discussion altogether.

[Mundell]

 .... and here are the latest scatter graphics for League One, summarising the season to date. Early days yet and individual matches can have a disproportionate impact on the averages, but some clear themes are beginning to emerge. Our attack isn't amongst the best in the division, but its not bad. The real weakness appears, perhaps surprisingly, to be at the back. We are conceding too many good shooting opportunities. If we continue like this we might score more goals than last season, but we'll probably also concede more. The net result, by my reckoning, would be an 'expected goal difference' of around what was achieved last season, if current trends continue.

It is early days though and its worth noting that the pattern of our fixtures has been quite skewed. We've already played against the four worst defences, based on their expected goals against (Gillingham, Oldham, Northampton and Bristol Rovers) and the three best attacks, based on their expected goals for (Wigan, Southend and Rotherham).

There is a long way to go yet. 

[newyorkaddick]

I wonder if the absence of any draws this season has negatively skewed our view of how things have gone (especially off the back of two defeats).

We have actually gone WLWWWWLL but had we gone say WDDWDWDD then we'd be unbeaten and probably feeling very bullish yet with one fewer point.

[Mundell]

For those who are interested in Experimental 3-6-1's analysis of how we are performing and how League One is shaping up I thought I'd post the latest update.

All of the usual health warnings apply, but with the season now ten games in a number of patterns are beginning to emerge. 

1. Based on this analysis we have one of the better attacks in the League, with only Shrewsbury, Rotherham, Peterborough, Southend and Wigan appearing to be more dangerous going forward. Indeed, only Peterborough have had more shots per game than us and this might suggest, as some have been arguing, that we have been creating lots of chances, but that our finishing isn't quite good enough. However, the evidence demonstrates that this is probably the wrong conclusion since we appear to be scoring at around the level predicted by our 'expected goals', i.e. our conversion rate is around average.

2. A comparison with Wigan's attack is tells us a bit more about how we compare with the very best in the League. Wigan are having about the same number of shots per game as we are, but their 'expected goals' per game is significantly higher. Anecdotally, this says that while we are taking 'wild long shots' Wigan are creating much better openings so that when they shoot they are more likely to score. Ironically, despite this, Wigan have actually only scored two more goals than we have suggesting, perhaps surprisingly, that our finishing has been better than theirs has.

3. I suspect that these observations fit with what many of us have said based on our own 'eye witness' reports. We look capable of scoring goals, but would be more dangerous and could score more goals with a bit more patience, better decision making and more precision in the top third. It may well be that a Marshall for Fosu switch will help to deliver that. More generally, with Marshall and Reeves yet to join the squad we can probably be confident that our attack is good enough to get us into the top six, all other things being equal.

4. Unfortunately, all other things are not equal because our defence is closer to bottom six than top six. Infact, only four teams have faced more shots against per match than we have and only six have a higher 'expected goals against'. It is perhaps no more than good fortune that we haven't conceded even more goals. Only Fleetwood in the top half of the table have actually conceded more goals than us, but they have 'suffered' a higher conversion rate against and appear to be a little harder (or 'less easy') to penetrate than we are. 

5. This has to be a major cause for concern and it means that our 'expected goals difference' is barely top half of the table. Last night's match at Walsall was a classic example of both our strength and our problem. We scored two goals away from home (despite having an expected goals for of only 1.2, i.e. our conversion rate was high), but conceded a deserved equaliser (Walsall had an expected goals for of 1.6). We simply have to do better defensively if we are going to finish in the top six. 

6. Again, the contrast with Wigan is striking and alarming. Their expected goals against in less than 0.9 per game and they've only actually conceded six goals. They played us off the park in the recent game at the Valley, running out 3-0 winners, of course, but its the ability to prevent the opposition creating good chances which is the biggest difference between the two sides. 

7. Finally, its interesting to reflect on our fixture list to date. Based on Experimental's analysis we've already played the four weakest sides in the League, but taken just six points from those four matches. Over an entire season that ratio would leave us with just sixty nine points and short of the playoffs. We've also played three of the better sides, losing to Wigan, but beating both Rotherham and Southend. Unfortunately, Experimental's data suggests that those two wins were a little against the odds, i.e. we could just as easily have lost. Our matches against the three 'slightly below par' sides we've played (Bristol Rovers, Bury and Walsall) have yielded five points which is obviously better, but still not great. 

As I've said on a number of occasions when posting on this thread, I hope I'm wrong because, unfortunately, the evidence appears to suggest that we have a lot to do if we are to finish in the top six. In particular, we need to do much better defensively. I suspect that our defensive problems are not about the back four per se, but even if that's true the injury to Jason Pearce couldn't have come at a worse time.      

[seriouslyred]

We are certainly creating more chances than last season and we are scoring more often. But the quality of the chances has not yet improved.

The latest graphic from experimental 361 hopefully illustrates the point I have been trying to make:
experimental361.com/2017/09/27/attack-breakdowns-league-1-27-sep-2017/
Magennis and Holmes are leading the way on actual goal while Magennis has a very high expected goals per game, i.e., the quality of the chances falling to him.

So if Reeves and Marshall are as good as their reputations and can match Holmes then our expected goals scored will continue to increase.

As you state Mundell Wigan are way ahead. And CAFC has some ground to make up before we consistently challenge the top six. We might over perform the stats but the real answer is to create better chances and/or address the issue below.

Meanwhile our defence is about the same as last season and it isn't going to improve with Pearce out... Analysis of how we are conceding so many chances required before any further comment - we can all see the goals at Walsall.


So here's to a 3-2 win on Saturday!

[seriouslyred]

Two weeks later and the latest scatter graphs from experimental 361 clearly show:
1) we are in the top six for creating chances and converting
2) our defending is distinctly mid-table
3) we are better than last season but there are no guarantees that we will stay in the top six

The die is cast until the next window opens and we will soon see whether our position on these graphs covering expected goals scored conceded moves closer to our league position, or not. Will our defensive capability improve as the back four and midfield become more familiar with the formation? And we will see whether Fosu's conversion from creator to precise finisher is a one-off or not.

Robinson has spoken of bringing in two more strikers in January. For now our fate rests with Magennis maintaining form and fitness, plus the six(!) attacking midfielders creating and converting chances.

Another welcome change is the consistency in selection. Squad usage shows that we have the most consistend first XI selection. Top teams have a higher number of minutes played by the most used players simply because they have less injuries and the managers will keep a winning team together.

But we haven't always been a winning team. Fans were quite critical when we went four games without a win and yet KR stayed with the same players. Did he have much of a choice? More importantly, did he need a choice?

This second chart is perhaps what Charlton fans have been looking for after a couple of years of turbulence: consistent team selection and the ability to walk down to the Valley next Saturday in the knowledge that one can predict ten of the starting XI. When we win it's a bonus... when we see a similar but better side than last season then that's progress even if we are not yet the finished article.

[Mundell]

Thanks for posting seriouslyred As ever, Ben Mayhew's graphics are interesting and insightful.

The good thing about data and analytics, provided the 'model' works of course, is that the conclusions are not subject to the behavioural and emotional biases our intuitive judgements are vulnerable to. Models can provide a useful, objective analysis of whatever it is we might be interested in. In this case, Charlton's prospects for the remainder of the season.

The bad news, on the other hand, is that while we can always construct a positive, optimistic narrative if left to our own devices, our model won't always tell us what we want to hear. It won't allow us to simply brush the negatives aside.

When our model tells us something we don't like, we often argue that it's wrong or, more subtly, that there are special factors at work which mean that, on this occasion, we don't need to be concerned about its results and conclusions.

That's probably the best strategy here because based on Experimental 3-6-1's analysis we appear to be an upper mid table side with a slightly better than average attack. The question, of course, is whether that's the right conclusion or, if not, what might the model be missing, other than the fact that we'd prefer it to be more optimistic?

Unfortunately, I have no real answer to that question, but there is, perhaps, good reason to believe that League One is going to be especially competitive this season and that, partly for this reason, performances and results haven't yet settled down. The good news here, supporting this hypothesis, is that Experimental's model has had a very mixed start to the season. The four worst teams, based on 'expected goal difference', are all in the bottom six, but while that's fairly impressive the rankings at the business end of the table are all over the 'scatter graph'.

Wigan, who completely outplayed us at the Valley, of course, appear to be the best side in the League by some margin, but the sides in first (Shrewsbury) and third (Bradford) places appear to be in false positions. If the model is right about them they'll both experience a disappointing run of results at some stage. On the other hand, Doncaster and Southend would appear to be much better than their results suggest. Indeed, Saturday's game against Doncaster should be very interesting; their expected goals, for and against, suggest that they are as strong as we are in attack, but much better in defence. We'll see.

So what can we conclude from the latest scatter graphs? First, unless something significant changes, Wigan will run away with the League. Second, second place is up for grabs with no obvious favourite at this stage. That said, based on an analysis of recent form (which I've linked below), Blackburn and Rotherham are probably best placed at this stage to secure the second automatic promotion spot, though there is a long way to go yet. Third, the four playoff slots are wide open, with perhaps as many as eight to ten teams in the mix. We're one of those. Let's hope we can maintain our challenge.

Recent form in League One

[seriouslyred]

That makes much sense Mundell as well as triggering some thoughts around models, optimism and where CAFC might be going.

We are 1/4 of the way through the season. After 23 games, everybody will have played everybody else and we will have had a chance to watch our side against our direct rivals. These are the six pointers which will define where we are on New Year's Eve and may even impact the club approach to the next window? These type of games last season against Millwall and Peterborough showed us how far short we were of a real challenge. Doncaster will be tricky opponents and the next chapter in this encounter.

At this time I would concur with the bookies in that we are a 50:50 shot for the top six. The stats suggest that one or more of our top six rivals might teach us a lesson. Conversely confidence and fitness together with the same 14/15 players representing us every week may lead to improved performances?

In all honesty, models like these will never be very successful at predicting individual results. But one would expect them to have a good success rate when predicting league position - perhaps a 75% correlation? At some point Ben Mayhew normally releases a different model which illustrates the chances of a top six finish - This link shows a summary throughout last season of how various clubs moved from a wide range of outcomes to a final position.

One can see how we had just a 10% chance of making the playoffs last January which quickly became a 90% chance of finishing in the bottom half. Contrast that with the likes of Millwall who had a consistent team selection throughout and perhaps we can see what it takes to finish sixth. Are we as good as Millwall were this time last year? I doubt it but it can be argued that we are at least halfway on the journey. The league table says we are there. The stats suggest that our attack is there, but...

On optimism: We are better than this time 12 months ago and we are going in the right direction. Stating that should not be confused with optimism nor entitlement. There is no certainty as to whether we will finish in the top six and we have no devine right to be promoted. Ultimately we need to concede less chances, less goals and the finishing and decision making of the three attacking midfielders needs to be more precise.

The difference today is that the manager has been given the playing budget and has shown that he is good at recruiting the right players. And he has just flagged up that he will be "shopping" for strikers in January. That doesn't mean that every signing is an unqualified success nor does it imply that we will get through the season without serious injuries. But we are going well so far and we have two players who are experienced top six performaers back in contention. The art of the possible if you will!

Depending upon your definition of upper mid-table, that is entirely consistent with my earlier description that we are better than last season (mid-table). And we are top six in the league table but certainly not on the defensive side of things. Do we improve defensively or just aim to outscore the opposition?!

Our "special factor" is that we have won ten in 16 league games but one must caveat this by stating that these have been against lower half opposition in the main. So we can say that we have the next 12 games to prove ourselves and that they are an entirely different proposition. Last season's table suggests that we need 37 or 38 points to be sixth at the half way mark. So are we up for 1.5 points per game? Forgetting draws, can we win every other game with this team and those stats?

Add to that the fact that we have used a core of 14 or 15 players with the minimum of changes every week and it is rational to presume we will continue as we are with the distinct possibility of some improvement as the team gels... and we sign more talent in January. That's not optimism, it is simply lining up a path to success which is possible.

And I should add that it is also a path towards RD selling up for if he funds the right players and KR gets it right then the value of the club goes up exponentially. It might even approach the £50M+ which RD has paid / loaned into CAFC? Again not a prediction, just a statement to suggest that straightening out the squad and management is a precursor to attracting decent bids.

I mention this because it is the antithesis of the CARD agenda. For their rationale is that encouraging commercial and season ticket boycotts along with expecting a decline in performance will somehow lead to RD giving up and selling CAFC on the cheap. In other words they expect somebody worth €750M(?) to write off £50M because it suits them.

In summary we are heading in the right direction but the next few games and the January window will tell us if we have real chances and if the owner is in a hurry to get us back to the Championship. Should experimental 361 release a model of expected finish then we might be able to discuss whether that calebrates with our view. As well as looking at what is needed to enhance our chances. And that is perhaps what is at the core of what I write?

What is great is that we watch the same games and we can look at the same data / presentations. And yet we can all discuss and dispute the performances and the outcomes with sobriety or passion. The models simply put an envelope around expectations - they might add precision to the conversation rather than bring any certainty.

But what we don't need to do is predict disaster to make our point nor delve into pejorative language about the management and the board to make our voice heard.

And finally, if anyone reached this far in another ramble, then thanks!

[stevebrown]

Oct 11, 2017 12:41:06 GMT seriouslyred said:

Are we as good as Millwall were this time last year ? I doubt it ..... 

Millwall were 19th at the start of October last season.

[Mundell]

Oct 11, 2017 19:03:01 GMT stevebrown said:

Oct 11, 2017 12:41:06 GMT seriouslyred said:

Are we as good as Millwall were this time last year ? I doubt it ..... 

Millwall were 19th at the start of October last season.

It's interesting stevebrown Millwall are a very good example of the potential value of Experimental 3-6-1's expected goals analysis.

Despite their dodgy start to the season, the 'model' always identified them as one of the strongest teams in the Division. Indeed, in a post on this thread during last October I categorised the League One teams into five groups and placed Millwall in the Strongest Teams group along with Bradford City and Sheffield United. Despite their lowly league table position at the beginning of October Experimental's analysis showed that they were a much better team, and a better bet for promotion, than we appear to be now, unfortunately.

[stevebrown]

Oct 11, 2017 21:10:38 GMT Mundell said:

Oct 11, 2017 19:03:01 GMT stevebrown said:

Millwall were 19th at the start of October last season.

It's interesting stevebrown Millwall are a very good example of the potential value of Experimental 3-6-1's expected goals analysis.

Despite their dodgy start to the season, the 'model' always identified them as one of the strongest teams in the Division. Indeed, in a post on this thread during last October I categorised the League One teams into five groups and placed Millwall in the Strongest Teams group along with Bradford City and Sheffield United. Despite their lowly league table position at the beginning of October Experimental's analysis showed that they were a much better team, and a better bet for promotion, than we appear to be now, unfortunately.

I wouldn't disagree, although I also think anyone that actually watches games are likely to have similar views.

I always thought Millwall would likely win the play offs, even when they were struggling.

When we beat Rotherham A and they slipped to roughly mid table, I thought they would pick up and be a play off challenger.

I don't think Shrewsbury will get automatic promotion and I feel Southend will end up higher than they are now.

[Deleted]

Oct 11, 2017 21:24:05 GMT stevebrown said:

Oct 11, 2017 21:10:38 GMT Mundell said:

It's interesting stevebrown Millwall are a very good example of the potential value of Experimental 3-6-1's expected goals analysis.

Despite their dodgy start to the season, the 'model' always identified them as one of the strongest teams in the Division. Indeed, in a post on this thread during last October I categorised the League One teams into five groups and placed Millwall in the Strongest Teams group along with Bradford City and Sheffield United. Despite their lowly league table position at the beginning of October Experimental's analysis showed that they were a much better team, and a better bet for promotion, than we appear to be now, unfortunately.

I wouldn't disagree, although I also think anyone that actually watches games are likely to have similar views.

I always thought Millwall would likely win the play offs, even when they were struggling.

When we beat Rotherham A and they slipped to roughly mid table, I thought they would pick up and be a play off challenger.

I don't think Shrewsbury will get automatic promotion and I feel Southend will end up higher than they are now.

My dog had a dump this morning.

[Mundell]

I think you're right stevebrown In general, you'd expect a well informed fan to recognise, regardless of the result on the day, which teams are potential promotion contenders and to identify the relegation candidates. For example, though their results might have suggested otherwise, Southend looked a decent side when they came to the Valley and will no doubt believe that they ought to have taken a point and could have had all three. In Southend's case, Experimental's model supports our intuition.

However, in a game of very small margins it's not always obvious who the best side is and, moreover, we all suffer from what's called 'confirmation bias'. Because we've been taught that 'the best team will win on the day', we tend to instinctively assume that the team that won was the best side, even though we also know that there is a significant element of randomness involved in the result of any individual game. In other words, we are often overly influenced by results, especially over a run of matches.

When the pundits talk about 'form', good or bad, what they're usually observing is randomness. For example, there are some interesting studies that demonstrate that the so-called 'new manager bounce' is nothing of the sort. It's simply reversion to the mean. It's hard not to be taken in by all this 'stuff' though. We are constantly fooled by randomness and that's where a 'model' can help, provided we think about what it's telling us and don't take the results too literally.

It is for these reasons that Saturday's match against Doncaster Rovers will be especially interesting. Based on their expected goals, for and against, they appear to be one of the best teams in the league. Significantly, Wigan apart, they have the best defence, by some margin, and appear to be much more difficult to create chances against than most of the sides we've played so far. If the model is 'right' about them it's going to be a really difficult game and one of the biggest tests we've faced so far. Yet they lie seventeenth in the table and the bookies, perhaps influenced by that, rate them at 6/1 to make the playoffs. That might be a very good bet.

If Doncaster do give us a very good game on Saturday mind you it will be no more than a warm up for what, on paper, looks to be a much more difficult run of fixtures than we've experienced so far.

We've now played eleven teams, of course, and their current league table positions are as follows;

2, 4, 9, 12, 15, 16, 19, 20, 22, 23, 24. Wigan, in second place, outclassed us let's not forget, but on the other hand we've taken two points per game from the remainder.

Once we've played Doncaster, we'll then face the other eleven teams in the division and their current league table positions are as follows;

1, 3, 5, 7, 8, 10, 11, 13, 14, 18, 21

I don't have a comparator to judge just how unlikely such a skewed fixture calendar is, but my intuition is that it's probably very unlikely. The teams we are yet to play, excluding Doncaster, have an average league table position around five places higher than those we've already played. We may be ranked seventh in the table based on expected goals for, but we've created many of those chances against the weakest teams in the league so we'll do very well to sustain that performance over the next twelve matches, though it's very helpful, of course, that both Marshall and Reeves are now available.

Returning to the bookmakers' odds, based on the results of Experimental's model, Doncaster, at 6/1, look a much better value bet for a playoff place than we do at 5/4. Let's hope that by the time we've seen the two sides play against each other on Saturday that's no longer the case. To paraphrase one of seriouslyred 's favourite lines, it will indeed be interesting to see where we are at 5pm on Saturday.

[stevebrown]

Oct 12, 2017 7:12:29 GMT Mundell said:

I think you're right stevebrown In general, you'd expect a well informed fan to recognise, regardless of the result on the day, which teams are potential promotion contenders and to identify the relegation candidates. For example, though their results might have suggested otherwise, Southend looked a decent side when they came to the Valley and will no doubt believe that they ought to have taken a point and could have had all three. In Southend's case, Experimental's model supports our intuition.

However, in a game of very small margins it's not always obvious who the best side is and, moreover, we all suffer from what's called 'confirmation bias'. Because we've been taught that 'the best team will win on the day', we tend to instinctively assume that the team that won was the best side, even though we also know that there is a significant element of randomness involved in the result of any individual game. In other words, we are often overly influenced by results, especially over a run of matches.

When the pundits talk about 'form', good or bad, what they're usually observing is randomness. For example, there are some interesting studies that demonstrate that the so-called 'new manager bounce' is nothing of the sort. It's simply reversion to the mean. It's hard not to be taken in by all this 'stuff' though. We are constantly fooled by randomness and that's where a 'model' can help, provided we think about what it's telling us and don't take the results too literally.

It is for these reasons that Saturday's match against Doncaster Rovers will be especially interesting. Based on their expected goals, for and against, they appear to be one of the best teams in the league. Significantly, Wigan apart, they have the best defence, by some margin, and appear to be much more difficult to create chances against than most of the sides we've played so far. If the model is 'right' about them it's going to be a really difficult game and one of the biggest tests we've faced so far. Yet they lie seventeenth in the table and the bookies, perhaps influenced by that, rate them at 6/1 to make the playoffs. That might be a very good bet.

If Doncaster do give us a very good game on Saturday mind you it will be no more than a warm up for what, on paper, looks to be a much more difficult run of fixtures than we've experienced so far.

We've now played eleven teams, of course, and their current league table positions are as follows;

2, 4, 9, 12, 15, 16, 19, 20, 22, 23, 24. Wigan, in second place, outclassed us let's not forget, but on the other hand we've taken two points per game from the remainder.

Once we've played Doncaster, we'll then face the other eleven teams in the division and their current league table positions are as follows;

1, 3, 5, 7, 8, 10, 11, 13, 14, 18, 21

I don't have a comparator to judge just how unlikely such a skewed fixture calendar is, but my intuition is that it's probably very unlikely. The teams we are yet to play, excluding Doncaster, have an average league table position around five places higher than those we've already played. We may be ranked seventh in the table based on expected goals for, but we've created many of those chances against the weakest teams in the league so we'll do very well to sustain that performance over the next twelve matches, though it's very helpful, of course, that both Marshall and Reeves are now available.

Returning to the bookmakers' odds, based on the results of Experimental's model, Doncaster, at 6/1, look a much better value bet for a playoff place than we do at 5/4. Let's hope that by the time we've seen the two sides play against each other on Saturday that's no longer the case. To paraphrase one of seriouslyred 's favourite lines, it will indeed be interesting to see where we are at 5pm on Saturday.

This is very thought provoking and yet the stats are "at odds" with the current reality regarding Doncaster.

Donny have let in 15 goals this season, which puts them equal 12th for goals conceded and only 10 teams have conceded more.

Donny have also scored 15, which puts them 15th for goals scored and only 8 teams have scored less.

I also think Donny are quite reliant on Marquis to tuck away many of their chances.

www.sportsmole.co.uk/football/league-one/table.html

Skybet appear to be offering 7/1 for Donny to make the play offs, but this still ranks them 15th in the league.

Regarding the fact that we've played more lower league teams, I'm hoping that this won't be too detrimental as we always seem to make hard work of beating teams that pack their defence and better against teams that attack us (Wigan apart).
So we perform better against many of the higher placed teams.

www.oddschecker.com/football/english/league-1/to-make-the-playoffs.

It's very nice conversing with someone that wants to talk about Charlton and football. Thank-you.

[stevebrown]

Perhaps Donny let in more goals in relation to chances against because of the goalie ?
Ian Lawlor has been in goal and Ross Etheridge was ruled out in the summer for 3/4 months after breaking his ankle.
Pure guesswork as neither keeper are known to me.

[seriouslyred]

stevebrown I can but concur that it is good to talk about football and future possibilities.

The point that we need to be clear about with expected goals (xG) is that this talks about the potential of a chance to be converted into a goal and not the quality of the finish nor the the defensive blocking / keeping. A chance from near the penalty spot or in the six yard box will rate far higher than a shot from outside the area.

Our xG for goals conceded is virtually the same as last season, i.e., mid-table. Is that because of the style of 4-2-3-1 which KR plays or is it down to the midfield conceding too much space... or the full backs allowing too many crosses in?

Can't answer that because we simply don't have the underlying data which Ben Mayhew uses to agregate into the xG for all of the clubs. In other words we don't know if the data might suggest that there are particular weaknesses which might be addressed.

In a similar vein, I have maintained that signing Dodoo / releasing Novak is not the biggest of our issues because Magennis has played most games and his finishing has been consistent. My point has been that it is the three behind Magennis who need to chip in with chances and goals. Holmes has been there all season but Clarke and Fosu's contribution was not as high to start with.

Obviously Fosu chipped in with a hat-trick the other day plus Marshall and Reeves are gaining fitness every week. We can have no idea whether Fosu has moved up a level in terms of finishing or this is just a one-off. But what we can say is that increasing the contribution from those three over 90 minutes will certainly raise our expectations of goals scored.

As Mundell states there is a whole lot of randomness going on with refs, posts, mishits and blocks. 30 chances from both sides might yield just three goals - one goal per ten chances is about par. So for a team to progress it needs to raise the quality of the chances and the calibre of the finishing. Doing both will inevitably lead to more goals.

Here's the interesting bit: how much of that can we expect to come from good coaching and developing the squad? And how much depends upon signing the right players in January? As it happens January is too late for the forthcoming games when we play our rivals so our hopes rest upon improving fitness, individual coaching and tactics / substitutions.

In relation to Saturday, the question we can ask is how many chances have Doncaster created and conceded per game as opposed to how many times did the ball cross the line. Typically they face just eight shots per match and yet Charlton normally create around 15 - so which end of that range will we end up with on Saturday? Similarly we concede 12 chances per game and they make 12 so it is rational to expect Donny to have 12 chances on Saturday. Will that convert to 0, 1 or 2 goals?

There are all manner of other factors which come in that make the outcome completely uncertain - and that's why we love the game! The point is that good management and serious cash behind the scenes can enhance the chances of improving. Some criticised the transfers over the summer and worry about the possibility of injuries - no harm there as that just shows concern for our club. Not my choice since it is the players wearing our shirt which make the difference. Not the ones who are out nor the ones who have left the club!

The data collated and graphics supplied on Experimental 361 are based upon the players wearing our shirt. For me the biggest improvement might come from our core 15 players improving as individuals and as a team. Ideally the better teams look for a win against us and that leads to open entertaining football. Time will tell if we can keep up with the top six, or not.

[stevebrown]

Oct 12, 2017 14:27:23 GMT seriouslyred said:

stevebrown I can but concur that it is good to talk about football and future possibilities.

The point that we need to be clear about with expected goals (xG) is that this talks about the potential of a chance to be converted into a goal and not the quality of the finish nor the the defensive blocking / keeping. A chance from near the penalty spot or in the six yard box will rate far higher than a shot from outside the area.

Our xG for goals conceded is virtually the same as last season, i.e., mid-table. Is that because of the style of 4-2-3-1 which KR plays or is it down to the midfield conceding too much space... or the full backs allowing too many crosses in?

Can't answer that because we simply don't have the underlying data which Ben Mayhew uses to agregate into the xG for all of the clubs. In other words we don't know if the data might suggest that there are particular weaknesses which might be addressed.

In a similar vein, I have maintained that signing Dodoo / releasing Novak is not the biggest of our issues because Magennis has played most games and his finishing has been consistent. My point has been that it is the three behind Magennis who need to chip in with chances and goals. Holmes has been there all season but Clarke and Fosu's contribution was not as high to start with.

Obviously Fosu chipped in with a hat-trick the other day plus Marshall and Reeves are gaining fitness every week. We can have no idea whether Fosu has moved up a level in terms of finishing or this is just a one-off. But what we can say is that increasing the contribution from those three over 90 minutes will certainly raise our expectations of goals scored.

As Mundell states there is a whole lot of randomness going on with refs, posts, mishits and blocks. 30 chances from both sides might yield just three goals - one goal per ten chances is about par. So for a team to progress it needs to raise the quality of the chances and the calibre of the finishing. Doing both will inevitably lead to more goals.

Here's the interesting bit: how much of that can we expect to come from good coaching and developing the squad? And how much depends upon signing the right players in January? As it happens January is too late for the forthcoming games when we play our rivals so our hopes rest upon improving fitness, individual coaching and tactics / substitutions.

In relation to Saturday, the question we can ask is how many chances have Doncaster created and conceded per game as opposed to how many times did the ball cross the line. Typically they face just eight shots per match and yet Charlton normally create around 15 - so which end of that range will we end up with on Saturday? Similarly we concede 12 chances per game and they make 12 so it is rational to expect Donny to have 12 chances on Saturday. Will that convert to 0, 1 or 2 goals?

There are all manner of other factors which come in that make the outcome completely uncertain - and that's why we love the game! The point is that good management and serious cash behind the scenes can enhance the chances of improving. Some criticised the transfers over the summer and worry about the possibility of injuries - no harm there as that just shows concern for our club. Not my choice since it is the players wearing our shirt which make the difference. Not the ones who are out nor the ones who have left the club!

The data collated and graphics supplied on Experimental 361 are based upon the players wearing our shirt. For me the biggest improvement might come from our core 15 players improving as individuals and as a team. Ideally the better teams look for a win against us and that leads to open entertaining football. Time will tell if we can keep up with the top six, or not.

I would say our defence is shipping goals because we attack with abandon and then don't get tight on the opposition.

The full backs aren't blocking/preventing enough crosses and the central defence are leaving the strikers far too much room/unmarked.

A number of the long range goals against us could have been prevented, if we hadn't given them a free shot and for a number of the headers, we have left the scorers unmarked. To be honest it's all pretty basic and preventable.

I like Bauer but he has been one of the worst offenders of late.

I also think Kashi has been playing too deep and almost confusing the central defence as to who should be marking who.

They are all good players at this level and we need to sort it out.   

I can recall unnecessary goals that cost us points at Gillingham, at home to Bury and away at Walsall. 

5 points lost and we could have been 4th.

[Mundell]

Thanks for your reply stevebrown You're quite right, of course. Doncaster Rovers have both scored and conceded fifteen goals this season and, as result of this apparently mediocre performance, they currently sit seventeenth in the table.

The question Experimental 3-6-1 is looking to answer is whether that's a fair reflection of their performances so far or whether they are better, or worse, than their current points tally might suggest. The underlying hypothesis in Experimental's analysis is that over a season 'expected goal difference' is a better predictor of final league table position than a simple extrapolation of results over ten to fifteen games. While it won't always be right, the evidence suggests the Experimental's model does a fairly good job, on average.

Doncaster represent an interesting conundrum. The model suggests they have a very strong defence and yet they've conceded more goals than most teams in the division. So how might we explain that?

As seriouslyred has noted, Doncaster only concede an average of around eight shots per game. That is fewer than any other side in the League, apart than Wigan. By comparison, we concede over twelve shots per match and that's probably about average for the division. Given this we might expect Doncaster to concede fewer goals than anybody else, except Wigan, of course, so why don't they?

One possibility is that the quality of chances they are conceding is higher than average, i.e. more shots from within their penalty area and fewer 'long shots', for example. However, the expected goals model allows for this by taking account of the position from which a shot is taken and then attaching a probability to a goal being scored dependent on that position. The sum of those probabilities, i.e. for each shot, is the 'expected goals'. Once this adjustment has been made Doncaster's performance is, if anything, even more impressive and they retain their position as the best defence, apart from Wigan.

As I see it, we are now left with three broad explanations for the apparent disconnect between the model and actual goals conceded. First, the data inputs to the model are far from perfect and hence the model's results are, inevitably, noisy. In other words, while data errors might 'wash out' over a full season they might distort results in the short-term. That said, while this is certainly a possible explanation, the eight shots against per game metric is probably a fairly robust statistic.

Second, it may be that Doncaster have a 'problem' which the model doesn't capture. They concede a goal every six or seven shots against. Only Southend and Oldham are clearly inferior on this metric so perhaps, as you suggest, Donny's goalkeeper simply isn't good enough, for example. That's clearly possible too, but it would be a little surprising at this level.

Third, we may simply be observing randomness, i.e. Doncaster may simply have been 'unlucky'. To put that in perspective, based on their expected goals against, Doncaster ought to have conceded 11-12 goals, i.e. three or four fewer than they've actually conceded. Had they shipped just eleven goals they'd have the fifth best defence in the league. The point here is that it's not hard to see how, over just eleven games, a team might concede four more goals than they deserve to; a couple of unbelievable finishes, a goalkeeping error and a lucky deflection, for example. Of course, sixteen goals over the course of an entire season is a different proposition and much more significant. The randomness should begin to even out over a long stretch of games.

Two more goals scored and four fewer conceded, as predicted by their expected goals, doesn't seem much when looked at this way, but such an outcome would probably have transformed their league table position and our perception of them as we approach Saturday's game. It's a game of very fine margins.

Where does this leave us? The great thing about football is its unpredictability. Anything could happen on Saturday. We could win, lose or draw. My point is really a very simple one. Based on their expected goals against I'm expecting Doncaster to be much stronger defensively than their record suggests and harder to create chances against than any other side we've played so far, except Wigan. That may be wrong, of course, but forewarned is forearmed and it's to be hoped that Karl Robinson's coaching staff know exactly what they'll be up against. Either way, I'm really looking forward to the game.

[Mundell]

Today's League One timelines