This figure appears in DeepMind’s instant-classic paper ‘Mastering the Game of Go without Human Knowledge’ (2017):
Figure 3b: 'Prediction accuracy on human professional moves. The plot shows the accuracy of the neural network at each iteration of self-play, in predicting human professional moves...
The accuracy measures the percentage of positions in which the neural network assigns the highest probability to the human move.'
It shows that AlphaGo Zero (AGZ) only predicts human pro moves with 50% accuracy, at best. That is, AGZ disagrees with human professionals on 50% of moves.
This perhaps has implications for human expertise in general, by the following argument:
1. AGZ plays far beyond peak human ability. 2. AGZ would play differently from a peak human in 50% of moves. 3. So a peak human makes suboptimal moves at least 50% of the time. 4. Go is an excellent environment for human learning (small ruleset, rapid objective feedback, amenable to intuition). 5. So, relative to more complex domains, human mastery of Go should be relatively complete. 6. So we can expect human experts in other, more complex domains to make suboptimal decisions at least 50% of the time.