I was interested in that... trying to design a machine that would collect a lot of experience and then make predictions about what was likely to happen next, and so the... this thesis ends up with that, a little bit theory about that... but then I decided that that was really the wrong way... or a wrong way to, think about thinking. And... I think in 1956, or was it 54? I’ll have to look it up again... another friend of mine named Ray Solomonoff started to produce a theory of how to predict... make predictions from experience based on a completely different set of ideas based on probability theory and, more important, based on Alan Turing’s idea about universal machines; and Solomonoff’s idea was that basically the best way to make a prediction is to find the simplest Turing machine that will produce a certain phenomenon.

Now, there’s no systematic way known where... and in fact it may be impossible to find the simplest Turing machine that produces a certain result, but still this is a wonderfully important idea about the foundations of probability and about how one could build very intelligent machines. So after I got this insight from Solomonoff, I stopped working on neural nets entirely because I felt that was a bottom-up approach, and one really couldn’t design a machine to do intelligent things unless one had some sort of top-down theory of what it is that… what kind of behaviour that machine should have.

Once you have a clear idea of what it is to be a smart machine, then you can start to figure out, well then, what kind of machinery could support that, and what kind of machinery would it take to do that? And then at the end of a rather long path we could ask, and how could we use neurons or things like we find... like the cells we find in the brain, to produce these functions?