Quantify intelligence

Now is the question when we know there’s different levels, we might want to quantify intelligence, maybe you heard already of ways to quantify human intelligence and very typical example is the IQ test. Many people have maybe tried already, but it’s really, really hard to quantify intelligence, human intelligence or artificial intelligence, and there is no clear way to do it. But there are some notable aspects, and I will now discuss about a few of them. So, for example, the ability to generalize that is a quite nice ability. So if you give just one example of an apple to recognize all the other apples that that it’s a quite nice ability and that something AI should fulfill.

On the other hand, the complexity is also important. If you have a very, very complex agent, which requires a lot of resources, then we would more or less assume it should be able to generalize. Sometimes actually, it doesn’t that over fits. So it would recognize every individual picture of an apple, but not recognize generally the concept of an apple. So there are some discussions already, and it is about the generalization ability of learning agents. And another it’s a quite nice paper from two thousand seven trying to define machine intelligence. And it says an AI is defined by the complexity of a set of environments where agents succeed.

So you can make maybe tests and they list several tests and they discuss these tests and say, OK, if an AI succeeds there, then it is more strong, than another AI, which can only solve very simple tests. And there are various aspects to measure this intelligence. So if we have a test, we can say the test can be a valid test that can be informative, can either cover a wide range of problems or just a small range of problems. Can be general can be a dynamic test, which changes maybe in a changing environment. Can be biased or unbiased fundamental. It can be a formal, it can be objective or subjective, and a very nice example for subjective tests.

When you learn a new language at the end, the teacher evaluates how good you are, and it’s typically quite subjective because it’s based also on the evaluator your outcome, and it can be a fully defined test or only partially defined. For example, there are concepts of ultimate intelligence tests which really test everything and say, and AI should be able whatever problem given to solve it, but it’s not really fully defined what kind of problems are there? So it can be universal or it can be not so universal. And one important aspect this is the practical or not. Can we actually do this test or is it just a concept which is to have what we cannot really perform these tests?

So that’s what they discussed in this paper. One very, very famous test is the Turing test. Maybe you have heard about that already. The idea is that an expert evaluates if the expert is speaking with an AI or not or chatting with AI or not. The original Turing test was actually that the expert is chatting was two people or two entities, one being a human person and the other being a computer and the original test actually even said that both of them should convince the expert that they are females. But that’s a small detail of the original tests, the general concept behind it. The AI should convince a human that it is the human

and the test isn’t. I mean, the original test is not yet solved. No, AI can yet do it. There are variants of the Turing test, which are in a similar setting where human evaluators, when they were not trained, already thought quite often that this is a human they are talking to and not an AI. An example is the cleverbot to which you can play out in the tasks later on. And the idea is to test the AI by asking questions which also ask for consciousness. So you will ask the human, Hey, are you really sure about what you are saying now? Do you really stick to that and give me some more explanation why you think like that?

And simpler computer chats cannot do that. More advanced ones can do that. And then there are expansions of those tests, some of them say you can also use a video signals so the human can really see it with whom it is interacting , even maybe in a physical environment. And finally, an ultimate test could be that the AI has to evolve its own manifestation. So it has to realize I am what it is, so it gets a kind of consciousness and then maybe even able to replicate itself to create more AI. That is maybe one of the ultimate goals of the AI.

But that’s not really practical and a better test, which is also practical and covers many, many of the aspects which I mentioned. This is a so-called C-test, and that is again, you have various problems, but all of these problems are and the domain of sequence prediction. Maybe you know that from numbers given the number one, three, five predict the next number seven and so on.

They can be more and more complicated sequences, some of them also incorporating natural language and whatsoever. And an AI, which is really, really strong, should be able, given any kind of sequence test to produce the next following symbols. So these are different levels of AI, and now we have also an idea how we can maybe measure AI. Thank you very much.