AI as a technology in a box

So one of the concepts when we speak about AI and ethics of AI, is how much do we actually know about what the AI system is doing? And we consider the AI as a box and there are various shapes of that box or ways how we can see that box. We can see it either as a black box, like in flying, it’s just

collecting data and maybe giving outputs, but it never tells us why. So somehow this creates insecurity. So it tells you and you are not eligible to get this credit and you ask why? And it says Because, but no explanation. That’s something no one wants to interact with. On the other hand, what would be like people would think that’s the ultimate goal of the glass box, and I was totally transparent. It tells me every individual reasoning step. It can tell me everything that I need. However, if it tells me every inch in detail, it becomes quite difficult to understand. Neural networks 100 million parameters. We know exactly the weight of each individual neuron, however. Do you understand what’s really happening inside?

No, it’s still, we cannot say because that number is bigger than that, and that is that 100 million different equations, that is too difficult to really understand what’s behind. So a reasoning on a more useful level for the user that will be the goal to achieve? But but that’s difficult to achieve. And one of the more recent research trends, it’s also quite highly covered here in Sweden by the WASP-HS the Autonomous System Program funded by Wallenberg , which focuses on the human

aspects of human science of autonomous systems: The idea of a white box. And there the idea is we have inputs and outputs, and the way how this system acts is evaluated and defined by rules. So either we have laws or we have some other guidelines, and it should be fully functioning according to these guidelines. And should be also explainable, and this is very similar to a human. If you have a human, human makes many actions and doesn’t explain every action why. But if you ask this human why it should be responsible for the action the human has taken, and then you can define tests for that.

And for example, a self-driving car could be evaluated like a driver when they have to drive and do their driving test. So we can just say OK, given on and maybe even not just 50, but a hundred thousand minutes of driving. If the AI performs typically as we expect it to perform, then we can say it absolves the test. And if it still makes an accident later on, it’s like all of us a human.

It has to be then responsible for that accident, and explain why has it done this decision leading to an accident? Maybe often

the case: “I didn’t see the human passing the road.” Maybe when the Tesla was a big problem a few years ago, where the person in the dark suit crossed. When it was rainy, it was a bicycle rather suddenly and then the car crashed against this person. There were some issues. OK, the Tesla didn’t see everything, but an issue was the car was driving too fast and that was really not good. And there the programmers at the end had to be responsible for what has been happening. Thank you very much.