"If the human brain were so simple that we could understand it, we would be so simple that we couldn’t."
~ Emerson M. Pugh

I don't necessarily think what follows is true, but I would like to talk about it to stimulate my own thoughts:

Imagine you have a system which does something simple, like adding up two numbers in base 10.

Is it possible for this system to conceptualise how it (itself) works?

Patently not, it can only add up.

What if we append to the original system another system, which is capable of comprehending the original adding system.

Can this new conjugate system comprehend how it works? Well, we got a bit closer, in that the system understands part of itself, but in doing so we had to add a new system, which the system cannot comprehend; moreover, though it's probably not that important, this new system is much more complicated than the original adding system.

So again we append a system capable of comprehending the original comprehending appendix, and of course we find ourselves with the same problem; that this new system cannot understand itself.

If we proceed by induction it would seem that we conclude that no system can understand how it works.

Thoughts?