All Things Techie With Huge, Unstructured, Intuitive Leaps

Dimension & Event Sorters & Classifiers - The Genesis of Artificial Conciousness & Abstract Machine Reasoning

Artificial Intelligence will remain stunted, and at best, a pale sort of intelligence until machines can have a degree of consciousness as to what they are doing.  The field of Artificial Intelligence is galloping forward in many different directions, doing amazing human-like things in the human domain, but it is still like monkeys typing Shakespeare. For a small discourse on the evolution of imperfect artificial intelligence, see the blog entry below called "Dawkins, Wasps, Artificial Intelligence, Evolution, Memorability and Artificial Consciousness".

Of course, one has to realize that not every application of Artificial Intelligence has to be perfect.  It can even be pseudo-AI if it does the job (Google searches are an example of kludged or pseudo AI as explained in the blog entry below as well.)  But I am talking here about pushing the boundaries of AI and making a case for Artificial Consciousness that can lead to abstract machine reasoning.

To accomplish these lofty goals, one needs to start with a practical framework and take the baby steps towards an Artificial Consciousness.  Therefore machines need to start collecting data and knowledge just for the sake of doing so. They must have event and state memory.  One could say that computer logs are a primordial event memory, in the form of action records. Indeed, academics like Professor Wil van der Aalst at the Technical University of Eindhoven in the Netherlands have made a science out of analyzing event logs. However their focus is to divine business processes from event logs.  They have made tools, some open source for doing so.  Their goal is to understand business processes using machine-generated data logs.   With just a little shift of focus, efforts such as those of van der Aalst could be adapted to do the analysis of the state of the computer at any given time.

State awareness is key to Artificial Consciousness. And the state transitions are events to be collected and analyzed as well.  The machine must have cognizance not only of it states but also of the actions that change states and drive state changes.

This process is akin to the development of a newborn baby. When it is born, it has just two main states, conscious and sleeping.  The sleeping state can be divided into the sleep stages - REM, etc and the state where the brain does neural net formation based on its sensory inputs and memory from the waking state.  The waking state initially has just two sub-states, comfort and discomfort.  The discomfort comes from hunger or the waste acids burning the baby skin from a full or wet diaper. The comfort comes from feeding or being cuddled.  All the while, the baby is hoovering up and collecting vast amounts of sensory inputs for processing while sleeping.

The baby's development of consciousness comes from state awareness. It begins with a classification of state and events.  Classification is demonstrated by the Sesame Street song where one of the things is not like the other -- a difference discriminator if you will:

Once you are able to discriminate difference, you are well on the way to having a classifier.  The concept of classification -- discriminating and sorting based on sameness and difference, the idea of collections of things and events becomes apparent. Although these are considered higher cognitive functions, many animals possess these capabilities.  My border collie used to sort his toys based on prime importance to him in holding his attention as playthings.

The formations of collections allows for the preliminary ability to abstract.  An abstraction can be as simple as choosing a parameter, property or function of everything in that set that all objects in that set exhibit.  In other words, the classification threshold can one of the abstract models of a thing common to everything in that set.

It's still a pretty dumb sort of abstraction, and the reasoning is limited to comparison, but it is a start.  Fortunately the operating systems of computers are fairly good at comparison and the sub-tools required for that, such as sorting and iteration through collections.  The computer does a decent job of collecting sets as well.

So the preliminary forms of artificial consciousness starts with data collection about internal self.  Then we advance to sorting and classifying the data.  From there we can get states.  Preliminary classification criteria becomes the abstract model of state awareness.  Once you have a good handle on the state, and an abstraction of that state, one can become aware of state change transitions or events.  It is just a short leap to start sorting and classifying events as well.  This adds power to predictive ability and is the first steps on the way to abstract, complex reasoning. Babies can do complex reasoning about state transitions at a very early age.  When the baby is negotiating say a bowel evacuation, there is a bit a stress that can be detected.  Then when the diaper fills, there is relief.  A full diaper is very enjoyable for the first thirty seconds as well, and this is reflected in the baby's emotional state.  Then when things begin to gel, so to speak, discomfort sets in and the baby reacts according.  One event drives possible three different reactions based on a timeline.  Time is the most important dimensions for prediction and reasoning.

The biggest cognitive asset to come out of a machine consciousness development protocol such as this, is that with more and more associations and abstractions comes the recognition of the dimension of the time.  States do not stay the same. They either transition due to external drivers or decay from one state to another.  These state changes or events allow for the cognition of the arrow of time.  The biggest step to autonomous reasoning will come from noticing and reacting to states and events as time goes by.

If the time dimension of states and events can be sorted and classified, then a machine will not only have a utile reasoning ability, but it will be able to do in with respect to time, and in real time. This will allow monitoring processes for things like self-driving cars and such.  The ability to abstract in the time dimension or domain allows for the reasoning ability to foresee consequences of action.  And in anyone's book that is both consciousness and intelligence -- and it won't matter whether it is artificial or not.

So what are the next steps to Artificial Consciousness? The machine must be able to discern its internal state through dimension and event sorters and classifiers.  Then it must be able to link up states on a time line uses probabilities for cause and effect and the interaction of events and states.  It will be a fairly primitive thing at first, but once you open that Pandora's Box, the evolution of artificial consciousness will be exponential in terms of progress.

Human consciousness has been explored by many throughout history, including pseudo-scientists. I have been fascinated by it ever since I once read that on an evolutionary scale, consciousness is nothing but a highly advanced tropism.  I know that many would disagree.

I once attended a meeting of Jung Society in Nassau, and paid $75 for the dubious privilege of attending.  A cheap Chinese buffet was included.  A psychologist and psycho-analyst was one of the speakers and he also happened to be a Catholic priest.  In his talk, he related the tale of how Carl Gustav Jung discovered the supposed human collective unconscious.  Jung was treating a patient with a severe mental disorder, and gave the patient a piece of paper and some crayons. The patient drew a rudimentary face on the page.  Gustav ruminated over the drawing and came to the conclusion that the patient had tapped in the collective unconscious and created a drawing of a Polynesian mask of some sort.  This was the germ of the idea of collective unconscious populated with instincts and archetypes, according to Jung.

At the appropriate time when it came to the question period, I gently pointed out that 99.99 percent of Freudian theory was debunked.  Brain physiology research had advanced to the point where we could identify substances such dopamines and other chemical receptors and inhibitors that were responsible for mood control and compulsive and unconscious ideations. I further pointed out that Jung's ideas had no scientific basis.  My observations and questions were about as welcome as strong bean flatulence in a crowded elevator.

One of the quotes on consciousness attributed to Jung was this: "There is no coming to consciousness without pain. People will do anything, no matter how absurd, in order to avoid facing their own Soul. One does not become enlightened by imagining figures of light, but by making the darkness conscious."

I would like to paraphrase that quote with scientific rigor. There is no coming to consciousness without sorting and classifying data about the internal state. People will believe anything, no matter how absurd to avoid facing the physio-mechanical nature of consciousness. One does not reach enlightened reasoning by imagining artificial constructs, but my making reasonable inferences using solid logic to achieve machine consciousness.  It's just the way it is, and I know that this view will be on the right side of history and human development.

Dawkins, Wasps, Artificial Intelligence, Evolution, Memorability and Artificial Consciousness

There were two items of interest in the past few weeks that came to my attention, which have relevance to artificial intelligence. The first was a Twitter feed that announced that MIT had created a deep learning neural net tool to predict the memorability of a photo at near-human levels. Here are some gifs that accompanied the article:

The ramifications of this algorithm, is that a whole bunch of feed-forward neural networks can accurately judge a human emotional response to an image.  These neural networks consist of artificial neurons that essentially sift though mounds of pics that have been annotated by humans for areas of memorability.

The ironic part of this whole artificial neural network stuff, is that mathematicians cannot adequately describe in integrated empirical detail,  how these things operate in concert to do incredibly complex stuff.  Each little artificial neural net sums the input multiplied by the weight and uses something like a sigmoid function as an activation function to determine whether it fires or not.  In supervised learning, after the entire mousetrap game of neural nets have finished their jobs, the result is compared to the "correct" answer and the difference is noted. Then each of the weights in the neural nets are adjusted to be a little more correct (back propagated) using stochastic gradient descent (the methodology to determine how much the weights should be adjusted by).  Eventually the whole claptrap evolves into a machine that gets better and better until it can function at near-human, human, or supra-human levels, depending on the training set, the learning algorithms and even the type of AI machine chosen to do the task.

This process actually reminds me of my early days in electrical engineering, using transistors to create logic gates, and then building up the logic gates to build things like flip-flops or data latches on a data bus. Once you had latches, and memories and such, you could go on to build a whole computer from first principles.  The only difference is that with transistors, one must map out the logic and scrupulously follow the Boolean formulas to get the functionality.  In Artificial Neural Networks, these things sort themselves out solely by back-propagating a weight adjustment.  One of the tenets of this universe that we inhabit, is that you can get incredible complexity from a few simple building blocks.

So the upshot for AI, is that this dumb machine has evolved its neural networks to be able to judge the human emotional impact of an image.

In the old days before all of this machine learning hoopla, if I had told you that a machine would be able to judge human emotional response, and then I asked you to theoretically describe how the machine did it, a natural assumption would be that one would have to teach the machine a limited range of emotions, and then one would have to catalog responses to those emotions. One would then have to teach the machines the drivers of those emotional responses in humans (ie cute kittens etc).  It would be a fairly vast undertaking with the input of many cognitive scientists, programmers, linguists, natural language processing freaks etc.  In other words, today's machine learning, deep learning, recurrent neural nets, convolutional neural nets do an end run around first principles learning with underlying knowledge. It's sort of a cheat-sheet around generally accepted terms of human cognition.

In the same way, with a more familiar example, take Google search.  If , twenty years ago, I laid out the requirements for a smart system of having a text box and as you typed letters it guessed what you wanted to search for with incredibly accuracy, one would think that there would be an incredible AI machine behind it with vast powers of natural language processing.   Instead, it is a blazing fast SQL database lookup coupled to probabilities.  Artificial Intelligence is not the anthropomorphic entity that was portrayed in the sci-fi movies.  It ain't no HAL like in 2001, A Space Odyssey. There is no person-like thing that is smart.  It's all a virtual mousetrap game coming up with the right answers, most of the time.

So this brings me to Richard Dawkins and wasps.  I am reading his new book called "Brief Candle In The Dark".  I never knew that he was a biologist first and foremost. The atheism is just a recent sideline.  In recounting his adventures in science, he talks about the waspmanship and games theory.  It goes like this.  A certain species of wasp digs a hole in the ground.  It then fetches insects known as katydids, and paralyzes them with venom. It hauls the living, paralyzed katydid into the burrow.  It lays in a supply of paralyzed katydids and lays eggs.  The eggs will hatch and the katydid will be fresh food  for the larvae when the eggs hatch, The katydids will not rot, because they are not dead, just paralyzed.

Occasionally, instead of digging a new burrow, it will find a burrow dug by another wasp. Sometimes the new burrow will have katydids in them from a previous tenant.  The wasp will go on catching more katydids, and filling the burrow.  This works out quite well, as the find is valuable from a energy budget and reproduction economics point of view.  The wasp has saved herself the task of fetching a pile of katydids.

However, if the burrow-originating wasp comes back, a fight ensues.  One would think that the winner would be random between the two, but the wasp who brought the least number of katydids to the burrow, is the one that is first to give up fighting.  The one who has invested the most, is the most prepared to conduct all-out warfare and keep fighting.  In decision theory and economics, this is called the sunken cost fallacy.  Instead of giving up and building a new "Katydid Koma" burrow, the wasps will fight and perhaps risk dying because of their previous investment.

So why have wasps evolved this way?  Further analysis and research has shown counting the overall katydid number in a burrow is computationally expensive from a biological point of view.  Running food-counting mathematics in that tiny brain takes more resources than simply counting the number of katydids that one personally has dragged back to the burrow.  One can be based on say memory, while the other requires mathematical abstraction. It is like generic brands of personal computers leaving out the fancy math co-processing chips that that the more expensive computers have.

 To quote Dawkins, animal design is not perfect, and sometimes a good-enough answer fills the overall bill better than having the ability to accurately and empirically give an accounting of the situation. Each wasps knows what they put into it, and they have a fighting-time threshold based on their investment only.  Even if the hole was fully of katydids, and was a true egg-feeding goldmine, if a certain wasp was the junior contributor to that stash, that lesser number is all that goes into their war effort computation.

That good-enough corollary has applications that we are seeing in AI.  Google didn't go and teach its search engine the entire dictionary, semantics, and natural language processing.  They do quick word look-ups based on probability.  MIT didn't teach their machine all about emotions.  They let the machine learn patterns of how humans tag emotions.  It is the dumbest intelligence that anyone would want to see, because it doesn't understand the bottom-up first principles. It just apes what humans do without the inherent understanding.  You cannot ask these kinds of intelligence "Why?" and get a coherent answer.

In essence, it is the easy way out for making intelligent machines. It is picking the low hanging fruit. It is like teaching monkeys to type Shakespeare by making them do it a million times until they get it right.  It may be Shakespeare once they finish the final correct training epoch (a training epoch in AI, is letting the machine run through an example, calculate what it thinks is the right answer, and correcting it using back propagation), but to the monkeys, it is just quitting time at the typewriter, and end result is not any different to them, than the first time that they typed garbage.

So, the bottom line is that artificial intelligence, with today's machines, is truly artificial intelligence. It can do human things a lot better than humans can do, but it doesn't know why.  Artificial neural networks at this stage of development, do not have the ability to abstract. They do not have the ability to derive models from their functioning neural nets.  In other words, they do not have consciousness and the ability to abstract yet, which is a pre-requisite for abstraction and learning, or self-learning from abstract first principles.

But suppose that development model for artificial consciousness resembles the same model that the wasps have and the same model that the current crop of machines doing human-like things possess.  Suppose that you faked artificial consciousness in a way now that machines fake intelligence and are able to do human-like task very well within a human frame of reference. Suppose that you developed artificial consciousness to cut corners of cognition and be parsimonious with compute resources.  For example, suppose you taught a computer to be worried when the CPU was almost plugged up with tasks and computation ability and bandwidth was stunted. It wouldn't know why the silicon constipation was a bad idea, and it couldn't, in its primitive state, reason it out.  It just knew that it was bad. These are the first baby steps to artificial consciousness.

The wasp can't count the total number of katydids in the burrow. It cannot make a rational fighting choice based on overall resources.  It's consciousness circuits have evolved using shortcuts and they are not perfect.  Yet the wasp has evolved a superb living creature capable of incredible complex behaviors.  In a similar fashion, we will have artificial consciousness.  Sure as shooting it will come sooner than you think. It will be primitive at first, but it will do amazing things.

So when you look for artificial consciousness, the seeds of it will be incredibly stupid.  However, you can bet that it too will evolve, and at some point, it will not matter whether it knows what it knows from human first principles.  It really won't matter.

And this is why Stephen Hawking says that Artificial Intelligence is dangerous.  Suppose that your AI self-driving car decides to kill you instead of four other people when an accident is inevitable.  Suppose a war drone, meant to replace attack infantry has a circuit malfunction and goes on a rampage killing civilians due to that malfunction.  Suppose that a fire suppression system decides that an explosion is the best way to extinguish a raging refinery fire, but can't detect if humans are in proximity or not.  Don't forget, these things can't abstract. They will be taught to reason, but like all other AI developments, both in the biological and silicon worlds, we will take huge shortcuts in the evolution of artificial intelligence and artificial consciousness. Those shortcuts may be dangerous. Or those shortcuts, like the wasp lacking the ability to do total counts, may be absurd, however they will be adequate for a functioning system.

The big lessons from Dawkins' book is that if I read between the lines, I can get some incredible insights into AI and biomimicry to create it.  As an AI developer, what Dawkins' example has taught me, that like Mother Nature, it is sometimes okay to skimp on computational resources when evolving complex AI.  This is the ultimate example of the end justifying the means.