AI: Revelations


For a long time, my AI strategy has been:

  • First, figure out the AI’s knowledge structure – the way knowledge is stored inside its mind. You’d think this would be easy, but the problem of knowledge representation turns out to be nontrivial (much like the Pacific Ocean turns out to be non-dry).
  • Once I know how to represent knowledge, I will begin work on knowledge acquisition, or learning.

To me, this order made sense. A mind must have a framework for storing information before you can help it learn new information.


Well, for the past week, I’ve tackled the problem from the opposite direction. I’ve pushed aside my 5,000+ lines of old code (for the moment) and started from scratch, building an algorithm that’s focused on learning.

The result is a little program (less than 200 lines long) that reads in a text file and searches for patterns, with no preconceived notions about what constitutes a word, a punctuation mark, a consonant, or a vowel. For instance:


This AI-in-training makes short work of Hamlet, plowing through the Bard’s masterpiece in about ten seconds. The result is a meticulous, stats-based tree of patterns. I can examine any particular branch that starts with any letter or letters I like.

Here I’m looking at all the patterns it found, that start with “t”:

python output

The full list is much longer, but already you can see it’s picked up some interesting patterns. It’s noticed “the” and “there”, and it’s noticed that both are often followed by a space. It’s even started picking out which letters most commonly start the next word. And it’s noticed a pattern of words ending in “ther”, presumably “mother”, “father”, “together”, “rather”, and their kin.

This algorithm is cool, but rather limited at the moment. It can notice correlations between letters, and fairly simple strings, but it doesn’t do well with more complex patterns. I won’t bore you with the details, but rest assured I’m working on it.

In the meantime, AI is fun again. I mean, it was mostly fun before, but I was entering a dry spell where the work had started feeling like a chore. Every now and then, a fresh perspective helps get you excited again.

In this case, it also showed me that I had my strategy backwards. Just as you can’t build an ontology and weld on input/output later, it turns out likewise that you can’t build an ontology and weld on learning. Learning, it appears, must come first. The how determines the what.

And this new direction is fun for another reason, too.

Till recently, I’d been coding in C++. Now C++ is a white-bearded, venerable patriarch of a language: time-tested, powerful, respected by all. But it’s also a grumpy old man who complains mightily about syntax and insists you spend hours telling it exactly what you want to do.

This new stuff, on the other hand, I’m coding in Python. Python is a newer language, not as bare-bones efficient as C++ but a hell of a lot simpler from the programmer’s point of view, shiny and full of features and full of magic. And I’m new to Python myself, so I’m still in the honeymoon phase. I’m not saying one or the other is “better” overall, but right now, Python is a lot more fun.

And really, programming ought to be fun. Especially if you’re building a mind.


9 responses to “AI: Revelations

  1. Yeah, what I’ve noticed about python is that you have to play some interesting coding gymnastics to keep your script file from becoming too large. Passing values between multiple classes can become very interesting…

    • When assigning variables and passing parameters, it’s been kinda tricky to figure out when Python assigns/passes by value, and when it’s by reference. Still getting the hang of that. Is that what you meant, or were you talking about something else?

      I’m curious what you mean about keeping the script file from getting too large. From what I’ve seen, the import command works pretty simply, but I haven’t gotten in-depth with it yet. What kinds of gymnastics have you had to do?

      • So if you have a single file as your ‘command script’ and that is the one in which you import everything into then you are fine. It’s like a central node, now if you want to pass values back and forth from the center it is fine. But there is no way to easily connect outer nodes. So if you need them to talk then it can get interesting. I basically had to start writing files to pass variables or rewrite my entire code. lol which is what I ended up doing.

      • Huh, interesting. Surprised none of the Python designers saw that coming. I’ll be on the lookout.

  2. Very interesting. Keep us updated.

  3. I am loving the updates on the A.I. Have you read The Ravenous Brain, by Daniel Bor? It’s an interesting exploration of consciousness in humans, and might also highlight a couple other features of our minds that allow us to have this oddity called consciousness.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s