A World Of Affect
    By Samuel Kenyon | August 1st 2013 11:30 PM | 5 comments | Print | E-mail | Track Comments
    About Samuel

    Software engineer, AI researcher, interaction designer (IxD), actor, writer, atheist transhumanist. My blog will attempt to synthesize concepts...

    View Samuel's Profile
    Back in the fall of 2005 I took a class at the MIT Media Lab called Commonsense Reasoning for Interaction Applications taught by Henry Lieberman and TA'd by Hugo Liu.

    For the first programming assignment I made a project called AffectWorld, which allows the user to explore in 3D space the affective (emotional) appraisal of any document.

    The program uses an affective normative ratings word list expanded with the Open Mind Common Sense (OMCS) knowledgebase. This norms list is used both for appraising input text and for generating an affect-rated image database. The affective norms data came from a private dataset created by Margaret M. Bradley and Peter J. Lang at the NIMH Center for the Study of Emotion and Attention, consisting of English words rated in terms of pleasure, arousal and dominance (PAD).

    To generate the interactive visualization, AffectWorld analyzes a text, finds images that are linked affectively, and applies them to virtual 3D objects, creating a scene filled with emotional metaphors.

    The image files were scraped from a few places, including Eric Conveys an Emotion, in which some guy photographed himself making every emotional expression he could think of and then started doing requests. I used OGRE for the 3D graphic engine.

    So what was the point? If I remember correctly, somebody asked that in the class and Hugo interjected that it was art. Basically the emotional programming looks to an outsider like a pseudo-random image selector applied to cubes in a 3D world...well, that's not completely true. With a lot more pictures to choose from (with accurate descriptive words assigned to each picture), I think that one could make a program like this that does give some kind of emotional feel that's appropriate from a text.

    Certainly stories are a kind of text that explicitly describe affect: the emotions of characters and the environments enveloping the characters. AffectWorld programs would never be perfect though, because stories themselves are just triggers, and what they trigger in any given person's mind is somewhat unique. This is perhaps the realm that film directors using published stories live in--creating a single visual representation of something that already has thousands or millions of mental representations. But in an AffectWorld, I simplify the problem but assuming from the beginning that the visual pictures are arbitrary. It is only the emotional aspects that matter.

    At the time of the demo, some people seemed momentarily impressed, but that was partially because I made them look at a bunch of boring code and then suddenly whipped out the interactive 3D demo. Otherwise, my first version of AffectWorld was just a glimmer of something potentially entertaining. I started another project for that class which I will talk about in a future blog.

    Part of the reason why I took the class was because I was skeptical of using commonsense databases, especially those based on sentences of human text. During my early natural language explorations I became suspicious of what I learned later was called the "hermeneutic hall of mirrors" by Stevan Harnad--in other words, computer "knowledge" dependent on English (or any other human language) is basically convoluted Mad Libs. However, I did witness other students making interfaces which were able to make use of shallow knowledge for unique user experiences. Just as Mad Libs lends itself to a kind of surprising weird humor, so do some of these "commonsense" programs.

    This is somewhat useful for interaction designers--in some cases a "cute" or funny mistake is better than a depressing mistake that triggers the user to throw the computer out the window. Shallow knowledge is another tool that is perfectly fine to use in certain practical applications. But it's not a major win for human-level or "strong" AI.

    The Semantic Web is a similar beast as far as I can tell. Despite recent good-intentioned articles full of buzzwords, the Semantic Web has been around for a long time, at least conceptually. Seven years ago I went to an AI conference where Tim Berners-Lee (the inventor of the World Wide Web) told us about how the Semantic Web was the new hotness and its relationship to AI (AAAI-06 Keynote Address). OWL, the web ontology language standard, had already been started. And now the semantic web is apparently finally here, sort of. Companies that rose to power since the concept of OWL like Facebook and Google have made massive semantic networks out of user data. These are great enablers and we probably have not even seen the killer apps to come out of these new semantic nets. And in some narrow contexts, semantic net powered apps could be smarter than humans. But they do not understand as human organisms do. Sure, there could be a lot of overlap with some level of abstraction in the human mind, and it is not necessarily true that all knowledge is grounded in the same way or at the same level.

    Someone will probably post a comment along the lines of "well that is ultimately how the brain works, just a big semantic net in terms of itself" which skips the issue of the nodes in computer semantic networks that depend on human input and/or interpretation for the meaning. Or somebody might argue that the patterns have inherent meaning, but I don't buy that for the entirety of human-like meaning because of our evolutionary history and the philosophical possibility that our primitive mental concepts are merely reality interfaces selected for reproductive ability in certain contexts.


    At the time of the commonsense reasoning class--and also Marvin Minsky's Society of Mind / Emotion Machine class I took before that--a graduate student named Push Singh was the mastermind behind Open Mind Common Sense. Although I was skeptical of that kind of knowledgebase, I was very interested in his approaches and courage to tackle some of the Society of Mind and Emotion Machine architectural concepts. His thesis project was in fact called EM-ONE, as in Emotion Machine 1, dealing with levels of cognition and mental critics. I didn't know him very well but I talked to him several times and he had encouraged me to keep the dialogue going. I recall one day when I reading a book about evo-devo in the second floor cafe at the Harvard Co-op bookstore, ignoring all humans around me, Push happened to be there and made sure to say hello and ask what I was reading.

    One day I went to his website to see if there was anything new, and found a message from somebody else posted there: Push was dead. He had committed suicide. Below that, stuck to my computer monitor, lurked an old post-it note with a now unrealizable to-do: "Go chat with Push."

    Image credits:

    Mad libs example by Becca Dulgarian via Emily Hill


    We are on the verge of semantically-correct AI, with the pace of development accelerating to accommodate the needs of Users-Mobile, Tablet, Desktop, any AAA device (AnyThing, AnyWhere, AnyTime)- for MAAO (Multiple Applications Always On- see the new Droid for a start).
    We still have a way to go.
    In my mind the major barriers are: evolving from Analog processing to Gestalt processing- "all at once" resource access and processing; and, a new architecture of (unlimited RAM) which includes "dynamic autonomous programming" able to create "software on demand" custom to User needs, to accomplish a task, combined with algorithms that access cloud-based databases, searches and selects needed information to add to the programming, and completes the task.

    In my mind the major barriers are: evolving from Analog processing to Gestalt processing- "all at once" resource access and processing
    I'm not sure what you mean by that. Computer-based processing is mostly digital (not analog) right now. The human component of a human-computer interaction is, of course, mostly analog.

    "dynamic autonomous programming" able to create "software on demand" custom to User needs, to accomplish a task, combined with algorithms that access cloud-based databases, searches and selects needed information to add to the programming, and completes the task.
    Yes, that is an exciting and perhaps liberating computer competency to strive towards. One part or stage towards that could be making it easier for different kinds of users who are not programmers or even power users to communicate with computers in order to generate new programs on the fly for that user's customized context. Some people--including Lieberman who taught the class mentioned in my post--have tried to conceptualize alternate forms of programming computers such as "end user development." Nothing revolutionary has come of that yet in the marketplace that I know of, but small steps have been made--I think that IFTTT and Rethink Robotics have made different small but very important steps towards interfaces that allow normal people to get computer-based devices to do what they want without needing a programmer.
    I guess I wasn't clear. The processing is linear-step-by-step- as I define Analog. I'm hoping for the power and speed of the human brain to be multiplied in an architecture that provides for the advantages of simultaneity, with all the synergism that would entail. Quad Core x (pick a number), plus some quantum leap in information access routes and processing to reach "best outcome" decisions.Think of every dataset compared to every other applicable dataset within-and extant to-the systemall at once, figuratively speaking.
    Think of the Total button on an old calculator.
    Just a thought.
    Barry Dennis
    Gerhard Adam
    Nice ideal, but not likely.  Unfortunately it seems that our "information age" is as much "misinformation" as anything.  Without a means of evaluating and validating this information, the majority of it simply serves confirmation bias, and wastes 10x [or more] of all resources simply to wade through the garbage.

    Mundus vult decipi
    Unfortunately it seems that our "information age" is as much "misinformation" as anything. The majority of it simply serves confirmation bias and wastes 10x [or more] of all resources simply to wade through the garbage.
    Hopefully, one day we'll enter a conceptual age. But the path requiring less energy per individual (since most people do not bother sifting) is consistently the more popular one.