Banner
    Random Reward Schedules and the Ambiguity of Language.
    By Patrick Lockerby | June 7th 2009 06:19 AM | 5 comments | Print | E-mail | Track Comments
    About Patrick

    Retired engineer, 60+ years young. Computer builder and programmer. Linguist specialising in language acquisition and computational linguistics....

    View Patrick's Profile
    Random Reward Schedules and the Ambiguity of Language.


    I had the clue from Pavlov: control your conditions and you will see order.
    B.F. Skinner, A Case History In Scientific Method.


    Skinner discovered that the rate with which the rat pressed the bar depended not on any preceding stimulus (as Watson and Pavlov had insisted), but on what followed the bar presses. This was new indeed. Unlike the reflexes that Pavlov had studied, this kind of behavior operated on the environment and was controlled by its effects.
    Skinner named it operant behavior.
    Julie S. Vargas A Brief Biography of B.F. Skinner


    Reward Schedules.

    The name B.F. Skinner deserves to be as famous as that of Sigmund Freud.  Skinner performed, first with rats and then with pigeons, experiments which throw light on learning behaviour.  His results in what he termed operant conditioning - conditioned responses to something in the environment - are both concise and repeatable.  Of most relevance to this article is the random reward schedule, wherein an animal performing a particular behaviour has to perform the same behaviour an unpredictable number of times before being rewarded.

    A response must first occur for other reasons before it is reinforced and becomes an operant. It may seem as if a very complex response would never occur to be reinforced, but complex responses can be shaped by reinforcing their component parts separately and putting them together in the final form of the operant. Operant reinforcement not only shapes the topography of behavior, it maintains it in strength long after an operant has been formed. Schedules of reinforcement are important in maintaining behavior.

    When reinforcement occurs after an average number of responses but unpredictably, the schedule is called variable-ratio. It is familiar in gambling devices and systems which arrange occasional but unpredictable payoffs.
    Source:  B.F. Skinner Foundation -  Science and Human Behavior.pdf


    Whilst Skinner's behaviourist theories have their applications, their unenhanced extension to language behaviour is, I feel, an open invitation to attack.  Human language is not amenable to analysis as mere chains of causality.   However, language is a type of behaviour, and behaviour is amenable to modification by the methods that Skinner describes.

    Noam Chomsky's review of B.F. Skinner's book Verbal Behaviour has become something of a classic. It is most commonly cited as a personal attack on Skinner.  It most definitely is not.  It is a well-written piece of constructive criticism.  Well-written indeed. 
    One would expect nothing less from the world's most widely-know linguist.
    My reason for discussing Skinner's book in such detail was that it was the most careful and thoroughgoing presentation of such speculations, an evaluation that I feel is still accurate. Therefore, if the conclusions I attempted to substantiate in the review are correct, as I believe they are, then Skinner's work can be regarded as, in effect, a reductio ad absurdum of behaviorist assumptions. My personal view is that it is a definite merit, not a defect, of Skinner's work that it can be used for this purpose, and it was for this reason that I tried to deal with it fairly exhaustively.
    Source: Noam Chomsky, A Review of B. F. Skinner's 'Verbal Behavior'



    Garden path sentences

    A 'garden-path' sentence is a sentence which, when analysed in a serial left-to-right manner, leads to an erroneous conclusion about the meaning of a term or terms.   The way in which the ambiguity of 'garden path' sentences arises strongly suggests that the eye-brain language-analysing system operates through parallel processes and cues.    Many words are classified according to a word or words which follow, used as one or more cues.   In order to misinterpret a word which has not yet been consciously read, cues must be picked up ahead of 'conscious time' so that the conscious mind may be fooled.   The parsing operation cannot be purely serial-sequential.   Consider the following examples:

        1.    The man who owned the green fell.

        2.    To the man who owned the cash money was no object.

    In  sentence 1, parsing appears to proceed in a linear fashion.   Sentences of the form  "the man who…" tend to cue what the man did, i.e., the form is a strong verb cue.   However, the term 'fell' does not carry the normal past tense cue '-ed' and the term 'green' is a strong cue for a following noun.   It seems that the absence of  a secondary instance of a verb cue, in spite of the tendency of such sentences to have two verbs, together with a strong noun cue, causes a parsing failure.  In sentence 2, the two words 'cash' and 'money' will most probably be read together as 'cash-money'.  However, to understand the sentence properly, a comma is needed: 'cash, money'.

    Although such failures of comprehension often result from wrong grammatical classification, the error is inherantly semantic rather than syntactic.    Where there is no such classification error, the parsing error must be based in semantics.   Consider 2., ante.   'cash', 'money' and 'cash money' are all nouns.   

    The initial perception is of an (objective) error in the sentence itself, rather than a parsing (subjective) error in the human reader.    In the case of typographical and grammatical errors, the error appears to be self-evidently objective.   It is demonstrably subjective, however, because the observer is using a 'personal semantic filter' to determine the meaning.   By way of example, consider:

    3.    shed like a garage (sic)

    In the absence of punctuation, the semantic filter which is applied during serial parsing determines that 'shed' has shared properties with 'garage'.   This result may obtain from back-tracking serially from 'garage' to 'shed' and determining a semantic link.   It may equally be the result of multi-tasking.   If the unconscious mind is aware of 'garage' before the term  'shed' has been fully evaluated, it may cause 'garage' to cue 'building' and thus influence the sense assigned to 'shed'.

    Sentence 3 appears incomplete. 'Completion' in a sentence appears to follow these simple rules:

    a.    Ensure that the correct semantic cues are present for each ambiguous term.
    b.    Ensure that any redundant cues are removed.

    Completion ensures that ambiguity is minimized.   

    One way in which example 3 may be completed is:

    4    A shed is like a garage.

    This form of completion, or somewhat equivalently 'correct grammar' provides additional noun cues, and the verb 'is' provides a strong association or set-membership cue.   The sentence is, at least grammatically, unambiguous.

    Consider, however, that example 3 is merely unpunctuated.    It can be completed simply by applying  punctuation, giving:

    5    She'd like a garage.

    If this produces a sudden 'of course' reaction in the reader then the point is well made.   This sudden moment of insight, this 'ahah reaction' this 'eureka moment' is highly rewarding.    It is the reward function of insight that is of interest in the acquisition of language.

    When one teaches a child to read, there are moments when one says "Well done" or some equivalent.   The proverbial 'pat on the back' works wonders if not overdone.   There comes a point, having once learned to read, when a child does not keep looking in the dictionary or asking about new meanings.   The child has learned to derive meaning from context, and finds this ability rewarding.   Gradually the process becomes ever more subconscious, but occasionally the 'jolt' of discovery is so great as to make itself known to the conscious mind.   In this way, the child has learned to 'pat itself on the back', and at randomly reinforcing intervals.   

    New and strange terms will usually be encountered in an entirely unpredictable fashion, and hence reinforced by 'random reward scheduling'.  But just give a child a book filled with 'hard words' and the frequency of occurrence of 'eureka moments' will become somewhat predictable.  The average child will soon cease to find the book 'challenging and hence  'rewarding', and will rapidly tire of reading it.  The reward schedule has become insufficiently random.  This observation applies also, I suggest, to adult readers, but to a degree which reduces with increases in literacy and related skills.


    A brief note on cues and filters

    On reading a new passage and coming across a new word, the child 'just knows' what it means.   All of the necessary cues are usually to be found in the sentence or later in the passage.

        "Twas brillig and the slithy toves
            did gyre and gimbal in the wabe …"

    All of the 'real' words in Lewis Carrol's Jabberwock are strong cues to the semantic categories of the nonsense words:   

    Twas{weather / season / time }          brillig
    and the _y {adjective}            slithy
    the + adjective{noun}                toves
    {noun} did >> {action}                    gyre and gimbal
    {action} in the {location / energy source}    wabe            

    Children (and adults) enjoy Jabberwock because they feel the joy of words.   This leads directly to the notion that we may memorise new terms because of the feeling of reward when we come to understand what those terms mean.    Would we remember nonsense terms under other conditions?   Would we recall many years later supercalifragilisticexpialidociousThe Pobble Who Has No Toes and The Akond of Swat?  Or not?    It is not essential to memory that the nonsense term be embedded in poetry.    Is there not  joy in the discovery of words such as antidisestablishmentarianism?   More to the point, who can forget such a word once heard?

    It is worthy of note that if a document contains occasional humour then its gist tends to be remembered more readily.   Aides memoires tend to be humorous:   "-mites grow up and -tites fall down." (geology).    "Senior officers have curly auburn hair 'til old age" (SOH CAH TOA, sines, cosines, tangents).   "My very educated mother artfully said 'you name planets cleverly' (Mercury, Venus, … Asteroids … Comets.)


    Cues and semantic filters

    The terms 'cue' and 'semantic filter' have been chosen to express something of the underlying mechanisms used in parsing.    My suggestion is that words either label mental representations of external reality or they label 'rules' which must be applied to labels in order to select from the contents of the sets that are labelled.    Words generally may be thought of as existing in an array of sets of sub-arrays.   In the x dimension they are arranged by grammatical rules, in the y dimension by semantic rules.  ( Other dimensions  are undoubtedly required. )

    The cue words and affixes represent rules of the form:  from all sets (x) select only set/s(y1, y2, …yN).   The cues include the function words. They also include affixes and inflections.   For example '-ing' is a strong verb cue, as is '-ed'.   Whilst 'un-' is a strong verb cue, 'non-' is a strong noun cue.    The affix  '-ly' may be thought of as both a strong adverb cue and a strong verb cue, since an adverb requires a verb for completion of its containing sentence.

    Apart from the cues already mentioned, it is inherent in the syntax of most sentences that words occupying specific slots are in some instances definitely, and in others  probably, of particular grammatical classes.   Again, known words tend to act as cues to unknown words via links to semantic cues.   A  verb of elevation carrying a sense of 'up' will provoke puzzlement if linked to a 'down' idea.   That is why "The city was razed to the ground." seems so odd.   At the instant of parsing the term 'razed', the idea 'up' is primed by the brain's 'sounds-like' analyser.   It seems that the 'semantic filter' is primed to accept only verb phrases of the class which have some semantic link to 'city'.   'Raise' is rejected as not logical when linked with the 'down' idea, whereas 'razed' is eventually accepted by the same filtering mechanism.

    It may be that as a sentence is parsed the mind adds to a template - a sequence of labels representing the x-y coordinates of valid sets.   A word is examined to discover if it is a member of the anticipated x-y set.    Usually, the labels are correct, the signaling function of the cues does its job.   Occasionally, a word is found which does not fit the template.   In such case, the template must be amended.

    It is suggested that the back-tracking action consists of a single leap, rather than a step-by-step process.    If a word does not fit the template at point p, then at least one cue has been wrongly interpreted.   That cue must be re-evaluated and the template amended at the cued point.   Parsing is then re-started at that point.


    Concluding remarks:

    Any large enough corpus of samples of language will contain 'garden path' elements.   There is  inherent ambiguity in almost all instances of language.   One's common-sense expectation is that this should make language acquisition more difficult than if language had evolved to completely eradicate ambiguity.   However, it seems that this ambiguity means that, perhaps at a subconscious level, readers experience a 'eureka moment' at somewhat random intervals.   Thus, by simply absorbing the rules of natural grammar without overt tuition we can derive much personal satisfaction from simply understanding novel instances of language use.   In a somewhat literal sense, language acquisition and use is its own reward. 

    Evolution, it seems, has ensured that for the benefit of the entire species, individuals can derive great enjoyment from using oddball-language skills to entertain small children, annoy language purists and confuse high court judges.


    Related articles in The Chatter Box:
    Strangeness And Ambiguity In Language
    Random Noise #14 : Colorless Green Syntax

    Comments

    "Noam Chomsky's review of B.F. Skinner's book Verbal Behaviour has become something of a classic. It is most commonly cited as a personal attack on Skinner. It most definitely is not. It is a well-written piece of constructive criticism. Well-written indeed."

    Really? His arguments were so confused that it took a decade before anybody could figure out who exactly he was attacking. His essay was supposedly a reply to Skinner's Verbal Behavior but Chomsky spent most of his time taking apart S-R psychology (something that Skinner had done many years before).

    MacCorquodale's (1970) reply to Chomsky is a well-written essay. After reading it you have to question whether Chomsky had actually ever read any of Skinner's work as he was entirely clueless as to what radical behaviourism actually was and is.

    http://www.pubmedcentral.nih.gov/pagerender.fcgi?artid=1333660&pageindex=1

    The rest of your article was very interesting though, unfortunately my lack of knowledge of linguistics prevents me from making any substantial comments on your main points...

    logicman
    Mike: thank you for your comment.

    My purpose in writing this article was to suggest that Skinner's behaviour-oriented work deserves to be incorporated into the body of linguistic theory.   I may be alone in this, since linguists have generally followed the path of syntax in pursuit of an understanding of how language works.  I would say that linguists, in rejecting Skinner's work  in its entirety would seem to have rejected the notion that language evolved so that brain may communicate with brain.    I am confident that a re-evaluation by linguists of Skinner's work will demonstrate its value to linguistics.  However, I think that a theory based mainly on behaviourist ideas would be just as wrong as a theory based mainly on syntax.

    A proper study of how language works must reflect knowledge gleaned from many different scientific and literary disciplines.   Such a theory must accept that language use is but one of many possible human behaviours.  A linguistic 'theory of everything' will need to be quite eclectic.  I am writing a series of articles highlighting various aspects of how language works, from the perspective of various disciplines, according to my understanding of the matter.  My over-riding view of language theory is this: 

    No theory of how any human language component works should be considered scientific unless it demonstrates a plausible basis in biology, evolution or psychology
    .
    "My purpose in writing this article was to suggest that Skinner's behaviour-oriented work deserves to be incorporated into the body of linguistic theory. "

    Yes, I had gathered that and it's a very brave position of you to take (one that I happen to agree with though). As I said in my previous comment, I unfortunately lack any deep knowledge of linguistics, however, I've never fully understood why behaviourist principles aren't ultilised more in the study of language.

    I don't see why a behaviourist theory of language would be wrong but maybe as I read through more of your series it will become clearer. I did notice in one of your articles you referred to Skinner as promoting a "stimulus-response" perspective of behaviour which is wrong, so maybe you have a mistaken idea of what Skinner's behaviourism actually is and perhaps it is more applicable to language that you first thought? (Not to mention the huge leaps and advances made in the field since Skinner's day).

    I agree that we'll probably find that language use is an amalgamation of a number of different fields but behaviourism seems like a front runner when dealing with the acquisition and usage aspect of it at least, especially considering its compatibility with genetics/biology.

    logicman
    Mike: thank you for your generous comment.

    behaviourism seems like a front runner when dealing with the acquisition and usage aspect of it at least, especially considering its compatibility with genetics/biology.
    I am confident that the pleasure derived by infants from just hearing speech indicates the operation of some sort of reward mechanism.  To me, that is an important factor in language acquisition and development.
    I did notice in one of your articles you referred to Skinner as promoting a "stimulus-response" perspective of behaviour which is wrong
    I take it you are referring to The Buzzword Blog #4 : Irrational.    I obviously need to clarify what I wrote.  I meant that S-R and operant-conditioning behaviours alone cannot explain human language.  As a simple instance, any utterance, or 'speech-act' may be followed by any other.  There is no hard and fast causal link between conversers' choices of words.
    There are so many components to human behaviour, and to speech in particular, that any attempt at a binary division of a behaviour as type X and type Y is bound to be wrong.  There are so many dimensions of human behaviour that our minds may , to borrow a metaphor from mathematics,  be said to operate in the dimensions of hyperspace.
    Ultimately, I suggest,  language as an evolved biological mechanism must either be understood from a biological/behaviourist/psychological perspective, or not understood at all.

    If you are interested - and I hope you are - most of my Chatter Box articles are about theories of language.
    Hi Patrick,

    (I'm just figuring out the quote function now, so apologies if this doesn't come out right..)

    I am confident that the pleasure derived by infants from just hearing speech indicates the operation of some sort of reward mechanism. To me, that is an important factor in language acquisition and development.

    I don't think it's so much 'hearing the speech' (even though I'm sure hearing funny noises can be quite rewarding for a baby), but rather it's the ability to interact with their parents and environment that is rewarding. For example, reaching forward and flexing their fingers may be fairly successful in getting somebody to hand them their milk bottle, but it's a lot of effort and nowhere near as effective as saying "drink". I think you're on the right track though, that's just how I'd interpret it.

    I take it you are referring to The Buzzword Blog #4 : Irrational. I obviously need to clarify what I wrote. I meant that S-R and operant-conditioning behaviours alone cannot explain human language. As a simple instance, any utterance, or 'speech-act' may be followed by any other. There is no hard and fast causal link between conversers' choices of words.

    This sort of sounds like Chomsky's argument about the black scorpion? That argument only works against simple stimulus-response relations, however, with operant conditioning you don't need a hard and fast causal link between specific choices of words - they may increase or decrease with probability though, depending on their reinforcement history. This also ignores work that's being done in stimulus equivalence and concept formation where they reliably demonstrate how emergent (untrained) behaviours are formed through conditioning - this is the behavioural interpretation of 'insight'.

    If you are interested - and I hope you are - most of my Chatter Box articles are about theories of language.

    Yeah I definitely will. And I'm sure you know about it already, but just in case, you should check out the Language Log blog. I find it quite an interesting read.