Saturday, December 22, 2018

Neurotribes and Authoring Autism - Part 2

Neuroqueer (n): an individual whose identity has in some way been shaped by their engagement in practices of neuroqueering
Neurotypical (n): a label for people who are non-autistic

Preface


In the second of this two-part blog post series about texts on autism, I explore the contours and concepts on offer in Melanie Yergeau's "Authoring Autism". But rather than offering an analytical reading, I have decided to let it speak for itself as much as possible. It is, after all, a text about authoring an identity, and who am I to add another layer to its language? It is full of quotable passages, and my main effort has been to capture essential ideas and serve them up as directly as possible.

Introduction


Melanie Yergeau's "Authoring Autism" is a text about autism that is simultaneously a queering of autism. If we think of autism as a diagnostic that was historically bestowed on the autistic subject from outside the subject, i.e. by the medical establishment, then authoring autism is a way of queering, of interacting with, of reclaiming as well as dismissing, autism by and for autists.

In stark contrast to "Neurotribes", which introduces autism in a way that is accessibly neurotypical, Yergeau's text is dense and more divergent. It is a challenging text defiantly centred in the writing of autistic practice.

The Legacy of Lovaas


If there is one subject that comes in for special and repeated criticism, it is the legacy of behaviourism in autism, and particularly the form pioneered by Ole Ivar Lovaas. He comes in for plenty of flack from Steve Silberman in "Neurotribes", but Melanie Yergeau takes it to a whole different level. That's because this time it's personal.

The author's own experiences of suffering as an autistic in society inform her critique, and no one is immune from criticism. She is not afraid to take on hallowed ideas and characters, even taking aim at the widely adulated Temple Grandin:

"Temple Grandin's routine proclamations that autism teachers should emulate the social practices of the 1950s is not a socially just nor revolutionary approach to neuroqueer sociality, but a demonstrably racialized orientation toward the world. Such autism awareness is better termed perilous than it is positive or gainful."

Grandin's mother also comes in for criticism for writing an alarmist piece that associates autists with child pornography:

"In 2013, Temple Grandin’s mother, Eustacia Cutler, wrote a horrific article for the Daily Beast in which she (quite unempirically and egregiously) warned that autistic men are potential child pornographers lying in wait"

These admonishings are necessary. Temple Grandin may be a hero to many, but her theoretical musings about autistic education do not fit well with progress made in identity politics during the early part of the 21st century.

Having established her willingness to take on all comers, Yergeau reserves the full force of her ire for Lovaas and his legacy of behavioural therapies, which includes the Applied Behaviour Analysis (ABA) offered by Autism Speaks. She introduces its purpose thus:

"To put it briefly, ABA represents a suite of therapeutic modalities whose end goal involves behavioral shaping toward the normative, toward the prosocial, toward compliance"

Applied Behaviour Analysis (ABA) isn't offering value in itself, because the potential it offers is mere "'cosmetic potential' ... in which behaviors are re/directed for 'being more socially acceptable than the behaviors they replace.'". Cosmetic is also masking:

"Recovery, then, is not the process of becoming straight or cisgender or nondisabled, but is rather the process of faking the becoming of normativity". 
If we consider the adage "fake it till you make it", we may consider the difference. Making 'it', in the case of ABA, is an endpoint at odds with the true potential of an autist.

In any case, the point could not be clearer. Behavioural recovery is not true recovery, it is superficial, serving the needs of allistics rather than autists.

The Curse of Zeno


Yergeau's thesis highlights the way in which autistic disability fulfills a rhetoric in which autists are prevented from participating in the world, fixing them to their designated disability like butterflies pinned to a wheel. She invokes the concept of a demi-rhetor, which can be seen as a type of logic that resembles Zeno's paradoxes: the bearer must accomplish an infinite number of moves before reaching a certain destination. Or to put it differently, the bearer will never reach her destination. Yergeau sees clinicians as using this type of logic to undermine autistic agency:

"As a construct, demi-rhetoricity enables clinicians to claim the best of both worlds when they respond to autistic rhetors: 
1. They can argue that autistic people are not autistic enough to make claims about autism.
2. They can likewise argue that autistic people are too autistic to make claims about autism. 
[...] 
While demi-rhetoricity often renders its subjects effectively non-rhetorical, its construction as a kind of rhetorical residue fosters the illusion that the demi-rhetorical are (or can aspire to be) participating bodies."

The implication of this criticism is one that would have confounded Lovaas, whose primary concern was to make autistic children appear non-autistic. However, he didn't see autism as a positive identity, but merely as a lack, as a disability.

Queer


Yergeau is making it clear from the outset that those who have dominated the conversation around autism for the longest time have almost always been non-autistic. It is time to reclaim autism by and for autists.

The establishment has used autism as a route to its proper destination, namely queerness, and Yergeau sees strong links to the mid century gay panic.  She sets out to analyse and deconstruct this harmful rhetoric, and asserts the activity of queering as a positive activity in which autistic subjects engage:

"This project, at root, aims to deconstruct cognitive studies scholarship that reifies the inhumanity and neurological passivity of autistics, while also claiming that autistic people queer the lines of rhetoric, humanity, and agency"

By being engaged in an activity of queering the existing field, which came into existence via rhetorical work done by clinicians and nonautistics (whom she variously calls allists or neurotypical), Yergeau is opening a space in which autists can be fully rhetorical without having to be rhetorical in a way that neurotypical people may necessarily be comfortable with.

"If clinical discourse on autism is, as Duffy and Dorner declare, storied around rhetorics of 'scientific sadness,' then autistic rhetorics, in all of their contrastive resonances, queer the motifs, structures, modes, and commonplaces of what nonautistics have come to narrate and thereby know about autism. To author autistically is to author queerly and contrarily." 

Rhetoric and Agency


She contends that the involuntarity ascribed to autists' actions (eg. stimming) is a way of saying that autists do not have access to and cannot participate in rhetoric. This assumption marginalises those pinned to demi-rhetorical autism:

"involuntarity’s stories are those of abuse, of disbelief, of suffering and non-agency and pain. Involuntarity is forcibly imposed onto autistic bodies [..] Involuntarity is a project of dehumanization"

Rhetoric, as a fundamental socialising activity in which humans participate, is something autists are by that very token excluded from:

"Nowhere is the syllogism clearer:
—One must be human in order to be rhetorical.
—Autistic people are not rhetorical.
—Autistic people are not human."

This double whammy of denying humanity and agency in the same clinical move is a theme Yergeau develops and counters throughout:

"the following remains my chief concern: the ways in which non-rhetoricity denies autistic people not only agency, but their very humanity"

As an antidote she proposes a positive definition of autism, as an embodied neuroqueering that strives towards entelechies, which is to say multiply divergent futures that do not follow a linear trajectory and are neither located at nor aiming at a singular point, but are instead embodied through a striving towards in "verbed forms" of "cunning movements" that demi-rhetorically subvert normal socialised rhetoric:

"queering ... 'confronts all of us with the incommensurabilities of desires and identities and socialities.' [...] To be autistic is to be neuroqueer, and to be neuroqueer is to be idealizing, desiring, sidling [...] a neurologically queer motioning that is asocially perverse, a lurching toward a future that imagines 'incommensurabilities of desires and identities and socialities,' a ticcing toward rhetorical residues [...] Autism is my rhetoric."

The strength of Yergeau's position emerges from seizing the allistic blindspot, the in-between where autistics supposedly lack intention, social rhetoric, and agency:

"Autistic people persist and insist in the narrativity of their tics, their stims, their echoed words and phrases, their relations [...] Autistic stories are interrelational, even if that interrelationality does not extend toward allists, or even humans more generally" 

In setting things up in this way, Yergeau does not opt for a neat and easy delineation, and acknowledges the complexity of this tapestry identity:

"these rhetorical webs of autism and queerness are not just notable for their horrors. They invoke all of the tough, meaty questions that any kind of intersectionality demands. How do we account for where queerness begins and disability ends? It may well be that I am queer only because my neurological disability predisposes me to queerness. But does that matter? What are the consequences of saying that I’m queer because I’m autistic—or, conversely, that I’m autistic because I’m queer?"

The task may be daunting, but as an autist the author brings the weight of her being to bear and  asserts her identity by writing it:

"For my part, I want a rhetoric that tics, a rhetoric that stims, a rhetoric that faux pas, a rhetoric that averts eye contact, a rhetoric that lobs theories about ToM [Theory of Mind] against the wall"

The Blessings of Demi-Rhetoricism


One of the keys that unlocks the power of this identity and reclaims autism lies are the queer concepts of demisexuality and demigenderednes:

"I suggest that demi-rhetoricity holds potential as a reclamatory strategy for those who publicly disclose an autistic identity. Rather than conceptualize identity or rhetoricity as points along a linear spectrum, deminess might instead be queerly viewed [as] about rhetorical attraction or rhetorical desire, and what it means to roll, crip-queerly, outside the bounds of rhetoric"

Rhetoric is a central concept to dismantle because of its close links to other more visible traits that autists supposedly lack. In other words, it is a kind of latent variable in this web of neurotypical assumptions:

"Rhetoric’s topographies shore up that which autistics are time and again claimed to lack: intentionality, symbolic capacity, sociality, and audience awareness, among other rhetorical means"

Rhetoric's links to the social is reinforced again and again: "rhetoric is firmly situated in the 'realm of the social'", "intentionality only becomes rhetorical when it is social", and "intention requires a theory of one’s own as well as other minds".

Yergeau frequently brings up allistics' preoccupation with the social, and their "assumption that allos and autos are binaristic poles, blips on a continuum that speak toward the autistic’s lack of sociality and thereby moral degradation". This is closely related to "rhetoric’s privileging of linear or developmental trajectories, of a social symbolic, and of normatively brained means and motives."

In this regard even Lorna Wing, one of the originators of autism as a spectrum, comes in for criticism:

"If rhetoric is the stuff of tricks and lies, then the unimaginative autist must surely not be the stuff of rhetoric. Indeed, in Lorna Wing’s original triad of impairments, impoverished imagination occupied its own specific domain, broadly encompassing pretense, play, and deception, as well as the ability to think about and predict the actions of others — all necessary preconditions for traditional conceptions of rhetorical manipulation."

The tripartite of "social, communicative, and motoric domains" are "god terms that unite this triad of impairments" and "overwhelmingly concern themselves with sociality and ToM [Theory of Mind]".

Yergeau's concern lies with the real psychological effects of diagnosis, not with the doors it might open with regards to educational assistance:

"Symptoms and test results might indeed represent only a portion of being human, but their rhetorical effects are essentializing"

She makes a similar point later:

"More people are becoming autistic; more people are becoming nonpeople. And so, what does fuck you look like on the part of nonpersons?"

In Kenneth Burke she finds a singular purveyor of the contrasting duality of, on the one hand, rhetorical sociality, and on the other, arhetorical less-than-human autism:

"Burke laments the ocular, arrhythmic style of mathematicians, visual thinkers, and hyperlexics, placing them in direct opposition to the psychogenic symbolic. Although he does not name autism in these examples, he does describe what resembles stereotypes of autistic perception—disconnection from the human and the rhetorical body."

A concept that Yergeau uses to stand this on its head is entelechy, usually seen as the "final destination" of some potential (comparisons with Hegel's Spirit as teleology is interesting but out of scope here).

"Building upon Aristotle, Burke described entelechy as the 'temporizing of essence.' Unlike Aristotle, Burke’s concern wasn’t with the innatism of biology, but rather how narrative comes to be understood, determined, or essentialized. In this way, entelechy is circumscribed in a narrative’s ending: how a story is fulfilled"

In other words, it has a single destination. Instead, following Byron Hawk she she sees the neuroqueering of rhetoric following a multiple and divergent entelechy:

"'Entelechy becomes not the striving for a single, predetermined goal but the striving itself that generates multiple lines of divergence as a residual effect.' Hawk’s embrace of motion rather than end point is intrinsic to the neuroqueering of rhetoric"

The link to queering and the subverting of the normal is later reaffirmed:

"These multiple paths of flight, to channel Hawk, are not equivalent or stationary, but are rather always-unfoldings of rhetoricities that frustrate norms"

Yet in contrast to "rhetorical impulses [that] are often imperialist impulses—whitening, converting, persuading, assimilating", autistic rhetoric is partial, often involuntary - yet no less rhetorical. "Symbolicity may be rhetoricity, but rhetoric is not constrained to the symbolic" (my emphasis):

"Rhetoric is not always narrated, despite narration always being rhetorical: the New York Times best-selling autie-biography is as much a font of rhetoricity as is the autistic child headbanging in a clinic. There might or might not be meaning; there might or might not be symbolic linguistic formation or representational intent; but there are rhetorical effects, there is invention at work, there is rhetoricity."

Yergeau wants to ensure that the reader understands that an autist can be rhetorical even if such rhetoric results from "involuntary" actions.

What emerges is a far more nuanced picture of rhetoric than Burke, or indeed much of mainstream prosocial narration, can afford. Yet in spite of the pain and suffering that autists experience at the hands (the "closed fists") of prosocial rhetoric, a richness of alternative communication and meaning begins to surface, a demi-rhetoric in which autists are the authors.

The Allistic Project Exposed


Autism studies take inspiration from crip and disability studies. For example Fiona Kumari Campbell calls for "an antisocial turn, a politics that refuses respectability", which stands in stark contrast to the demands of for example ABA, in which prosocial behaviour is the goal.

Yergeau also draws attention to gender differences in diagnosis:

"Autism researchers continually debate whether autism’s gendered ratios—ranging from 4:1 to 10:1 male-to-female—are a matter of biology or phallocentric and ciscentric conceptions of developmental disability."

She favours the view of Elaine Day, who points out that girls are taught from a young age to behave more socially, and are therefore able to mask there autism better.

"As autie-biographer Elaine Day describes, 'Social reciprocity, eye contact, and even mannerisms are actually physically taught to us from an early age in an attempt to make sure that we develop into appropriately mannered young women, and the simple fact is that it can make diagnosing AS [Asperger Syndrome] at a young age almost impossible'"

She associates ABA's practices with a variety of paternalistic and political strategies to expose the allistic ideology. First up is the language of surveillance:

"The rhetorical training of ABA might be best understood as a kind of 'we are always watching you.'"

Next is the language of governance and manipulation:

"What practitioners trip over themselves to call a 'science of learning' is in fact a science of regulation and social control"

Then the language of erasure:

"ABA doesn’t remove the neuroqueer — it overwrites it"

And then the language of disability, and keeping disabled:

"Rhetoricity cannot be fully realized in neuroqueer subjects, for neuroqueerness resides. The brain’s capacity for trainability—more plastic and malleable in children, but still plastic into adulthood—requires that intervention be a lifelong endeavor."

Finally, she calls this project out for coercing its subjects:

"if neuroqueer lives are to have meaning, it is through behaviorist meanings, through bending neuroqueer bodies to neurotypical wills."

As an aside, while reading this I could not help but wonder how allism's ideology impacts society more generally, beyond the realm of autists. As Yergeau mentions, "autistic traits, taken together, represent everything that allistics devalue in an audience or social exchange". The autistic problem is a particular allistic construction, but the constituent components of that creation seems to affect all spheres of life.

For example, in the context of experiencing peer pressure to conform to normal prosocial behaviour, to what extent do private individuals, especially those who identify as introverts,  experience and perhaps resent this demand? Conversely, is queering its flipside, the willingness to be unsociable? The answer, I feel, is twofold - firstly in the extent to which that individual is perceived after an initial social encounter: once a social lack has been established, and it is deemed possible to overcome with a carrot and stick, the game begins. But secondly, neuroqueering is a way of attracting rhetorical desire outside the bounds of neurotypical rhetoric:

"autistic rhetorics might be regarded as a way of thinking not about 'how much rhetoric or how much autism can my brain hold,' but rather about rhetorical attraction or rhetorical desire, and what it means to roll, crip-queerly, outside the bounds of rhetoric."

Modes of Rhetoric


The blurring of lines when it comes to rhetoric is lent further support by thinking about rhetorical failure in the absence of autism:

"If we drop autism from the conversation, for example, it is very easy to state that all rhetorical exchanges might be characterized as failure or, at the very least, failable"

Yet the author frequently encounters resistance when suggesting that autism might be rhetorical. She relates a poignant exchange during a graduate student conference. After she suggests that autistic people are rhetorical the backlash ensues.

"One such comment came from the mother of an autistic child ... 'I am deeply uncomfortable in thinking about autism as rhetorical.'"

The author went even further and outed herself as autistic, yet to no avail.

"I was an autistic person declaring myself to be rhetorical, and a posse of Professional Type people fervently swooped in to deny me that right. How dare I consider myself among them, the rhetorical? How dare I insinuate that people like me can rhetorically act?"

Autistic rhetoric is not necessarily convergent to neurotypical rhetoric. In fact, it would seem that more often than not it diverges. This is perhaps the key to understanding neurotypical disorientation, and no doubt mutual misunderstanding. For this reason Yergeau rejects reformed neurotypical rhetoric as amenable to autistic rhetoric and social exchange. She compares and contrasts persuasion and invitational rhetoric, and finds both wanting. Invitational rhetoric, which some find appealing, the author mistrusts:

"it is better termed 'invitation only' wherein marginal bodies are immanently excluded rather than immanently valued" because, as she notes from experience, "an invitation to take antipsychotics might seem optional or self-effacing, but when that invitation is posed by a lab-coated doctor ... invitation might be more accurately read as demand"

Invoking Lovaas again, this distrust is amplified:

"When the stakes are as severe as bodily annihilation, do those without power really wish to invite the presence and purposes of those who would do them harm? What peace is there to be made? When is rhetoric ever safe?"

This point is supported by the fact that "Such interlocutors [of invitational rhetoric] are often white, able, cisgender, and/or straight, agilely drawing upon inventional resources available only to those most rhetorically mobile."

In other words, there is a clear structural power imbalance. As with racism, the process is institutionalised.

"while rhetoric demands reciprocity, it also operates from asymmetrical standpoints"

Diplomatic rhetoric, another rhetorical mode critiqued, in a way encompasses both invitational rhetoric and persuasion:

"Rhetors employ diplomatic methods as a structural means of sometimes persuading, sometimes inviting audiences to a given table."

Diplomacy is also interesting for the way in which it "functions as a metonym for rhetorical exchange, or the kinds of sociality that rhetoric privileges".

Yergeau counters the logic of diplomacy by invoking "a different Zeno, not Zeno of Elea, but Zeno of Citium. A fourth-century philosopher, Zeno regarded rhetoric as a diplomatic exercise, what he termed an open hand. Logic or dialectic, however, was represented by a closed fist." She proposes that, despite expectations to the contrary, "there is value in the clenched hand, the antidiplomatic disclosure. As in, 'Fuck this. I’m autistic.'"

As another aside, in her book on racism Beverly Tatum talks about allies in the white community. I wonder how that might fit into Yergeau's critique of rhetoric.  In other words, how do autists find allies within the allistic community? This might seem problematic given the characterised differences between allistic and autistic rhetoric, and there not be any simple common or neutral ground. However the reasonable starting point would seem to be for allists, who are structurally privileged, to become more aware of the damage done to autistic lives through institutionalised rhetorical and prosocial practices, and to proactively modify it. Perhaps this could even lead to deeper insight into the harm that the general allistic ideology is doing to other allists...

But onwards. The reference to rhetoric's open hand and closed fist goes back to Edward Corbett who

"famously refigured the contours of Zeno’s analogy, suggesting that rhetoric’s open hand often serves as a gatekeeper, preventing marginal bodies from participating in civic spaces [...] [his] reimagining of the closed fist — as provocation rather than logic — was especially indebted to activist strategies, such as boycotts, sit-ins, marches, riots, and even tactical acts of aggression, namely vandalism and violence."

Corbett describes the closed fist rhetoric as "muscular rhetoric" or "body rhetoric". Whereas diplomacy is "for the few", closed fist rhetoric is for the "'dispossessed, the disenfranchised in our society - poor people, students, minority groups - people who do not have ready access to the established channels of communications'".

This applies to autism and racial intersectionality, where "white autistic children might be represented as hapless victims of neurology, [but] autistic children of color are often represented more deterministically and violently, as products of bad parenting or as volcanoes waiting to explode".

Invention


The moment of being diagnosed as autistic is the moment when a life changes forever. "For the neuroqueer ... diagnosis tends to be all consuming and temporally contingent". Countering such diagnosis calls for a way that "promotes a crip-queer ethos", a "kakoethos" that "entails opposing, countering, and neuroqueering that which is typically framed as authoritative and credible."

As we've seen, allism is aligned with normative and prosocial rhetoric and therefore serves as a marker, a "mechanism for regarding the neurotypes of the nonautistic - for calling attention to both a neurological ideal and a neurological ideology".

Kakoethos is the counter value. If allism is associated with centre and linearity, "the kakoethos that attends counter-diagnostic disclosures holds radical potential because it queers perceptions of center, linearity, residence, and rigidity".

Yergeau has deep misgivings about allistic strategies, and her persisting aim is to reclaim a space for autists by neuroqueering allistic rhetoric:

"autistic subjects stake and deny rhetoricity by queering what rhetoric is and can mean, by in/voluntarily middling and absenting themselves from rhetoric’s canons." 

The act of neuroqueering suggests "stepping out of rhetoricity altogether, and questioning the desirability (and at times tyranny) that rhetoricity imparts." It is a uniquely inventive activity, which helps to define autism's positive identity:

"to be autistic is to negotiate inventional movements, movements that straddle the rhetorical and the non-rhetorical, that muddle and murk. Like any inventional movement, autism’s is configured by its coalitional histories."
Following Bre Garrett, Denise Landrum-Geyer and Jason Palmeri, invention is seen as "the 'process of making connections, rearranging materials (words, images, concepts) in unexpected ways.… [Invention] manifests through the body, for a given body actively participates as an inherent material, alongside other materials, other bodies … in the ever-becoming, ever-shifting engulfment of semiosis.'"

The "embodied communication is not a site for intervention, as many clinicians would have us believe, but is rather a site of invention". Such embodied inventiveness also finds traction in the concept of motor schemes, rhetorical schemes that  are "embodied, echoed, and cripped, a perverse way of involuting (i.e., making involuntary) discourse on elocution or rhetorical gestures".

Autistic invention can, and often does, happen at the expense of allistic rhetoric, which tends to be more linear:
"Neuroqueer rhetoricity doesn’t orient on a line but instead collapses shapes and diagrams of all kinds — disorienting and unorienting participating bodies."
This disorientation follows from the activity of demi-ness which has "a queer attachment to remnants". For example gender and sexuality are "fragmentary and oscillating. To claim a demigirl identity, for instance, is to suggest that one holds a partial or shifting relation to some construct of girlness." The demi figure is borrowed from queer communities, and in autism echophenomena like echolalia would be examples of how demi-rhetoric navigates.

Such invention is invention even if it does not expressly intend to invent:

"invention need not be positive to be considered invention ... Echoes proliferate, jump-start, and interrupt... Each echo can constitute its own discursive unit, and, as I’ve been suggesting, it can also serve as a placeholder for multiple meanings. These multiple meanings might spread or shift over time, or they can fill one singular echoing act, signifying multiple meanings or feelings simultaneously." 

In other words, it's what they signify that's important - not what they intend. They do not even require an audience:

"Can we declare echoes as rhetorical failures when they do not seek human interlocutors to begin with? (Answer: hell no.)"

These invented acommunications follow a logic that do not have to conform to language and gesture as understood through the neurotypical lens. For example "echolalia’s meanings lie more in affect or anxiety than they do in the bounds of syntactic units". (The language of dreams comes to mind as vaguely analogous, a 'language' whose meaning relies on an immersive experience rather than syntactical understanding.)

Autistic invention arises from autism's necessary straddling of two worlds, where "autism is a negotiation between rhetorical and arhetorical worlds. And, while at times these worlds may be idiosyncratic or mutually unintelligible, these worlds hold value, meaning, and at times meaninglessness."

Understanding this multimodal experience is to understand the danger inherent in a diagnosis of "autistic spectra", which only serves one of the two worlds: the allistic one.

In a more clinical sense, and following the embodiedness of demi-rhetorical neuroqueerying, it might be better to consider autism as sensorimotor divergence in the way Silvio Savarese proposes:

"Savarese contends that regarding autism as sensorimotor divergence more accurately represents autistic movements between volition and avolition, and it does so in a manner that respects and maintains autistic people’s humanity. In other words, sensorimotor approaches resist spectra and diagnostic fixity."

Activities usually seen as "symptoms" of autism, for example stimming and echolalia, can gain depth and dignity in such a definition. More generally, it also permits a broader view for rhetors in general, by looking at signification rather than meaning:

"Echophenomena do not symbolically represent so much as they immediately, and often acontextually, signify."

Conclusion 


In "Authoring Autism" Melanie Yergeau asserts autism's identity and freedom positively, but never trivially. Autism is a complicated, interwoven subject with a painful history. She borrows from other disciplines, including disability studies and queer theory, and builds her thesis with great purpose. The reader acquires a rich appreciation of both autistic pain and potential:
"Neuroqueer demi-rhetorics are demi-rhetorics of gain and of pain. And without these queer ticcings toward queer futures, we all stand to lose a great deal"

Sunday, December 16, 2018

Neurotribes and Authoring Autism - Part 1

Neurodiversity as a social and political movement is something that's only come to my attention fairly recently, through the lens of autism. Even six months ago I thought of autism in much the way that it was popularised in Rain Man and Little Man Tate: as a specific type of giftedness. The term Asperger's and the phrase 'on the spectrum' has entered the popular lexicon, but is stil often used in a euphemistic rather than a clarifying way.

I now have a much better understanding of what is at stake thanks to two books I've been reading the last few months: "Neurotribes", by David Silberman, and "Authoring Autism", by Melanie Yergeau.

In this blog post I will kick off with "Neurotribes", a wonderfully accessible introduction to the field. In the next blog post I will look at "Authoring Autism", which is a much more theoretical take - and also where things get really interesting.

"Neurotribes" tells the story of autism from the days before psychiatry and the medical establishment entered the fray. It contrasts the careers of socially awkward scientists like Henry Cavendish and Paul Dirac, who were likely to have been high functioning autists, with the fate of children under the psychiatric supervision of later diagnosticians like Leo Kanner. In the latter case the kids usually ended up in psychiatric wards.

It is a heartbreaking story, all the more so for the sensitive progress made in Vienna before the war years, until Hitler's policies tainted everything. Hans Asperger and his colleagues, including Sister Viktorine, constitute one of very few clinical teams to emerge with an enhanced reputation by the end of the book. Asperger afforded his charges dignity by not focusing solely on their disability, but by recognising their unique characteristics and potential, realising that they required different learning methods. He called them "little professors".

The reputations of later autism pioneers like Leo Kanner, Ivor Lovaas, and Bruno Bettelheim fare considerably worse. Through a combination of parental blame (Kanner and Bettelheim) and behaviourist normalisation (Lovaas) they ensured a fraught atmosphere in which the medical establishment inevitable became at odds with the families of autistic children. American psychiatry's narrow focus on the negatives and judging parents as complicit in their becoming autistic meant that families were left alienated and feeling disempowered.

The war years complicates the picture. There is plenty to suggest that Kanner's breakthroughs may not have come without the help of colleagues who immigrated and joined him from Asperger's Viennese clinic - including Sister Viktorine herself.

However the American establishment's belief in a top-down application of medical science over an empathetic, empowering approach meant that exasperated parents eventually started taking matters into their own hands. Bernard Rimland, who founded both the Autism Society of America (ASA) and the Autism Research Institute (ARI), was instrumental in shifting the balance more in favour of parents.

Parents were starting to realise that autistic children didn't need more therapy, à la Lovaas' horrifying behaviourist methods, but instead more recognition for the learning requirements of their children. Meanwhile Lorna Wing, over in the UK, rediscovered the work of Hans Asperger and saw that autism was more of a spectrum than the restricted version Kanner had postulated. Her influence on the expansion of the DSM diagnostic criteria meant that soon more children were being diagnosed than ever before, affording them the official educational assistance they needed. In DSM-V, the most recent version, autism has been redefined to include a much wider variety of related disorders under the bracket of Autism Spectrum Disorder (ASD).

"Neurotribes" also takes a look at the contribution made by "Rain Man", a film that did so much to bring autism to public consciousness. After reading Yergeau, however, it is clear that the next revolution properly starts when autistic people are finally empowered to speak for themselves, and be themselves. The precursor to this revolution comes in the form of an unlikely autism hero, Temple Grandin. Her story is heartwarming in a completely unexpected way, and I can definitely recommend the film version of her life in which Claire Danes stars as Grandin.

What makes Grandin's contribution unique is that, with the help and belief of her mother, she uses her own creativity and ingenuity to find a way to exist in the world that is in line with her autistic nature. The story of how she became an unlikely spokesperson for autism is now legendary, and it is easy to see how her story inspired many who never had a role model to look up to.

"Neurotribes" delineates a clear arc from extreme disempowerment to relative empowerment. This is an important realisation. We start with the pre-medical establishment, when inherited wealth and status in society (Henry Cavendish's parents were Lady Anne de Grey and Lord Charles Cavendish) is practically all that could save a handicapped child from a place like the Bethlem Royal Hospital (commonly known as Bedlam).

The next stage starts with the medical establishment's evolution of psychiatry, as early as Eugen Bleuler's designation of autism as a type of schizophrenia in 1911. However it is only during World War II that psychiatry finally provides autism with a more formal and independent diagnostic, albeit still under huge clouds of confusion and misunderstanding. In the following phase we see ordinary parents increasingly demanding better lives for their autistic children, and gradually taking control from the establishment authorities by forming their own societies like the ASA. Finally, autistic people themselves - people like Temple Grandin - begin to find ways to succeed on their own terms and make their voices heard.

In the internet era, Silberman further suggests, many people living with autism began to see ways of establishing community in the online world - ways that didn't rely on the usual types of stressful socialisation prevalent in "normal" society. In this empowering and interconnected world people with autism have increasingly found ways to assert themselves and make their voices heard. Neurodiversity, both as a social movement and as a part of wider identity politics, has now become part of a global conversation.

Against the backdrop of this brave new world Melanie Yergeau brings her theory rich thesis full of fresh and radical ideas. I discuss "Authoring Autism" in the next blog post.

Monday, August 20, 2018

The Role of Gender in the Hong Kong Film "After This Our Exile"

!WARNING: SPOILERS!

After This Our Exile won numerous film awards, including Best Film at both the Golden Horse and Hong Kong film awards in 2006. It is a touching, often tragic film. A family falls apart when a father indulges his character flaws at the cost of his family. He gambles and borrows, losing money and failing to repay his debts.

His wife (Lin) realises he won't change his ways change and decides to leave him. Their poor young son is caught in the middle. He spots her attempt to leave the first time around, but is blamed when she gets away the next time around. Referred to as "Boy" (at least in the English translations), Lin abandons him too. Her role is simplified, no doubt as a way to focus on the father and son's relationship.

Boy misses her and experiences conflicting feelings of loyalty. However his Dad's influence prevails, and he soon adopts his dad's negative view about his mom. Only when it is too late does he realise  his dad is the real bad apple of the family. His father, impulsive and unwilling to work, forces the boy to steal for money. At this point the boy gets caught and thrown into a correctional facility.

FAfter This Our Exile is therefore a cautionary morality tale. Now that China has an up and coming middle class, the film is perhaps saying don't throw away your parental responsibilities to chase your dreams.

While I found the film genuinely affecting, I lament the missed opportunity to realise the potential in the mother's role. Her character is at first wonderfully interesting, full of passionate restraint as she schemes to escape a dead end life. Sadly this is marred by the father's view of her as merely an unfaithful woman, which Boy believes and is reinforced when she exchanges parental love for a kind of naive yet inconsequential sentimentality about Boy. And so she finally transforms into yet another stereotype, of the lover turned domesticated wife. For someone so strong-willed this doesn't make a lot of sense.

Shing, the dad, is portrayed as a weak-willed character. He is all the more dangerous for having once possessed a dream of success that may have been within reach had he worked at it. He doesn't want to lose face completely and looks for easy solutions. However the interest of the tragic story is based on more than character flaws. A central part of the plot revolves around the particular way in which gender roles play out in the narrative.

The Chinese version of the title is 父子, which literally means "father son". We should therefore be under no illusion that Shing and Boy are the central characters in this story. The moral seems to be that only a father can give his son the right education in life, and when he fails to do so, tragedy will follow.

It is worth remembering that Chinese culture is largely paternalistic, so this moral injunction isn't a surprise to Chinese audiences, nor even the strong filial loyalty, as filial piety is an essential part of Confucian teaching. However to Western audiences such a paternalistic morality is more likely to meet with disapproval as they would expect a more equal, nuanced message about gender roles, such as I have expressed above. However it is precisely the strict partitioning of roles that proves instructive about the forces that drive the story.

Lin, the mother, runs away but fails (the first time) due to a premonition the boy has. In a powerful and dramatic early scene Shing apprehends her and takes her home in a fury. He treats her badly in what appears to be a domestic pattern of abuse, he also ignores her accusations about his bad habits. Instead he becomes very emotional - almost histrionic (a character trait that, in the West at least, has a long and unhappy association with women).

On the other hand Lin keeps her cool and gets to the point, even if it takes her a while to open up. It is a very powerful scene. Shing remains in denial at first, but eventually succumbs to the truth. It appears to be out of deep love, but we soon learn that he also needs Lin to help him pay his debts. She is the provider. His love has a dark side.

She, on the other hand, manages to win back his trust to win herself time. He clearly believes in his own masculinity, as they make love that same night in a scene that gives an insight to Lin's precarious position. She still finds him attractive enough to give in to their passion, but the viewer is aware that she may have something up her sleeve and probably needs to keep him on her side.

It is this 'cunning' element of her character that is juxtaposed with his more straightforward bad character. The question is hinted, if not exactly asked: is it worse to be honest and emotional, yet a hopeless case, or more capable but a bit cunning and hypocritical? Given the way things play out, it is clear that Shing's character is judged in the negative. His is a bad sort who fails his own son in the worst kind of way. But what of Lin? Could she have saved them by staying on?

I think the answer is no, she was always more ambitious, and the context is about the father-son relationship being the backbone of society. However this also shows the way in which Lin's character is problematic. She has to leave because by staying and saving her family she would be fulfilling the father's role. She would become the backbone that rises through the slackness of her husband's lack of moral fibre. And this will not do. She can't be the man, she can't be the one to wear the trousers. Instead, it is better that she disown her family in a double negative, moving from female victim to active seeker of happiness in the arms of another man who happens to be rich and successful - even if it makes her look flighty. She is not even evil, incapable of real evil - just inconsequential.

Given the film's ending it is probably safe to assume that the film doesn't directly acknowledge society's role in Lin's decision. Or to put it differently, it doesn't acknowledge that her choices are by default highly constrained. It's a lose-lose situation, morally speaking, so she might as well choose the option in which she gains something.

Although she does not want to cut herself off from Boy completely, she acts in her own interest for the new baby, and the narrative turns her parental care into distracted sentimentality. She completes a double negative leaving Shing and Boy to their circle of masculinity while she pursues a new motherhood. She is diminished by being seen as a giver of children, not a giver of souls, which is the role demanded from Shing.

It is perhaps slightly unfair to suggest that the film intentionally sets out to paint Lin as mere stereotype. After all, it was meant to be about the father and the son. But as pointed out, the film does not acknowledge the society's influence in her options. The film's limited view of gender roles has repercussions when, ultimately, the father fails to be a father - to be a man of substance - and we are left to wonder whether his excessive masculine posturing isn't partly responsible for his failure in fatherhood.

Boy, then, is the only character left to fulfil the expectation of being a man, and, being a boy, he cannot do so. It is only at the end of the film, once he has grown up, that he sets out to right those wrongs. He shows some of the backbone that both his parents lacked.

Sunday, August 12, 2018

Why Are All the Black Kids Sitting Together in the Cafeteria? : Learning about Racism

"Why Are All the Black Kids Sitting Together in the Cafeteria?", a book on race relations first published in 1997, has opened my eyes about the nature of racism and what we can do about it. I can't really do the book justice in a blog post, but I would like to highlight some of the points that have made an impression on me.

By way of quick introduction, the author Beverly Daniel Tatum, PhD, is a psychologist as well as an educator. She originally wrote the book in answer to questions about race she would often receive, in particular the one in the title, from well-meaning White teachers who were concerned and perplexed upon seeing black kids sitting together in the cafeteria and not mingling.

The book draws on a variety of research studies while avoiding abstract theory, focusing instead on concrete examples. This makes the book very accessible without skimping on credibility. The version I read is the recently updated 20th anniversary edition. The first version is already a classic in the field.

I'm sure everyone who reads it will have their own a-ha! moment. I personally had several. Yet there was one that really stood out. It was the realisation that my fundamental assumption about racism was inadequate. Like many other people apparently, I equated racism with a kind of prejudice towards people of other races. Being a good citizen, this understanding of racism meant I could check my attitude and behaviour and feel confident that, yes, I am not being racist in my daily life.

However that is a fairly superficial definition of racism that does not get at the heart of the problem. To put this in perspective, my understanding of how racism developed has forever been changed by a recent trip to Washington DC during which I visited the African-American Museum. It tells the story of the global slave trade during the colonial era, the immense suffering of the slaves who were sold and bought as chattel, and their resistance and perseverance over centuries to find a better way and a better life. In one single afternoon I learned more about slavery and its consequences than I'd ever known before.

As a UK citizen born in South Africa I also had occasion to ponder how slavery and racism have manifested in different ways on three different continents, but that is a whole analysis unto itself. For present purposes, suffice it to say that a historical perspective, of slavery and its consequences in particular, is essential to understanding what racism is. Those consequences are an ideology of White privilege that have been ingrained in culture and set in laws over centuries.

What that means is that racism is structural in nature, and that existing societal structures are racist inasmuch as they favour White people over Black people (and other races). This is the crux of the matter, and that is why the idea of racism as a prejudice is of limited use. To dismantle racism we can't only check our attitude and biases, we have to go much, much further.

Tatum quotes David Wellman, who defines racism in this sense as a "system of advantage based on race" (p. 87). Another definition of racism often used is that of "prejudice plus power", which explains how the structural inequality comes into being and has been enforced:

"Racial prejudice combined with social power - access to social, cultural, and economic resources and decision-making - leads to the institutionalization of racist policies and practices" - p. 87

However this definition, Tatum concludes, has one drawback that on a practical level many White people feel that they do not have the 'power' that is being alluded to. She therefore prefers Wellman's definition. Nevertheless, I would add that this definition does indeed describe how racism was *initially* instantiated. It is now maintained simply through the ongoing maintenance of the status quo that was perviously established by those with social power. Only when the status quo is challenged can this historical reality be uncovered and seen for what it is, namely the construction of racism.

The second big idea I encountered was that of intersectionality. This is a term that has had a fair amount of coverage in the popular media, but I never really looked at it closely. In practical terms a person's identity may form along several axes of distinctiveness, or otherness, of which Tatum highlights seven: race or ethnicity, gender, religion, sexual orientation, socioeconomic status, age, and physical or mental ability (p. 103). Following another book I'm currently reading (Neurotribes by David Silverman) I would probably expect neurodiversity to eventually join this list.

The point is that each of these categories of otherness has a form of oppression associated with it: racism, sexism, religious oppression / anti-semitism, heterosexism, classism, ageism, and ableism (p. 103).

As an example of intersectionality, one's identity might form along the following manifested centres of experience: black, female, Christian, lesbian, working class, middle aged, and healthy. Any part of the identity not in the dominant or normal side of the category means that the individual will experience oppression or discrimination in some way or form.

When it comes to race, White is the dominant and normal race. As a result it is not uncommon for a White person to not really self-identify in terms of race. Tatum quotes Debby Irving in her memoir Waking up White:

"The way I understood it, race was for other people, brown and black-skinned people. Don't get me wrong - if you put a census form in my hand I would know to check 'white' or 'Caucasian'. It's more that I thought all those other categories, like Asian, African American, American Indian, and Latino, were the real races. I thought white was the raceless race- just plain, normal, the one against which all others are measured" - p. 186

However there is "a hidden cost of racism for Whites" (p. 187), namely the experience of psychological discomfort whenever racism is brought up - guilt, shame, frustration, even anger. The absence of a racial identity in the case of Whites, Tatum contends, is the root cause of this discomfort. She notes that a common reaction for the White person, once they become aware of racism and their own role in it as a White person, is to conclude that they need to have more interactions with Black people or make friends with Black people.

She explains that a more fruitful approach is to develop a positive White identity first of all. This point was another big a-ha! moment for me, because I have experienced that psychological discomfort myself. Combined with the perception of racism as a form of prejudice, silence often seems like the safest route in the face of uncertainty - and yet of course it does not change anything. But with a positive racial identity there would be a foundation to work from and things can begin to fall into place.

While acknowledging that there is no set recipe, Tatum offers practical advice to encourage White intragroup conversations and help develop such a positive White identity:
- find other Whites who have already progressed along the way and can show you what to do
- read autobiographies and biographies by White anti-racist activists, like "A Season of Justice" by Morris Dees, or "White Like Me" by Tim Wise
- participate in White anti-racism consciousness raising groups (eg. Showing Up for Racial Justice (SURJ))

She provides perspective on the need for all-White support groups and the function they fulfill:

"Particularly when Whites are trying to work through their feelings of guilt and shame, separate groups give White people the 'space to speak with honesty and candor rarely possible in racially mixed groups'. Even when Whites feel comfortable sharing these feelings with people of color, frankly, people of color don't necessarily want to hear about it" - p.205

The onus is on the White individual to do the work and develop his or her White racial identity, making it into something positive.

The question of identity is also central on the side of Black people to gain insight into the question in the title: why are all the Black kids sitting together in the cafeteria? During pre-adolescence, race isn't viewed in the same way by kids because their identity has not been fully formed yet. But during adolescence new social factors come into play (who is dating who, who is friends with who, what is my future?, etc.) that become increasingly important. Black kids are then often drawn together by their shared experiences of being institutionally othered and oppressed by the rest of society. In other words, by the sort of structural racism that does not affect the dominant White group much.

"In the prepuberty stage, the personal and social significance of one's REC[racial-ethnic-cultural]-group  membership has not yet been realized, and REC identity is not yet under examination [...] During adolescence their understanding evolves to include not just more about themselves but also more about their group, including an 'understanding of a common fate or shared destiny based on ethnic or racial group membership and that these shared experiences differ from the experiences of individuals from other groups" - p. 135

In the face of such experiences, being part of a larger group of people who understand one's situation and experiences is a benefit. Therefore the concerns of those White teachers who asked why they are sitting together are valid, but misplaced. The root of the problem should be sought in the institution of racism, and how to dismantle it, and not directly in the behaviour and thoughts of those being othered and oppressed.

This brings us to another important point, namely what those of us who are on the privileged side of the equation can do about it. The book points out on more than one occasion that those who are oppressed do not want us to speak for them because they have their own voices. So what is it that we can do? Plenty, as it turns out. For starters, by starting in our own sphere of influence and pointing out when someone has made a racist comment or joke can change awareness.

One of my favourite examples is actually in the context of sexism, but it could as easily have applied to racism. It happens when Andy Murray corrects a journalist for referring to Sam Querrey as "the first US player since 2009 to reach a major semi-final since 2009". Of course, Serena Williams (and other US women tennis players) have been winning plenty since 2009. The counter-argument that the context was implicitly "men's tennis" is almost the point, because the same can be said about all institutional racism and sexism: the existing context, or status quo, can only be exposed by drawing attention to it.

As White people we have more social power than we often realise, and even simple interventions, like the way Andy Murray used his influence in the media, can make a powerful statement.

There is tons more excellent material in the book, and I've glossed over much at the expense of nuance. Three more worth mentioning in passing include the need for affirmative action and for goal setting in affirmative action programs; the challenges of aversive racism; and how to counter the influence of bias in decision-making.

Rather than go into all of them, I want to highlight one last point that really stuck out for me. In the final section of the book Tatum discusses racism and the experience of racism in the context of other ethnic and racial groups in the US, including Native Americans, Asians, Latinx, and others.

In the case of Native Americans - a catchall name for many different communities - researchers like Paul Ongtooguk have noticed that such communities have been reduced to static stereotypes in the public mind. Even when their traditions have been preserved it is usually presented in terms of how things used to be once upon a time. In other words, it gives them no sense of their current existence, nor of their future.

While the traditional arts and crafts were worthy of study, the curriculum embodied a "museum" perspective whereby the traditional life of Alaska Natives was studied "as an interesting curiousity commemorating the past." Ongtooguk explained, "The most disturbing picture of Inupiaq culture, then, was of its static nature - something that had happened 'back then' rather than something that was happening now. Did this mean that the people living in the region now were like a cast of actors who had run out of lines?" - p. 267

Ongtooguk focuses on creating study materials that allow American Native students to see themselves in the future. This future oriented imagination is an important part of the continuity of community identity, and therefore of their cultural survival as a distinct group, and almost certainly of their capability to thrive again in the future.

It is worth summarising these insights once more:

1. Racism is institutionalised and structural, not just a question of conscious prejudice
2. White people should develop a positive White racial identity that does not deny the reality and history of racism, but acknowledges, addresses, and helps to dismantle it
3. Personal identity formation is a complex process influenced by highly individual combinations of intersectionality
4. The survival and prosperity of a community lies not only in preserving its past, but also in connecting to its present and actively imagining its future


This has without doubt been an eye opener for me. From a practical point of view, and from my personal perspective as a middle class White male, the second point is a clear call to action.

Sunday, July 22, 2018

Detecting Similarity of Textual Style and Content

similarity.py performs rudimentary detection of textual style and content. Basically it uses the predictive capability of the pytorch-char-rnn autoencoder to check the likelihood of a character in a provided input text against character sequences in an existing trained model (trained on some other text).

The average of likelihoods across the provided input text is calculated to provide a broad indication of the similarity of style and content of the input text compared to the original text on which the model was trained. In particular it provides a similarity score as a percentage (higher means more similar).

For example a sentence from the original modelled text should come up with high similarity, typically scoring over 97%. A text in the same language, but written in a very different style might score over 90% but not as high.

An input text written in a totally different language should score significantly lower, eg. 80-85%. If the texts do not share all the textual characters, for example the Turkish alphabet compared to the Roman alphabet, the score will drop even more.

Under the hood the script actually detects variance, and then converts it to a similarity score for convenience. The lower the detected variance, the more like the original text the provided input text is.

The script is provided as part of my pytorch-char-rnn repo.

Below are some examples:

Example 1: Compare English text from Jane Austen's Persuasion with a model trained on Jane Austen's fiction.

python2.7 similarity.py \
--text "Sir Walter Elliot, of Kellynch Hall, in Somersetshire, was a man who, for his own amusement, never took up any book but the Baronetage; there he found occupation for an idle hour" \
--checkpoint checkpoints/austen_checkpoint.cp \
--charfile data/austen_chars.pkl 
Parameters found at checkpoints/austen_checkpoint.cp... loading

Detected similarity: 99.15%

Example 2: Compare German text from the Bible with a model trained on Jane Austen's fiction.

python2.7 similarity.py \
--text "Am Anfang schuf Gott Himmel und Erde. Und die Erde war wüst und leer, und es war finster auf der Tiefe; und der Geist Gottes schwebte auf dem Wasser." \
--checkpoint checkpoints/austen_checkpoint.cp \
--charfile data/austen_chars.pkl 
Parameters found at checkpoints/austen_checkpoint.cp... loading

Detected similarity: 83.84%

In principle the technique can be improved by creating a larger window for comparison. In other words not just character by character, but character sequence by character sequence across a moving window. A bit like LSTM in reverse. It isn't clear whether all the information is available to make this possible, I'll have to do a bit of digging around the model's saved state.

I'll leave that as an exercise for another day.

Monday, February 19, 2018

Syntax Char RNN for Context Encoding

Summary


Syntax Char RNN attempts to enhance naive Char RNN by encoding syntactic context along with character information. The result is an algorithm that, in selected cases, learns faster and delivers more interesting results than naive Char RNN. The relevant cases appear to be those that allow for more accurate parsing of the text.

This blog post describes the general idea, some findings, and a link to the code.

Background


As both a writer and a technologist I have for some time now been interested in the ability to programmatically generate language that is at once creative and meaningful. Two of my previous projects in this context are Poetry DB and Poem Crunch. I also wrote a novel that incorporates words and phrases generated by a Char RNN that had been trained on the story's text.

Andrej Karpathy's now-famous article on RNNs was a revelation when I first read it. It proved that Deep Learning can generate text in ways that at first appear almost magical. It has afforded me a lot of fun ever since.

However, in the context of generating meaningful text and language creatively, it ultimately falls short.

It is helpful to remember that Char RNN is essentially an autoencoder. Given a particular piece of text, let's say this blog post, training will build a model that, if fully successful, will be able to reproduce the original text exactly: it will generate exact copies of the original text from which it learned and created the model.

The reason for Char RNN's widespread employment in fun creative projects is its ability to introduce novelty by either tuning the temperature hyperparameter or, more commonly, as a side effect of imperfect learning.

To be sure, imperfect learning is the norm rather than the exception. For any text beyond a certain level of complexity, a naive Char RNN will reach a point during training when it can no longer improve its model.

This naturally leads to the question, can the Char RNN algorithm be enhanced?

Context encoding


Char RNN encodes individual characters, and the sequence of encodings can be learned using for example LSTM units to remember a certain length of sequence. Aside from the relative position of the character encodings, the neural network has no further contextual information to help it 'remember'.

What would happen if we added other contextual information to the character encodings? Would it learn better?

Parts of Speech


Parts of Speech are structural parts of sentences and a fairly intuitive candidate for the problem at hand. Although POS parsing hasn't always been very accurate, SyntaxNet and spaCy have been setting new benchmarks in recent times. Even so, accuracy might still be a problem (more on that later), but they certainly hold promise.

So how does POS parsing fit into Char RNN?

Let's take a look at the following sentence and its constituent parts.

Bob   bakes a  cake
PROPN VERB DET NOUN

We can see that the 'a' in 'bakes' and the 'a' in 'cake' are contextually different. The first is part of a verb and the second is part of a noun. If we were able to encode the character and POS together, for each character across the whole text, we would cover a sequence longer than is practical for an LSTM to remember. In other words, the model would understand syntactical structure in a more generic sense than with naive Char RNN.

B + PROPN
o + PROPN
b + PROPN
[space] + SPACE
b + VERB
a + VERB
k + VERB
e + VERB
s + VERB
[space] + SPACE
a + DET
[space] + SPACE
c + NOUN
a + NOUN
k + NOUN
e + NOUN

One way of achieving this is, for each character,  to create a new composite unit that captures both the character and the pos type. So if we create separate encodings for the characters and the pos categories, eg. a = 1, b = 2, etc. and NOUN = 1, VERB = 2 etc., then we could do something along the lines of:

a + VERB = 1 + 2 = 3

However this creates a new problem, namely one of duplicates. I.e. we'd end up with lots of cases that have the same final encoding (in this example, the final encoding is 3):

a + VERB = 1 + 2 = 3

and

b + NOUN = 2 + 1 = 3

A better solution would have to ensure the encodings are completely separate before sorting them back into consecutive indexes to ensure uniqueness.

But the real problem with this solution is that, although we now have an encoding influenced by both characters and types, we've lost each unit's individual quality. In other words, the importance of a character encoded along with one type of POS unit is no longer properly weighted against a character of a different type of POS unit. Instead, it has simply become a composite type of its own.

An improved approach would be to encode both the character and the type as independent data, albeit of the same unit, and let the LSTM do the rest.

A Syntax Char RNN tensor might then look as follows:

[[ char, type ], [ char, type ] ... ]

However this heavily favours the type encoding over the character encoding, which in turn will skew the weightings.
A more balanced encoding might be:

[[ char, char, char, char, type ], [ char, char, char, type ] ... ]
              word 1                          word 2

A sense of unevenness remains, because some words are longer than others: why should each word receive just one type encoding? This is something left as a refactoring improvement for later.

For the time being, experimentation showed a kind of optimum results from adding two type encodings per word, as follows:

[[ char, char, char, char, type, type ], [ char, char, char, type, type ] ... ]
                  word 1                              word 2

Results


Char RNN can generate surprising turns of phrase and novel combinations of words, but longer extracts often read like gibberish. The hope was that context encoding might improve this state of affairs by strengthening the overall sentence structure represented in the model.

The SyntaxNet installation also installs DRAGNN and a language parsing model. Due to problems I had getting consistent results from SyntaxNet, I eventually settled on DRAGNN instead.

DRAGNN


Shakespeare


The first benchmark was based on the Tiny Shakespeare corpus. The following snippets are from checkpoints with equivalent validation loss, trained using the same hyperparameters (allowing for the proportionately longer sequence length in the Syntax Char RNN due to the additional type encodings).

Naive Char RNN (temperature: 0.8)

Capulet:
with him; there were not high against the nurse,
and i, as well as half of his brother should prove,
thou ask my father's power,
in this life of the world command betwixt
of greep and displent, rup in manrown:
and thou dost command thy stors; and take our woes,
that star in the sea is well.
 Now is a cunning four gloves of all violer on
himself and my friend traitor's jointure by us
to be holy part that were her horse:' the miles
for this seat with me in the island from scards
shall have stone your highness' weech with you.
 And unjust while i was born to take
with hardness from my cousin when i forget from me.

 Shepherd:
the unrelery, reign'd with a virtuous tongue,
to blush of his harms, and as sweet, if they
cape of england's purple say well true song hence,
shall appetite straight hath the law with mine?
 The composition should know thy face of my heart!

 Second huntsman:
i know him, and with my chast my mother.

Syntax Char RNN (DRAGNN parser; temperature: 0.8)

ITrue words, his son, fair despair is, and the guard brother; 
always you tell, 
though to see so many trees; i come!
 
Pedant:
 would you have no redress of joy of the march?
 
I, what i come up.
 
Menenius:
 i not a gentlemen cred; and as this old farewell.
 
Lucio:
 sir, she's a duke.
 
Capulet:
 so straight is the tyrant, shape, madness, he's weigh; 
which of the confail that e'er said and gentle sick ear together, 
we will see the backs.

Note: Syntax Char RNN output has been reformatted, but remains otherwise unaltered


I think it is easy to agree that the naive Char RNN generated text reads significantly better. There is a somewhat interesting punchiness to Syntax Char RNN's shorter dialogue sections, but that's about all it has in its favour.

This was, frankly, disappointing.

However, there is a reasonable chance that inaccurate parsing could be influencing the results. The DRAGNN model probably doesn't generalise well to Shakespearian English.

Would prose offer a better benchmark?

Jane Austen


The works of Jane Austen was used next. They would almost certainly parse more accurately.

The results this time were rather surprising. Syntax Char RNN raced away, reducing its loss pretty quickly. After one hour and fifteen minutes on my laptop CPU, Syntax Char RNN hit a temporary minimum of 0.771.

After the same time frame and with the same hyperparameters (again allowing for a slight adjustment of sequence length due to the extra type encodings for POS), naive Char RNN went as low as 1.025 - still nowhere near the Syntax Char RNN checkpoint.

I left it running overnight and it still reached only 0.944 after just over 8 hours.

This was interesting.

What about the quality of the generated text?

Naive Char RNN (loss: 0.944; temperature: 0.8):

"i beg young man, nothing of myself, for i have promised to be whole 
that his usual observations meant to rain to yourself, married more word
--i believe i am sure you will allow that they were often communication 
of it, but that in that in as he was in the power of a hasty announte 
by print,  to have given me my friend; but at all," said lady russell, 
so then assisted to  his shall there must preferred them very ill--
considertaryingly, very pleasant, for my having forming fresh person's 
honour that you bowed really satisfied we go.

Syntax Char RNN (DRAGNN parser; loss: 0.771; temperature: 0.8)

Mr. Darcy was quite unrescrailous of place, which and want to carry 
by the attractions and fair at length; and if harriet's acknowledge by 
engaging it for a sorry day he must produce ithim. I could be 
mr. Knightley's compray of marrying in the rest of the disposition, 
and i was particularly successful. He has been much like her sister 
to julia, i wishin she believed he may want pause by the room with 
the same shade. "" but indeed you had obliged to endeavour to concern 
of a marriage of his yielding.""


The results are roughly comparable, neither are special. If pushed I'd say I prefer the latter over the former, it reads a little better.

spaCy


spaCy is an amazing set of tools made available by the good folks at Explosion AI. Unlike SyntaxNet, or even DRAGNN, it is a breeze to use.

The spaCy parser's data had an interesting effect on training. A training run with DRAGNN data reached a loss of 0.708 after just under 6 hours, then failed to go lower for the rest of its total run of over 16 hours. The spaCy parser achieved 0.686 after 5.5 hours, and its best loss of 0.650 after just under 8 hours.

Here are snippets from the relevant checkpoints.

Syntax Char RNN (spaCy parser; loss: 0.686; temperature: 0.8)

You beg your sudden affections and probability, that they could not 
be extended to herself, but his behaviour to her feelings were 
very happy as miss woodhouse--she was confusion to every power 
sentence, and it would be a sad trial of bannet, but even a great 
hours of her manners and her mother was not some partiality of 
malling to her from her something considering at mr. Knightley, 
she spoke for the room or her brother.

Syntax Char RNN (spaCy parser; loss: 0.650; temperature: 0.8)

In the room, when mrs. Elton could be too bad for friendship by it 
to their coming again. 
She was so much as she was striving into safety, and he knew 
that she had settled the moment which she said," i can not belong 
at hints, and where you have, as possible to your side, 
i never left you from it in devonshire; and if i am rendered 
as a woman," said emma," they are silent," said elizabeth," 
because he has been standing to darcy, and marianne?"" oh! 
No, no, well," said fitzwilliam, when the subject was for a week, 
that no time must satisfy her judgment.

Note: Lines have been wrapped, but formatting remains otherwise unaltered

Both are quite readable, except for the injudicious use of quote marks (a problem that is most likely the result of redundant spaces picked up during pre-processing).

Even allowing for over-fitting, it is quite clear that in the case of Jane Austen's text the snippets generated by Syntax Char RNN are more readable and cohesive than those from naive Char RNN. Among the former, those produced via spaCy also show a marked improvement over the results produced via DRAGNN parsing.

Since the only significant difference between the Syntax Char RNN runs trained on Jane Austen texts were the data from the two different parsers, these findings suggest that accuracy of parsing between DRAGNN and spaCy likely accounts for the difference in performance and readability between the runs. This in turn suggests that a lack of accurate parsing accounts for the poor results achieved with Tiny Shakespeare.

Code


The code is available on github. Comments and suggestions welcome.

The majority of work was around pre-processing (parse and prepare). For training and sampling I was able to build on the existing Pytorch Char RNN by Kyle Kastner (which in turn credits Kyle McDonald, Laurent Dinh and Sean Robertson). I altered the script interface, but the core process remains largely the same.  

Concerns and Caveats


The approach and implementation isn't ideal. Below are some considerations.

  1. Pre-processing is complex. It has to make assumptions about the parser input, bring character and syntax encoding together, and try to remove data that can skew the weightings.
  2. Pre-processing can be slow. It can take anything from a few seconds to tens of minutes, depending on the size and complexity of the file.
  3. POS parsing is imperfect. The results suggest spaCy is doing a better job than DRAGNN, but even at its best it will have errors.
  4. Applicability is limited to text that is at least consistently parseable by an available parser. Poetry, archaic language, social media messages etc. likely fall outside this scope.
  5. Some POS units are known to cause skewing by introducing extra spaces. For example "Mary's" becomes:
Mary : PROPN++NNP
's    : PART++POS
All punctuation are separated out as well, for example:
, : PUNCT++,
The effect is that these parts of speech remain tokenised when encoded, resulting in redundant spaces on one side of each token. Unless they are subsequently removed, spaces become overrepresented in the resulting encoding, affecting weightings for all character representations ever so slightly. The code manages to remove some of these redundant spaces - with occasional side effects - but not all.

To Do


Avenues to investigate, features to add.
  1. Reduce the number of redundant space encodings
  2. Calculate a more granular weighting for type
  3. Investigate further candidates for context encoding, over and above syntax
  4. Investigate more elegant ways of grouping encoding contexts
  5. Add validation loss for comparison to training loss
  6. Estimate parser accuracy for a specific text
  7. Run on GPUs!

Conclusion


The findings suggest that in some cases where parsing is accurate and consistent, Syntax Char RNN trains faster and achieves better results than naive Char RNN. This lends support to the hypothesis that accurate contextual encodings, over and above syntactic Parts of Speech, can improve Char RNN's autoencoding.

While they come with several caveats, the findings nonetheless warrant further experimentation and clarification.