Saturday, May 02, 2020

BERT for Short Short Stories

As a creative writer I'm always on the lookout for new developments in NLP and language modelling. With the advent of the new Age of Machine Learning there was a lot of promise that creative breakthroughs might be around the corner. There was an early burst with the work of people like Ross Goodwin's Sunspring in 2016, and Botnik Studios' Harry Potter and the Portrait of What Looked Like a Large Pile of Ash in 2017.

However this momentum appears to have stalled more recently, and the most interesting AI collaborations have been in the visual arts instead, highlighted by Obvious' auctioned Portrait of Edmond Belamy, but even more so by the avant-garde work of serious artists like Mario Klingemann.

With incredible language models like GPT-2 and XLNet now openly available, it is disappointing to note a comparative lack of collaboration between creative writing and these advances in AI. Is it perhaps a case of more not really being better, when it comes to language generation? Like that scene in The Matrix Reloaded where the CGI was amazing for its time, but not quite convincing enough to carry the story.

But predictive generation isn't the only NLP game going at the moment, and BERT is another model that has garnered a lot of interest. In short, its relative success in language understanding has made it suitable for various related tasks.

One such task is text summarisation. I recently discovered the Bert Extractive Summarizer, which makes this incredibly easy to do (there's an online version you can try out - although it has some limitations). I decided to play with a selection of famous short stories, and the results are quite fun - a bit like micro stories in their own right.

Here are 5 examples. Some of the originally longer stories required a smaller ratio than the default (0.2).

The Garden Party, by Katherine Mansfield


They could not have had a more perfect day for a garden-party if they had ordered it. "That's right, miss," said the tallest of the men, a lanky, freckled fellow, and he shifted his tool-bag, knocked back his straw hat and smiled down at her. " Laura's upbringing made her wonder for a moment whether it was quite respectful of a workman to talk to her of bangs slap in the eye. She crouched down as if to warm herself at that blaze of lilies; she felt they were in her fingers, on her lips, growing in her breast. Laura caught hold of her sister's sleeve and dragged her through the kitchen to the other side of the green baize door. If some one had died there normally - and I can't understand how they keep alive in those poky little holes - we should still be having our party, shouldn't we?" "I don't understand," said Laura, and she walked quickly out of the room into her own bedroom. What did garden-parties and baskets and lace frocks matter to him?

The Mask of the Red Death, by Edgar Allan Poe


Ratio : 0.2

The "Red Death" had long devastated the country. Blood was its Avatar and its seal--the redness and the horror of blood. The external world could take care of itself. The prince had provided all the appliances of pleasure. There were buffoons, there were improvisatori, there were ballet-dancers, there were musicians, there was Beauty, there was wine. There was a sharp turn at every twenty or thirty yards, and at each turn a novel effect. The second chamber was purple in its ornaments and tapestries, and here the panes were purple. The seventh apartment was closely shrouded in black velvet tapestries that hung all over the ceiling and down the walls, falling in heavy folds upon a carpet of the same material and hue. There was no light of any kind emanating from lamp or candle within the suite of chambers. But in the corridors that followed the suite, there stood, opposite to each window, a heavy tripod, bearing a brazier of fire, that projected its rays through the tinted glass and so glaringly illumined the room. But when the echoes had fully ceased, a light laughter at once pervaded the assembly; the musicians looked at each other and smiled as if at their own nervousness and folly, and made whispering vows, each to the other, that the next chiming of the clock should produce in them no similar emotion; and then, after the lapse of sixty minutes, (which embrace three thousand and six hundred seconds of the Time that flies,) there came yet another chiming of the clock, and then were the same disconcert and tremulousness and meditation as before. It was necessary to hear and see and touch him to be _sure_ that he was not. To and fro in the seven chambers there stalked, in fact, a multitude of dreams. The dreams are stiff-frozen as they stand. And the rumour of this new presence having spread itself whisperingly around, there arose at length from the whole company a buzz, or murmur, expressive of disapprobation and surprise--then, finally, of terror, of horror, and of disgust. The whole company, indeed, seemed now deeply to feel that in the costume and bearing of the stranger neither wit nor propriety existed. And the life of the ebony clock went out with that of the last of the gay.

Ratio : 0.1

The "Red Death" had long devastated the country. No pestilence had ever been so fatal, or so hideous. The external world could take care of itself. In the meantime it was folly to grieve, or to think. The second chamber was purple in its ornaments and tapestries, and here the panes were purple. There were arabesque figures with unsuited limbs and appointments. And the rumour of this new presence having spread itself whisperingly around, there arose at length from the whole company a buzz, or murmur, expressive of disapprobation and surprise--then, finally, of terror, of horror, and of disgust. There was a sharp cry--and the dagger dropped gleaming upon the sable carpet, upon which, instantly afterwards, fell prostrate in death the Prince Prospero. And Darkness and Decay and the Red Death held illimitable dominion over all.

The Darling, by Anton Chekhov


Olenka, the daughter of the retired collegiate assessor, Plemyanniakov, was sitting in her back porch, lost in thought. They want a clown; what they ask for is vulgarity. In the evenings and at night she could hear the band playing, and the crackling and banging of fireworks, and it seemed to her that it was Kukin struggling with his destiny, storming the entrenchments of his chief foe, the indifferent public; there was a sweet thrill at her heart, she had no desire to sleep, and when he returned home at day-break, she tapped softly at her bedroom window, and showing him only her face and one shoulder through the curtain, she gave him a friendly smile. "AWAITING IMMATE INSTRUCTIONS FUFUNERAL TUESDAY." "Vassitchka and I have no time to go to theatres," she would answer sedately.  Little by little the town grew in all directions. "I have resigned my post, and have come to settle down and try my luck on my own account. Besides, it's time for my boy to go to school."

The Haunted House, by Virginia Woolf


Whatever hour you woke there was a door shutting. And then, tired of reading, one might rise and see for oneself, the house all empty, the doors standing open, only the wood pigeons bubbling with content and the hum of the threshing machine sounding from the farm. The windowpanes reflected apples, reflected roses; all the leaves were green in the glass. If they moved in the drawing room, the apple only turned its yellow side. "Safe, safe, safe," the pulse of the house beat gladly. But the beam of the lamp falls straight from the window. Wild beams of moonlight cross both floor and wall, and, meeting, stain the faces bent; the faces pondering; the faces that search the sleepers and seek their hidden joy.

The Kiss, by Guy de Maupassant


My Little Darling: So you are crying from morning until night and from night until morning, because your husband leaves you; you do not know what to do and so you ask your old aunt for advice; you must consider her quite an expert. You say that you are all attention, love, kisses and caresses for him. Perhaps that is the very trouble; I think you kiss him too much. To tell the history of Love from the beginning of the world would be to tell the history of man himself: Everything springs from it, the arts, great events, customs, wars, the overthrow of empires. A preface which can always be read over again, whereas one cannot always read over the book. One caress alone gives this deep sensation of two beings welded into one --it is the kiss. Therefore, my dear, the kiss is our strongest weapon, but we must take care not to dull it. After describing the expectancy of a lover, waiting in a room one winter's evening, his anxiety, his nervous impatience, the terrible fear of not seeing her, he describes the arrival of the beloved woman, who at last enters hurriedly, out of breath, bringing with her part of the winter breeze, and he exclaims: Oh! The taste of the kisses first snatched through the veil. Therefore, the value of this caress being entirely a matter of convention, we must be careful not to abuse it. Well, my dear, I have several times noticed that you are very clumsy. You had been paying no attention to it, and it was almost out. Then when you freed him, you began to grumble: "How badly you kiss!"

On the whole the effect is interesting and often pleasing. The digests retain the language, which in the originals are unfailingly elegant, and often a discernable strain of their meaning too. A digest, prosaic as it may seem, is creative in its own way. Synthesis and understanding requires a path through the heart of a text. This tends to stand in opposition to novelty, but they can also form two parts of a larger storytelling process.

What if we combined them to come up with something new?

No comments: