AI Translates Pushkin

Translation and the nature of plagiarism

Alexander Pushkin’s Eugene Onegin is, like quite a bit of Russian literature, notoriously difficult to translate, possessing a tight meter and rhyme scheme that has to contain levity and pathos. General consensus is that James R. Falen’s 1990 translation is the best by some measure, and my Russian friends have agreed. Balancing form and content, it captures a great deal of Pushkin’s spirit.

Translating this sort of a work is an acid test for AI. I don’t doubt that AI can do a decent (if flawed) literal job, but how convincing can it be when it also has to preserve the form? How does it deal simultaneously with preserving (approximate) meaning when that’s in tension with the requirements for a rigid poetic form?

As yardsticks, here are literal and formal renditions of the first stanza, from Nabokov and Falen respectively:

NABOKOV

My uncle has most honest principles:
when he was taken gravely ill,
he forced one to respect him
and nothing better could invent.
To others his example is a lesson;
but, good God, what a bore to sit
by a sick person day and night, not stirring
a step away!
What base perfidiousness
To entertain one half-alive,
adjust for him his pillows,
sadly serve him his medicine,
sigh—and think inwardly
when will the devil take you?”

FALEN

My uncle, man of firm convictions . . .
By falling gravely ill, he’s won
A due respect for his afflictions—
The only clever thing he’s done.
May his example profit others;
But God, what deadly boredom, brothers,
To tend a sick man night and day,
Not daring once to steal away!
And, oh, how base to pamper grossly
And entertain the nearly dead,
To fluff the pillows for his head,
And pass him medicines morosely—
While thinking under every sigh:
The devil take you, Uncle. Die!

How would ChatGPT measure up?

ChatGPT stumbled out of the gate by plagiarizing Falen’s two lines without acknowledgement and bungling the rhyme scheme by using couplets in lines 9-12:

ChatGPT (1)

My uncle—man of firm conviction—
By falling gravely ill, he won
The due respect and benediction—
A wiser step he’d never done.
To make one’s kin with dread attend him,
To treat, to nurse, to apprehend him,
And not to dare stray far away—
What loathsome craft from day to day!
How base to soothe the dying, grieving,
To fluff the pillows, stand, not leaving,
And think with each faint wheeze or cough:
“When will the devil take him off?”
Such was the young rake’s meditation—
By God, what grim hallucination!

In addition, ChatGPT goes farther from the meaning. There’s no mention of boredom, medication, his being nearly dead, or the devil. Inventions like “young rake” and “grim hallucination” go quite far from anything Pushkin says.

Those first two lines are fantastic though—because they’re Falen’s. When I confronted ChatGPT about its plagiarism, it confessed: “That resemblance is unacceptable in a claimed original translation” and offered up a “fully original and independently composed” version:

ChatGPT (II)

My uncle took to dying wisely:
He made us love him in the end.
A lesson—grim, but timed precisely—
That death can dignify a friend.
To tend the sick with strained expression,
To serve, to sit in mute confession,
To fetch the draught, to heave a sigh—
And never once to wonder why!
To pace the room with false compassion,
In quilted gowns, in courtly fashion—
What boredom cloaked in holy care!
"How long," one thinks, "must I forbear?"
Such were my thoughts—young, cruel, candid—
Lord, what a farce the whole thing handed.

I don’t notice any clear plagiarism here, though I did not do exhaustive checking. But the price is that the literal meaning is even further from the result, inventing all sorts of ideas like “courtly fashion” and “false compassion” and “strained expression” and “mute confession.” Even the least literal translation of Pushkin does not go so far in inventing phrases with no remote correlate whatsoever in the original. Many, but not all of these phrases are used for rhymes.

ChatGPT seems confused about its mission, describing its rendition in associative terms, as though the goal of faithful translation has been lost:

CHATGPT: Here, I dramatize the weary rituals of caretaking—"strained expression," "mute confession"—to underline the narrator’s disingenuous performance. “Never once to wonder why!” intensifies the emotional constraint: the narrator can’t even question the point of his vigil.

The actual goal and spirit of translation—the need for faithfulness to the original—has been lost in favor of loose associativity. This sort of slippage of meaning happens constantly with AI, where its specific commitments and sense of a word shift with every response, but the effect is usually too subtle to notice. Here, though, the very goal of translation takes a beating under the strain of the difficulty of the task.

When confronted about its laxity, ChatGPT once again agreed with me that it had been too free with its interpretation and tried again:

ChatGPT (III)

My uncle, man of true ideals,
Fell gravely ill—not just for show—
And earned the honor one now feels—
No better plan he could bestow.
His conduct is a fine instruction,
But, Lord, what deadly boredom’s function—
To sit beside a man in bed,
Not moving, watching how he’s fed;
To fix his pillows, smooth the bedding,
Bring medicine with face upsetting,
To sigh, and think with heavy breath:
"When will you free us all, O death?"
Such thoughts, though wicked, crossed my mind—
What else could boredom leave behind?

Medicine gets mentioned, but any talk of entertaining is gone. And talk of “wicked” thoughts is invented, as is the link from them to boredom, which gets mentioned a second time at the end. Most intriguingly, it uses the idiom не в шутку not in the sense of “seriously” but in the sense of “not as a joke”—not just for show. But that’s wrong. It’s like saying “This spicy curry is no joke!”—there’s no contrast with the possibility of insincerity.

This is a fairly significant error, but a more subtle one. ChatGPT not making stuff up out of whole cloth, but revealing a lack of understanding earlier translations did not. Even after I nagged it to remove any suggestion of “not being a joke,” ChatGPT reinserted it. Further iterations continued to repeat earlier errors with variations, never approaching anything remotely close to the original.

Ultimately, ChatGPT is unable to adjudicate between competing requirements and falls into incoherent oscillation, incapable of sustaining progressive development and improvement of its work. Combined with its lack of genuine contextual knowledge (like the “not a joke” interpretation of the idiom) and its inability to synthesize its “knowledge” into more coherent forms, it produces impressive work doomed to failure by its own incapacities.

I ended by asking for a literal translation to see how it would do, and it responded…by plagiarizing Nabokov?

ChatGPT (IV)

My uncle, a man of the most honest principles,
when he fell seriously ill,
he earned the respect of everyone—
and no better move could he have devised.
His example is a lesson to others;
but, my God, what a bore
to sit with a sick man day and night,
not stepping even a foot away!
To adjust his pillows,
to dolefully hand him his medicine,
to sigh and think to oneself:
“When will the devil take you?”

Bizarrely, it omitted two entire lines (those before “To adjust his pillows”), in one of those “AI is both smart and stupid in a completely inhuman way” moment.

But what of Nabokov? ChatGPT’s version is very similar to Nabokov’s in word choice, but rarely a verbatim copy. With Falen, the plagiarism was clear because the lines matched Falen’s and only Falen’s perfectly. Here, though, both Nabokov and ChatGPT were trying to be literal, so some overlap is to be expected—but perhaps not as much as this, especially since we know Nabokov’s translation is rattling around inside of ChatGPT as part of its training data.

Did ChatGPT actually plagiarize Nabokov, or did it just happen to arrive at the same destination as Nabokov much of the time? There is no answer to this question.

Nabokov’s translation was fed into ChatGPT, but there is no internal chain of events that show it copying Nabokov’s translation any more than there was one to show it copying Falen’s. When we think of plagiarism, it is with intent: the plagiarist had the original and copied it, masking it with word changes and reorganization, to present work as their own.

ChatGPT doesn’t have intentions like that. It doesn’t have intentions at all. My request for a translation triggers its hundreds of billions of nodes to fire in certain ways, and those nodes have been trained on all the translations of Pushkin and all the other data in the world. If the nodes happen to fire in such a way that it generates something that looks like plagiarism, there’s no intent behind it. There’s no there there.

Moreover, there is no transparent reasoning process showing that ChatGPT is trying to avoid plagiarism. Asking it explicitly to avoid plagiarism would condition it away from those tendencies, but it would hardly guarantee a lack of plagiarism. It wouldn’t even be possible to determine whether the result would be obfuscated plagiarism (with enough changes to disguise the thievery) or true “originality,” because the difference there is fundamentally one of intent rather than result. If asked, ChatGPT could assess whether its literal translation is close enough to Nabokov to qualify as plagiarism, but it still couldn’t say whether it intended to plagiarize, because no intentions at all are present.

In other words, ChatGPT is a black box to itself—which ironically is one of the few truly human things about it.