And the Pulitzer Goes to …

I found an interesting article on the Fox News app on my tablet the other day.  Now, before anyone launches into a political bias tirade I also peruse headlines on US News and World Report, Real Clear Politics, The New Republic, and The Huffington Post.  No, not MSNBC.  Every libertarian has his limits.  Anyway, the article reported that 122 articles in technical journals, many in publications of the Institute of Electrical and Electronics Engineers (IEEE), were recently found to be computer generated gibberish like, This section provides a complete description of the canceller output spectrum in terms of Discrete Prolate Spheroidal Sequences and their eigenvalues.  Just kidding, of course.   That is a sentence from my doctoral dissertation.  Try this: Application and Research of Smalltalk Harnessing Based on Game-Theoretic Symmetries.  Therein lies part the problem … many scientific papers may sound like gibberish to laymen and even to those in a slightly different scientific field.  That’s why technical papers are supposed to be peer reviewed by people working in the same field.

During my Big Industry and Graduate School days (which happened to coincide), I published a number of papers in IEEE journals and the peer-review process was sometimes excruciating.  It would take months of back-and-forth with reviewers to resolve issues with terminology, notation and interpretation of results.  Even what we saw as clear mathematical results were often questioned.  One reviewer warned of disastrous consequences if our particular choice of terminology were to propagate in the literature, as if anything we did in our arcane corner of engineering could have disastrous consequences.   I also reviewed papers and took my responsibility seriously. Such journals are known as refereed journals and at the time, only publications in refereed journals carried any academic weight.  Other journals (for instance, conference proceedings) were simply reviewed by an editor or editorial board, so it was easier to for work to slip by with less scrutiny.  Still, shouldn’t someone be able to spot gibberish?

According to to Slate.com the heart of the matter is SciGen, a publicly available program developed in 2005 by students at MIT, partly for their own amusement and partly to demonstrate how easily phony papers could be slipped into reviewed publications.  The 122 papers were discovered by Cyril Labbe using a program he developed to detect papers generated by SciGen or similar programs.  Labbe describes the spate of fake papers as a symptom of the spamming war started at the heart of science in which researchers feel pressured to rush out papers to publish as much as possible, coupled with the profits of the academic publishing business.  As more papers are submitted, there are more papers for editors and reviewers to evaluate.  The need for more reviewers may mean less capable reviewers are used.  There’s more motivation to try to slip a nonsense paper into a journal … and less chance of getting caught.   And the credibility of science suffers.

But wait.  According to the NY Times, these words (from the Big Ten Network) … Wisconsin jumped out to an early lead and never looked back in a 51-17 win over UNLV on Thursday at Camp Randall Stadium. The Badgers scored 20 points in the first quarter on a Russell Wilson touchdown pass, a Montee Ball touchdown run and a James White touchdown run … were written by a computer using an Artificial Intelligence (AI) program called Quill developed by Narrative Sciences using only game scores.  It ain’t exactly Shakespeare but I’d guess I’d have a hard time distinguishing it from the work of the average internet hack sportswriter.  Initially, Narrative Sciences is targeting markets seeking low cost “journalism” and has over 20 clients, including the Big Ten Network.  But Narrative’s goals are less modest.   In five years, says Kris Hammond, one of the company’s co-founders, a computer program will win a Pulitzer Prize — and I’ll be damned if it’s not our technology.  I think that Mr. Hammond is spending too much time in the computer lab and not enough time on his Kindle reading some Pulitzer prize winners.  If a computer ever wins the Pulitzer Prize, it will reflect the kind of drop in standards that has brought us reality TV and gibberish scientific papers, not artificial intelligence with human-like investigative and creative capabilities.  As Stuart Russell and Peter Norvig write in their leading textbook on the subject, Artificial Intelligence: A Modern Approach:  AI has become too much like the man who tries to get to the moon by climbing a tree: “One can report steady progress, all the way to the top of the tree.”  But then I’ve been wrong (once or twice) before.

What do you think?

Explore posts in the same categories: technology

Tags: artificial intelligence, computers, curmudgeonly rants, postaday, science, technology, writing

You can comment below, or link to this permanent URL from your own site.