The Past and the Present of Natural Language Generation

Rome wasn’t built in a day. Well, so wasn’t natural language generation. It has taken years for computers to exhibit the level of intelligence they do today and be able to produce text that sounds and read human-like.

It’s time to appreciate this revolutionary journey.

In 2007, The first step was taken by Robbie Allen, who was a veteran engineer at Cisco. He created an online college basketball website that automatically published game reviews, real-time updates, recaps, and incidents of injury. The name of the platform was StatSheet, and for years, it was the No. 1 website for college basketball statistics. 

The NLG journey

In 2010, StatSheet created the first piece of NLG-powered content- called StatFix- in-game note format. Simultaneously, the public was questioning the usefulness of NLG and the effort it required.

Between 2011 and 2013, though, NLG expanded in the sports and real estate market. In 2011, StatSheet became Automated Insights and partnered with the National Football League to release fully-automated and real-time coverage content. In 2012 Yahoo!, also came up with Yahoo! Sports Fantasy Football and provided personalize drafts and game recaps. 

In 2013, NLG took its first step in the BI industry when Allstate adopted Automated Insights’ NLG technology. In 2014, Automated Insights announced the launch of its NLG software- Wordsmith. A partnership with Associated Press was formed, which revealed the real power and potential of NLG.

Since the cat was out of the bag, Gartner declared NLG as its own domain in 2015. After that, the fire has only continued to spread, especially in the BI sector. NLG has been paired with data visualization software like Tableau and MicroStrategy to increase the value derived from data. Concurrently, Automated Insights continued to solidify itself as an NLG leader by winning the Tableau Conference Hackathon in 2016 and hosting the Amazon Alexa Hackathon in partnership with Amazon. The company was also awarded the SOC 2 Compliance.

Another AI-based startup, Narrative Science came into limelight in 2017. This company sets itself apart from others by leveraging deep learning rather than existing template-based generation. Their platform named Quill is making strides with 11 patents in their bag, and partnerships with big analytics firms to integrate Quill into their platforms. 

Differences between NLP, NLU, AND NLG

Other fields related to computational linguistics, machine learning, and artificial intelligence also grew, with NLU and NLP being 2 of them. The relationship between the 3 can be expressed as:

NLP = NLG + NLU

Pictorially,

NLP deals with processing language and turning text into structured data.

NLU deals with reading data and is used in profanity filters, topic classification, and sentiment detection. Alexa and Siri rely on NLU.

NLG deals with writing language and turning structured data into text- whether written or oral. 

Evolution of algorithms used in NLG

The earliest algorithm used for language generation was the Markov Chain. It formed sentences by predicting the next word based on the current word. The downside was that structure and context weren’t taken into account, leading to incorrect formation.

The next stage was RNN or recurrent neural networks. These neural networks were inspired by the human brain and thus had a memory, which could remember the sentence context. They were limited when the sentence length increased too much as they could not remember words encountered far back.

LSTMs were introduced to tackle this problem, but they, too, gave out when faced with longer sentences. The current model being tested is that of Transformers, which uses self-attention to focus on previously seen words that might be relevant to predicting the next one. Longer sentences can now be tackled without expending more computational power.

In conclusion, NLG is ushering in a new era of working- especially in the BI sector. The only thing that remains to be seen is how quickly businesses- including yours- adapt to this change and leverage it for greater success.