To understand just how vital NLP will be to the future of SEO, it’s worth looking at how Google – in particular – implements NLP and just how central it is to their mission.

One of the issues that researchers have faced when it comes to convincing businesses of NLP’s value is that the idea of using these models to read real-life text is inherently dated. As Dixon Jones, CEO of Inlinks.net, recently put it, though, this is a huge mistake: “When people realize NLP stands for Natural Language Processing instead of some 1970s hypno-mumbo-jumbo,” he said, “they’ll realize that not only is it here to stay, it’s the very bedrock of the mantra organizing the world’s information.”

Google’s NLP approach is built on a ground-breaking language processing model: BERT (Bidirectional Encoder Representations from Transformers). BERT is outlined in a recent paper published by researchers at Google AI Language. Its publication caused quite a stir in the Machine Learning community by presenting state-of-the-art results in a wide variety of NLP tasks, including Question Answering (SQuAD v1.1), Natural Language Inference (MNLI), and others.

BERT’s key technical innovation is applying the bidirectional training of Transformer, a popular attention model, to language modeling.

While that might seem technical, BERT’s impact on SEO can be put pretty simply: Google no longer looks at words or phrases individually, as we understood in the past when it would traditionally run keyword research. Now it looks at sentences, paragraphs, and the query as a whole. In other words, the algorithm looks at the sentiment or overall intent rather than focusing on individual words.

Google’s official Search Liason recently revealed on Twitter that Google BERT is now helping with one out of every ten google searches in the US in English, with plans to expand soon to include searches in more countries and languages.

For Google, all of this research and investment in NLP is geared toward one outcome: improved search quality. Just as it always has, this search engine juggernaut continues to push for a better way to provide the exact information that users are after.

There are many challenges in this pursuit. According to the information on Google’s blog, 15% of search queries are used for the first time. In other words, people are using more long-tail searches to find an answer to their question, especially with the rise of voice search.

Another is that computers – even those equipped with advanced AI systems – are still not that good at assessing the relevance and authority of information on web pages. This capability is increasing all the time, of course, and it might well be that in 2030 we regard the NLP capabilities of today as primitive. However, the fact remains that the SERP for most pages is still more a product of the keywords it contains than its genuine utility to most users.

NLP is an attempt, on Google’s behalf, to change that. According to recent tweets from the Stanford NLP Group, NLP is now a key component of AI and integral to SEO practices.

A second way in which NLP is currently impacting SEO is in internal linking structures and content recommendation tools.

These two functions may seem fairly distinct, but a closer look at how NLP engines work reveals that they are inherently (pardon the pun) linked. When an NLP system encounters a term that it doesn’t understand, it attempts to resolve its meaning. If it can do so by referring to the material on your site, it grants you an SEO boost in recognition of this fact. To see this in action, take a look at how The Guardian uses it in articles, where the names of individuals are linked to pages that contain all the information on the website related to them. Robert Weissgraeber, CTO of AX Semantics, notes that NLP boosts brand visibility with no additional effort by creating huge quantities of natural language content.

The same type of process underpins more advanced forms of content recommendation. There are now several tools available that can assess the meaning of the content that appears on a user’s list of visited sites and takes advantage of this to suggest other content. While some of this functionality can be limited by browser privacy modifications, it provides a powerful way for websites to increase the “dwell time” on their sites.