Howard Lerman, founder of Yext, designed and built technology to support enterprise search engines based on artificial intelligence and natural-language processing. By running through this history and how search engines work, he made the case during a virtual roundtable to rid the internet of misinformation it requires search engines that run on NLP.
Keyword search gives people hyperlinks from one algorithm, whereas AI search -- with help from NL -- gives people direct answers in multiple languages, he said.
Lerman said the World Health Organization (WHO) made it clear during the past year-and-a-half that the internet’s ability to efficiently serve up authoritative information is critical, especially during the pandemic. It can save lives.
“One of the biggest issues we have, we pour out too much content,” said Christopher Strebel, manager of DCX at the World Health Organization (WHO), during a virtual conversation with Yext Founder Howard Lerman. “About 1,000 pieces of content are created weekly. It’s so fast that sometimes it slows the system.”
If things change so fast, he said, how the “hell” are people going to find the content needed? This is not working well with traditional search engines, he added.
Strebel suggests that the easiest way to help people find information is to highlight the content in the search engine query results, or have the engine direct the person to the exact place where the important answer lives. The relevant information, typically, is buried in the page of content. People often times need to dig into the page to find the answer to the question.
Google takes content from the WHO's website and attempts to surface that in its search query -- but it's not enough, Strebel said.
The WHO has created knowledge graphs to help search engines pick up verified information -- the correct information. Strebel provided an example on how the organization uses Yext to drive search within its own environment to ensure that numbers are accurate.
The WHO is trying to use Google to capture data on sites outside of the organization, as well as using Yext to capture data and build a structured knowledge graph to display the content more quickly. The content that comes from within the WHO organization.
At the moment, Strebel said, the WHO does not have a person dedicated to search, but we’ve “flagged it,” and he hopes soon to have a dedicated person in the position to work on this.
Sidebar: Here’s the backstory of search and why Lerman and Strebel believe ML is the future to rid the web of misinformation. 1994 marked the explosion of keyword search from companies like Alta Vista, Lycos, and Yahoo. That is, until Google launched an algorithm called PageRank, and it became one of the most important page ranking algorithm.
In 1999, at Xerox Park an engineer names Doug Curly launched as open-source engine called Apache Lucene, which went on to become important, but not many people heard of it, he said.
While search engines of the ‘90s focused on keyword search, that was the old way because it is focused on ranking millions of results.
AI search uses a list only as a last resort. It focuses on NFL. But as Google went on to power consumer search, the industry left enterprise search stagnant. The biggest change, NLP understands the way humans ask questions, and it’s absolutely necessary these days for the it to power search. Many other sites use keyword search technology based on the outdates Lucene technology.
Searches on Google trigger several algorithms simultaneously. Enterprise search engines require the same as commercial search — both must run on NLP, multiple algorithms, and have a knowledge graph.
Lerman believes that in order to make it work, not only did Google train algorithms to answer precise questions, but the company trained people how to ask questions that the algorithms can answer.