In a radical departure from what has been done in the past, Bing has begun to implement a process to speed search engine queries and ad-serving through programmable hardware, with a goal of completing the rollout this year.
Microsoft Bing will use field programmable gate arrays (FPGAs) to speed search and advertising services, and other compute intensive services like translation, image understanding and voice search will benefit the most. And while this may not mean much on the surface to the average marketer, it will likely earn brands more money in coffers through increased real-time data processing and ad serving speeds.
The most interesting part of this new technique involves the ability to process the tasks in the semiconductor chip rather than the software. Typically companies creating search engines will write the software and run processes on hardware. This technique allows the hardware to do more of the work. Tests show FPGAs can accelerate large-scale services in the data center.
Search engines have a multistage process to field queries. Queries come in. The technology quickly aggregates the indexed documents with the term, and gets scored and sorted to determine the ones that serve up in the front of the list. This process-intensive service requires technology to understand the search query requirements in a fraction of a millisecond before serving the content. Processing part of the step in the hardware, rather than software, helps to speed the task.
The ability for Microsoft and Bing engineers to program the semiconductor chips ultimately speeds the process. In early tests, tens times faster, said Doug Burger, director of hardware, devices, and experiences at Microsoft Research. He said it will let Bing save money by using fewer servers, do the same amount of work in less time, and gain better results.
For nerdy marketers, Burger explains it this way: FPGAs uses "spatial computing," meaning the engineer maps the functions to a physical area, and it takes several seconds to reprogram or change those functions. So the number of processes is a function of the amount of logic, the space, and the size of the processes. "We disclosed designs with dozens of separate units computing different things, and have built FPGA images where completely separate software processes are communicating with distinct functions on the FPGA, mapped to different parts of the chip,” he said.
In the past, the technology industry, including search, has been driven by Moore's Law, but that's coming to an end. Search engines and ad servers will need to use new techniques. "Without the free jumps in silicon, you need to take risks," Burger said. "It's going to become a very exciting time."
Burger and a large team of researchers at Microsoft began testing the system last year, and in June 2014, released a paper describing the process of testing this with 1,600 servers. It ran "tens of times" faster with help from the FPGAs. The Microsoft team also managed to run Bing ranking with half the number of servers.