In a radical departure from what has been done in the past, Bing has begun to implement a process to speed search engine queries and ad-serving through programmable hardware, with a goal of
completing the rollout this year.
Microsoft Bing will use field programmable gate arrays (FPGAs) to speed search and advertising services, and other compute intensive services like
translation, image understanding and voice search will benefit the most. And while this may not mean much on the surface to the average marketer, it will likely earn brands more money in coffers
through increased real-time data processing and ad serving speeds.
The most interesting part of this new technique involves the ability to process the tasks in the semiconductor chip rather
than the software. Typically companies creating search engines will write the software and run processes on hardware. This technique allows the hardware to do more of the work. Tests show FPGAs can
accelerate large-scale services in the data center.
Search engines have a multistage process to field queries. Queries come in. The technology quickly aggregates the indexed documents with the
term, and gets scored and sorted to determine the ones that serve up in the front of the list. This process-intensive service requires technology to understand the search query requirements in a
fraction of a millisecond before serving the content. Processing part of the step in the hardware, rather than software, helps to speed the task.
The ability for Microsoft and Bing engineers
to program the semiconductor chips ultimately speeds the process. In early tests, tens times faster, said Doug Burger, director of hardware, devices, and experiences at Microsoft Research. He said it
will let Bing save money by using fewer servers, do the same amount of work in less time, and gain better results.
For nerdy marketers, Burger explains it this way: FPGAs uses "spatial
computing," meaning the engineer maps the functions to a physical area, and it takes several seconds to reprogram or change those functions. So the number of processes is a function of the
amount of logic, the space, and the size of the processes. "We disclosed designs with dozens of separate units computing different things, and have built FPGA images where completely separate
software processes are communicating with distinct functions on the FPGA, mapped to different parts of the chip,” he said.
In the past, the technology industry, including search,
has been driven by Moore's Law, but that's coming to an end. Search engines and ad servers will need to use new techniques. "Without the
free jumps in silicon, you need to take risks," Burger said. "It's going to become a very exciting time."
Burger and a large team of researchers at Microsoft began testing the system last
year, and in June 2014, released a paper describing the process of testing this with 1,600 servers. It ran "tens of times" faster with help from the FPGAs. The Microsoft team also managed to run Bing
ranking with half the number of servers.