Commentary
Wolfram's Project Catalogs All Electronics Data
- by Laurie Sullivan , Staff Writer @lauriesullivan, January 8, 2014
Can you imagine how hard it will be to curate data from every device that makes up the Internet of things? Wolfram Research, a division of the computational search engine, Wolfram/Alpha, took on the project. When complete, it will allow the company to store and use data from all Internet-connected devices.
The platform will let device manufacturers do complex queries about devices, similar to the queries made about consumer products such as price, weight, and model number. The project is similar to Google's Knowledge Graph or Microsoft Satori, two databases that connect similar things across the Internet and teach the Web how to understand connected concepts.
Wolfram/Alpha founder Stephen Wolfram says the company aims to collect and curate all objective data about connected devices; implement every known model, method, and algorithm; and make it possible to compute everything about anything from an open Web site. In a blog post, Wolfram defines a connected device as one that "connects to a general-purpose computer using some standard connector or connection technology."
advertisement
advertisement
Results from queries in the Wolfram Alpha "computational" knowledge engine give precise answers from structured data about gadgets, but it requires precise queries that have been formatted. For now the company will omit objects that have complex custom electrical systems, such as sensors or integrated circuits that plug into something.
The goal to store data on all Internet-connected devices requires teamwork, and Wolfram's Mathematica team works with device manufacturers and the technical community to add information about connected devices into the database.
Laurie Sullivan is a writer and editor for MediaPost. You can reach Laurie at lauriesullivan@gmail.com.
I don't have to imagine it.
This has been my career at DetectRight.com for over seven years.
It's been our job to catalog and detect devices (over 45,000 devices, 12,000 possible datapoints for each one and 2 million user-agents). We not only have to maintain a consistent catalogue and keep it proactively updated, we have to track how the devices can be detected in real-time for websites, and implement a search engine with capability search, and the ability to assign to devices arbitrary group membership such as device family (e.g. Samsung Galaxy Tab), country of origin, Carrier, countries seen in the wild, etc.
If Wolfram imagine all they have to do is ask "the community" and "device manufacturers", this project will be a total failure.
This is partly because there's too much mess in the data, different data schemas, interacting nested datapoints, capability change upon OS upgrade, model numbers within model numbers, interaction in capabilities between the baseband chipsets, OS, Browser and UI....
These are problems we've sorted that Wolfram won't even see coming if they've just started. I had to invent a new way of storing data in trees with quantum nodes to resolve conflicting heterogenous data, for example.
There have already been community projects to collect data of this type, and they were failures as community projects because data maintenance requires consistency, accuracy and persistance, and the community is good at creativity and fragmentation. Meanwhile, manufacturers tend to be disorganised, possessive of data, or simply disinterested in documenting their devices: especially regional vendors.
That's awesome, Chris. Thanks for sharing. It's an incredible task. I did it (in a slightly different way) while working at Ingram Micro. The data is a mess. So glad you can relate.
Oooh, I feel for you there Laurie...
There was a time when the number of devices people thought there was (which is always 25% of the number there actually are) was small enough so that companies felt they could do their own database... I think those times are gone now... it's an absorbing field, but not for the faint hearted!