Diffbot’s Data Repository, a New Attainment for Artificial Intelligence

Diffbot is working on a new concept of serving independent Artificial Intelligence application developers with Artificial Intelligence. Unlike big companies such as Google, Facebook etc., most of the small independent AI developers do not have the resources to store, process and transform the data into usable format. But Diffbot has come up with an effective solution that will let the AI application developers to use a convenient medium for storing, processing and converting data by utilizing some distinctive AI algorithms.

Image source: processor.com

Mike Tung, the chief executive of Diffbot said, “We’ve been working on this technology for quite a few years. It was really last year that 90% to 95% accuracy was reached. And hitting profitability last year as one of the first AI startups to do so was a turning point”. He also said that “they want to make the world’s largest database of structured knowledge”.

Unlike other big companies that utilize humans to classify web pages, Diffbot uses some special algorithms created by the company itself to sort the important web pages. This is a major reason why use of Diffbot databases costs much less than the other companies.

Diffbot initialized a new era of computing with this new improved database software that can organize data in the most efficient manner. This technology is equipped with powerful advance chipsets that can process the complex data quickly and effectively. And, last but not the least, there are pervasive arrays of sensors for collection of the most relevant data.

Tung mentioned that “From day one we made it an on-demand service. You pass a URL and we will process that. For every hit to our server we earn .008 cents”.

Diffbot initializes the process of web spidering, with a motto to speed up the process of data collection. The algorithms were not able to spider the web from day one, but the process has started from last year, 2015. The process of data identification was done with an accuracy of 90% to 95% by Diffbot algorithms. It has been one year since the company began the spidering project and its taxonomy already contains more than 1.2 billion objects, continuing its growth adding 10 million objects daily. The venture is also attracting many big tech giants of the industry and earning some good profit.