“We get a lot of catalogue data from sellers on Flipkart and one of the problems they face is that they have to adhere to our standards of describing something. We want to make that process simpler for sellers and want to ask them for the least amount of information, but at the same time provide our customers with as much data about a product as possible to make a decision,” says Nishant Khurana, Senior Engineering Manager at Flipkart, who is working on the new project.
It’s this gap between asking for little but delivering a lot that Flipkart is trying to address at first and wants to use NLP techniques developed by Professor Talukdar and his team to analyse text written by a human to extract relevant information about a product from it. This data can then be used to link two or more products to each other no matter how disparate they may seem using other ML and AI techniques.
Essentially Flipkart is trying to use these smarter ways of scraping data to build a knowledge graph, a term for the underlying base technologies that was made popular by search engine giant Google to enhance results it shows users from a variety of sources.
“Our research has been focused on how we can read unstructured data at scale which is written in natural language. We have done quite a bit of work on reasoning and canonicalization which is applicable to this space. The way to understand it is that Flipkart has some data and consumers are trying to access this to discover products, so you need to have one common language to put everyone on the same page,” says Talukdar.
Collaborating with academia, Flipkart gets access to years of research and some of the leading minds in the field to build such technologies. For researchers, the access to vast amounts of data which are up-to-date is among the biggest lures. Talukdar says such projects allow him and his team to get out of the “ivory towers” and work on solving real-world problems.
Several of the world’s largest technology companies
including Google, Facebook and Microsoft also work with top-notch universities to develop intellectual properties. The credits for such research is generally shared between the companies
and the researchers.
Mayur Datar, who heads Flipkart’s data sciences division, has a team of several such researchers building future solutions that will drive the company’s growth. However, Flipkart continues to look outward for innovation as there are far too many problems to solve and there are certain areas of research where internal teams might be lacking in expertise.
However, working with internal teams or external researchers, Datar says he’s clear that the future of Flipkart is going to be determined by how well it adopts ML and AI. As the Vice President of Engineering, he says he’s pushing more young engineers to dip their feet in newer technologies and to take up courses in AI and ML.
“We have obtained certain amount of scale and profitability using human intelligence and domain knowledge that people bring. But we can continue down this road for maybe another year or two, but beyond that if we do not embrace AI and ML, it’s absolutely clear that we will not be able to sustain as a company,” said Datar.