Why is it easy to find the information you are looking for on the Web, and so hard to find it at work? I continue to be amazed that you can find the diameter of the moon, or an inexpensive yet highly rated TV, or the largest city in Ohio, with just one or two queries and a few clicks. Now contrast your experience on the Web with your experience at work, where finding a spreadsheet, or the latest planning memo, or the most applicable sales deck, can take forever and a day.
Big web sites, such as Google, Amazon, Netflix, or Pinterest, are faced with the seemingly impossible task of sifting through thousands, millions, or even billions of items, to find just the few that are most relevant. They do this by constructing and then analyzing a graph of interconnections between items. So, for example, Amazon can learn that two TVs are related, because people that browsed the first TV went on to buy the second. Amazon uses this graph of related products to recommend similar TVs as you are browsing and searching. Similarly, Google can learn that a particular site is great for vacation planning because thousands of people have linked to that page and millions have clicked on it.
Part of the magic is that usage data is used to infer the meaning and relationships between items. There are no teams hand-tagging web pages, or hand-authoring a graph of related products. Compare this with prevalent schemes for enterprise search and knowledge management, where hand-tagging is the central mechanism for retrieval and browsing. These tagging approaches are much like Yahoo! of 1996: lots of hand-tagged web pages, and a frantic race to keep up with the pace of change. That approach to web search was quickly swept away by the data-driven approach pioneered by Google. We’re still in the 90s era of enterprise search. Our goal at Highspot is to help usher in an experience more akin to Google, but inside the organization.
Mining usage to learn which documents are most important or which products are related is one type of machine learning. I’ve been working on machine learning for a long time, having taught it to MIT grad students and run the Bing search engine relevance team. I’m looking forward to writing a series of posts detailing these approaches and how Highspot is bringing the power of machine learning to the workplace.
One key insight we’ve had is that within the enterprise, information and people can be linked together into a knowledge graph of interconnections. Using this knowledge graph, Highspot can help you and your colleagues find, discover, and share knowledge at work. The knowledge graph is a central part of everything we do, and every part of our product uses it to return better results. Conversely, every feature has been carefully designed to collect the usage data needed to build a comprehensive knowledge graph for your organization. By using techniques from machine learning and data science, we compute the authority of information and the influence of people to deliver ever more relevant results.
Highspot makes the enterprise knowledge graph a first-class citizen.
The knowledge graph is much more valuable than the conventional information available within an organization. Imagine we’ve imported the organizational chart and all the documents from a company. We’d have a lot of information, sure, but we wouldn’t know which documents were the most authoritative or which people were the most influential. In order to understand a person’s expertise and interests, or a document’s category and authority, we need to collect and mine information about patterns of usage. As you learn more about Highspot I think you’ll find that it is a great tool for collecting, organizing, and publishing information within your organization. And along the way, as you use Highspot, we can collect information about your interests and expertise.
Take a hypothetical user: Susan. Using Highspot, she reads, comments, collects, and recommends documents on the Marketing Spot. Highspot machine learning algorithms use this data to discover that Susan is someone that is interested in marketing, and, more specifically, the marketing of particular products and areas. If we further observe that others read the docs that Susan created, or comment on them, or recommend them, then we can conclude that Susan is an expert on marketing. If new users search or browse for marketing information, documents created by Susan should be at the top of the list. We can also recommend Susan and other similar users as experts that they might want to follow.
The various features of Highspot play two simultaneous roles: the features help Susan publish and find information, and these features also help us to collect the usage information necessary to build a graph of relationships between users, information, knowledge, and expertise. Before Highspot, most of this usage data was not collected or was lost.
At a higher level, I think creating this closed loop — between ML, on the one hand, and an easy-to-use experience on the other hand — is unique in the industry. And ultimately it is this closed loop that allows Highspot to deliver a great knowledge management service.
