(Not) Following The Crowd

By Haris Dindo, PhD (Yewno Chief Data Scientist)

Big data. Global search. Keyword everything. While today’s search engines have chosen to “go big”, Yewno sees the world differently. The most valuable knowledge lies at the atomic level. A work is more than the sum of its keywords, and cannot be described in a handful of controlled terms. The true nature of any given work, in any format, rests in each and every idea and concept within, and how those ideas and concepts relate to all other known ideas and concepts.

Without a doubt, there is information that is best crowd-sourced, and the more often it occurs the more likely it is correct. “What works has an author published?” fits well into this model, and today’s search engines handle this with ease.

Research and learning isn’t about discovery of the obvious…

However vigorous research isn’t about the keywords that appear most frequently in large data sets. Research and understanding calls for finding the concept, no matter how small or deeply buried, and knowing how it relates to all other known concepts. More importantly, when launching into new research exploration, one often is looking for the blanks in our knowledge map — the concepts and relationships that have yet to be identified and explored. It isn’t about the keywords that occur most often, it’s about the hidden concepts that have yet to be found and that eventually correlate to form ideas.

We have developed Yewno differently than today’s search engines for these very reasons. We don’t simply count the keywords. We look to the atomic units of knowledge within each work, and extract all of the key concepts. We then use machine-learning algorithms to connect the concepts with each other, and the known universe of concepts from all of the works within the Yewno inference engine.

This next generation design seeks to elegantly apply artificial intelligence to enhance human capabilities, providing the user with a highly useful knowledge map that can be navigated and explored. Instead of an endless, dry list of results based on frequency, the user can carefully explore the interconnections of ideas and concepts, and often find unexpected relationships that were previously overlooked by simple keyword frequency engines.

Just as astronomers must filter out the noise of the universe at-large when seeking the smallest sign of a habitable planet in a distant universe, researchers, teachers and students can quickly filter out the noise of irrelevant information in their pursuit of one pure concept or relationship, and no matter how faint that is within the “big data” universe, they are able to quickly identify and explore it.

Research and learning isn’t about discovery of the obvious. It’s about finding and amplifying concepts and their relationship to each other, especially those previously undiscovered and/or under-explored.

Knowledge & DataJun Ge