What is Knowledge Management?

I don't know how many different definitions of knowledge management are out there. Too many, for sure. Most of them are too long as well. Would you consider this very short definition of Semantic Knowledge Management?

• Knowledge Management is connecting knowledge sources with knowledge needs through knowledge attributes.

So what is KM all about? It is not about people, per se. It is not about change management. It is not about content management, organizational culture, communities, etc. etc.

It is simply about knowledge and about ways of effectively making use of it in an organization. And the hypothesis here is that such effective use is only possible in the long run through knowledge attributes, i.e., by taking KM to the semantic level.

In doing so, some of those other things - people, communities, culture, change management, etc. - do come into the picture, of course. Nevertheless, our focus should remain on the core problem: knowledge and its effective use.

What are Knowledge Attributes?

Consider a book, a traditional source of knowledge.

It has many physical attributes: its length, width, and other dimensions, its weight, type of binding, whether it has a removable jacket, whether it has a built-in bookmark, etc.

A book also has several data-level attributes: its size in words, the number of illustrations it contains, what writing system and fonts are used in it, whether it is in multiple volumes, other formats (audio, e-book, ...) in which it is available, etc.

A book has many more attributes at the information level: its title, names of authors, the publisher, year when it was published, edition, number of pages in it, its ISBN number, cover price and currency, its table of contents, etc.

Conventional knowledge management systems such as library and bookstore management systems as well as most KM solutions make good use of many of the above attributes. However, they barely touch the next level: attributes at the knowledge level.

Knowledge attributes of a book include its subject matter (a.k.a. topic), its intended readership, the background knowledge it assumes, how it is meant to be read or taught, how well it explains some concepts, how popular it is, what reviews it has received, how is it rated by particular user communities, how its matter is related to knowledge contained in other sources, etc.

Notice that some of the above knowledge attributes are often contained in the book in textual form such as in a preface or foreword, and as such aren't useful to systems. What we need are structured representations of knowledge attributes of which our library/bookstore/e-commerce/KM systems can make good use.

How else could it pick the right knowledge source to satisfy a user's knowledge need effectively? By counting hyperlinks? By launching Clusty the Clown? I for one don't think so.

Without knowledge attributes, KM will continue to barely touch the semantic or knowledge level.

What is the K in KM? I in IR?

What is the K in KM (Knowledge Management)? What is managed in KM? Can we really manage knowledge?

In other words, how is KM different from information management?

Before trying to answer these questions, we might as well ask what is the I in IR (Information Retrieval)? When have you really retrieved information from a search engine? I seem to always retrieve data, text, or occasionally images or audio/video.

The data or text that is retrieved is then processed in our brains to extract information from it. If it is data that is retrieved, we are talking about good old query processing in databases. If it is text or other multimedia, that is content retrieval, not quite information retrieval.

How come then we are comfortable talking about the I in IR being information? And why are people more skeptical of the K in KM being knowledge?

Well, there are a few subtle but important differences between IR and KM. In IR, we are more comfortable with the notion of extracting information from the retrieved data or text. We are not so sure of the meaning of extracting knowledge from the same stuff.
Next, IR emphasizes retrieval, a smaller problem than all of information management. KM is being more ambitious in talking about managing knowledge. Also, for IR, it seems alright to believe that managing reasonable kinds of meta data (that librarians have been using for ages) enables the engine to find relevant data and text. What needs to be represented for KM? And how is the meta data in KM different from that in IR?

To see the additional kinds of meta data that may be needed for KM, consider the following types of questions that a librarian may get: "Who has written books on database management?" is a simple data retrieval question. "Which book on databases covers database tuning?" is an IR question. And "Which database book explains tuning well?" is really a KM question. For the first one, a simple database schema will help us translate the question into an appropriate database query (in SQL, for instance). For the IR question, we can come up with a reasonable schema for storing the tables of contents of books (or keyword or other classification data) to meet the requirements. For the KM query, what needs to be represented and managed so that the system can pick books that explain tuning better than others?

The K in KM is not really knowledge itself, but a representation of it. This knowledge representation is mostly meta data but it is rich with "knowledge level attributes."

What are knowledge attributes? In my next posting, if you don't mind ...

Semantics: How Deep is Meaningful?

Semantic approaches appear to be in fashion nowadays. However, not every technique that is labeled “semantic” indeed deals with meaning. These approaches come in all forms and depths, some more pseudo-semantic than others.

How do we measure the “depth” of a semantic approach? A simple way to characterize the depth of semantics is to assess its closeness to the level at which humans can be expected to capture and analyze the meaning of an utterance, a piece of written text, or a situation. The closer it is to a human explanation of the meaning, the deeper the semantics. Otherwise, we call it shallow semantics.

Of course not all humans are equal in the background knowledge and commonsense that they bring to the situation or even in their linguistic abilities or cognitive bandwidth. Perhaps we can apply the same measure of depth to characterize some people as “deeper” than others, but this is not important to our present topic.

Another way to characterize depth is in terms of the types and amounts of knowledge that the approach demands in order to extract and process meaning. Shallow approaches can work with statistical or other empirical techniques that demand little knowledge of the domain. Deeper methods require large amounts of declarative (as well as procedural) knowledge creating which is often the biggest challenge in implementing deep semantic solutions. We may also note here that the types of knowledge required for shallow approaches are very un-human-like and are incompatible with cognitive models of reasoning, memory, and language processing.

How deep should we go?

Early work in Artificial Intelligence in the pre-WWW era, in what was popular then under the name of Natural Language Processing (NLP), attempted very deep approaches and consequently failed to scale up to faster performance or larger domains, corpora of texts, and knowledge bases. Later battles belonged to shallow approaches that have dominated most of the development of information retrieval and search engines as well as other Internet technologies. Even those exceptions that started out with deep ideas, the primary example being Yahoo!, soon tilted towards shallower approaches.

Google is a good example of the success of shallow semantics. It is a wonderful platform and search engine; I use it everyday. Nevertheless, links that point to a page do not always tell you what a page is really about. They are at best poor cousins of the actual meaning of the contents of the page. Yet, by analyzing links, Google was able to show far better precision and ranking of search results than other search engines. Many other such pseudo or shallow semantic tricks, heuristics, and hacks have been very successful… but only when compared to their non-semantic counterparts.

Friends, it is time once again to set aside quick wins and short-term ROI and seek far deeper semantic approaches. Who is going to do it? Well, that is a separate discussion for another day.

We do not want to repeat the mistakes of AI/NLP, i.e., promising human-like behaviors and being unable to deliver them in any but the smallest micro-worlds. At the same time, shallow approaches will only help us climb taller trees, not reach the moon ever.

Where then is the middle ground? There are a couple of ways to characterize this. One is in terms of the size of text units at which meaning is analyzed. For example, we do want to consider meanings of words and short phrases; we do want our systems to try to disambiguate them and pick the most contextually appropriate meaning. Perhaps even the meanings of individual sentences. But, we do not want to claim that they can deal with meanings of entire texts or figure out the intentions of the writer or speaker.

Another way to characterize the middle ground is in terms of the types of knowledge needed and inferences or reasoning that they support. We want to build semantic systems that are endowed with comprehensive lexicons that map words and phases to nodes in a comprehensive ontology. We do not want, at this time, systems that attempt to make elaborate inferences from meanings, the kinds of which did not work very well in AI. As an example, we are not trying to build a robot that can search the web, read and understand web pages, and make decisions or solve problems from its understanding. We are at a stage where it is meaningful only to attempt solutions that employ small units of meaning to improve, for example, the user experience of web search by providing better precision, recall, or navigational and visualization abilities.

The next time you hear or read about claims of semantic approaches, please judge for yourself whether it is indeed deep (enough) semantics, or just yet another poor cousin of true semantics.

Interactive Search: A Case for Semantics

Are you happy with the quality of web search engines?

Try searching for "mouse" in your favorite search engine. You are most likely to get a good mix of hits that are about rodents, the computer mouse, a rock band and a few other senses of the word mouse. Instead of just throwing all those hits at you, wouldn't it be nice if the search engine came back to you and asked "Which kind of mouse do you mean?" Isn't this what a human assistant would do?

What does it take for a search engine to engage users in such interactive dialogues? First of all, it must know that there are different meanings for "mouse." It must not treat the "keyword" entered by the user merely as a juxtaposition of the characters "m" "o" "u" "s" "e." Rather, it must have a dictionary entry for the word "mouse" that maps it to multiple concepts in its ontology: one for the rodent mouse, another for the computer peripheral known as the mouse, and so on.

In other words, the search engine needs an ontology with concepts (or nodes) for the above kinds of mice. In addition, it also needs a lexicon that maps words or phrases (like "mouse") to one or more concepts in the ontology. One of challenges is who is going to build this ontology? Can we get all human beings to agree on one ontology, one view of the world, and one way of classifying all concepts?

Even if we have an ontology and a lexicon, figuring out the intended sense of the word (a process called disambiguation) is not an easy task. In the case of a search engine where there is hardly any textual or conversational context available, we may have to exploit knowledge of the user's profile to guess the intended meaning.

Or else, ask the user.

In any case, as long as search engines continue to ignore meanings, they will keep throwing too many hits at users to whom many of them are irrelevant.

The problem of diambiguation is in fact more complex. For example, if a user searches for "dog" instead of "mouse," the search engine must not throw a dialogue box at the user asking "Which type of dog do you mean? Alsatian, poodle, dalmatian, ..." This would be seen as a silly thing to ask. In fact the expectation is that pages that are about any of these or other types of dogs are relevant to the query. The reason is that these are all (sub)types of the same concept of a dog (other than "hot dog", etc.) whereas there is no single concept of mouse that subsumes rodents and computer mice. At least, most peolpe do not view the world that way... wherein the shape of the entity makes it a mouse and then you classify it into biological mice and computer mice.

In summary, we need search engines that are capable of engaging the user in an effective dialogue to improve the precision and recall of a search. They should also be able to figure out when to throw a question at the user and when to simply return hits.

Why Semantics for Knowledge Management

Greetings!

Human communication is all about conveying meaning from one person to another. Language, be it spoken or written, is a medium used to carry the meaning. It should have been obvious from this that any attempt to process texts or manage documents, web pages, or other forms of repositorized knowledge must deal with meaning, i.e., the processing must go on essentially at the semantic level.

Why then is this not so? From the early days of computational linguistics to the development of information retrieval, search engines, and knowledge management systems, we have mostly been processing at the syntax or symbol level... as though there is no meaning underlying the texts. We have built systems and shown results with techniques such as statistics that ignore meaning, or at least do not deal with it directly. Every now and then we have come back to attempt a semantic approach to things, but only with very limited success.

Of course semantics is hard. So why do I think it is time once again to bring to the fore the unpopular semantic approaches to managing text, document, content, and knowledge?

This blog will present a series of articles dedicated to this issue. Being a blog, I might occasionally take the liberty of discussing topics less directly related to semantic approaches to knowledge management.

Your comments and criticisms will be most helpful in debating these issues. May the better approach win!