Natural language computer interfaces were introduced commercially about 15 years ago*.  They failed miserably.

*I.e., the early 1980s

For example, Artificial Intelligence Corporation’s Intellect was a natural language DBMS query/reporting/charting tool.  It was actually a pretty good product.  But it’s infamous among industry insiders as the product for which IBM, in one of its first software licensing deals, got about 1700 trial installations — and less than a 1% sales close rate.  Even its successor, Linguistic Technologies’ English Wizard*, doesn’t seem to be attracting many customers, despite consistently good product reviews.

*These days (i.e., in 2009) it’s owned by Progress and called EasyAsk. It still doesn’t seem to be selling well.

Another example was HAL, the natural language command interface to 1-2-3.  HAL is the product that first made Bill Gross (subsequently the founder of Knowledge Adventure and idealab!) and his brother Larry famous.  However, it achieved no success*, and was quickly dropped from Lotus’ product line.

*I loved the product personally. But I was sadly alone.

In retrospect, it’s obvious why natural language interfaces failed. First of all, they offered little advantage over the forms-and-menus paradigm that dominated enterprise computing in both the online-character-based and client-server-GUI eras.  If you couldn’t meet an application need with forms and menus, you couldn’t meet it with natural language either.

Even worse, NL actually had a couple of clear disadvantages versus traditional interfaces.  First of all, it required (ick!) typing, often more typing than the forms and menus did.  Second, forms and menus tell the user exactly what he can do. Natural language, however, lets him give orders the computer doesn’t know how to follow.  This is inefficient, not to mention frustrating.

However, even in 1983, it was obvious that the typing objection would go away some day, because of speech recognition — once desktop computers reached 100 MIPs or so.  (Effective keyboard-replacement speech recognition – as opposed to true natural language understanding — is mainly a matter of processing power.)  15 years later, standard PCs exceed 100 MIPs (assuming that 1 MIPs = a couple of megahertz for these purposes), and speech recognition is indeed getting practical.

In fact, as become increasingly evident recently, speech recognition is now a hot technology.  Bill Gates has been talking it up for a couple of years.  Increasingly, the press has swung to believing him … And my parents just bought a PC with two speech recognition products on it.

That said, speech recognition is as misunderstood (no pun intended) as most artificial intelligence technologies.  Yes, it beats typing, in a number of circumstances:

• On the telephone (duh!)
• “Busy hands” and/or “busy eyes” applications and locales (doctors‘ offices, trading floors, warehouses, etc. – and, some day in the future, your kitchen and car)
• People simply reluctant to type (e.g., anybody with sufficient wrist or back problems, and many males over the age of 45)

But before our computers talk back and forth with us in the voice of Majel Barrett Roddenberry, applications are going to have to add several important elements required for truly functional natural-language  interfaces:

• Intuitively clear names for everything on (or just behind) the screen
• Application-specific disambiguation logic

For most practical purposes, the latter requirement equates to

• A new generation of document selection technology

THE RULE OF NAMES

According to legend, knowing something’s name gives you power over it.  When that “something” is a button or menu choice on a speech-enabled computer, the legend is literally true.  But when a feature doesn’t have an obvious name, you can’t easily invoke it.

When applications consisted mainly of forms and menus, this was rarely a problem.  Everything had a clear role and label.  But web pages are less organized.  Hyperlinks can be scattered all over the place, with little rhyme or reason.

Frankly, I don’t think this is a hard problem to solve.  It wouldn’t take a lot of XML to divide the page into clear regions, so that commands like “Show me article #3″ (on a search results list) could be interpreted in the obvious way.  But it does take at least some discipline; random web pages will not necessarily be easy to “talk” to.

CYBERNETIC LISTENING SKILLS

The bigger challenge is to make sure that the application can respond in some useful way, no matter what command it’s given. This is even more difficult than it was 15 years ago, because of the radical increase in “casual” computer usage.  In the old days, we could assume the user had some clear business reason for using the application, and if necessary that s/he had time to be trained (even if people rarely sat still for as much training as they really needed).  Therefore, we could at least assume that the users had at least a general idea of what the application did, and hence of which commands the computer could obey.  From an NL standpoint, we could assume that what they actually “said” (which in those days meant “typed”) was at least reasonably close to what they were “supposed” to say.

Now, however, some of the most important applications are internet e-commerce and portals, competing and begging for the user’s attention.  The user is there strictly on a voluntary basis, and if he doesn’t get immediate gratification, he‘s gone, history, hasta la bye-bye.  Site-specific training isn’t even a consideration. And even if somebody did actually take a class on “How to use Excite,” the knowledge would be obsolete in six months.  So applications, if they are to have natural language interfaces that please and respond to users, have to be able to respond pretty much to any command.

Ideally, voice-enabled systems would be like the computers on Star Trek, which can return information from vast archives, brew a pot of Earl Grey tea, play three parts of a quartet, create self-aware life forms, or answer questions like “Computer, what is the nature of the universe?”  More realistically, they should be able, for example, to respond to a command like “Tell me about flights to Miami” by automatically giving the user a travel-reservation application or web page, and entering Miami in the appropriate form field.

If one thinks about the complications in such a system, it becomes clear that there are only two possible ways an application system can be designed to respond meaningfully to an enormous range of reasonable possible requests.

1. It can do the equivalent of saying “I’m sorry, I didn’t understand that,” “I’m sorry, I can’t do that,” and so on.

2. It can interpret many commands as text-search strings, and return appropriate results.

The first strategy – application-specific disambiguation logic, clear responses to “errors,” etc. — is absolutely necessary.  No software is perfectly intelligent; the user will have to be asked for disambiguation help from time to time (just as clerks today ask customers to repeat their requests!). I’m not going to go into much detail about how that works because, frankly, it’s a tricky thing to get right.  Users hate unnecessary disambiguation steps. They also hate the incorrect responses that result from ambiguity, and do tolerate being asked for help when it’s truly needed.  In short, whatever you build the first time around will probably be wrong.  So build something fast; then run, don’t walk, to the nearest usability lab, find out how you screwed up, and redo your system until you get it right.

I’m convinced that the second strategy — heavy reliance on text search technology — is a requirement as well. Just try to name a major web site that doesn’t use text search.  True, text search has gotten a bad rap recently, mainly because a whole generation of search engines didn’t really work.  But it will stage a comeback.

Related links

• My December, 2007 survey of speech recognition technology
• Star Trek fun

]]> http://www.texttechnologies.com/2009/05/30/men-are-from-earth-computers-are-from-vulcan/feed/ 2 http://www.texttechnologies.com/2009/05/29/google-wave-finally-a-microsoft-killer/ http://www.texttechnologies.com/2009/05/29/google-wave-finally-a-microsoft-killer/#comments Fri, 29 May 2009 09:49:24 +0000 Curt Monash http://www.texttechnologies.com/?p=330 Google held a superbly-received preview of a new technology called Google Wave, which promises to “reinvent communication.” In simplest terms, Google Wave is a software platform that:

• Offers the possibility to improve upon a broad range of communication, collaboration, and/or text-based product categories, such as:
  • Search
  • Word processing
  • E-mail
  • Instant messaging
  • Microblogging
  • Blogging
  • Mini-portals (Facebook-style)
  • Mini-portals (Sharepoint-style)
• In particular, allows these applications to be both much more integrated and interactive than they now are.
• Will have open developer APIs.
• WIll be open-sourced.

If this all works out, Google Wave could play merry hell with Microsoft Outlook, Microsoft Word, Microsoft Exchange, Microsoft SharePoint, and more.

I suspect it will.

And by the way, there’s a cool “natural language” angle as well.

For starters, here are some basic links:

• Google has naturally set up a home page for the Google Wave project.
• Featured on that page but also separately available is an 80-minute video introducing Google Wave.
• Techcrunch has two highly detailed posts on Google Wave, one summarizing what’s in the main Google Wave video and one reporting on a Google Wave Q&A.

Here are some reasons I think Google Wave could actually live up to its promise:

• The email problem Google Wave purports to solve is real and critical. The email paradigm assumes linear conversations, and what actually happens is that they branch. Google Wave’s message-board-like paradigm is simply better, and more flexible (e.g., not limited to a single enterprise!) than Microsoft Exchange or Lotus Notes.
• The instant messaging problems Google Wave purports to solve are also major. Instant messaging is slow, tedious, disjointed, and ephemeral. Fully integrating IM with email solves most of those problems. And Google Wave’s UI interactivity solves most of the rest.
• Twitter needs to be integrated with other forms of communication. What’s more, Twitter’s functionality needs to be drastically extended. Google Wave is the best hope I know of to meet those needs.  Enterprise Twitter is just a special case of that.
• Workgroups (enterprise or otherwise) need light-weight mini-portals that can be created on the fly by non-technical users, to ease collaboration. Microsoft SharePoint, SAP Rooms, et al. don’t really meet that need.  Google Wave could.
• In particular, collaboration on documents, presentations and so on needs to be more cloud-based and generally easier than is the case in Microsoft Office. Google Wave has the potential to provide that.
• Google + open source is a potentially potent combination, especially versus Microsoft.

One note: Google of course needs to improve the reliability and customer service of its cloud-based offerings to make a huge dent in Microsoft’s market. But even with its flaws Google has already been a good alternative for a while.

As for the “natural language” angle: At the 44:30 mark of the main Google Wave video is a demo of some cool, very grammar-sensitive spell-checking technology. Google’s spell-checking technology is further discussed in a separate, short video.  The basic idea is that Google uses its vast library of web pages — and email and chat? — not just to model intended word usage but also kinds of mis-spelling behavior as well.

Thanks for your mention to LanguageWare on your blog, albeit a skeptical one I totally understand your scepticism as there is so much talk about text analytics these days and everyone believes they have solved the problem. I guess I can only hope that our approach will indeed prove to be different and offers some new and interesting perspectives.

The key differentiation in our approach is that we have completely decoupled the language model from the code that runs the analysis. This has been generalized to a set of data-driven algorithms that apply across many languages so that you can have an approach that makes the solution hugely and rapidly customizable (without having to change code). It is this flexibility that we believe is core to realizing multi-lingual and multi-domain text analysis applications in a real-word scenario. This customization environment is available for download from Alphaworks, http://www.alphaworks.ibm.com/tech/lrw, and we would love to get feedback from your community.

On your point about performance, we actually consider UIMA one of our greatest performance optimizations and core to our design. The point about one-pass is that we never go back over the same piece of text twice at the same “level” and take a very careful approach when defining our UIMA Annotators. Certain layers of language processing just don’t make sense to split up due to their interconnectedness and therefore we create our UIMA annotators according to where they sit in the overall processing layers. That’s the key point.

Anyway those are my thoughts, and thanks again for the mention. It’s really great to see these topics being discussed in an open and challenging forum.

By the way — I by no means want to single out IBM’s natural language efforts for especial bashing. The AI industry’s unit of bogosity has long been the “microlenat,” and Doug Lenat’s life work is, approximately, solving natural language access. I sat next to Doug at dinner at an IJCAI/AAAI conference in 1985 or so. So far as I can tell, what he told me about then still hasn’t been delivered in real life. I’m not aware of any connection between Lenat and IBM.

What’s different this time, apparently, is a rigorous focus on performance. Marie and her team seem to believe that what has held natural language processing back in the past has been poor performance. That’s not as crazy as it sounds, since natural language may be one of those artificial intelligence problems in which brute force outperforms sophisticated heuristics (Lenatesque or otherwise). Still, I have to wonder if performance has really been the main problem.

One interesting side note is that a reason given for this great performance is that processing is done in one pass rather than several. Since seems to directly contradict the philosophy of UIMA, IBM’s proposed general-purpose text analytic industry standard. And it’s tough to see how that architectural choice alone can produce enough of a performance advantage to be a game-change.

The link I gave above already has quite a bit of material. Marie tells me that more and/or fresher material is coming soon.

I don’t think I’ll want to play with it for a while (I’m heading off on vacation for a while), so I thought I’d post it here to see if anybody else had any thoughts about or familiarity with it.

Related link

• Russian chatbot apparently passes Turing Test

Let me begin by explaining what I mean by some of that list (taken from Slide 2), starting from the bottom.

• Machine translation is a small business, with small specialized vendors. Lernout & Hauspie attempted to combine it with voice recognition in a complex financial play, but that collapsed in a miasma of stock fraud. The remnants turned into what became Nuance Communications.
• Nuance is a roll-up of most of the important independent voice recognition vendors. So far voice recognition has worked best in two areas: “Hands-free” computer use/dictation, and IVR (interactive voice response). While both are important, neither is exactly a mainstream enterprise computer software business. So voice recognition is not closely integrated with the other market segments.
• “Natural language processing” other than voice recognition isn’t much of a business at this time (with apologies to Progress EasyAsk). It doesn’t make the list at all.
• Spam filtering is obviously a major business, whether or not it is getting combined into more general security and/or messaging product suites. Antispam vendors actually perform a lot of machine learning, much like text miners do. But the types of rules they wind up with are quite different. And their hardest problems aren’t linguistic ones, usually, as the spammers have gone beyond text to, e.g., words depicted in graphical images. Besides, even where linguistics are involved, it’s a very different problem to identify words used by bad guys trying to spoof you (and the rest of the world) than it is to understand your particular users.

Why and to what extent I see the other five as separate markets was explained in connection with the subsequent 17 slides.

1. Web search

2. Public-facing site search

3. Enterprise search and knowledge management

4. Custom publishing

5. Text mining and extraction

Three are more standalone:

6. Spam filtering

7. Voice recognition

8. Machine translation

This list comes from a talk I gave Monday at the Text Analytics Summit called The Text Analytics Marketplace: Competitive landscape and trends. In half an hour, I covered the first five areas (in Sue Feldman’s word, at a “gallop”). The slide deck has been uploaded to the link below. I plan to break out the material from the talk into a series of blog posts over the next few (or perhaps not-so-few) weeks.

Slides:

• The Text Analytics Marketplace: Competitive landscape and trends

Other posts based on those slides:

• Three specialized markets for text analytics (based on Slide 2)
• 6 trends that could shake up the text analytics market (based on Slide 19)
• Why search technologies are going to recombine (in A World of Bytes, based on Slide 19)
  

Here’s much of what I know or believe about speech recognition:

• Most major independent commercial speech recognition efforts have wound up being merged into Nuance Communications. That goes for both desktop and server-side stuff. None was doing particularly well before its respective merger.
• A folk dance buddy (Jonathan Young, once of Dragon Systems) taught me the essential principle of developing speech recognition systems, which probably applies more broadly to other language-understanding technologies as well: “How do you make a good speech recognition product? You start with a bad one and keep incrementally improving it.”
• Linda tells me that a lot of novelists use dictation software, to reduce repetitive strain from typing. However, this often leads to repetitive use strains on their throats. I don’t know whether it makes a difference if one uses better microphones, talks more softly, and/or has access to software that is less demanding of carefully enunciated gaps between each word.
• Perhaps due to accuracy concerns, and perhaps also due to concern about noise pollution in the workplace, ordinary computer control via voice is rare. Most applications focus on specialized-circumstance dictation (hands-free, disabled users, users who are being harmed by typing, etc.) or telephone interaction.
• Rich semantic technology isn’t yet used in speech recognition to nearly the extent it is in text search/mining/analytics. The grammar in speech recognition systems is primitive at best. And while there may be some hand-built semantic networks with small numbers of nodes, ala Sybase AnswersAnywhere, nobody’s ever hooked up (say) a WordNet equivalent or a good entity-extraction engine as part of a mainstream commercial speech recognition product. (Please correct me if I’m wrong about this part!)
• There are real challenges in voice recognition via remote microphones in small enclosed places (e.g., automobiles), especially when noisy. But wearing headsets while driving is generally frowned on by the traffic police. EDIT: It seems that those challenges are being overcome.
• Overall, I can’t think of anything wrong in this Wikipedia article on Dragon NaturallySpeaking. That said, the article is a bit sloppy, so I’d encourage people to see if they can edit it a bit and spruce it up.

Any thoughts? In particular, what version of Dragon NaturallySpeaking or a competitive product should Linda use, and why?

One second after the phone hears speech in Japanese, the cellphone with the new technology shows the text on the screen. One second later, an English version appears. …

“We would need to study how to recognise [sic] voices on the phone precisely. Another problem would be how the person on the other side of the line could know if his or her words are being translated correctly,” he said.

Stay informed! No hassle, no spam — all it takes is an email address or an RSS subscription! Get all our research, or just the text analytics part, or just get a very few notifications of our most important news.

Technorati Tags: NEC, machine translation, speech recognition, NIPNY

*Other company or product names in the mix at various times include AI Corp and English Wizard. Not inappropriately, it seems that Larry has quite an affinity for synonyms …

EasyAsk is still a small business. The bulk is still in enterprise query, but new activity is concentrated on e-commerce applications. While Larry thinks that they’ve solved most of the other technical problems that have bedeviled him over the past three decades, the system still takes too long to implement. His rough rule of thumb is that implementation — i.e., building the thesaurus – takes 10% as much effort as overall database design did in the first place. That comment leads to what seems to me to be a pretty obvious suggestion: Focus on selling to sites that have already installed a “semantic layer” (Business Objects’ term) or the equivalent while setting up their BI system – whether or not EasyAsk can get actual partnerships with BOBJ, Cognos, et al. And I’ll stop right there, because I’m not sure whether Larry’s comments on what they have or haven’t done in that regard were meant as general briefing material, or were under NDA in our client relationship.

In the e-commerce area, EasyAsk is a direct competitor to Mercado, with a lot of analogous functionality. As previously noted, they claim their users enjoy particularly strong revenue benefits. After talking about it with Larry, I now feel it’s a sincere and credible claim. Neither he nor I would claim the case has been proven with shining statistical rigor (and my doubts as to its fundamental merits remain greater than his). But Larry is a smart and honest man, and when we discussed it I didn’t happen to catch him in any obvious and uncharacteristic thinking errors.

Takeaways from the conversation included:

• Customers are often software OEMs, especially in the health care and human resource areas.

• SQL generation times are down to a millisecond or so.

• MDX is a future direction for them, as an alternative to SQL. To the extent MDX syntax is ugly, that’s a plus for them, not a minus!

• Larry feels they’ve gotten a lot better at navigating people to canned reports and the like, rather than just re-executing queries for them. On the other hand, when I told him a vision I first pitched on his behalf (and to him) in 1984 about application command-and-control, I didn’t get a lot of recognition. Too bad. That kind of thing is central to natural language’s potential ability to solve some of the inherent problems of dashboards.

• The obvious voice recognition/natural language pairing now works pretty well for known users, but isn’t good yet for web applications.

• Since being acquired by Progress they’ve become much more multilingual. (I guess the name “English Wizard” gives a clue as to how multilingual they used to be.)

Keep getting great research about text analytics, data management and related technologies. Get a FREE subscription by RSS/Atom or e-mail!

Technorati Tags: Progress Software, EasyAsk, natural language, Mercado