Beyond that, Alessandro denies all the rumors that have grown out of TEMIS being hard to reach recently. He reports that pharma is still TEMIS’s big market, but stresses that this covers a range of apps, from research to Voice of the Market. I do get the sense that TEMIS’s sentiment extraction capabilities are less sophisticated than some of the other vendors’ — but the other vendors I’m thinking of are pretty focused on English, SPSS aside. If you need sentiment analysis in non-English languages — e.g., French or Italian — TEMIS should definitely be on your vendor shortlist.

• Voice of the Customer/Market applications require less linguistic sophistication than other text mining applications.
  
• Hence, Voice of the Customer/Market apps are easier to get running than other text mining applications, which he conjectures is a big part of the reason for burgeoning sales.

I’m guessing most text mining vendors would agree with those views, although they might not agree with his elaborations, which include:

• Attensity’s knowledge extraction technology is more sophisticated than Clarabridge’s or most other competitors’.
• In particular, Clarabridge’s extraction is little more than bag-of-words.
• There’s a good match between companies he thinks have less-sophisticated extraction (e.g., Clarabridge, SAS, SPSS) and companies whose text mining sales are heavily concentrated in Voice of the Customer/Market applications.

So the question arises: Just how much linguistic sophistication is needed in these market-trend-oriented text mining applications?

I actually got onto this subject not just because of what David said, but also via a conversation an hour earlier with Brooke Aker of Expert System, who proposed linguistic sophistication as a key reason for beating the competition (which, however, didn’t include Attensity or Clarabridge) at two accounts. The point Brooke was stressing is that it’s important to be able to extract multiple facts or indicators of sentiment from the same sentence. E.g., “I just had a crummy Chevy, but at least the seats were comfortable” is both a negative indicator about Chevrolet and a positive indicator about Chevrolet’s seats. Attensity captures both of those too, and I think Clarabridge would as well. (If you do comprehensive/ exhaustive extraction, you extract — well, you should extract comprehensively.)

Anyhow, my first-best answer to the question I posed is:

• Sentiment analysis is hard, at least in venues where you have to deal with slang, metaphor, or irony (the real biggie). The more sophisticated, the better.
• Otherwise, the linguistics of customer/marketing applications is pretty straightforward. Just put together the right list of wacky synonyms, and you’re good to go.

But what do you think?

• Expert System now has 145 employees.
• 2 of the employees are in the US (plus at least one more full-time equivalent on a contract basis). Brooke believes the US operation will eventually be the biggest part of the company.
• Expert System has sold its market intelligence SaaS offering to two global auto manufacturers. Competitors were Nielsen BuzzMetrics, somebody whose name sounded like “flexilytics” (I presume that would be Lexalytics Edit: But see Lexalytics CEO Jeff Catlin’s comment below), and somebody whose named sounded like “Truecast” (I haven’t yet guessed who that is).
• If I understood correctly, Expert System acquired that product by picking up Brooke’s tiny company Acuity Software. Acuity was/is a user of Expert System’s technology, having replaced Coveo’s with it so as to get better semantics.
• Brooke is optimistic about Expert System’s prospects in the intelligence market. New semantic networks in Arabic and English (joining one Expert System already had in Italian) are a big part of the reason. Brooke says the intelligence community is now actively interested in technology that’s been validated by the commercial market, on the theory it’s apt to be more complete than research/government-only products. Expert System is also working on a semantic network in another undisclosed Middle Eastern language; Brooke stoically refrained from confirming the blindingly obvious guess that this would be Farsi.
• Expert System’s third effort in the US market, coming soon, will be a semantic ad platform.

Once again, however, I made it through an Expert System briefing without gaining a real understanding of its technology. I gather they’re proud of their in-memory data structure for their semantic network, but I haven’t a clue (beyond the obvious guesses) as to what that data structure is. Similarly, Brooke said that a distinguishing feature of Expert Systems semantic network is that words have lots of attributes, which are the same thing as categories, and supplied a list of the 11 top-level categories: Objects, animals, plants, people, concepts, places, time, natural phenomena, state, quantity, group. But it’s easy to come up with a lot of things that don’t seem to fit that list very well (especially events, such as numerous different word-senses of “strike”), so absent further elucidation I didn’t find that particularly instructive either.

As of last year, Expert System also has English-language products, and a couple of minor OEM sales in the US (for mobile search and semantic web applications). German- and Arabic-language products are in beta test. The company says that its market focus going forward is national security – surely the reason for the Arabic – and competitive intelligence. It envisions selling through partners such as system integrators, although I think that makes more sense for the government market than it does vis-a-vis civilian companies. In February the company is introducing a market intelligence product focused on sentiment analysis.

Expert System is a bit of a throwback, in that it talks lovingly of the semantic network that informs its products. This semantic net was assembled in the usual way – start with WordNet, add a huge number of proper nouns, license a bunch of domain-specific dictionaries, and handcraft further as individual customers require it. In English the whole thing has 300,000 nodes and 1.2 million relationships.

Expert System insists that there’s a secret sauce in how the semantic net is organized, to optimize performance. But I haven’t gotten the slightest hint of what that magic data structure is — despite having asked more than once – and so have to reserve judgment on that part.

On the search side, Expert System sounds fairly rich in terms of deciding relevancy, going beyond Term Frequency/Inverse Document Frequency + synonyms. Terms are also assigned importance by their grammatical roles, such as whether they’re sentence subjects, sentence objects, in paragraph topic sentences, and so on. Of course, the whole concept of exploiting grammatical structure is a bit old-school. Specifically, it presupposes you’re looking at decently grammatical documents in the first place, which can be a dubious assumption in the era of “im in ur cuzztom3r bazz eatin ur r3venue$z.”

The most interesting application Expert System told me about was one for Pirelli Tire, scanning the web for prices Pirelli products were sold at, to detect gray market activity. Web-crawling for text analytics and price detection are both major activities – e.g., QL2 is active in both – but this is the first I’ve heard of the areas being so tightly integrated.