Notwithstanding the manifest virtues of centralization, there are numerous reasons you might want data marts,  in the tabular and document worlds alike.  For example:

• Price/performance. Your main/central data manager might be too expensive to support additional large specialized databases. Or different databases and applications might have sufficiently different profiles so as to get great price/performance from different kinds of data managers. This is particularly prevalent in the relational world, where each of column stores, sequentially-oriented row stores, and random I/O-oriented row stores have compelling use cases.
• Different SLAs (Service-Level Agreements). Similarly, different applications may have very different requirements for uptime, response time, and the like. (In the relational world, think of operational data stores.)
• Different security requirements. Different subsets of the data may need different levels of security. This is particularly prevalent in the document world, where security problems are not as well-solved as in the tabular arena, and where it’s common for a search engine to index across different corpuses with radically different levels of sensitivity.
• Integrated application and user interfaces. In the relational world, there’s a pretty clean separation between data management and interface logic; most serious business intelligence tools can talk to most DBMS. The document world is quite different. Some search engines bundle, for example, various kinds of faceted or parameterized search interfaces. What’s more, in public-facing search, a major differentiator is the facilities that the product offers for skewing search results.
• Different text applications require different thesauruses or taxonomy management systems. Ideally, those should all be integrated — but the requisite technology still doesn’t exist.

Bottom line: Text data marts, much like relational data marts, are almost surely here to stay.

Related link

• The future of data marts

*Care and feeding of celebrity tweeters. Celebrity tweeting is actually a subject I’ve written and even been interviewed about several times.

As of this writing, the full list is:

Job Title	Location	Department
Operations Engineer	Technical Operations
Operations Engineer – Search	Technical Operations
Product Manager	Product
Product Manager – Platform (Consumer Internet)	Product
Product Manager – Search	Product
UI Designer/Developer	Product
User Experience Designer	Product
Communications Associate (Contract to FTE)	G&A
Founder Associate	G&A
Software Engineer – Business Intelligence	Engineering
Software Engineer – Front-End	Engineering
Software Engineer – Ruby on Rails	Engineering
Software Engineer – Search	Engineering
Software Engineer – Systems	Engineering
Software Engineer – Systems Spam	Engineering
Director, Strategic Partnerships	Business Development
Japan Country Manager	Business Development

Related link

• Twitter’s analytics job opening

My opinions about the applicability of semantic technology include:

• The big bucks in web search are for “transactional” web search, and semantics isn’t the issue there. (Slides 3-4)
• When UIs finally go beyond the simple search box — e.g. to clusters/facets or to voice — semantics should have a role to play. (Slide 5)
• Public-facing site search depends — more than any other area of text analytics — on hand-tagging. (Slide 7)
• “Enterprise” search that searches specialized external databases could benefit from semantic technologies. (Slide
• True enterprise search could benefit from semantic technologies in multiple ways, but has other problems as well. (Slides 10-11)
• Semantics — specifically extraction — is central to custom publishing. (Slide 12 — upon review I regret using the word “sophisticated”)
• Semantics is central to text mining. (Slide 18)
• Semantics could play a big role in all sorts of exciting future developments. (Slide 19)

So what would your list be like?

• There are plenty of times when voice is a more convenient form of input than typing.
• Voice recognition was good but far from perfect.
• Editing search strings was annoyingly difficult.
• Search results themselves aren’t 100% perfect.

No big surprises there.

Below are some quotes — fair use!! — from the Authors Guild official statement on the deal (emphasis mine):

Our proposal to Google back in May 2006 was simple: while we don’t approve of your unauthorized scanning of our books and displaying snippets for profit, if you’re willing to do something far more ambitious and useful, and you’re willing to cut authors in for their fair share, then it would be our pleasure to work with you.

…

The payments will flow through the Book Rights Registry, a new independent entity that can be thought of as the writers’ equivalent of ASCAP. Much as ASCAP tracks the uses of songs and collects royalties for songwriters and musicians, the Registry will serve the interests of authors and others who own the rights to books appearing online as a result of this settlement. The Registry will be controlled by a board of authors and publishers; as part of the settlement, Google will pay $34.5 million to get the Registry up and running, notify rightsholders of the settlement, and process claims.

Readers are also big winners under the settlement of Authors Guild v. Google. Readers will be able to browse from their own computers an enormous collection of books. We hope this will encourage some readers to buy full online access to some of the books. Readers wanting to view books online in their entirety for free need only reacquaint themselves with their participating local public library: every public library building is entitled to a free, view-only license to the collection. College students working on term papers will be able to point their computers to resources other than Wikipedia, if they’re so inclined: students at subscribing institutions will be able to read and print out any books in the collection.

This is what writers have been — or at least should have been — awaiting for over a decade, ever since it became clear that the Web would transform media and publishing. With print-on-demand plus an online book registry, authors get complete access (starting in the US, at least) to readers, paying and otherwise.

So let’s review what publishers are good for. In some order, their role is:

• Marketing through branding/imprimatur.
• Marketing through advertising, book tours, and the like.
• Marketing/sales/distribution through the physical supply chain.
• Editing, art, and other actual contributions to the quality of the product.

It’s a publishing industry open secret that advertising and such like are pretty useless, doing more for the egos of all concerned than they do for actual book sales. Amazon is obsoleting most physical book stores, airport locations (for impulse purchases) and the like perhaps excepted.

As for branding/imprimatur: The backing of a major publisher can be worth a few thousand hard copy sales to libraries. But where else does it matter? I was going to suggest that it might in the academic world. But then I looked over at the math books on my shelves, a number of which are bound in familiar Springer Verlag yellow and white. Is there one book there I wouldn’t own if it weren’t a Springer Verlag publication? Probably not. I suspect that your initial reputation boost in academic publishing comes from your institution and peers much more than from an actual academic press.

So for the most part, book publishers and music publishers are left with just one marketing function — getting the ball rolling. Published products ultimately sell through word of mouth, but if you don’t start out with listeners or readers, how can the word of mouth build? The answer, of course, will increasingly be online promotion. If your natural audience is scattered around the country or the world, without being concentrated in one particular geographic location, online marketing is the obvious way to go. And in this world of search engines, YouTube, blogs, and the like, ever more channels for marketing are opening up.

Other than promotion and aggregation (the latter applying more to news/blog publishers than books/music), I do see one other role for publishers — actually creating product. Movies/TV and video games are both far bigger businesses than book publishing, and in both cases products are produced by large teams of people. Music is generally produced by small teams of people. And by the way, books can spin off from other kinds of entertainment (I suspect that a large fraction of all science fiction book sales at this point are Star Wars/Star Trek/etc.). Or vice-versa — some day the economic model for trying an ambitious new comic book project may factor in hoped-for movie and other spin-offs from the getgo.

But traditional publishers aren’t generally set up to do anything that ambitious. As for lesser but still important functions — editing, art direction, etc. — who needs large firms for that? The music industry seems to get by just fine with small recording studios and independent record producers; book producers could spring up just as well. Indeed, they exist already, but suffer from the problem that nobody wants to pay them much because book sales overall are so weak.

If I were a best-selling author, I’d hire away my favorite editor, put her on the payroll directly, and then send her and my agent to squeeze a few extra percentage points out of the big publishers, more than covering the cost of her paycheck. At least, I would if I were a best-selling author and also had my current personality. Now, I’m not, and if I were that kind of author I’d probably be quite fixated on being left alone to write, and might find it easier to leave money on the table than to take that kind of business responsibility myself. That kind of consideration is probably a big reason why traditional publishers are allowed to stay in business.

But the times sure are a’changin’.

MOUNTAIN VIEW–Information search giant Google, Inc. announced Thursday the release of Google Body, a search service aiming to index the internal and external anatomy of every living creature on the planet. …

Early testers have remarked upon a fuzzy-logic “match my organ” feature, which helps users get in touch with the nearest, most suitable donor for multiple organ systems. …

Responding to criticism from privacy groups, Google’s Hind pointed to the program’s opt-out policy. “We are very concerned about user privacy, and that’s why we will not make publicly available any information about anybody who let’s us know they do not want to participate by wearing an Opt-Out headband when in public. Google archives information about those individuals, but does not make it searchable.” The yellow and black vinyl headbands can be requested free of charge by writing to the company at its Mountain View headquarters.

I think this could be a really big deal. On most sites, it is already the case that Google gives much better results than the native search engine. (I generally use the site name or URL rather than the “site:” command to get this effect, but I imagine the difference is small.) Now that it’s easy to use, I expect increasing numbers of Web users to, like me, use the system that works better.

What’s more, there’s nothing stopping Google from co-opting the one way native site search is commonly made to work well, which is by hand-tagging of pages. Most metatags are devalued in search when used to compare pages from different sites, for very good reasons of adversarial information retrieval. But there’s little reason not to take metatags into account within a site, albeit with a little care to avert the worst potential abuses.

Large e-commerce sites running something like Inquira, Endeca, or Mercado will do better with their specialized search tools than with Google search. For most of the rest, however, Google is likely to win on quality, relevance, and utility.

Yes, there’s an obvious problem about ad placement and revenue, as Kee Hinckley recently noted to me. But:

• Site owners don’t really have a choice about cooperating with Google, as the searcher sees Google before she sees the target site. Going nuclear and removing oneself from the Google index isn’t a realistic option.
• Most large site owners do a lot of SEO (Search Engine Optimization) for Google, and get a lot of benefit from Google already.
• Google could cut special ad deals for single-site search, changing their usual policies to let site owners buy all placements, for a suitable fee.

Please sign up for our feed!

Given the Youtube explosion, I find it hard to argue with his claim.

That said, here’s the essence of Baynote’s story, as best I could figure it out.

• Baynote’s secret sauce is a set of 20+ behavioral metrics to identify whether, if somebody clicks on a page, they are SATISFIED with the content.
• Based on that, Baynote provides a “content recommendation” engine. (For now, the distinction between “content” and “web page” is not important, but the concepts are in my opinion diverging over time.) This is manifested in two forms (a typical installation uses both). One is just a list of recommendations. The other is in a search engine – “social search” with an “implicit folksonomy” — and its results pages. Both sit on web pages as boxes/widgets.
• Baynote’s first markets were online support and eMarketing. The company is now rolling out eCommerce as well. I didn’t get clarity about what was different in the nature of the recommendations, if anything, that underlies any small separation between these apps. (Baynote was clear about saying that the differences were indeed small.)
• The whole thing is SaaS, built on a LAMP stack. MySQL 4.something seems to suffice, which makes sense given that Baynote’s system is not handling any significant transactions directly. That said, I didn’t push to understand what it means for a search engine to be built on MySQL. This wasn’t the kind of conversation in which one could elicit substantive detail.
• Baynote claims that a sample size of as few as 7-10 visitors liking a particular piece of content suffices to provide a good basis for predicting who else will like it. I’m not in a position to assess the credibility or, more to the point, limitations of this claim.
• Baynote has the philosophy that they try to watch a user’s behavior on a site and map that to a “context.” I like that approach.
• The company cites tested stats of 20% net lift (revenue increase), with 50% of sales being touched by its recommendations. Those numbers don’t sound terribly impressive, perhaps unless they’re truly additive to those provided by, say, Endeca, which is an announced partner.