Google's Inflection Point
2009-04-05 12:31PM PDT/Home
There's an interesting inflection point for Google, and to a lesser extent all search engines, Yahoo! included (disclaimer: where I'm employed). My posited inflection occurs when the value of a piece of content appearing in Google's index equals the value of the traffic that Google sends to view that content, and the publisher realizes it. Go past the inflection point, and the situation becomes unstable. When publishing content is worth more to Google, or Yahoo! or any search engine, than to the publisher itself, sooner or later they'll want to be properly compensated.
That's what I was thinking of as I read Google is just an amoral menace. Here's a choice quote:
… what we are seeing is a much older and toxic capitalist model - the classic monopoly that destroys industries and individual enterprise in its bid for ever greater profits. Despite its diversification, Google is in the final analysis a parasite that creates nothing, merely offering little aggregation, lists and the ordering of information generated by people who have invested their capital, skill and time.
I've been waiting for this point, and have started to see hints we're almost on top of it. The question then becomes, well, what happens next?
I think the next logical step is for newspapers and publishers to start charging to allow their content to be indexed. A mechanism for excluding index programs already exists. It's what a robots.txt file is for, and it's very straightforward. Going a little further along this line of thought, one could imagine automating the robots.txt file across hundreds of publishers creating much larger leverage. Going even further and one could imagine publishers themselves interfacing directly with search engines, controlling how their content gets indexed and creating a marketplace for their content.
In this future marketplace, a publisher and indexer could negotiate and possibly indexer bid over what keywords a piece of content is allowed to be indexed under. A higher quality (to the publisher) traffic generating keyword, but low advertising value keyword might be very cheap for a search engine to purchase. A lower quality traffic generating, higher ad value keyword might be much much more expensive.
Update: Another data point, Robert Thomson the editor of the Wall Street Journal is quoted in The Austrailian
“Google argues they drive traffic to sites, but the whole Google sensibility is inimical to traditional brand loyalty,” he said.
“Google encourages promiscuity — and shamelessly so — and therefore a significant proportion of their users don't necessarily associate that content with the creator.”
“Therefore revenue that should be associated with the creator is not garnered.”
While he understands that the value is now residing in Google's index and not with the publisher, he mentions e-readers as a possible solution. Unfortunately, I don't think that's going to work out well.
Update: Alexander Macgillivray, a lawyer for Google responded.
We drive traffic and provide advertising in support of all business models -- whether news sources choose to host their articles with us or on their own sites, and whether their business model is ad-supported or based on subscriptions. In all cases, for news articles we've crawled and indexed but do not host, we show users just enough to make them want to read more -- the headline, a "snippet" of a line or two of text and a link back to to the news publisher's website.
This was largely a we're-not-breaking-any-copyright-laws style response, but he does mention robots.txt, and meta tags as an option for publishers to opt out of Google's index.
My disclaimer: I work for Yahoo! My opinions are mine, and mine alone.
Update: Danny Sullivan fires back with Google's Love For Newspapers & How Little They Appreciate It
Newspapers get special treatment, both with First Click Free and with the extraordinary amount of traffic they get from Google. And while their top managers go off on renewed Google rampages, they still continue to work to get even more traffic. It is stunning hypocrisy, and certainly not what you'd expect from smart business people. But given how badly their papers seem to be going, I suppose they aren't so smart.
He argues, and not in these words, but he argues that the value Google sends to Newspapers is greater than the value of the article appearing in Google's index. And I'd agree, Google and Yahoo, and all the other search engines do send a lot of traffic to Newspapers. My only quibble is that he leaves out the ugly reality that newspapers are dying from lack of ad and subscription revenue.
Update: Nicholas Carr wrote perhaps the best analysis of the situation so far: Google in the middle.
Google's overriding interest is to (a) maximize the amount and velocity of the traffic flowing through the web and (b) ensure that as large a percentage of that traffic as possible goes through its search engine and is exposed to its ads. One of the most important ways it accomplishes that goal is to promote the distribution of as much free content as possible through as many sites as possible on the web. For Google, any concentration of traffic at content sites is anathema; it would represent a shift of power from the middleman to the supplier. Google wants to keep that traffic fragmented. The suppliers of news have precisely the opposite goal.
He argues that the solution is to drastically reduce supply, meaning lots of newspapers and publishers need to go out of business before the supply side will once again gain the upper hand.