"The Long Tail": a review

I've just finished reading "The Long Tail": Chris Anderson has written a book that will have a significant impact on the way we think about markets.

Or at least, about some markets.

Anderson posits that while we were so far living in a world of reduced choice, of "scarcity", mainly due to the constraints of the physical world (think: distribution costs), the digitization of cultural and entertainment products (think: music downloads) or at least the digitization of their findability (buying a book through Amazon rather than at the local bookseller) is creating a world of endless choice, of innumerable niches, "of abundance". That, in turn, creates "unlimited demand": if you look at *the true shape of demand in our culture, unfiltered by the economics of scarcity", he writes, "basically everything you put out there finds demand".

Anderson first wrote about the "Long Tail" in the magazine he edits, Wired, in October 2004. It was an 8-pages article subtitled as follows: "Forget squeezing millions from a few megahits at the top of the charts. The future of entertainment is in the millions of niche markets at the shallow end of the bitstream". It was a groundbreaking text - and it was received as such, becoming one of the most-discussed articles Wired has ever published. Analyzing data from companies such as online merchant Amazon, DVD service Netflix and music service Rhapsody, Anderson basically suggested that the culture of "hits" and "bestsellers" in which we have been living is unnatural:  by artificially limiting choice, it negates the individuality of each customer/user/consumer, their diversity, the real scope of their interests and passions and lifestyles. This culture is mainly the creation of two major physical constraints: the limited shelf space (there is only a finite number of products that a bookstore, record store, movie store, newsstand, supermarket or specialty store can carry at any given moment - and of course they tend to carry the products that sell best) and the impossibility to cost-efficiently aggregate dispersed audiences or customer groups (shop owners decide on the allocation of that limited shelf space to a specific product based on the existence, locally, of a critical mass of customers interested in it). That's why only a fraction of the books published and records released every year (not to mention those released in the past) are available at the bookseller or record store. That's why the average local newsstand in, say, Liverpool is not likely to carry Austrian daily papers or Finnish fashion magazines. That's why it's difficult to find Asian food stores or sushi restaurants in villages in the Swiss Alps, while they abound in Zurich. That's why every movie theater within driving distance seems to be showing the same Tom Cruise movie: because you may want to see that great old Turkish movie tonight, but with a public of one the theatre owners would not recoup his costs, so he allocates his limited screen space to movies that may attract a critical mass of viewers (that same old Turkish movie isn't to be found at the DVD rental either, for the exact same reason: limited shelf space).

The Long Tail is about that Turkish movie: it's about the infinite number of niches and micro-niches and nano-niches that match individual and small-group interests and passions but reside beyond the limited shelf spaces of physical life. In the book he calls it "a market of multitudes": "increasingly, the mass market is turning into a mass of niches".

That a market structure generally looks like a power-law distribution (a continuum with a few big products/service/players/etc at the "head" and many others progressively smaller in the "tail") is no news. Anderson's insight is that in the physical world, the Long Tail is rather short and fiinite: after so many CDs even the biggest record stores runs out of shelf space. But in the digital world that shelf space becomes infinite: the Long Tail can stretch endlessly, creating or awakening a potentially limitless number of niche markets that either didn't exist, or used to uneconomic to serve.

Example: If, in a given city, only two people are interested in a specific music subgenre, it's unlikely that the local music store will carry those CDs. Such is the tyranny of geography. But iTunes and its competitors can offer all the available music (because the marginal cost for them for doing so is close to zero: it's just some bytes stored on a server) and consolidate the demand from people living thousands of miles apart, making that niche commercially interesting ("suddenly, popularity no longer has a monopoly on profitability", Anderson writes).

In a way, by analysing the way economic and cultural systems evolve, Anderson makes a strong case for how culture has been so far shaped by the physical infrastructure of distribution and access ("Many of our assumptions about popular taste are actually artifacts of poor supply-and-demand matching"), reflecting classical Marxist thinking that the "infrastructure" dialectically determines the "superstructure".

If the Long Tail is, basically, about reducing the costs of reaching niche markets, then the digitization of distribution ("democratizing distribution") is only one of three forces at play here, writes Anderson. The others are the democratization of the tools of production (basically, the computer hardware and software that allows people to self-produce any kind of content at almost-professional quality levels) and, on the other side of distribution, the new ways to connect supply and demand: Google and other search systems; recommendation engines ("customers who bought this book also purchased…"), personalized online radio channels and other such filters.

This last point is about findability, and is a crucial one: the larger the choice, the more we need help in finding the stuff we may like, and finding it quickly. Recommendations act as a shortcut through the growing mountain of information and content. Anderson has a telling anecdote on how this works. In 1988, UK climber Joe Simpson wrote "Touching the Void", an account of a near-death experience in the Peruvian mountains. The book got good reviews but reached only a limited readership, and was soon forgotten and went almost out of print. Until ten years later, when John Krakauer wrote "Into Thin Air", also a book about a mountaineering tragedy, this time on the Everest, which became a bestseller. Suddenly, "Touching the Void" found a new life. Anderson ascribes that basically to Amazon's "Long Tail filters": their readers-authored reviews, and their recommendation engine. "When "Into Thin Air" came out, a few readers wrote reviews of on Amazon.com that pointed out the similarities with the then lesser-known "Touching the Void", which they praised effusively. Other shoppers read those reviews, checked out the older book, and added it to their shopping carts. Pretty soon the Amazon software noted the patterns in buying behavior - "Readers who bought ITA also bought TTV" - and started recommending the two as a pair. People took the suggestion, agreed wholeheartedly, wrote more reviews. More sales, more algorithm-fueled recommendations - and a powerful positive feedback look kicked in". Simpson's book was later turned into a movie and at a certain point it was outselling "Into Thin Air". Writes Anderson: "A decade ago, readers of Krakauer would never even have learned about Simpson's book - and if they had, they wouldn't have been able to find it. By combining infinite shelf space with real-time information about buying trends and public opinion, they created the entire "Touching the Void" phenomenon".

It is clear from this example that the future economics of entertainment will be radically different from the past - hits aren't going away, at least the good and genuine ones, but more and more resources and attention will be diverted towards an increasing number of niches. (In the original article though, Anderson wrote that the sum of the niches in the Long Tail would be "a market bigger than the hits"; in the book, he has scaled that down a bit, and he speaks only of "a significant market").

One of the least convincing portions of the book is when, using as examples amateur astronomers discovering major phenomena or Wikipedia writers compiling that amazing encyclopedia, Anderson tries to make the point that "we are at the dawn of an age where most producers in any domain are unpaid". That's a radical statement. He seems to be assuming that while the whole economics of entertainment and culture will be reshuffled, the distribution of work won't: but as attention moves down the Tail, so will rewards (although a big part of the Tail will continue to exist in a non-monetary reality).

I've some disagreement also with the notion (mentioned at the beginning of this post) that "everything you put out finds a demand". That's definitely not the case, despite Anderson's numbers referring to music services such as Rhapsody. Consider for example the analysis of SourceForge open-source software projects released by Clay Shirky at the Aula06 conference in June: 75% of the 122'000 software programs available on that platform are never downloaded, not a single time. These data contradict frontally Anderson's, and tend to suggest that the entertainment market that he has analyzed so well may actually behave differently than others. Anderson does try to apply his theory to other fields - "the long tail of fashion", "of national security", "of beer" - but does so unconvincingly (I'm wondering whether he threw in those few unsubstantiated paragraphs following an editorial suggestion to "generalize the theory").

Another limitation of "The Long Tail" in my eyes, is that all - ALL - the data exposed and analyzed in the book are from US firms, markets, and players, and all the examples (websites, newspapers, rock groups, blogs, television series, etc) he quotes are from the US as well. Anderson does actually mention the Encyclopedia Britannica, only briefly in the chapter about Wikipedia; and has fleeting mentions of Skype and Korea's OhMyNews and India's Rediff; and, yes, he devotes two pages to Lego, which was invented in Denmark (detail not mentioned in the book), but that's it for the book's global view.

"The Long Tail" is a strongly-researched, insightful and important book, which discusses much more than I've been able to include in this post. However, as the subtitle of the original article said, it is a book about the changing economics of entertainment and culture in the United States. It contains very little to suggest that the theory applies to other markets - "other" both by sector, structure and geography. It is likely, but the data substantiating it are not in this book.

PS: the discussion of the Long Tail continues on Chris Anderson's blog.

Comments

[Peter Warne]

The encyclopaedia may have a British name, but that's a false friend; it's been firmly in United States hands for more than a hundred years (http://corporate.britannica.com/company_info.html). So even that isn't giving the non-US perspective that the book might.
I don't personally think there is anything (currently) to suggest that the phenomenon under discussion exists anywhere outside "The changing economics of entertainment in the United States."

[kreardon]

Bruno: I was writing a blog the other day about the limitations of blogs – the interactivity limits (blogs followed by comments to which responses usually don’t follow). In this sense, blogs, like all communication channels have their limits depending on their purposes. If our purpose, for example, is to identify a means of bringing about peace, then blogs are useful as vehicles to convey suggestions but fall short in terms of the kinds of dialogues we really need to accomplish the goal. A blog is posted, people respond, but the blogger rarely responds to them or to other bloggers. In fact, I was musing about how we’ve yet to invent a medium that is superb at enabling that kind of dialogue – of facilitating access to great ideas out there from people who are less known. So, while the long tail concept is indeed intriguing, we are still quite limited in terms of where we get information and certainly how it comes to our attention, despite Google. Even as Chris Anderson noted on his quite interesting site, his book, ironically, sold the old fashioned way – an author fortunately getting the right publicist or a crucial review in the mainstream media. I’ll really believe in the power of the “long tail” when I see that contrived limited access highway to visibility split into a thousand small avenues and some truly surprising ones followed by some great success stories. And we’ll all rejoice when we find a way to bypass current media limitations to create a blog-like medium that holds our attention on particular subjects, provides true dialogue, and thereby facilitates the solving of the most significant crises of our time. (Here’s a site that’s way at the tip of the tail – a niche for sure – that may help people if they find it -- http://web.mac.com/reardon_on_painting ). Or maybe the mainstream media will just have to find it! Kathleen

[BG]

Kathleen: I totally agree with the premise of your comment: in their current form, blogs aren't really a medium for "conversations". They're somehow just an inch beyond the definition of self-publishing tools, but still way on this side of becoming a platform for a sustained dialogue. I had a discussion on this very topic earlier this year with Robert Scoble (which I blogged here: http://giussani.typepad.com/loip/2006/02/blogging_frustr.html) and we put forth some very initial ideas to increase the "conversationality" ratio of blogs. And of course there are also non-tech limitations to global dialogue: language and time in particular. But some bloggers are starting to, as you say, "hold our attention on particular subjects" by becoming the center of small or big communities of interest, by directing the traffic within those communities. That's, I believe, also the new role of journalists ("mainstream media").

[LP]

Readers might be interested in this talk by Chris Anderson, provided by itconversations.com:
http://www.itconversations.com/shows/detail477.html
Quote: "Economics of the Long Tail: [snip]
The talk by Chris Anderson is followed by a conversation with Joe Kraus, the CEO of Jotspot, in which they discuss how the failure to understand the significance of catering to the long tail of the advertising market led to the downfall of the Excite search engine. Joe also talks about his experience of currently running a company whose products are targeted at the long tail of the market."