P2P Streaming and Long Tail: Like Oil on Water

...or Misunderstanding the Long Tail

Here is another problem I have with P2P streaming. It purportedly delivers long-tail content.

I have already discussed how P2P streaming does not work. In fact, it does work, well… kinda, sorta, but it is “the weakest link” among all streaming and all P2P technologies. I believe that means it is destined to end up on the fossil heap thanks to the propensity of the marketplace toward natural selection of better technologies. The 40% extra bandwidth usage required for P2P streaming, and the fact that most connections are asymmetric will kill any chance that P2P streaming will actually work in the marketplace. No ISP will incur 40% additional costs on your behalf even if you are a brilliant billionaire (okay that was a low jab, but appropriate).

Let us for the moment assume that Joost manages to circumvent the laws of either Physics or Economics, and manages to make P2P streaming technology work. Let us assume that Joost or Vudu, or their users are able to convince ISPs to change their offering – make connections symmetric and allow P2P on their networks. (Unlikely – even the University of Ohio has banned P2P on their campus, despite a lot of criticism – but bear with me). Or that Vudu has some technology (Voodoo?) up their sleeve that will make P2P streaming consume a lot less bandwidth. Again, we would have to rewrite our college Physics books, but really have faith, will you? :-) To be fair, Vudu may use P2P streaming only for quick starting of streams, and not for the entire stream.

So, now that we have P2P streaming, let us put some Long Tail content out there to stream. Let us say someone (lets call him Mustafa from San Diego) connects to the Turkish Television Channel on Joost and brings up a Turkish movie. Now, in order to serve this customer in the P2P streaming model, we need peers! Someone, anyone, watching the same movie at about the same time (or within the 2 hours built-in to the buffer)? In an on-demand world? Not too many! Besides Akimbo has cornered the market on Turkish Television! (Sorry, another low jab).

OK, lets suspend our belief one more time. Let us say there are a couple of other people watching the exact Long Tail content within plus or minus one hour from Mr. Mustafa. Mr. Taimur (and of course has a “lame” 400kbps connection) is on the same ISP (defying all logic, he lives right next to Mr. Mustafa) but Mr. Ataturk is on a different ISP (it doesn’t matter where he lives). If you look at the diagram below (extremely simplified for all you “non-technical” people) you will see that Mr. Mustafa and Mr. Taimur connect through the same router or DSLAM, or whatever their ISP uses at the POP.



Unfortunately as you can see, Mr. Ataturk connects to this peer network through the Internet and saves no bandwidth (nor does his ISP). Of course, his ISP is extremely unhappy since all it means is more bandwidth usage. So if you were the ISP, what would you do? I suggest that you have two choices – cap the bandwidth or charge more for it.

In other words, the P2P streaming provider has increased the bandwidth (in comparison to straight streaming delivered from a server) and has then shifted the bandwidth off their network on to that of the users’ ISPs. If you ran an ISP, would you put up with it?

Actually the news is even worse. Most Telco based ISPs use DSLAMs, and the traffic shaping is usually done on the DSLAM itself. So even though they are on the same ISP, Mr. Mustafa and Mr. Taimur, probably would not be able to do much for each other in P2P streaming.

We have not even talked about the QOS (Quality of Service) issues of Streaming Video which of course apply to P2P Streaming Video. I have argued ad nauseum that Streaming quality cannot be guaranteed on the public Internet because of all the nodes in between over which no one has control (and Joost’s technical paper admits that). And in the above example, we have assumed that all the peers are fairly close – what if the majority of these peers lived across the Atlantic? QOS would be non-existent – you can’t wish away latency at those distances.

Of course, Joost could forget about the long tail and focus more on the Fat Tail (the other end of the graph where the big numbers are). Yes, that would work, wouldn’t it? It might even work with P2P streaming assuming some of the physics were taken care of. Wait! There is another technology that serves the Fat Tail quite well – its called Cable TV!

Incidentally, L3 Technology works with both the Long Tail and the Fat Tail. And it uses P2P but in a download and play scenario. I will write next about how P2P really works in a real word scenario. Stay Tuned!

Misunderstanding P2P

Mea Culpa. I have been guilty of some lazy thinking about P2P. And it is unforgivable because my own technology L3, relies greatly on P2P.

If I seem fixated on Joost, please forgive me; they just happen to be in the news a lot these days. Plus, I have used Joost (as well as Jaman) and so my examples often use them. However, Joost will not be alone in having problems with P2P; Vudu, Zattoo, UUSee (China) and a host of others will have similar issues. Jaman, on the other hand is at least technologically sounder (in my opinion), because they use download-and-play and P2P works great with that. But Jaman has some other weaknesses. By the way, I quite enjoyed the videos I have been watching on both Joost and Jaman.

But, I am getting ahead of myself. Lets back up.

A quick pop-quiz. Of the following count those you would classify as P2P:

Kazaa, Skype, Napster, BitTorrent, Joost, Hotmail, YouTube, AIM

If you said all, give yourself a pat on the back. Now, it may sound like semantics, but yes, Hotmail – most email for that matter – is a Peer-to-Peer application. When it is not, it is usually Spam. But I am trying to make a point here. There are P2P applications and there are P2P technologies. Email and to a lesser degree a site like YouTube, are P2P applications. BitTorrent is primarily a P2P technology. Of late, BitTorrent is positioning itself as a distributor of video so it is morphing towards being both – a P2P application as well as a P2P technology.

In my earlier post (7 reasons why Joost could fail), I had mentioned that the creators of Joost were P2P people and so they used it in Joost. In fact, Skype is a P2P application and Joost uses P2P technology – in a sense they are apples and oranges. Hence, I was guilty of intellectual laziness. However, I still stand by the point of the argument in that article – that people use what they know, and that sometimes blindsides them.

Now that we can tell a P2P application from a P2P technology, let us look at two types of P2P technologies that can be used for Internet Video.

P2P Downloads – this is what BitTorrent does. You exchange files with others through your Internet connection. You can download the files any time and watch the video whenever you want (except instantly) – aka download and play.

P2P Streaming – what can I say? This is dead before you can say “Peer”. I had not really given it a lot of thought when I wrote the aforementioned article, yet I could tell there was square-peg-round-hole problem there. Which is why I simply relied on someone else’s quote that they found significant overhead in P2P streaming. I was right, but guilty of not following the thought process through to its logical conclusion.

It is easy enough to do the math. A typical home broadband connection (Cable or DSL) is asymmetric. My connection at home is 1536x256kbps. Which means I can download six times faster than I can upload. For the sake of simplicity, assume that everyone has the same connectivity. This means that if I were to use my DSL connection to watch Joost it would take six peers to service my viewing! Joost freely admits in this presentation by Colm MacCarthaigh – Joosts Network Architect that

Joost servers “top-up” the DSL“bandwidth gap”

In other words Joost will never be a purely P2P application. They will always need servers providing a majority of the bandwidth. My guess is that they may be able to save about 10% of their bandwidth by using P2P (assuming there are peers).

Now this is why the Download and the Streaming models of P2P are very different. In P2P streaming this situation requires six peers as shown above, whereas in P2P downloading you can have as few as one peer – it will simply take six times as long as to get the file. But in the latter case, when you are done downloading you can watch a flawless playback. In the former, you have no guarantee of the quality.

Also, unlike other streaming video offerings, Joost cannot make effective use of CDNs (Content Delivery Networks) to improve quality or save bandwidth. From the same presentation:

We’re willing to peer, but is there much point? Only portions of the long-tail are
peerable.

And

Joost servers also handle the “longtail” (which is still pretty long)

I may be mistaken, but it sounds like they don’t want to peer into other ISPs and so a CDN (which does exactly that) would not be of use to them.

There is another problem with P2P streaming. Since it is streaming, and streaming is real time, other peers have to be watching the same content at the same time as you are. Okay, you can stagger it a little bit, and perhaps that will help the first problem mentioned above (that is, reduce the number of peers I need). Again, since I am familiar with Joost, I know they use a 2GB buffer on my hard disk. So, some of the stream (I would guess about an hour and a half based on the quality) can be saved locally and this could be used to help out all those peers who are watching the same content at the same time as you are. Wait! Stop the presses! That sounds suspiciously like prime time! I wonder if we can get a Walter Cronkite clone!

I hope I have convinced you that P2P works best with downloading. With streaming P2P works hardly at all and if some people are to believed, it is even counter-productive. In fact I am a big fan of Peer-to-Peer distribution, because used correctly it does address the Long Tail. That is what my next article will be about. Watch this space!

P.S. Do read about the Long Tail of Bollywood here.