Peer-to-Peer Technology to Trump Traditional Cloud Storage

On a lovely Thursday afternoon in June, I sat down with Sarah Koo, a data scientist with BitTorrent. I grew up in a time and place where Peer-to-Peer (P2P) transfer was almost commonplace, and I was fortunate enough to have the opportunity to learn more about this unique technology.

 

When Peer-to-Peer transfer is mentioned, the subject of copyright protection is the first thing to come to mind. Though P2P transfer is used, at times, to download Game of Thrones, the technology itself is an innovative protocol designed to efficiently transfer large files. I want to make an explicit statement that neither I, Sarah, nor BitTorrent, condones any forms of illegal file transfers. I am advocating the technology itself, which is commonly used in both academia and business. 

 

Bram Cohen created the Peer-to-Peer transfer protocol about a decade ago. This genius literally coded the program by himself in his bedroom, like many of our city’s tech entrepreneurs. Traditional file transfer protocols connected downloaders to a single server for direct file transfers. If there were 100 users downloading the same file, all these users would connect to the same server. Such traditional methods resulted in significant speed reductions and heavy server strain as demand for the file increased. Cohen's application created a distinct virtual network per file that allowed any invited users to help with distribution.

a protocol that created a virtual network based on the file itself, thus anyone within the network can help distribute the file

Lets say UC Berkeley is working on a massive database, and it needs to distribute it to 10,000 researchers across the country. If each of these 10,000 researchers downloads the file directly from the Cal server, the strain on the single server is intense and extremely inefficient. P2P transfer creates a network that links all invited researchers together, and once any researcher finishes downloading the database, he can act as a file distributor for other researchers within the network. Thus, the strain of distributing the database is crowd sourced to all its participants. 

BitTorrent, the most popular P2P transfer application, breaks down large files into small segments for even more efficient distribution. If the database described above is 20-gigabytes; BitTorrent could break down the file into 20 equal sized 1-gigabyte files. “If I request the file from peers, I can be getting it from 20 different people at the same time.” Thus, each peer only needs to distribute 1/20th of the overall load.

breaks down large files into small segments for more efficient distribution

Because BitTorrent breaks down files into many individual segments, peers can act as distributors the moment they finish downloading any piece of the overall file. Once all the segments are downloaded, BitTorrent reassembles the original file as whole. The ability to deconstruct files and then reassemble the segments after the transfer protocol is complete is groundbreaking, especially in the field of cloud storage.

BitTorrent has utilized this technology to build sync applications that are commonly used in industries such as video editing. In Hollywood, uncut video files run in the hundreds of gigabytes. If multiple video editors are working on the same film, it’s inefficient to retransfer the whole video anytime a minute change is made. Many editors utilize this segmentation technology to transfer and sync only the portions that were altered. Instead of retransferring the whole 100gb video, Hollywood only transfers the 1gb scene that was actually edited. Now that’s efficiency.

 

In our current age of NSA surveillance and CISPA interrogation, privacy and security has become a major concern. BitTorrent is unique among cloud storage applications, because it’s only a protocol, and not a storage system. “There’s no copy of (your files) on our cloud.” “If the government would be to subpoena our records, we wouldn’t have anything to turn over.”

“If the government would be to subpoena our records, we wouldn’t have anything to turn over.”

The lack of centralized storage acts as both a blessing and a curse. On one hand, your files never leave your possession; on the other hand, there’s no backup other than your own. Obviously, there are ways to alleviate this issue, but it’s something to keep in mind. I want to emphasis again that BitTorrent is only a protocol, not a storage service. 

Finally, I want to mention another firm that utilizes a similar technology, or at least one that I found to be of some interest. Bitcasa acts as a traditional cloud storage service, but it’s developing a technology called Infinite Drive: When a duplicate file is uploaded, the protocol searches its database for preexisting files, and earmarks the original without uploading the duplicate. 

If two teenage girls upload the same Justin Bieber album, only one copy is actually stored on Bitcasa's servers. When the album is downloaded, both users access the same file. The technology is interesting, but there are significant areas of concern. If one user alters the album, how would it affect related users who share the same download address. I don’t think this technology is ready for widespread use, but it’s something to look out for in the future.

the word “traditional” to describe cloud storage

It’s funny; it seems like it was only yesterday that Dropbox launched, but this Generation Y technology has already become old news. Earlier in this article, I, inadvertently, used the word “traditional” to describe cloud storage; might as well get out the cane and monocle while at it. The future of file storage will take advantage of Peer-to-Peer transfer protocol. It’s an ingenious method that enables crowd sourcing to distribute large files, and it's capabilities are greatly magnified by BitTorrent’s segmentation technology.

Peer-to-Peer file transfer is secure; but more importantly, it’s efficient.

  • A Farewell To Jobs

    Mar 28, 2015 | 08:00 am

    A Farewell To Jobs  Few subjects elicit more skepticism than the so-called “sharing economy.” Kevin Roose argues: “The Sharing Economy Isn’t About Trust, It’s About Desperation.” Catherine Rampell warns: “there’s a dark side to these work arrangements … the shifting of risk off corporate balance sheets and onto the shoulders of individual Americans.” Read More

  • While You Were Offline: Some One Direction Punk Just Ruined the Whole World

    Mar 28, 2015 | 06:00 am

    While You Were Offline: Some One Direction Punk Just Ruined the Whole World If the Internet taught us anything this week it's that it pays to think about domain registrations in advance. Also, people get really sad about pop stars. The post While You Were Offline: Some One Direction Punk Just Ruined the Whole World appeared first on WIRED.

  • Preview the Military Fantasy Story ‘Rules of Enchantment’

    Mar 28, 2015 | 05:55 am

    Preview the Military Fantasy Story ‘Rules of Enchantment’ Read "Rules of Enchantment" from the military fantasy anthology Operation Arcana here. The post Preview the Military Fantasy Story ‘Rules of Enchantment’ appeared first on WIRED.

  • Two Jurors On Opposite Sides Share Their Pao Vs. Kleiner Perspectives

    Mar 28, 2015 | 00:38 am

    Two Jurors On Opposite Sides Share Their Pao Vs. Kleiner Perspectives  It’s been a very suspenseful day for all of those involved in the Pao vs. KPCB trial (well, maybe not “Sleeping Guy”). We had a verdict, and then suddenly we didn’t, because a juror had changed his mind on the walk up from the jury room to room 608. When the final verdict did come in, again, many were stunned and many, especially those on the KPCB side, were relieved. Read More

  • TC AppleCast 11: If You Had To Choose Just One – Apple Hardware Or Apple Software?

    Mar 27, 2015 | 20:37 pm

    TC AppleCast 11: If You Had To Choose Just One – Apple Hardware Or Apple Software?  On this week’s episode, we discuss both a lengthy profile of Tim Cook, as well as a full biography of Steve Jobs. We also cover some concrete announcements, including the open-sourcing of React, Facebook’s (potentially better than UIKit) UIKit alternative, and the acquisition of database company FoundationDB by Apple, and what that might mean. But wait, that’s not all! We… Read More

  • We Have No Way to Screen Every Pilot for Mental Illness

    Mar 27, 2015 | 19:59 pm

    We Have No Way to Screen Every Pilot for Mental Illness The regular physical exams pilots go through don't seriously address the risk of mental illness. The post We Have No Way to Screen Every Pilot for Mental Illness appeared first on WIRED.

  • RadiumOne is in dire financial circumstances, sources say — company denies it

    Mar 27, 2015 | 19:25 pm

    Late this afternoon, an email from the CEO went to all RadiumOne employees denying the bad news.

  • Ellen Pao loses Kleiner case, but says if she helped women and minorities in VC, then ‘battle was worth it’

    Mar 27, 2015 | 19:08 pm

    There was plenty of support for Pao online after the end

  • Ellen Pao’s Statement On Losing The Kleiner Perkins Case: “The Battle Was Worth It”

    Mar 27, 2015 | 18:55 pm

    Ellen Pao’s Statement On Losing The Kleiner Perkins Case: “The Battle Was Worth It”  Directly after the jury announced its final verdict in Ellen Pao Vs. Kleiner Perkins Caufield, in which it ruled against all four of Pao’s claims of gender discrimination and retaliation, Pao held a very short press conference at San Francisco’s Superior Court. Flanked by her lawyers Alan Exelrod and Therese Lawless, Pao read a short statement. Embedded above is video (amateur… Read More

  • Google Blasts Media Implications That It Runs That Town

    Mar 27, 2015 | 18:45 pm

    Google Blasts Media Implications That It Runs That Town  Google, a search company, fired back today at a string of reports published in the Wall Street Journal over the past few weeks that have brought the company and its business practices under scrutiny. The collection of pieces implied that Google has decision-tilting influence in the federal government, and that the Federal Trade Commission (FTC) all but went against its own internal views of… Read More

  • Follow these 7 steps if you’re talking to a possible acquirer

    Mar 27, 2015 | 18:15 pm

    Guest:A call from another company's CorpDev team can signal they're interested in acquiring you. But it could also lead to costly dead-ends. If you do decide to take the call, here are seven steps I recommend for converting those conversations into potential business.

  • Genomics Needs A Killer App

    Mar 27, 2015 | 18:14 pm

    Genomics Needs A Killer App  Genomics has undergone a major shift in the last year. The drop in price in the past decade from $3B to $1K to sequence a genome is the 10X force likely to cause an inflection point in a number of health-related industries. But despite the price, right now genomics is a niche product in large part used in and promoted by academia. Read More