Peer-to-Peer Technology to Trump Traditional Cloud Storage

On a lovely Thursday afternoon in June, I sat down with Sarah Koo, a data scientist with BitTorrent. I grew up in a time and place where Peer-to-Peer (P2P) transfer was almost commonplace, and I was fortunate enough to have the opportunity to learn more about this unique technology.

 

When Peer-to-Peer transfer is mentioned, the subject of copyright protection is the first thing to come to mind. Though P2P transfer is used, at times, to download Game of Thrones, the technology itself is an innovative protocol designed to efficiently transfer large files. I want to make an explicit statement that neither I, Sarah, nor BitTorrent, condones any forms of illegal file transfers. I am advocating the technology itself, which is commonly used in both academia and business. 

 

Bram Cohen created the Peer-to-Peer transfer protocol about a decade ago. This genius literally coded the program by himself in his bedroom, like many of our city’s tech entrepreneurs. Traditional file transfer protocols connected downloaders to a single server for direct file transfers. If there were 100 users downloading the same file, all these users would connect to the same server. Such traditional methods resulted in significant speed reductions and heavy server strain as demand for the file increased. Cohen's application created a distinct virtual network per file that allowed any invited users to help with distribution.

a protocol that created a virtual network based on the file itself, thus anyone within the network can help distribute the file

Lets say UC Berkeley is working on a massive database, and it needs to distribute it to 10,000 researchers across the country. If each of these 10,000 researchers downloads the file directly from the Cal server, the strain on the single server is intense and extremely inefficient. P2P transfer creates a network that links all invited researchers together, and once any researcher finishes downloading the database, he can act as a file distributor for other researchers within the network. Thus, the strain of distributing the database is crowd sourced to all its participants. 

BitTorrent, the most popular P2P transfer application, breaks down large files into small segments for even more efficient distribution. If the database described above is 20-gigabytes; BitTorrent could break down the file into 20 equal sized 1-gigabyte files. “If I request the file from peers, I can be getting it from 20 different people at the same time.” Thus, each peer only needs to distribute 1/20th of the overall load.

breaks down large files into small segments for more efficient distribution

Because BitTorrent breaks down files into many individual segments, peers can act as distributors the moment they finish downloading any piece of the overall file. Once all the segments are downloaded, BitTorrent reassembles the original file as whole. The ability to deconstruct files and then reassemble the segments after the transfer protocol is complete is groundbreaking, especially in the field of cloud storage.

BitTorrent has utilized this technology to build sync applications that are commonly used in industries such as video editing. In Hollywood, uncut video files run in the hundreds of gigabytes. If multiple video editors are working on the same film, it’s inefficient to retransfer the whole video anytime a minute change is made. Many editors utilize this segmentation technology to transfer and sync only the portions that were altered. Instead of retransferring the whole 100gb video, Hollywood only transfers the 1gb scene that was actually edited. Now that’s efficiency.

 

In our current age of NSA surveillance and CISPA interrogation, privacy and security has become a major concern. BitTorrent is unique among cloud storage applications, because it’s only a protocol, and not a storage system. “There’s no copy of (your files) on our cloud.” “If the government would be to subpoena our records, we wouldn’t have anything to turn over.”

“If the government would be to subpoena our records, we wouldn’t have anything to turn over.”

The lack of centralized storage acts as both a blessing and a curse. On one hand, your files never leave your possession; on the other hand, there’s no backup other than your own. Obviously, there are ways to alleviate this issue, but it’s something to keep in mind. I want to emphasis again that BitTorrent is only a protocol, not a storage service. 

Finally, I want to mention another firm that utilizes a similar technology, or at least one that I found to be of some interest. Bitcasa acts as a traditional cloud storage service, but it’s developing a technology called Infinite Drive: When a duplicate file is uploaded, the protocol searches its database for preexisting files, and earmarks the original without uploading the duplicate. 

If two teenage girls upload the same Justin Bieber album, only one copy is actually stored on Bitcasa's servers. When the album is downloaded, both users access the same file. The technology is interesting, but there are significant areas of concern. If one user alters the album, how would it affect related users who share the same download address. I don’t think this technology is ready for widespread use, but it’s something to look out for in the future.

the word “traditional” to describe cloud storage

It’s funny; it seems like it was only yesterday that Dropbox launched, but this Generation Y technology has already become old news. Earlier in this article, I, inadvertently, used the word “traditional” to describe cloud storage; might as well get out the cane and monocle while at it. The future of file storage will take advantage of Peer-to-Peer transfer protocol. It’s an ingenious method that enables crowd sourcing to distribute large files, and it's capabilities are greatly magnified by BitTorrent’s segmentation technology.

Peer-to-Peer file transfer is secure; but more importantly, it’s efficient.

  • Why Online Retailers Continue To Open Brick-And-Mortar Stores

    Oct 31, 2014 | 12:00 pm

    Why Online Retailers Continue To Open Brick-And-Mortar Stores  In an age when new technology and the growth of pure online-only retailers have industry analysts questioning the future of brick-and-mortar stores, what are online retailers doing to grow their businesses and gain market share? Why, opening up physical storefronts of course. The benefits that physical spaces provide make up three of the top reasons why online retailers are setting up shop… Read More

  • App Annie Fills The Void Left By Facebook’s Onavo Acquisition With Its New Company Smart Sense

    Oct 31, 2014 | 11:46 am

    App Annie Fills The Void Left By Facebook’s Onavo Acquisition With Its New Company Smart Sense  When Facebook scooped up mobile data analytics powerhouse Onavo last year, it was only a matter of time before someone rebuilt its service. And that seems to be just what the now-leading mobile app analytics firm App Annie has done. The company, which used some portion of its $39 million in funding to snap up its competitor Distimo earlier this year, has quietly released a new app called… Read More

  • On the downswing, chat room giant Paltalk plans pivot inspired by YouTube & Twitch

    Oct 31, 2014 | 11:36 am

    On the downswing, chat room giant Paltalk plans pivot inspired by YouTube & Twitch Paltalk is a strange company. All startups risk it all to realize their vision, but 15-year-old Paltalk will live or die for something you probably haven’t thought about in a while: chat rooms. Paltalk launched during AOL’s prime. In 1998, the company released a group chat tool for the desktop and grew it for the […]

  • Why Facebook Just Launched Its Own ‘Dark Web’ Site

    Oct 31, 2014 | 11:31 am

    Why Facebook Just Launched Its Own ‘Dark Web’ Site Now the world's least anonymous website (Facebook) has just joined the Web's most anonymous network (Tor). The post Why Facebook Just Launched Its Own ‘Dark Web’ Site appeared first on WIRED.

  • 3D-printed T. rex roars water onto your body

    Oct 31, 2014 | 11:28 am

    Transport your bathroom back to the dinosaur age with a shower head modeled after a toothy Tyrannosaurus rex skull.

  • “VideoSelfie” Scores $1.2 Million For Its App Offering Real-Time Filters, GIFs And More

    Oct 31, 2014 | 11:18 am

    “VideoSelfie” Scores $1.2 Million For Its App Offering Real-Time Filters, GIFs And More  But I thought we were recording our video selfies on Instagram and Snapchat now? A soon-to-launch mobile application called VideoSelfie, whose founders participated in the 500 Startups accelerator last year, has raised $1.2 million for its forthcoming mobile application that lets you edit your videos (of yourself, duh) in real-time. You can add filters, decorations and GIFs, hit record,… Read More

  • Whovians: Bake a Pudding Brain Cake in time for Halloween

    Oct 31, 2014 | 10:55 am

    All you pudding brains who have been keeping up with the latest season of "Doctor Who" can make your very own pudding brain for All Hallows' Eve.

  • How Google tricks itself to protect Chrome user privacy

    Oct 31, 2014 | 10:42 am

    An open-source project called Rappor uses randomly muddled data to let Google gather information about people's software usage while keeping individuals' behavior private.

  • Sony posts $1.2 billion loss as mobile business falls off a cliff

    Oct 31, 2014 | 10:35 am

    Dragging down the rest of the company Sony's mobile communications unit posts a quarterly operating loss of $1.6 billion on $2.8 billion in revenue.

  • Android stays 'unbeatable' in smartphone market -- for now

    Oct 31, 2014 | 10:32 am

    Google's mobile OS reigned supreme in the third quarter with 84 percent of global shipments, says research firm Strategy Analytics. And then there are the warning signs.

  • Android Founder Departs Google, Adding to Executive Shakeup

    Oct 31, 2014 | 10:26 am

    Android Founder Departs Google, Adding to Executive Shakeup Google's head of robotics and former head of Android, Andy Rubin, is stepping down from the company to launch a new incubator for hardware startups. The post Android Founder Departs Google, Adding to Executive Shakeup appeared first on WIRED.

  • Funding Daily: Today’s biggest tech funding news, in one place

    Oct 31, 2014 | 10:19 am

    Funding Daily: Today’s biggest tech funding news, in one place Get all the tech funding news of the day delivered straight to your mailbox! Sign up for Funding Daily and never miss a deal. Here’s a list of today’s tech funding stories, updated as the day unfolds. Tip us here if you have a deal to share. Slack raises $120M from Google & Kleiner Perkins at $1.12B valuation […]