Peer-to-Peer Technology to Trump Traditional Cloud Storage

On a lovely Thursday afternoon in June, I sat down with Sarah Koo, a data scientist with BitTorrent. I grew up in a time and place where Peer-to-Peer (P2P) transfer was almost commonplace, and I was fortunate enough to have the opportunity to learn more about this unique technology.

 

When Peer-to-Peer transfer is mentioned, the subject of copyright protection is the first thing to come to mind. Though P2P transfer is used, at times, to download Game of Thrones, the technology itself is an innovative protocol designed to efficiently transfer large files. I want to make an explicit statement that neither I, Sarah, nor BitTorrent, condones any forms of illegal file transfers. I am advocating the technology itself, which is commonly used in both academia and business. 

 

Bram Cohen created the Peer-to-Peer transfer protocol about a decade ago. This genius literally coded the program by himself in his bedroom, like many of our city’s tech entrepreneurs. Traditional file transfer protocols connected downloaders to a single server for direct file transfers. If there were 100 users downloading the same file, all these users would connect to the same server. Such traditional methods resulted in significant speed reductions and heavy server strain as demand for the file increased. Cohen's application created a distinct virtual network per file that allowed any invited users to help with distribution.

a protocol that created a virtual network based on the file itself, thus anyone within the network can help distribute the file

Lets say UC Berkeley is working on a massive database, and it needs to distribute it to 10,000 researchers across the country. If each of these 10,000 researchers downloads the file directly from the Cal server, the strain on the single server is intense and extremely inefficient. P2P transfer creates a network that links all invited researchers together, and once any researcher finishes downloading the database, he can act as a file distributor for other researchers within the network. Thus, the strain of distributing the database is crowd sourced to all its participants. 

BitTorrent, the most popular P2P transfer application, breaks down large files into small segments for even more efficient distribution. If the database described above is 20-gigabytes; BitTorrent could break down the file into 20 equal sized 1-gigabyte files. “If I request the file from peers, I can be getting it from 20 different people at the same time.” Thus, each peer only needs to distribute 1/20th of the overall load.

breaks down large files into small segments for more efficient distribution

Because BitTorrent breaks down files into many individual segments, peers can act as distributors the moment they finish downloading any piece of the overall file. Once all the segments are downloaded, BitTorrent reassembles the original file as whole. The ability to deconstruct files and then reassemble the segments after the transfer protocol is complete is groundbreaking, especially in the field of cloud storage.

BitTorrent has utilized this technology to build sync applications that are commonly used in industries such as video editing. In Hollywood, uncut video files run in the hundreds of gigabytes. If multiple video editors are working on the same film, it’s inefficient to retransfer the whole video anytime a minute change is made. Many editors utilize this segmentation technology to transfer and sync only the portions that were altered. Instead of retransferring the whole 100gb video, Hollywood only transfers the 1gb scene that was actually edited. Now that’s efficiency.

 

In our current age of NSA surveillance and CISPA interrogation, privacy and security has become a major concern. BitTorrent is unique among cloud storage applications, because it’s only a protocol, and not a storage system. “There’s no copy of (your files) on our cloud.” “If the government would be to subpoena our records, we wouldn’t have anything to turn over.”

“If the government would be to subpoena our records, we wouldn’t have anything to turn over.”

The lack of centralized storage acts as both a blessing and a curse. On one hand, your files never leave your possession; on the other hand, there’s no backup other than your own. Obviously, there are ways to alleviate this issue, but it’s something to keep in mind. I want to emphasis again that BitTorrent is only a protocol, not a storage service. 

Finally, I want to mention another firm that utilizes a similar technology, or at least one that I found to be of some interest. Bitcasa acts as a traditional cloud storage service, but it’s developing a technology called Infinite Drive: When a duplicate file is uploaded, the protocol searches its database for preexisting files, and earmarks the original without uploading the duplicate. 

If two teenage girls upload the same Justin Bieber album, only one copy is actually stored on Bitcasa's servers. When the album is downloaded, both users access the same file. The technology is interesting, but there are significant areas of concern. If one user alters the album, how would it affect related users who share the same download address. I don’t think this technology is ready for widespread use, but it’s something to look out for in the future.

the word “traditional” to describe cloud storage

It’s funny; it seems like it was only yesterday that Dropbox launched, but this Generation Y technology has already become old news. Earlier in this article, I, inadvertently, used the word “traditional” to describe cloud storage; might as well get out the cane and monocle while at it. The future of file storage will take advantage of Peer-to-Peer transfer protocol. It’s an ingenious method that enables crowd sourcing to distribute large files, and it's capabilities are greatly magnified by BitTorrent’s segmentation technology.

Peer-to-Peer file transfer is secure; but more importantly, it’s efficient.

  • Glass a failure? Luxottica and Google may turn ‘Glassholes’ into fashion plates

    Apr 25, 2015 | 16:47 pm

    Analysis: Successful CMOs achieve growth by leveraging technology. Join us for GrowthBeat Summit on June 1-2 in Boston, where we'll discuss how to merge creativity with technology to drive growth. Space is limited. Request your personal invitation here! The eyewear giant Luxottica said this week it’s working on a new embodiment of Google’s Glass wearable computer, and the result of that development will […]

  • Tesla’s Site And Twitter Account Hacked

    Apr 25, 2015 | 16:42 pm

    Tesla’s Site And Twitter Account Hacked  Looks like it’s going to be a rough Saturday for Tesla’s IT department: they’ve just had both their website and Twitter account hijacked. Read More

  • Game Of Angels Maps Tech Influence Networks

    Apr 25, 2015 | 16:00 pm

    Game Of Angels Maps Tech Influence Networks  Ever wondered who influences the influencers? Developer Pierre-Jean Camillieri did, so he put together a tool that scrapes public Twitter and Angel List interactions to map inbound and outbound connections between technology investors, journalists and analysts to foreground influence networks. Read More

  • A change in Facebook’s top paid execs speaks volumes about the company’s priorities

    Apr 25, 2015 | 15:00 pm

    Chris Cox, Facebook's Chief Product Officer, made his debut appearance on the company's proxy statement today, with a total 2014 compensation package worth $12.47 million.

  • Slideshow: The Cinder Sensing Cooker Won’t Burn Your Food To A Crisp

    Apr 25, 2015 | 14:15 pm

    Slideshow: The Cinder Sensing Cooker Won’t Burn Your Food To A Crisp  I’m a lucky lad. Every so often the creators of a really cool, really revolutionary cooking devices come to my house and make me food and this time it was Cinder. Think of Cinder as a George Foreman Grill that went to MIT. Designed to cook everything perfectly, the system can cook, sear, and grill automatically, allowing you to place food in the machine and have it cooked to perfection… Read More

  • Apple Watch teardown reveals deactivated blood oxygen sensors

    Apr 25, 2015 | 14:00 pm

    A teardown of the Apple Watch by iFixit reveals that the device indeed does contain the sensors needed to measure blood oxygen levels, not just heart rate.

  • The live-stream video tweet battle: Meerkat’s still fighting, but Periscope is winning

    Apr 25, 2015 | 13:00 pm

    Guest: Periscope easily drives more adoption than Meerkat if you know how to market your streams, and it seems to have won out on most of the reviews of the two apps in the tech press. But here are some stats that tell more of the story.

  • Gillmor Gang: Time Machine

    Apr 25, 2015 | 12:16 pm

    Gillmor Gang: Time Machine  The Gillmor Gang — Danny Sullivan, Robert Scoble, Dan Farber, John Taschek, and Steve Gillmor. Recorded live Friday, April 24, 2015. We got nothing but time on our hands waiting for the one on our wrists. Plus, the latest G3 (below) with Halley Suitt Tucker, Denise Howell, Mary Hodder, Rebecca Woodcock, and Tina Gillmor. @stevegillmor, @scobleizer, @dannysullivan, @jtaschek, @dbfarber Read More

  • Skype for iPhone gets Apple Watch support, but no voice or video calling just yet

    Apr 25, 2015 | 11:47 am

    Skype comes to Apple Watch.

  • Is Seattle Silicon Valley’s Next Favorite Stop?

    Apr 25, 2015 | 11:30 am

    Is Seattle Silicon Valley’s Next Favorite Stop?  In the last ten years, the Seattle tech scene has changed quite dramatically. In the 1990s and early 2000s, the only gigantic software companies in town were Microsoft and Amazon. For a time it even became common knowledge that most Seattle-based startups had only two viable exit strategies: go public, or get acquired by Microsoft. Read More

  • Plug-In hybrid HUMMERs headed for the South Pole on biodiesel

    Apr 25, 2015 | 11:00 am

    The mission is called Zero South, signifying the group's goal of being the first to reach the South Pole without using any fossil fuels.

  • The Apple Watch Can Be A Sixth Sense

    Apr 25, 2015 | 10:00 am

    The Apple Watch Can Be A Sixth Sense  The world around us is screaming with signals our original five senses don’t detect. Unlike some snakes, we can’t sense infrared with our naked eyes and unlike ducks and geese, we don’t have a native intuition for magnetic fields. But we’ve got pretty incredible brains, which we’ve used to craft ways to augment our five basic senses for a very long time. Read More