Peer-to-Peer Technology to Trump Traditional Cloud Storage

On a lovely Thursday afternoon in June, I sat down with Sarah Koo, a data scientist with BitTorrent. I grew up in a time and place where Peer-to-Peer (P2P) transfer was almost commonplace, and I was fortunate enough to have the opportunity to learn more about this unique technology.

 

When Peer-to-Peer transfer is mentioned, the subject of copyright protection is the first thing to come to mind. Though P2P transfer is used, at times, to download Game of Thrones, the technology itself is an innovative protocol designed to efficiently transfer large files. I want to make an explicit statement that neither I, Sarah, nor BitTorrent, condones any forms of illegal file transfers. I am advocating the technology itself, which is commonly used in both academia and business. 

 

Bram Cohen created the Peer-to-Peer transfer protocol about a decade ago. This genius literally coded the program by himself in his bedroom, like many of our city’s tech entrepreneurs. Traditional file transfer protocols connected downloaders to a single server for direct file transfers. If there were 100 users downloading the same file, all these users would connect to the same server. Such traditional methods resulted in significant speed reductions and heavy server strain as demand for the file increased. Cohen's application created a distinct virtual network per file that allowed any invited users to help with distribution.

a protocol that created a virtual network based on the file itself, thus anyone within the network can help distribute the file

Lets say UC Berkeley is working on a massive database, and it needs to distribute it to 10,000 researchers across the country. If each of these 10,000 researchers downloads the file directly from the Cal server, the strain on the single server is intense and extremely inefficient. P2P transfer creates a network that links all invited researchers together, and once any researcher finishes downloading the database, he can act as a file distributor for other researchers within the network. Thus, the strain of distributing the database is crowd sourced to all its participants. 

BitTorrent, the most popular P2P transfer application, breaks down large files into small segments for even more efficient distribution. If the database described above is 20-gigabytes; BitTorrent could break down the file into 20 equal sized 1-gigabyte files. “If I request the file from peers, I can be getting it from 20 different people at the same time.” Thus, each peer only needs to distribute 1/20th of the overall load.

breaks down large files into small segments for more efficient distribution

Because BitTorrent breaks down files into many individual segments, peers can act as distributors the moment they finish downloading any piece of the overall file. Once all the segments are downloaded, BitTorrent reassembles the original file as whole. The ability to deconstruct files and then reassemble the segments after the transfer protocol is complete is groundbreaking, especially in the field of cloud storage.

BitTorrent has utilized this technology to build sync applications that are commonly used in industries such as video editing. In Hollywood, uncut video files run in the hundreds of gigabytes. If multiple video editors are working on the same film, it’s inefficient to retransfer the whole video anytime a minute change is made. Many editors utilize this segmentation technology to transfer and sync only the portions that were altered. Instead of retransferring the whole 100gb video, Hollywood only transfers the 1gb scene that was actually edited. Now that’s efficiency.

 

In our current age of NSA surveillance and CISPA interrogation, privacy and security has become a major concern. BitTorrent is unique among cloud storage applications, because it’s only a protocol, and not a storage system. “There’s no copy of (your files) on our cloud.” “If the government would be to subpoena our records, we wouldn’t have anything to turn over.”

“If the government would be to subpoena our records, we wouldn’t have anything to turn over.”

The lack of centralized storage acts as both a blessing and a curse. On one hand, your files never leave your possession; on the other hand, there’s no backup other than your own. Obviously, there are ways to alleviate this issue, but it’s something to keep in mind. I want to emphasis again that BitTorrent is only a protocol, not a storage service. 

Finally, I want to mention another firm that utilizes a similar technology, or at least one that I found to be of some interest. Bitcasa acts as a traditional cloud storage service, but it’s developing a technology called Infinite Drive: When a duplicate file is uploaded, the protocol searches its database for preexisting files, and earmarks the original without uploading the duplicate. 

If two teenage girls upload the same Justin Bieber album, only one copy is actually stored on Bitcasa's servers. When the album is downloaded, both users access the same file. The technology is interesting, but there are significant areas of concern. If one user alters the album, how would it affect related users who share the same download address. I don’t think this technology is ready for widespread use, but it’s something to look out for in the future.

the word “traditional” to describe cloud storage

It’s funny; it seems like it was only yesterday that Dropbox launched, but this Generation Y technology has already become old news. Earlier in this article, I, inadvertently, used the word “traditional” to describe cloud storage; might as well get out the cane and monocle while at it. The future of file storage will take advantage of Peer-to-Peer transfer protocol. It’s an ingenious method that enables crowd sourcing to distribute large files, and it's capabilities are greatly magnified by BitTorrent’s segmentation technology.

Peer-to-Peer file transfer is secure; but more importantly, it’s efficient.

  • Disrupt 旧金山峰会邀请全球华人参会 (Announcing a Disrupt SF attendee-friendly for Chinese speakers)

    Jun 28, 2016 | 01:03 am

    Disrupt 旧金山峰会邀请全球华人参会 (Announcing a Disrupt SF attendee-friendly for Chinese speakers)  It’s no news that China’s startup scene is huge and Chinese investors and founders are keen to learn more about the Silicon Valley. And of course American investors and entrepreneurs have been working hard in China for many years. It seemed like a natural to make TechCrunch’s Disrupt SF (Sept. 12-14) more accessible to visitors from China. So we’re announcing a few… Read More

  • Attention Asian startups: Apply for the Startup Battlefield at Disrupt SF!

    Jun 28, 2016 | 00:30 am

    Attention Asian startups: Apply for the Startup Battlefield at Disrupt SF!  Calling startups in Asia and across the world — apply for our Startup Battlefield competition at TechCrunch Disrupt San Francisco this September! Startups are at the core of what we do here at TechCrunch. Beyond writing about them every day on TechCrunch, they are a hugely important focus of our events whether it’s TechCrunch Shanghai, where our team is right now or Disrupt SF… Read More

  • YCombinator makes every SimCity player’s dream reality with new research initiative

    Jun 27, 2016 | 22:20 pm

    YCombinator makes every SimCity player’s dream reality with new research initiative  Ever wanted to design a city from scratch? All those years tinkering with SimCity and Legos are going to finally start paying dividends for a lucky few. YCombinator is looking for urban mavericks to address everything from the creation of affordable housing to the design of public spaces in its new research undertaking. The buzz-word smart-city is rapidly becoming platitudinous but YC will… Read More

  • Samsung's Gear 360 camera now available in Australia

    Jun 27, 2016 | 21:54 pm

    Samsung's Gear 360 camera now available in Australia It looks like an eyeball on three legs, and it is now yours to own.On Tuesday, Samsung launched its new camera, Samsung Gear 360, in Australia. The device, which is able to capture 360-degree video and images, has a retail price of A$499.SEE ALSO: Samsung Galaxy Note 7 may unlock with a glance, report saysWhile the camera has been available in South Korea and Singapore since April, it still has only a limited release in the U.S., making this a rare opportunity for Australians to get their hands on something their American friends cannot.The slick Samsung Gear 360.Image:SamsungThe camera uses two fisheye lenses, both with 15-megapixel image sensors, to capture 360-degree video that can then be experienced using the Samsung Gear virtual reality headset. Footage can be previewed in real-time on a number of Samsung smartphones, including the Galaxy S7 and Galaxy S7 Edge, Galaxy S6 Edge+, Galaxy Note 5, Galaxy S6 Edge and Galaxy S6. Read more...More about Samsung Gear 360, Samsung, Australia, Tech, and Gadgets

  • Google's Project Bloks designed to get kids programming early

    Jun 27, 2016 | 21:08 pm

    Google's Project Bloks designed to get kids programming early Increasing its dedication to early learning through technology, Google has announced Project Bloks, a tool designed to get children started on the path to programming and making cool gadgets.SEE ALSO: Google teams up with MIT to make programming easier for kidsProject Bloks consist of three components: Base Boards, Pucks and the Brain Board. Pucks can be used to initialize any number of functions (on, off, turn, go, etc.) and work by connecting to square Base Boards via capacitive sensors. Those two components are given power and connectivity (Wi-Fi and Bluetooth) by hooking up to the larger Brain Board Read more...More about Stanford, Google, Coding, Programming, and Education

  • How the ancient art of origami is inspiring cutting-edge technology

    Jun 27, 2016 | 21:00 pm

    How the ancient art of origami is inspiring cutting-edge technology  The future is always being shaped by the past. Long before the 3D printer, origami was the original genius at creating lifelike forms out of a flat surface. Folding brings with it the ability to collapse, flex and unfurl structures at will, which has huge potential for a variety of engineering applications. From digestible origami pills that could provide alternatives to invasive surgery to… Read More

  • Airbnb sues San Francisco over new rental legislation

    Jun 27, 2016 | 20:51 pm

    Airbnb sues San Francisco over new rental legislation  The short-term rental company filed suit today over a new law that requires Airbnb to verify that its hosts have registered with the city before showing ads for their homes online. The suit aims to block the law from going into effect as scheduled on August 1.San Francisco legislators passed the law earlier this month in an effort to combat the housing crisis in the city, but Airbnb and… Read More

  • Writing aid for the blind provides a case study for “compassionate engineering” at Carnegie Mellon

    Jun 27, 2016 | 20:23 pm

    Writing aid for the blind provides a case study for “compassionate engineering” at Carnegie Mellon  New mobile games and robot butlers are all well and good, but there are also many applications for the latest technology in poverty-stricken school districts and in the service of the disabled. A Carnegie Mellon project that targets both of those things is described by its leaders as an exercise in what they call “compassionate engineering.” Read More

  • Sweden creates world's first 'electric road'

    Jun 27, 2016 | 20:08 pm

    Sweden creates world's first 'electric road' Sometimes vehicles will be too big and heavy to make them ideal candidates for full electrification. That doesn't mean, though, they can't be powered solely by electricity.Proving that point, Sweden — in conjunction with truck manufacturer Scania — has created the world's first "electric road." It's a two-kilometer stretch of freeway in central Sweden with power lines stretched over the righthand lane.SEE ALSO: Tesla signs $9 billion agreement to build factory in China, report saysHybrid trucks specially fitted with a device called a 'pantograph power collector' can drive underneath the lines and tap into the power. When connected to the lines, they run completely on electricity. When they drive out of the right lane, the onboard engine kicks on, as it reverts back to hybrid mode. Read more...More about Transportation, Cars, and Tech

  • Thrive Market raises $111 million for its online organic grocery store

    Jun 27, 2016 | 19:46 pm

    Thrive Market raises $111 million for its online organic grocery store  Buy organic groceries online, at an affordable price. That’s the mantra of Los Angeles-based Thrive Market, which has raised a significant $111 million Series B funding round, led by Invus. Thrive, which competes with similar online grocery services like Vitacost, believes there is a significant market opportunity to make natural foods more accessible to consumers and they’ve… Read More

  • So, does anybody want Twitter?

    Jun 27, 2016 | 19:44 pm

    So, does anybody want Twitter?  Twitter is in trouble. In a move out of a middle school flag football game, Microsoft selected LinkedIn to join its team. The sheer size of the transaction has reignited conversations about who will be picked next to join the super special circle of public mega-cap tech companies. Vultures have been circling Twitter for well over a year, but the company has defied all experts who have… Read More

  • Pocket Tripod is a credit-card-sized phone holder that fits in your wallet

    Jun 27, 2016 | 19:06 pm

    Pocket Tripod is a credit-card-sized phone holder that fits in your wallet  The Pocket Tripod is repeating its crowdfunding success from a few years ago, creating an updated, universal version of its incredibly popular credit-card-sized tripod that fits in your wallet. Now available for pre-order for Android users and people who like to stick cases on their phones, it’s a tremendously useful piece of kit for smartphone photographers on the move. Read More

Loading ...