Peer-to-Peer Technology to Trump Traditional Cloud Storage

On a lovely Thursday afternoon in June, I sat down with Sarah Koo, a data scientist with BitTorrent. I grew up in a time and place where Peer-to-Peer (P2P) transfer was almost commonplace, and I was fortunate enough to have the opportunity to learn more about this unique technology.


When Peer-to-Peer transfer is mentioned, the subject of copyright protection is the first thing to come to mind. Though P2P transfer is used, at times, to download Game of Thrones, the technology itself is an innovative protocol designed to efficiently transfer large files. I want to make an explicit statement that neither I, Sarah, nor BitTorrent, condones any forms of illegal file transfers. I am advocating the technology itself, which is commonly used in both academia and business. 


Bram Cohen created the Peer-to-Peer transfer protocol about a decade ago. This genius literally coded the program by himself in his bedroom, like many of our city’s tech entrepreneurs. Traditional file transfer protocols connected downloaders to a single server for direct file transfers. If there were 100 users downloading the same file, all these users would connect to the same server. Such traditional methods resulted in significant speed reductions and heavy server strain as demand for the file increased. Cohen's application created a distinct virtual network per file that allowed any invited users to help with distribution.

a protocol that created a virtual network based on the file itself, thus anyone within the network can help distribute the file

Lets say UC Berkeley is working on a massive database, and it needs to distribute it to 10,000 researchers across the country. If each of these 10,000 researchers downloads the file directly from the Cal server, the strain on the single server is intense and extremely inefficient. P2P transfer creates a network that links all invited researchers together, and once any researcher finishes downloading the database, he can act as a file distributor for other researchers within the network. Thus, the strain of distributing the database is crowd sourced to all its participants. 

BitTorrent, the most popular P2P transfer application, breaks down large files into small segments for even more efficient distribution. If the database described above is 20-gigabytes; BitTorrent could break down the file into 20 equal sized 1-gigabyte files. “If I request the file from peers, I can be getting it from 20 different people at the same time.” Thus, each peer only needs to distribute 1/20th of the overall load.

breaks down large files into small segments for more efficient distribution

Because BitTorrent breaks down files into many individual segments, peers can act as distributors the moment they finish downloading any piece of the overall file. Once all the segments are downloaded, BitTorrent reassembles the original file as whole. The ability to deconstruct files and then reassemble the segments after the transfer protocol is complete is groundbreaking, especially in the field of cloud storage.

BitTorrent has utilized this technology to build sync applications that are commonly used in industries such as video editing. In Hollywood, uncut video files run in the hundreds of gigabytes. If multiple video editors are working on the same film, it’s inefficient to retransfer the whole video anytime a minute change is made. Many editors utilize this segmentation technology to transfer and sync only the portions that were altered. Instead of retransferring the whole 100gb video, Hollywood only transfers the 1gb scene that was actually edited. Now that’s efficiency.


In our current age of NSA surveillance and CISPA interrogation, privacy and security has become a major concern. BitTorrent is unique among cloud storage applications, because it’s only a protocol, and not a storage system. “There’s no copy of (your files) on our cloud.” “If the government would be to subpoena our records, we wouldn’t have anything to turn over.”

“If the government would be to subpoena our records, we wouldn’t have anything to turn over.”

The lack of centralized storage acts as both a blessing and a curse. On one hand, your files never leave your possession; on the other hand, there’s no backup other than your own. Obviously, there are ways to alleviate this issue, but it’s something to keep in mind. I want to emphasis again that BitTorrent is only a protocol, not a storage service. 

Finally, I want to mention another firm that utilizes a similar technology, or at least one that I found to be of some interest. Bitcasa acts as a traditional cloud storage service, but it’s developing a technology called Infinite Drive: When a duplicate file is uploaded, the protocol searches its database for preexisting files, and earmarks the original without uploading the duplicate. 

If two teenage girls upload the same Justin Bieber album, only one copy is actually stored on Bitcasa's servers. When the album is downloaded, both users access the same file. The technology is interesting, but there are significant areas of concern. If one user alters the album, how would it affect related users who share the same download address. I don’t think this technology is ready for widespread use, but it’s something to look out for in the future.

the word “traditional” to describe cloud storage

It’s funny; it seems like it was only yesterday that Dropbox launched, but this Generation Y technology has already become old news. Earlier in this article, I, inadvertently, used the word “traditional” to describe cloud storage; might as well get out the cane and monocle while at it. The future of file storage will take advantage of Peer-to-Peer transfer protocol. It’s an ingenious method that enables crowd sourcing to distribute large files, and it's capabilities are greatly magnified by BitTorrent’s segmentation technology.

Peer-to-Peer file transfer is secure; but more importantly, it’s efficient.

  • Apple TV app curates Facebook videos just for you

    May 25, 2016 | 18:30 pm

    Apple TV app curates Facebook videos just for you A lot of videos are published on Facebook. But finding videos to watch is a lot more difficult.Which is why it's notable that the best solution I’ve seen for finding great videos on Facebook comes by way of Rheo, a new app for Apple TV.SEE ALSO: Facebook just made it easier to live stream your kittens indefinitelyRheo is the brainchild of Copper Studios, a startup founded by Alan Cannistraro. Before going out on his own, Cannistraro spent 12 years at Apple, where he built the very first iPhone app and was involved with the original Apple TV at its earliest stages. He joined Facebook in 2012 and led the team that built its AutoPlay video feature. Read more...More about Facebook, Rheo, Apple Tv Apps, Apple Tv, and Tech

  • Nuheara’s IQbuds are somewhere between wireless headphones and a hearing aid

    May 25, 2016 | 18:02 pm

    Nuheara’s IQbuds are somewhere between wireless headphones and a hearing aid  I went into the Nuheara demo not knowing what to expect. Not in the sense of, “How good will these be?” so much as, “I’m not exactly sure what these things do.” Like, are they headphones? Are they hearing aids? The answer, interestingly enough, is a little bit of both. The IQbuds are, ostensibly, Bluetooth earbuds. That they’re completely wireless is enough… Read More

  • Terrapattern is a neural net powered reverse image search for maps

    May 25, 2016 | 17:55 pm

    Terrapattern is a neural net powered reverse image search for maps  Terrapattern is a visual search engine that, from the moment you first use it, you wonder: why didn’t Google come up with this ten years ago? Click on a feature on the map — a baseball diamond, a marina, a roundabout — and it immediately highlights everything its algorithm thinks looks like it. It’s remarkably fast, simple to use, and potentially very powerful. Read More

  • A lending case involving top VCs moves toward a trial

    May 25, 2016 | 17:29 pm

    A lending case involving top VCs moves toward a trial  Elevate, a venture-backed company that uses big data to assess loan applications from people with low credit scores, has been called out as a predatory lender, including in Fortune last year. One reason among others is that the APR on some of its loans is a stunning 349 percent. Yet the company’s predecessor, Think Finance, which was founded in 2001 and  quietly spun out Elevate into… Read More

  • Facebook will shut down FBX, its desktop ad exchange

    May 25, 2016 | 17:03 pm

    Facebook will shut down FBX, its desktop ad exchange  Facebook plans to shut down FBX, the ad exchange that allows advertisers to buy retargeted desktop ads using third-party tools like Criteo and AppNexus. The news was first reported in The Wall Street Journal and we’ve confirmed it with Facebook. In an emailed statement, Vice President of Monetization Product Marketing Matt Idema suggested that this is part of Facebook’s shift to… Read More

  • Apple is looking into electric vehicle charging, report says

    May 25, 2016 | 16:17 pm

    Apple is looking into electric vehicle charging, report says How long do you think can Apple deny its car project, while evidence of its existence continues to mount?SEE ALSO: Google establishes self-driving tech development center in MichiganApple is reportedly in discussions with electric vehicle charging infrastructure companies, investigating the ins and outs of EV charging and hiring engineers with expertise in the field, according to Reuters.Citing anonymous sources close to the matter, Reuters reports Apple is specifically inquiring about EV charing companies' underlying technology. This, as one source pointed out, indicates Apple is focused on a car. Read more...More about Evs, Transportation, Apple, Cars, and Tech

  • Preparing for infrastructures of the future

    May 25, 2016 | 16:00 pm

    Preparing for infrastructures of the future  There are many abstract notions about what the future of infrastructure will look like, but the truth is that the future is staring companies in the face. The adoption of hybrid cloud models, containers and microservices architectures are only the beginning stages of preparing for infrastructures of the future. Read More

  • Oculus Rift goes on sale on Best Buy site, immediately sells out

    May 25, 2016 | 15:38 pm

    Oculus Rift goes on sale on Best Buy site, immediately sells out On Wednesday, electronics store Best Buy quietly launched the online purchase page for the Oculus Rift virtual reality headset, just a few weeks after the retailer made the device available in limited quantities at its brick and mortar locations.SEE ALSO: Oculus VR faces a consumer revolt as the Rift heads to retailThe device is listed at the normal price of $599.99 and offers free shipping, however, anxious VR fans obviously acted quickly, because the page already lists the device as "sold out." And while the scarcity of the device might lead some to believe that the Oculus is already a breakaway hit, the company has admitted that the inability to meet demand is due to a component shortage.Read more...More about Best Buy, Virtual Reality, Vr, Oculus, and Oculus Rift

  • Heavy Internet use leads to school burnout in teens

    May 25, 2016 | 15:30 pm

    Heavy Internet use leads to school burnout in teens  As if you didn’t need more proof that your darn kids need to get off the computer and into the back yard, researchers at the University of Helsinki and Department of Psychology have found that excessive Internet use — essentially bordering on addiction — leads to school burnout in teens. There isn’t much detail on the definition of excessive use (you’ll know it… Read More

  • Can Xiaomi dominate the drone market?

    May 25, 2016 | 15:15 pm

    Can Xiaomi dominate the drone market? How does Xiaomi do it? Feature-for-feature parity with market leaders, but at a fraction of the cost?The Chinese manufacturer's products are, however, like the worst kind of teases. Showing you all the good stuff, but keeping you — if you live in the U.S. — at arm’s length, since the brand doesn't distribute many of its products here (though there are some). It’s a promise without fulfillment. Will the new Mi Drone be different?SEE ALSO: You can now stream Facebook Live video from DJI dronesSurely, the drone is Xiaomi’s best trick yet. The lightweight, "prosumer" drone matches virtually all of the key features of its main competitor, the DJI Phantom 4. The Mi Drone can fly for almost a half an hour, lets you send it to a point in the distance with a tap on the screen, plan routes, circle subjects while keeping them in camera view, fly up to 2 kilometers (1.25 miles) away, shoot 4K video and return home with a slide of a button (and automatically when it gets low on power). Read more...More about Mi Drone, Xiaomi, Drones, Tech, and Gadgets

  • Google quietly discontinues the Nexus Player

    May 25, 2016 | 15:11 pm

    Google quietly discontinues the Nexus Player  The Nexus Player may have been the best TV media player no one ever bought. Google and Asus introduced the Nexus Player back in November 2014. It was one of the first Android TV devices, and a more-than-capable device. And yet, despite Google’s weight behind this device, it hasn’t been a mainstream success, especially when you compare it with the Chromecast. Read More

  • Austin and Seattle TC Pitch-off applications close in two weeks

    May 25, 2016 | 14:47 pm

    Austin and Seattle TC Pitch-off applications close in two weeks  You have about two weeks to apply for the pitch-offs in Austin and Seattle, going down June 14 and June 16, respectively. Ten companies will have exactly 60 seconds each to pitch their wares to a panel of expert judges, including TC editors and local VCs. After all the presentations, the judges will determine our winners. First place will receive a table in Startup Alley at TechCrunch Disrupt… Read More

Loading ...