This is the story of the Deep Dive sessions at IPFS Camp 2019 as told from the perspective of a DWeb historical researcher in the far future.
In 2019, the intergalactic dweb uprising began. The pillars of what had become the foundations of human civilisation started to crack under their own weight and the very beginnings of a new empire began emerging from the shadows. An empire forged in the flames of discontent of central servers and "do-no-evil" monopoly. Peer to peer communications - the heart of the dweb - slowly formed efficient and autonomous networks that expanded beyond the reach of the blue planet and far into the galaxy, the likes of which the old world order had never seen before.
The uprising began modestly as a "space camp" in the ALPHA region 5P4-1N. From fear of ambush the exact coordinates were only divulged to the heroes that enlisted. What happened at that camp laid the foundations for the world as we know it today, but between then and now, many battles took place and we will forever honour the unbeknown dragon slayers that emerged from camp to lead the uprising and shape our new world.
This story attempts to recount a small part of camp, what was referred to back then as "Deep Dives" but what we later came to know as The Great Calamity Circumvention Assembly.
At this time the uprising was in its infancy. Incredible progress had already been made expanding peer to peer communications, content addressing and information permanence, but as unbelievable as it sounds, the dweb had still not yet expanded beyond the reach of the blue planet (what we once referred to as "Earth").
Networks in those days were siloed and inaccessible. Data was equally unavailable and fragile and temporary. The transition from analogue to digital was still in living memory and appropriate steps had not yet been taken to ensure human history was permanently recorded. The worldwide data erosion seen in the Global Electromagnetic Fire Disaster of 2027 (just a handful of years after camp) proved this point and some estimates suggest that over 83% of all human history for that period was wiped out.
As you can imagine, there were a great many problems the dweb uprising was facing at that time. The Calamity Circumvention Assembly ("Deep Dives") was formed at camp to address as many of those issues as possible.
Around 160 space rangers formed somewhere in the region of 30 splinter groups to focus on specific open problems around IPFS. As you know, IPFS - the InterPlanetary File System is one of the biggest proponents of the dweb and is currently the most widely used file system across the universe, used by over 31,000 known alien races. It was one of just a handful of dweb technologies at the time and later went on to play a monumental part in the dweb uprising.
The plan was to come up with new and inventive solutions to these problems. The enlisted space rangers were vetted to ensure they were experts in the dweb field and were suitably well placed to solve the issues at hand. In just 1 hour and 30 minutes (that's 1000 hexsec (opens new window)s in our time) the splinter groups intensely discussed their assigned topics and documented a wealth of knowledge, ideas and observations that are still referenced in academia today.
The outcomes from each splinter group's discussions were "pull requested" to the camp repository (A "pull request" was a term used at that time to submit additions/deletions or changes to a set of files). The following day splinter groups presented their solutions to the assembly for intergalactic approval.
Transcripts and videos from The Great Calamity Circumvention Assembly were salvaged from the Global Electromagnetic Fire Disaster of 2027 and we've transcoded them into formats appropriate for today's dweb browsers. Enjoy!
| Topic | Owner | Description | Recording |
|---|---|---|---|
| Scaling up the DHT 📦 (opens new window) |  | What can we do to optimise the performance of our DHT? Are there alternatives? | Video (opens new window) |
| IPID - Master key security and user-experience 📦 (opens new window) |  | IDM (opens new window) (Identity Manager) aims to support several DIDs (opens new window) methods and IPID (opens new window) was chosen as the first DID method to be integrated. | Video (opens new window) |
| Package Managers: What Now, What Next? 📦 (opens new window) |  | Community evaluation and ideation session to recap the current (as of late June) state of the IPFS-enabled package manager landscape, discuss happiness and pain points, and do some ideation/co-creation around the direction of our next rounds of package manager efforts. | Video (opens new window) |
| IPFS on Mobile (opens new window) |  | Increasingly, everyday computing is happening in mobile environments. At the same time, many features of IPFS and related technologies work best on an 'always-connected' network that simply doesn't exist in the mobile world. In this deep dive, we will explore the issues inherent to p2p systems operating in mobile environments, discuss some existing solutions, and hopefully outline a research/development agenda that will help catalyze a wave of new IPFS-based mobile solutions. | Video (opens new window) |
| Private content on IPFS 📦 (opens new window) |  | Can we have a private group chat implemented on IPFS? | Video (opens new window) |
| Building the IPFS Community (opens new window) |  | Community, evangelism, engagement, on ramp. | Video (opens new window) |
| IPFS for websites (static, dynamic, search, etc) (opens new window) |  | What are the best practices and open questions for using IPFS for static / dynamic websites? What about indexing and searching? | Video (opens new window) |
| Revocation/Rotating of IPNS Keys (opens new window) |  | The idea of using a public key to identify mutable data is cool, but what happens when that key becomes compromised? | Presenter Opted Out of Recording |
| How can IPFS support sharding 📦 (opens new window) |  | How to best pin an arbitrary part of a DAG. | Video (opens new window) |
| Replication protocol 📦 (opens new window) |  | How can we more quickly sync a DAG or part of a DAG between two nodes? A compliment to bitswap, affectionately overloaded as GraphSync | Video (opens new window) |
| Do not replicate lists - DNR (opens new window) |  | How to stop your node from replicating content due to take down notices or other reasons | Video (opens new window) |
| Fast IPNS 📦 (opens new window) |  | How to increase the IPNS resolution speed | Moved to Lightning Talk |
| IPFS Education and Documentation (opens new window) |  | What are the biggest improvements we can make to documentation for IPFS? | Video (opens new window) |
| Mounting an IPFS filesystem 📦 (opens new window) |  | Is there a way to efficiently mount IPFS as a filesystem? FUSE support is here (opens new window). What are current limitations, challenges? What needs to happen to enable mounting of MFS? WebDAV (opens new window) is often used in network storage drives, and mainstream operating systems come with native support for mounting WebDAV as drives. It could supersede Writable Gateway and create a surface for interop with apps that provide built-in client-side support for WebDAV. | Video (opens new window) |
| IPNS link rot (opens new window) |  | Right now, IPNS needs to be refreshed by original publisher every day, otherwise it stops working. There should be a way to pin IPNS address so that other peers keep refreshing the last valid entry when original node goes offline, keeping the last version alive even if the original publisher disappears forever. | Video (opens new window) |
| Deterministic CIDs! Reproducible File Imports! Verifable HTTP Gateways! (opens new window) |  | How to deterministically get the same CID for the same data, even if default parameters of ipfs.add change? How to validate data read via HTTP Gateway without running full IPFS node? | Video (opens new window) |
| IPFS in Web Workers (opens new window) |  | Where we are, limitations, known problems. | Video (opens new window) |
| UnixFS v2 📦 (opens new window) |  | Latest spec iteration (opens new window) and implementation work (opens new window) | Video (opens new window) |
| Using CRDTs to build real-time collaborative DApps (opens new window) |  | What are the hurdles to using CRDTs in dapps with IPFS? How can we make peer-base easier to understand and use? | Video (opens new window) |
| Better NAT traversal so that Relay servers are a last (not first) resort (opens new window) |  | The current cost of running relay servers for IPFS is unsustainable. We need to implement better NAT hole punching mechanisms in libp2p to reduce the usage of relay servers by NAT’d hosts. TCP hole punching has a success rate of ~60%, so we should be preferentially hole punching. TCP simultaneous open is one example. STUN servers are another example; yet another is making every non-NATed host in a libp2p network a relay node. | Video (opens new window) |
| libp2p message orientation (opens new window) |  | libp2p is currently stream-oriented. We want to enable message-oriented communication (e.g. UDP). Help us envision what the APIs would look like, and how an application could simultaneously hold stream-based and message-based conversations with peers in the world via libp2p. | Did not Present |
| IPFS metrics and usage statistics 📦 (opens new window) |  | How to collect telemetry like application metrics and usage in a distributed world | Video (opens new window) |
| Decentralized Data Processing (opens new window) |  | How could we process large amounts of data in permissionless networks? Use cases, verifiable state transitions, aggregations and bulk updates, etc. | Video (opens new window) |
| Building Dynamic P2P Applications (opens new window) |  | IPFS and its associated technologies are mostly static. Radicle is an attempt to build a dynamic P2P web on top of IPFS; one which can host applications with the same abilities of the centralized servers of today to receive inputs from various people or sources. | Video (opens new window) |
| Better file chunking 📦 (opens new window) |  | Within the IPFS stack/ecosystem, just as within computing as a whole, an uncompressed stream of untagged octets is a fundamental unit of exchange. As a general-purpose data storage system IPFS needs to handle an unbounded variety of content represented by such streams. Handling the maximum amount of this variety efficiently ( ideally by default ) would likely have an outsized impact on the future adoption of IPFS as a long-term data interchange medium/format. | Video (opens new window) |
| Dynamic DAG traversal (opens new window) |  | Extend IPLD to cross between immutable and mutable references | Video (opens new window) |
| Design Patterns for DWeb Apps (opens new window) |  | Most IPFS apps have UX issues. We will pick a feature of an existing app, and improve its UX by simplifying how to interact with it, how it works. Textile.io lowered the permission and roles system from ~15 methods to 2 this way! | Video (opens new window) |
| IPFS and Guix 📦 (opens new window) |  | Exploration into existing or potential issues with integrating IPFS with Guix | Video (opens new window) |
# There are more to come, keep posted!
More content from IPFS Camp is being salvaged and transcoded.
If you are interested in receiving an update when the next batch of videos are available, you can do this in one of the following ways:
- Watch and Star the ipfs/camp repo (opens new window), where all the content will live, in addition to other other locations (IPFS 😄) shielded from the Global Electromagnetic Fire Disaster of 2027
- Subscribe to the RSS feed of this blog (opens new window).
- Subscribe to the IPFS Weekly Newsletter (opens new window) if you are interested in general IPFS updates.
- Subscribe to the IPFS Events Newsletter (opens new window) if you are interested in IPFS events.