# 🧘‍♀️ Purpose

JavaScript (and its typed variant TypeScript) is the most ubiquitous programming language with the largest community of developers. Our goal is to enable all those developers to reap the benefits of IPFS and do so productively.

This blog entry intends to give an update on what is happening with IPFS development in JS. It’s been a while since we’ve done this, so there’s a lot to cover. This is the first update of more to come. It isn’t a full roadmap, but we aim to give clarity to some of the history of IPFS development in JS, decisions by some maintainers on what to do going forward, and ways you can help.

# 📇 Names and Terms

To help with this update, the following names and terms will be used to aid with clarity:

# ⏳ IPFS-in-JS Development History

The js-ipfs project started in 2014 and sought to mirror the functionality and API of Kubo (then called go-ipfs). This manifested in the concept of the JS Core API (opens new window), which was effectively a conversion of the Kubo RPC API (opens new window) into JavaScript. There were multiple implementations of this interface including:

This effectively had Kubo and js-ipfs’ development awkwardly interlinked. You would find support for certain functionality in Kubo that wasn’t supported in js-ipfs, and js-ipfs would often be held back in experimenting on a feature because the change wouldn’t get added to the JS Core API without it also being implemented in Kubo.

In addition to interlinking with the API and design choices of Kubo, the top-level js-ipfs project followed a similar project structure to Kubo of being a kitchen sink of interaction options with:

  1. Daemon to be able to run js-ipfs as a standalone process
  2. A CLI to the core implementation
  3. Two RPC client and server implementations:
    1. RPC-over-HTTP API that matched Kubo
    2. gRPC-over-websockets API for bidirectional streaming
  4. HTTP Gateway (opens new window) server (though it lagged behind Kubo and did not support the latest specs for things like subdomain isolation and verifiable block/CAR responses (opens new window))

This “kitchen sink” aspect was also antithetical to the “JS way” of being lean and modular. For example, by importing js-ipfs, you always get MFS, IPNS, etc. even if they don’t use them, which leads to bigger bundles and more dependencies with more attack vectors.

“… You wanted a banana but what you got was a gorilla holding the banana and the entire jungle.“ — Joe Armstrong, creator of Erlang progamming language

This is unlike the more flexible js-libp2p model where you can extend the capabilities of the node by configuring additional modules.

# 🔀 Historical IPFS JS Usage Patterns

Given the flexible nature and multiple runtime support of JavaScript, several usage patterns emerged over the years. With the development history above, the IPFS network and protocols were accessed using JavaScript primarily with the following approaches.

# Remote Control of Kubo

With this approach, a long-running Kubo node be controlled by ipfs-http-client (opens new window) via Kubo's RPC API.

Based on npm download metrics, this is by far the most popular approach with ~10x more npm downloads (opens new window).

# 👍 Pros

  • Gives access to a full-featured IPFS implementation in Kubo with active maintenance and feature development.

# 👎 Cons

# js-ipfs in a Node.js Context

With this approach, you either embed ipfs-core (opens new window) directly in your application by importing and instantiating the IPFS object or accessing a shared js-ipfs daemon. With the former approach, the Node.js process of the application would also open IPFS peer-to-peer connections.

# 👍 Pros

  • Having the full application stack in JavaScript makes for a simpler architecture and easier debugging.

# 👎 Cons

  • Historically lacking in some functionality that was only more recently added like a working DHT client and server.
  • The API is large and hard to grasp. It contains many methods translated directly from the Kubo CLI, designed to be quick to type but often redundant.

# js-ipfs in a Browser Context

Because browsers impose constraints on the number of network connections opened and storage space used by each browser tab, they don’t make for reliable IPFS nodes. For this reason, many of the responsibilities of an IPFS node which include: routing, retrieval, and providing have all been handled in a delegated fashion in the browser.

Unfortunately, the default delegation setup in js-ipfs today is overly complicated and not well-engineered. Specifically, this happens via preload and delegate nodes hosted by Protocol Labs that support secure WebSockets (WSS) that browsers can safely and reliably connect to.

For delegated routing/retrieval, js-ipfs asks the delegate nodes to perform DHT operations on its behalf using the Kubo RPC API (/v0/dht/*). The delegate nodes query the DHT for peers and content providers, and then they fetch the content, which they provide for the browser nodes that are connected to them.

For delegated providing from the browser, js-ipfs contacts the preload node using Kubo RPC API (/v0/refs) to preload the entire dag from the browser. The preload node fetches the data from the browser over Bitswap and then advertises it on the DHT.

Note: Preloading could be understood as ephemeral pinning. The CIDs' data is fetched by a preload node for a short period during which it is available to the IPFS network, and after which it is garbage collected.

js-ipfs nodes running in the browser can connect to other js-ipfs nodes via the WebRTC Star transport, but this requires a centralized signaling server (opens new window) to do the initial handshake and cannot be used to connect to Kubo nodes as support for this transport has never materialized.

# 👍 Pros

  • It’s useful and works for multi-user apps with small amounts of ephemeral data transmitted in real-time.
  • Even though js-ipfs with WebRTC requires a centralized signaling server and is not the ideal long-term solution (opens new window), it enables direct browser-to-browser communication (after the initial SDP manifest (opens new window) is exchanged over signaling server).

# 👎 Cons

  • Availability and performance are heavily dependent on non-trivial centralized infrastructure:
    • If PL shuts down the hardcoded HTTP endpoints at preload.ipfs.io (opens new window), js-ipfs delegation is dead, unless someone also runs their own Kubo instance somewhere in the cloud with a TLS cert for HTTPS and sets it up as a preload in js-ipfs config. This is a non-trivial ask and a steep barrier to adoption.
    • This was designed as a temporary "hack" anticipating a proper libp2p protocol. Unfortunately, years later we’re still doing the original Kubo RPC /v0/refs hack.
  • Inefficient design:
    • It doesn’t scale well because certain preload nodes get disproportionate traffic due to hardcoding in old js-ipfs versions.
    • Preload via /v0/refs is very wasteful (opens new window).
  • Not specified or adopted by other implementations: the WebRTC Star transport is only implemented in js-libp2p and neither WebRTC Star nor the delegate/preload APIs are specified.

# 🧑‍💻 IPFS-in-JS Development the Last 18 Months

The last 18 months on the JS front have been spent on:

  1. General maintenance of the project including mostly maintaining parity between the JS Core API with Kubo.
  2. Modernizing js-ipfs and js-libp2p to TypeScript and ESM only. 😅
  3. Improving security and performance of js-libp2p given its criticality for other projects outside of IPFS such as Ethereum Lodestar. For example, a set of DoS and eclipse attack mitigations (opens new window) were added.
  4. Adding a working DHT client/server implementation to js-libp2p. (Yes, in a NodeJS context you can read from and write to the public IPFS DHT.)
  5. Expanding connectivity options of IPFS-in-JS implementations. Notably, we introduced a WebTransport transport (opens new window) and are adding a new WebRTC transport (opens new window) that does not require a centralized signaling server to enable both browser-based IPFS nodes to dial Kubo nodes directly and browser-to-browser connectivity. (This is discussed more below.)

At this point, we have a solid peer-to-peer library in js-libp2p that can be built upon, and many lessons learned for how to better progress with Javascript/Typescript in a world of multiple IPFS implementations (opens new window).

# 💡 The Future of IPFS-in-JS in 2022 and 2023

With all the background of IPFS-in-JS over the past covered, this section will transition into our practical plans for 2022 and 2023.

# Go and JS Development are Decoupling

We never had full compatibility between Kubo and js-ipfs, we don’t think we can, and we don’t think it’s worth investing more down this path. At least for the implementations maintained by Protocol Labs’ EngRes (Engineering & Research) group (opens new window), the Go and JS implementations will diverge and develop the APIs that are best for their respective user bases.

In practical terms, this translates to:

  1. ipfs-http-client will remain the RPC-over-HTTP API for controlling js-ipfs (you can also use ipfs-grpc-client over WebSockets). The current js-ipfs RPC APIs will be maintained until js-ipfs support ceases (discussed below).
  2. With investment in Helia, a new RPC API for Helia will emerge.
  3. We won’t test that ipfs-http-client has compatibility with recent versions of Kubo.
  4. If you want to control a Kubo node via JS, use the Kubo-specific library js-kubo-rpc-client (opens new window) (api).

# Seize and Leverage New Browser-Friendly P2P Transports

We are transitioning to a world where browsers can connect to other libp2p nodes (including other browsers) without Central Authority TLS certs thanks to new transports like WebTransport and WebRTC. See connectivity.libp2p.io (opens new window) for more details.

This means browser nodes have more optionality to which long-running IPFS nodes they delegate routing, retrieval, and providing. For example, making DHT or Bitswap requests to other nodes on the network is now viable.

Note: browser nodes will still want to delegate providing content to a node with more longevity since the new transports won’t stop browser nodes from disappearing from the network when the user closes a tab or puts their laptop to sleep.

We’ll lean into realizing these breakthroughs and remove the more convoluted mechanisms from the past that relied on the Kubo RPC API and preload nodes discussed in js-ipfs in a Browser context.

# Support Fully Speced Delegated Routing Protocols and Endpoints

While it will be possible from a connectivity perspective to make DHT queries from a browser, we expect various applications will want to still delegate out routing. Reframe (opens new window) HTTP Routing V1 (opens new window) is a protocol for delegated routing that other IPFS implementations like Kubo have implemented. While it currently uses HTTP as a transport, it is speced (opens new window) and not tied to the Kubo RPC API. If/when there is a speced protocol for ambient discovery of “Limited Delegated Routers” provided by libp2p, we will support that as well.

# PL Delegate and Preload Nodes Will Be Shutting Down

Given the new browser-friendly p2p transports discussed above, we’ll shut down the complicated “song-and-dance” with the legacy delegate/preload nodes and the Kubo RPC API described in js-ipfs in a Browser context. This yields a simpler setup for one’s application and removes centralized infrastructure.

For delegated routing, one can configure /routing/v1 (opens new window) endpoints. When it comes to providing content from a browser node, it will be up to developers to account for user behavior like closing tabs or laptop lids. The general recommendation is to either run your own preload node or upload content explicitly to a pinning service for providing.

# Release Helia in 2023

Helia is the to-be-developed IPFS implementation with all that we’ve learned over the last 8 years while leveraging what is available to us in JS runtime.

Some defining attributes include:

  1. Web-first isomorphic API - run in browsers, electron, node, deno, bun, etc - no node.js APIs, only standard JavaScript (e.g. web streams, not node streams, Uint8Arrays not Buffers). Node APIs will only be considered for special cases like mDNS.

  2. Leaner API not tied to the legacy “core API” concept - Helia will not have API compatibility with js-ipfs. It will expose a more ergonomic JS-developer-first API than what we have with the js-ipfs “core API” that was heavily influenced by Kubo. (One can also create an adapter from the “core API” to Helia’s API if they want to drop Helia to their existing application using js-ipfs’ ipfs-core.)

  3. ESM and TypeScript only - There’s no more debate on the utility of these for JS development. We’ll adopt them from day one.

  4. Leverage existing interplanetary libraries - While we’re moving away from the interface and composition in js-ipfs, we’re not abandoning the underlying layers like js-libp2p, js-bitswap, etc. Those libraries have received a lot of maintenance attention in the last 18 months (including TypeScript and ESM updates) and are battle-tested in production. We will depend on them in Helia as well.

  5. Unified file API - high-level commands that act like a filesystem and return CIDs. For example:

    const dirCid = await ipfs.mkdir('/foo')
const dirCid2 = await ipfs.touch(dirCid, 'file.txt')
const dirCid3 = await ipfs.pipe([
  'file content',
  ipfs.open(dirCid2, 'file.txt'),
]) // or something
const content = await ipfs.cat(dirCid3, 'file.txt') // could be a network request

  6. Expose a block API for low-level IPLD operations.

  7. Focus on the browser use case - We won’t do anything that precludes operating in NodeJS or a cloud service worker, but by default, we will prioritize delivery paths that deliver browser functionality sooner. This is because the browser runtime is the unique runtime the Helia IPFS implementation can enable that other implementations can’t. As a result, it means there aren’t plans to invest in things like a JS implementation of the HTTP Gateway spec. We’ll let other implementations like Kubo, Iroh, etc. pursue that use case.

  8. Enable configurable levels of delegation. With routing, retrieval, and providing there will be varying levels of delegating from none (all handled by the local Helia node) to full (all handled by HTTP Gateways (opens new window) and Pinning Services (opens new window)).

# Pause js-ipfs Maintenance Once Helia Is Released

Shortly after you can add and cat files across the network with Helia, PL EngRes (opens new window) will cease maintenance on js-ipfs. In the absence of an established group with a credible track record to take js-ipfs over, the community is welcome to fork js-ipfs and maintain the fork. (We want to avoid issues that can occur with casually giving away publishing rights.)

As discussed before, we are not ceasing support and development of many of the libraries that js-ipfs depends on like js-libp2p and js-bitswap. These projects will be actively maintained as core dependencies to Helia and other projects.

# A New Name Is Coming

As outlined here (opens new window), Protocol Labs wants to make space for additional IPFS implementations to be made, including in JS. We want to make it clear that js-ipfs is not IPFS and that js-ipfs is not the IPFS implementation in JS. go-ipfs successfully made this transition earlier in 2022 with its minimal rename to Kubo (opens new window). We will certainly not make the same name-squatting mistake with a new implementation like Helia. Details and plans will be shared here (opens new window) and in the IPFS forums (opens new window).

# Doc Updates Galore

From dedicated websites (opens new window), examples (opens new window), to official docs (opens new window), and courses (opens new window), many places will need updating in light of new names and implementations. This is going to be a sizable undertaking that hasn’t been scoped out yet. This will be tracked here (opens new window).

# 🗺 Timeline

The timeline for enacting all of the above is still actively being figured out. We’ll be updating the proposed roadmap (opens new window).

# 🤝 Ways You Can Help

  1. 🗳 Propose a name for the new “Helia” JS library (opens new window) and cast your votes.
  2. 🗣 Give feedback on the Helia roadmap (opens new window). Let us know how you’re using js-ipfs now so we can see if/how your use case would be supported with Helia in the future.
  3. 🫂 Join the team - we’re hiring and need more JavaScript and TypeScript developers who are eager to make the vision above a reality. It’s ideal if you have experience working at the protocol/bytes/streams level. Please learn more here (opens new window).
  4. ✋ Contribute - Open source contributors welcome. Have a great idea and need some funding? Consider a grant request (opens new window).

Thank you for reading and being on this journey to make IPFS exceptional in JS runtimes!

Note: An earlier version of this blog post referred to Helia as Pomegranate. The blog post has been updated to reflect the name chosen by the community. (opens new window)