Testground is a platform for testing and benchmarking distributed systems. We used it before to work on massive improvements to IPFS DHT, Filecoin Network, and libp2p. This year, the IPDX team (opens new window) re-ignited the project. We started two collaborations, created a new use case for interoperability testing, and welcomed new users and contributors.
Now that the end of the year is near; it is time for a “bragging session” and to share our master plan to take over the CIs of the entire Protocol Labs Network!
# 🔌 A new use case: interoperability testing for libp2p (opens new window)
Using Testground, the libp2p team is now testing interoperability between many implementations and versions. The newly introduced tests cover go-libp2p and rust-libp2p versions already, and js and nim support is in the works.
Milestones:
- Mixed Builders Feature: We can now write a test that depends on multiple builders and languages (opens new window).
- Interoperability Support Improvements in Go: Testground can rewrite mod files (opens new window), and overwrite dependencies (opens new window). This enables users to test many different versions of a library using a single test source.
- sdk-rust (opens new window): We introduced the sdk-rust (opens new window) and improved it over the year (opens new window).
- sdk-js (opens new window): Thanks to Little Bear Labs’ (opens new window) support, the Testground SDK for JavaScript now works in the browser (opens new window) too.
We also presented how Testground powers libp2p interoperability testing at IPFS Camp (opens new window) (Youtube).
# 🤹 Massive stability improvements for supported SDKs
Testground has an issue with flakiness. The project's stability has been on our minds throughout the year, and we have made significant strides in this area.
Milestones:
- A refreshed Docker Runner: to support new languages better and fix concurrency issues, we rewrote the Docker Runner (opens new window). In our tests, false negatives went from 40% to <1%.
- Benchmarking and stability improvements: we've implemented tooling and fixes (opens new window) that help with CI testing (opens new window). Some of this work will soon become part of the Testground Stability Dashboard and custom CI actions.
# 🐘 EKS Support
Testground is also designed to run large network tests spanning far beyond what a single machine can handle. With Testground on EKS (opens new window) (managed Kubernetes service from AWS), we can support tests with more than 10k nodes.
Milestones:
- Preview Support for EKS: The Bloxico team (opens new window) has been working hard on Testground EKS support (opens new window) and EKS infrastructure setup for Testground (opens new window). A working alpha setup is now available. Other teams, such as Celestia (opens new window), are already building on top of it.
# ✨ And sooooo much more
✨ The team also dusted off SDKs (opens new window) and examples (opens new window), helped improve ARM support (opens new window). We introduced new features (opens new window), bug fixes (opens new window), and brought many more improvements (opens new window).
# 🫶 New Users
New users and contributors are joining the project!
- Celestia (opens new window) is using Testground to test their network (opens new window). Building on top of our EKS Preview, they simulate between 4 and 6 thousand nodes.
- Magmo (opens new window) is using Testground to test and benchmark their payment channel. They had success with stability and performance measurements. They shared an excellent presentation during Retrieval Markets Summit (opens new window) (Youtube).
- status.im (opens new window) is working (opens new window) on adding their nim implementation to the libp2p interoperability repository.
- Sigma Prime (opens new window) is using Testground to test their implementation (opens new window) of Ethereum's Peer Discovery Protocol.
- number0 (opens new window) uses Testground to benchmark iroh (opens new window), an IPFS implementation written in Rust focused on efficiency. They share numbers on their website (opens new window).
- Moving the Bytes Working Group uses Testground to benchmark and test bitswap and new protocols for data exchange. You can follow along in Notion (opens new window) and join the #move-the-bytes-wg (opens new window) channel on FIL Slack.
# 🚀 What's Next
We shared a Roadmap (opens new window) for what is coming to Testground in 2023. Here are the highlights.
- Better Network Simulation & New Features: We plan on improving network simulation (opens new window) for libp2p and the Move The Bytes WG. We are also adding the ability to run more complex test plans (opens new window) and browser support (opens new window). Testground improves the fastest when it’s “customer-driven”.
- Testing, Stability, and Workflows Improvements: the team will invest (opens new window) in bringing Testground building and testing tools to a state where it's easy & efficient to contribute. We want to improve the scale and stability of changes we can make, encourage external contributions, and communicate clearly about stability & regressions.
✨ Testground is like that kid in the back of the class, "great potential, needs more attention.” We believe it's brilliant, and we'd like to encourage you to join us (opens new window)!
Whether you already know Testground and can't wait for improvements, or you are discovering the project and thinking about using it, we'd like to hear more from you:
- Join the Community (opens new window) and build the future of Testground with us!
- Take two minutes to fill out our User Survey Form (opens new window) and help us identify pain points.