EF-Supported Teams: Research & Development Roundup

We all have a lot to be thankful for this holiday season, from the first birthday of the beacon chain, to the successful upgrades over the past year and the incredible progress across the entire Ethereum ecosystem.

As a highly productive year draws to a close, there are a few final gifts to deliver in the form of updates from many (15+!!) EF-supported teams that are always working to improve the network. And there's a lot of substantive material here, so take some time to sort through the table of contents, and dig in!

As always, this roundup series focuses on EF-supported teams whose members are working to grow and improve Ethereum as a whole. Included in this edition are updates from many teams highlighted in the previous report, and other new and rotating groups.

Enjoy! 🦄

Consensus R&D (aka EF Research Team)

Authored by Hsiao-Wei Wang

In the second half of this year, the milestones in Ethereum consensus R&D included:

Delivered the first beacon chain hard fork Altair on October 27th. The main features are:
- sync committees to support light clients
- incentive accounting reforms to reduce spec complexity
- penalty parameter updates towards their planned maximally punitive values
Delivered The Merge consensus-layer specs for interops and testnets:

To align with The Great Renaming, we shifted components to “Consensus Layer” from “Eth2” for clearer communication in the long term.

In early 2022, our team will focus on helping to deliver the “The Merge”, the most significant consensus protocol upgrade ever. Fortunately, we have great support from client teams, and others in the community working to make this a reality! In the meantime, we will be continuing our research work on data availability, L1 scaling, and the features of the “clean up” fork after The Merge.

Ecosystem Support Program

Authored by ESP Team

We published our Q2 Allocation Update with grants totalling $7,794,000 for the quarter - and Q3 is coming soon! You can also check out recent monthly roundups here and here for more detailed progress updates from a few of our fabulous grantees.

Behind the scenes, we’re working on a major website overhaul which should make it easier than ever to understand ESP’s mission and priorities, and for eligible builders to apply for funding or other support. We can’t wait to launch the new site in early 2022!

Ethereum.org

Authored by Sam Richards

To make our work more accessible and to foster more community collaboration, our team publishes an overview of our quarterly roadmap goals. See our Q4 product roadmap here.

Happy holidays to all from the ethereum.org team 😀🎄 As always, our vision with ethereum.org is to create the best portal for Ethereum’s growing community and to serve as the front door to Ethereum for millions of new visitors each month.

Content updates

Ethereum moves fast! As well as updating hundreds of pages to ensure our content stays up to date and accurate, we’ve also released a bunch of new content:

Community growth

Ethereum.org exists thanks to hundreds of content and code contributors from the community. In Q3/Q4, we put emphasis on ways to increase contributions and acknowledge the community for the work they were doing:

Some stats (Aug - Dec)

Our GitHub contributors increased by 57%, from 396 to 621 😲
Our Discord community nearly doubled, from 6,500 to 12,200 members 🎉
We’ve hosted two community calls and started office-hours for contributors
We added 3 new community guides (trusted members answering questions and moderating chat) 😎
We introduced tiered POAPs to gamify contributions for content, code, and translations

Translation Program

Since hiring our new Translation Lead in July, the Translation Program has really ramped up!

Some stats:

Between July and November, the community collectively translated 1,373,046 words for ethereum.org, over 10x the amount of this same period last year! To put this in perspective, this is equivalent to translating the entirety of almost 20 books 📚!
We've received translation support from over 2,500 community contributors 🤯
We have 37 languages live on ethereum.org 🌍
We launched an initiative to better acknowledge our translators, including a leaderboard and translator certificates!

CLR funding

We're supporting a clr.fund round on Layer 2! After over 6 months of contributing to clr.fund's quadratic funding stack, we've merged our changes to the upstream repo, which adds L2 network support and a variety of web app enhancements.

clr.fund plans to deploy a funding round on Arbitrum One geared towards the staking ecosystem in January, and the EF is excited to offer matching funds to the round. We hope you participate! Stay tuned for details. Yay public goods!

We've been building on the shoulders of giants. Thank you to the clr.fund team, the MACI team & clr.fund's community of contributors who continue to push innovations in the ZKP & quadratic funding space.

What's next?

creating a learning hub to allow non-technical users to become proficient Ethereum users
building additional resources on running nodes and staking to improve accessibility
further automating our translation pipeline to release translated content out faster
expanding the Translation Program beyond ethereum.org
updating content to transition away from the Eth2 terminology as the merge approaches

How does that sound?

We appreciate feedback on our roadmap. Our guiding principles are based on delivering the most value in the shortest time, so if there’s something you think we should work on, please let us know! We welcome ideas and contributions from anyone in the community.

Ipsilon

Authored by Alex Beregszaszi

We have created a team website to provide a clear explainer and to comprehensively list our current and past work.

It is easy to tell from the following headers that in the second half of the year the team mostly focused on the EVM. It’s also important to mention that we have teamed up with the Geth Team to improve the performance of the EVM interpreter.

EVM Object Format (EOF)

The first step towards this, EIP-3541, went live with London and we have conducted a survey across many testnets and EVM chains to find a fitting prefix for EOF.

Continued clarifications were made to EIP-3540 (including the chosen prefix), and we also proposed further work building on top of it:

EIP-3670 to introduce code validation at deploy time
EIP-3690 to replace JUMPDEST opcodes with a JUMPDEST-table
EIP-4200 to introduce two new opcodes, RJUMP and RJUMPI, facilitating static jumps

During October we presented EOF at Liscon (slides here, but the recording is not available) and at the Ethereum Meetup in Berlin (slides, recording).

Other EIPs

EIP-2681 (Limit account nonce to 2^64-1)

One of our old proposals, EIP-2681, was accepted during ACD#120. It codifies a restriction, which was already (partially) implemented in practice in most clients. After acceptance we have extended the Ethereum State Tests suite and adjusted the geth implementation.

EIP-3855 (PUSH0 instruction)

EIP-3855 proposes to introduce a PUSH0 instruction which pushes 0 onto the stack. This is a frequently used feature, mostly accomplished today via inefficient or repurposed instructions.

Our analysis found that substantial resources could have been saved with this opcode:

To put the “waste” into perspective, across existing accounts 340,557,331 bytes are wasted on PUSH1 00 instructions, which means 68,111,466,200 gas was spent to deploy them.

EIP-3860 (Limit and meter initcode)

EIP-3860 is a proposal to set a boundary and introduce metering for initcode. This would allow for more optimised analysis and execution, because implementations would need to deal with less unknowns.

geth

As a collaboration with the Geth Team, we started to work on measuring and improving the performance of the EVM interpreter in geth.

On the analysis side two reports are noteworthy:

Geth vs evmone compares the speed of Geth and evmone using the benchmarking suite in evmone.
Geth & Go compiler explores the effect of the Go compiler version on the speed of geth.

Based on these initial results, we have looked into profiling geth, and finally contributing several improvements to the codebase, most of which has been already merged. A non-comprehensive list of relevant PRs: 23952, 23970, 23974, 23977, 24017, 24026, 24031, 24120.

Follow this link to see every PR. We plan to continue this work in the next quarter.

evmone

Two bugfix releases of evmone were made: 0.8.1 and 0.8.2.

ethash

The team also maintains a C++ ethash/keccak256 library, which is used by evmone and Silkworm.

The latest 0.8.0 release introduces a new method for verifying the final Ethash hashes against the block difficulty. This is both a usability and speed improvement. The method has been posted on Ethresear.ch.

Additionally, ProgPoW has been deprecated in the library.

Fizzy

The team also participated in the Wasm in Web3 conference during September. We gave two presentations:

Fizzy — A deterministic interpreter (slides) gave a comprehensive overview of what Fizzy is, how it compares to other engines, and also explains the reasoning behind many of the design choices we made.
Weird quirks while testing WebAssembly shows a good selection of edge cases we have encountered while developing Fizzy. The talk also gives some potential solutions and explainers for these edge cases, as well as how we have extended the official WebAssembly test suite to cover them.

Formal Verification

Authored by Leo Alt

In the second half of the year the FV team continued to focus on our existing tools:

Act:

We finally released Act 0.1! You can read the amazing tutorial at https://fv.ethereum.org/2021/08/31/act-0.1/ to check what’s possible currently and how to use it.
We’re currently refactoring error handling to improve usability.

Hevm:

SMTChecker:

Track the balances of contracts precisely, including msg.value sent to and from the analyzed contracts.
Also support the low-level call function as an unsafe external call.
Improve counterexamples by reporting block.*, msg.* and tx.* values that are important for failed verification targets.
Report contract and reentrancy inductive invariants back to the user.

Geth

Authored by Felix Lange

In the second half of 2021, we published 9 geth releases. As usual, our time has been split between EIP review/implementation, client optimization/maintenance and reviewing code changes proposed by the community.

In July, the London hard fork, which included EIP-1559, was activated. The new gas pricing scheme defined by this EIP required many changes across all subsystems of geth. We are still discovering and fixing corner-case issues related to EIP-1559 now, six months after its introduction.

Two security vulnerabilities were discovered in the past six months. For both of them, we followed our security advisory policy: we immediately assigned a CVE number to the issue and published a hotfix release. Technical details about the vulnerability were published 6-8 weeks later.

In the last quarter of 2021, our work has mostly shifted towards implementation and testing of The Merge. We are on track to turn geth into the 'execution layer client' of the merged execution+consensus (fmr. “eth1+eth2”) layers. In preparation for The Merge, we have re-written much of the sync code to operate under control of the consensus layer. Geth also participates in Merge testnets.

Additionally, the geth team has been working on several long-term projects, such as implementation of Verkle Trees, a beacon chain light client, and a new database storage scheme for the Ethereum state.

Javascript Team

Authored by Holger Drewes

In the last two quarters of 2021, preparations for “the big transitions” on the Ethereum network was a strong focus of our work. We participated in the Merge Interop in Greece and released the first Merge-testnet ready versions of our client, VM and related libraries (see e.g. the EthereumJS client v0.2 release). We also started on an exciting experiment with the Go-Ethereum Verkle/Stateless team to natively test stateless block execution based on a verkle proof served along a modified block header via devp2p within our client. If you are interested you can see the following tracking issue to watch our progress.

A bit more applicable for the end user right now: the support of our libraries for the emerging L2 networks like Polygon, Arbitrum or Optimism has been improved. These and some other networks can now directly be referenced to e.g. send a transaction to a certain L2 network. See e.g. the Common v2.6.0 release for the latest Optimism L2 network integration.

And last but not least: there is a VM ArrowGlacier release available and on the Ethers.js front. Richard has just posted an exciting overview on the upcoming Ethers.js v6 library changes and updates on his blog.

Privacy & Scaling Explorations

Authored by Thore Hildebrandt

The Privacy & Scaling Explorations team works to bridge the gap between cutting-edge research in zero-knowledge proofs, and application development on Ethereum.

zkEVM

The goal of zkEVM is to run smart contracts in a zk-rollup. Unfortunately, the EVM was not designed to run in a zk circuit which makes it a challenge. We want to implement the full set of EVM opcodes directly into the zk circuits so a smart contract running on L1 can be deployed to L2 with minimal modifications. This will allow full compatibility with existing tooling and enable us to leverage knowledge of the EVM that the ecosystem has built up over the past years. We are making good progress on specification of the opcodes and implementation of the circuits, and we have early benchmarks and an important goal going forward to bring prover time down.

ZKOPRU

ZKOPRU (zk-optimistic-rollup) is a layer-2 scaling solution for private transactions using zk-SNARK and optimistic rollup. It supports private transfers and private atomic swaps within the layer-2 network between ETH, ERC20 and ERC721. It also provides instant withdrawal with pay-in-advance features and compliance compatibility using spending key and viewing keys. ZKOPRU has recently launched on testnet - go ahead and check it out. We're working on improving sync-times and on a private exchange feature.

UniRep is a private and non-repudiable reputation system. Users can receive positive and negative reputation from attesters, and voluntarily prove that they have at least a certain amount of reputation without revealing the exact amount. Moreover, users cannot refuse to receive reputation from an attester. We are using Unirep to build Unirep Social: a Reddit-like platform that allows users to privately accumulate karma. Building the Unirep Social website was our focus in the past months. Proofs in Unirep are now indexed so that they can be referred many times and prevent one proof from being submitted twice. Unirep can now handle an initial reputation airdrop and user state transition airdrop. We are also improving the efficiency in generating user state and Unirep state.

Basic functions, frontend design, frontend and backend of Unirep Social are complete, and we are planning a closed pre-alpha release. Check out this blogpost if you want to learn more.

CLR.fund for Everyone

The goal of the project is to make it easy for any community to run their own CLR round with clr.fund. This project has been very busy. You can now deploy your own quadratic funding application with the clr.fund Deployer. Empower your community to choose and fund its own future, in a fully decentralized manner. Check out our Subgraph and Documentation.

InterRep

Reputation is the key to trust. People spend years building up their reputation on centralized social platforms, but they have to start from nothing whenever they start using a new app. InterRep aims to make reputation portable to expand the compounding benefits of trusted human interactions across the web. Check out this blogpost for the initial announcement and the repo. In the past quarter we have expanded the range of social proof sources, to POAP and email and have curated groups: on-chain, and off-chain via a Telegram bot. We are doing a UI redesign, improving interaction with client applications and preparing for a live launch.

Semaphore / ZK-Keeper

Semaphore is a zero-knowledge gadget which allows users to prove their membership of a set without revealing their original identity. At the same time, it allows users to signal their endorsement of an arbitrary string. It is designed to be a simple and generic privacy layer for Ethereum dApps. Use cases include private voting, whistleblowing, mixers, and anonymous authentication. With ZK-Keeper we are focussing on keeping Semaphore up to date with the latest zk tools and integrating it with other projects like InterRep. We have new libs for handling semaphore proofs and identities. Implementation is now done on top of Halo2 and we’re getting it ready to be used in the browser.

RLN

RLN (Rate Limiting Nullifier) is a construct based on zero-knowledge proofs that enables spam prevention for decentralized, anonymous environments. In anonymous environments, the identity of the entities is unknown. We have recently published an explanatory blog post to get more people excited about the idea. We have finished research around “Feasibility analysis for ETH2 Validator privacy using RLN”. We're working on productionalizing the “Private instant chat app using RLN and Interrep” project. We're also helping to integrate the ZK-Keeper plugin into the RLN projects.

Protocol Support

Authored by Tim Beiko

The Protocol Support (PS) team was formed in 2021 to grow the number of ways in which the teams building or interacting with the Ethereum base layer are supported. The team’s main focus is enabling core developers to ship network upgrades on Ethereum’s execution layer.

To this effect, Berlin, London and Arrow Glacier were deployed this year. Beyond these, PS spent significant efforts working towards The Merge, first with Rayonism, then the Amphora workshop and now the Kintsugi Devnet!

This accelerating pace and scope of change has required more outreach to the Ethereum community, which led our team to organize frequent Community Calls. During these, application, infrastructure and tooling developers were invited to discuss how to best support protocol upgrades and provide a smooth transition for their users. Alongside these calls, the team has given several talks and published several posts about the changing Ethereum roadmap, such as this recent all core devs update, this piece in Bankless and this recent post on the Merge and the application layer in the EF Blog.

Beyond protocol upgrades, the PS team has taken on two major initiatives to ensure clients teams are well supported. First, a Client Incentive Program was announced to give teams Ethereum-aligned long-term incentives. The program provides client teams with a set of 144 validators that they must run using their software. Assuming teams keep meeting certain performance benchmarks on mainnet, these validators are gradually vested to the teams, which are free to either liquidate them or keep them running to collect rewards and fees. This program aligns teams with Ethereum, ensures they are “dogfooding” their clients on mainnet, and that they keep delivering performant software.

Second, a Core Developer Apprenticeship Program was launched. This program provided stipends and mentorship to self-directed individuals who wanted to dive deep into protocol development. CDAP was launched as an experiment which proved to be incredibly successful! Two cohorts were run, with over 25 participants. Of these, at least 5 are now working full time in the ecosystem. These initial cohorts have taught us a lot about what was good and what could be improved with the program. Expect a revamped CDAP in 2022!

Lastly, the team experimented with providing infrastructure to the client teams and broader community. To that end, crawler.ethereum.org was shipped and open-sourced. We hope that having an additional crawler running and available for the community to improve, modify or fork helps provide better views of the network’s topology.

Remix

Authored by Rob Stupay

Over the past 6 months, the Remix team has unscrewed the back of our app to do some extensive rewiring. First among these changes was continued work on moving our code to React. We also expanded our reach by dialing in some effective channels to new communities, and onboarding new users with a basic product “tour” of our IDE. We’ve plugged in projects into our “experience”, integrating Slither, and Hardhat, as well as updating the Remix VSCode extension.

And, if that’s not enough, we jammed on some tools for collaborative coding, improving Decentralized GIT and integrating Github. Last but not least, we’ve updated our existing plugins. In short, we’ve maxed it out to 11.

See more details in our article.

Robust Incentives Group

Authored by Barnabe Monnot

The RIG (Robust Incentives Group) welcomed new team members and participated in many important milestones for the Ethereum protocol. For a quick refresher on what the RIG is and what we care about, you can check out Protocol cryptoeconomics with the RIG, presented by Barnabé at EthCC in July.

On the Proof-of-Stake consensus side, Caspar, who joined us as full-time research scientist earlier this year, found an issue with the current fork choice, written up as Three attacks on Proof-of-Stake Ethereum. Fortunately, there is a strong candidate fix that was recently merged in the consensus specs, after many productive discussions with Stanford's Tse Lab, who co-authored the "Three attacks" paper. Caspar and others also proposed a different mitigation ("proposer view merge"), that is still under research. Check out Caspar at Liscon presenting his results!

Shyam, who joined us first as research intern last summer and is now a research assistant at the RIG, released a series of notebooks exploring the beacon chain statistics from many unique angles, including oceanic games and inequality. Shyam has also been working on an extension to our Beacon runner PoS simulation engine that includes reinforcement learning. Check out his talk at EDCON!

Block 12,965,000, August 5th, 12:33:42 PM UTC, was an important date for us: the London hard fork activated, and with it, EIP-1559. Over the last year, we've released a series of notebooks presenting various simulations of the new fee market mechanism, which set the stage for further analysis after launch. Barnabé reported some results early after, and with co-authors (including Shyam) wrote up a longer-form paper, Transaction Fees on a Honeymoon: Ethereum's EIP-1559 One Month Later. The paper is motivated by the behaviour of the 1559 update rule in the real world, and opens new research directions for improvements to the rule.

The RIG was also closely working with the cadCAD Edu team in preparing an online masterclass in validator economics, supported by a fully extensible model of Ethereum economics (in Python).

Snake Charmers [Python Ecosystem]

Authored by Keri Clowes

In the second half of 2021, the Snake Charmers team completed the changes needed across the ecosystem to support the London hard fork. This involved wide-ranging, fundamental changes throughout our stack, especially in Py-EVM, Ethereum Tester, Web3.py, and eth-account. There were also two bug bounties submitted for Py-EVM that have been fixed. We’ve ramped up efforts to generate educational content, and placed a bigger emphasis on developer relations. And as always, there is ongoing community support, issue triage, and bug squashing across our Python tools.

Fe-lang

Authored by Grant Wuerker

Over the past 6 months, the Fe team has cut the following releases:

0.11.0-alpha “Karlite” (2021-12-02)

multi-file support
function definitions on structs

v0.10.0-alpha “Jade” (2021-10-32)

module-level constants and functions
unsafe support

v0.9.0-alpha “Iridium” (2021-9-29)

self declarations in function signatures

v0.8.0-alpha “Haxonite” (2021-8-31)

query-based analysis using Salsa

0.7.0-alpha “Galaxite” (2021-07-27)

Solidity ABI decoding checks

0.6.0-alpha “Feldspar” (2021-06-10)

no notable features

If you would like to know more about our progress over the last 6 months, you can checkout the following resources:

Portal

Authored by Piper Merriam

This year has been a big year for the Portal Network. We started this year with an idea and only a loose plan for how to build a peer-to-peer network that could deliver lightweight access to the Ethereum protocol. We now have three independent teams and implementations and are well underway to launching the initial testnet which should evolve into a fully functional network by the end of 2022.

The EF Portal team has been working hard on Trin, a portal client written in Rust. The EF Javascript team has also been working on Ultralight, a portal client written in Typescript aimed at being runnable in the browser. The team from Status.im has also been working on Fluffy, a portal client intended for integration with the Status ethereum client and wallet solutions.

During this year we have solved the previously unsolved problem of how to distribute the current Ethereum State in a manner that is conducive to efficient storage and retrieval. We established the Portal Wire Protocol, an extensible base protocol that is the foundation of all of the networks making up the Portal Network. We also had the pleasure of working with multiple participants of the Core Developer Apprenticeship Program who used the Portal Network projects as a jumping off point for getting into Core Protocol development.

Security [Security / Consensus Tests]

Authored by Security (Security / Consensus Tests) Team

On the security and testing side, a lot of attention has been given to the London upgrade and the upcoming merge. We’ve made updates on tooling for test authoring and continued to improve the reference tests.

Solidity

Authored by Franziska Heintel

In the second half of this year, we released Solidity versions 0.8.8, 0.8.9, 0.8.10 and 0.8.11:

Solidity 0.8.8 brings you user defined value types as a major new feature. It also improves overriding interface functions, reading from immutables, and more.
Solidity 0.8.9 is a pure bugfix release and fixes two important, but low severity, bugs:
- Signed Immutables Bug
- User Defined Value Types Bug
Solidity 0.8.10 contains external function call optimizations, enables the new EVM code generator for pure Yul mode and can report contract invariants and reentrancy properties through the SMTChecker.
Solidity 0.8.11 adds a first implementation of a Language Server and allows a safer way to perform ABI-encoding.

Moreover, several Solidity team members presented at ETHGlobal’s Developer Tool Summit:

Hari with “What’s New in Solidity”
Franzi with “Next Level Source Verification with Sourcify”

The Solidity documentation got a few upgrades, most notably, we...

updated the resources section with general resources, Ethereum IDEs, editor integrations, Solidity tools, Solidity parsers and grammars.
added the functionality to open code examples in the documentation directly in Remix.

Lastly, we launched our yearly Solidity Developer Survey. If you are a Solidity developer, please take 10 minutes to share your feedback and take part in the survey here. The survey will be open until 31st of December 2021.

Oh and we’re hiring! Have a look at our C++ Engineer Solidity opening.

ZoKrates

Authored by Thibaut Schaeffer

In the second half of 2021, ZoKrates advanced on different fronts:

Language

Type aliasing, as well as the ability to make function calls in constant definitions
Support for the ternary expression syntax
Enable constant generics on structs

Proof systems

Reduction of the deployment cost for some Solidity verifiers
Expose recursive verification in the standard library
Add support for Groth16 MPC ceremonies (coming soon)

Compiler performance

Extensive work on reducing memory and time requirements of the compiler (coming soon with metrics!)

For a full list of the changes, check out the changelog