Lido Node Operator Portal
đź“–

Lido on Ethereum Validator Exits

Intro

Table of Contents

  • Intro
  • Table of Contents
  • Summary
  • Validator Exits Policy
  • NO Responsibilities
  • Monitoring
  • Penalties
  • NO Tooling
  • Other tooling / scripts / resources
  • Resources
  • Follow & Contribute

Summary

The purpose of this document is to provide a collaborative resource to aid Node Operators participating in the Lido protocol to timely and effectively execute any updates to their infra or processes necessary for the smooth execution of the Lido V2 upgrade and resulting changes w.r.t. validator exit processing.

Validator Exits Policy

NO Responsibilities

Expectations and requirements are detailed in the Lido Validator Exits Policy.

Validator exit requests are signaled to all Node Operators via the on-chain Ejector Oracle reports multiple times a day. Node Operators have a duty to set up infrastructure to capture signaled validator exit requests and process relevant (i.e. pertaining to their validators) requests as soon as possible.

Node Operators participating in the Lido curated registry have a duty to exit validators in a timely manner when they are requested to do so. The actual mechanisms for validators to be exited are at the discretion of Node Operators. Open source tooling is available to aid Node Operators in identifying signaled validator exit requests, as well as in processing these requests.

Monitoring

The below are excerpts from the policy. Please consult the full policy for details.

If Node Operators are processing signaled validator exit requests as soon as they are available, the shortest possible time for a validator exit request to go from “signaled” to “processed” will be somewhere within the range of a few minutes to an hour. With respect to validator exit performance, each Node Operator may be considered to have one of the below three statuses.

  • In good standing - validator exit requests are being processed fully, correctly, and timely.
  • Delayed - validator exit requests are being processed incompletely, incorrectly, or not within the desired time frame.
  • Delinquent - validator exit requests are being processed incompletely, incorrectly, or not within the maximum acceptable time frame.
    • If a Node Operator has a status of Delayed, contributors from the NOM workstream will raise an issue in internal communications with the Node Operator and request remediative action.
    • If a Node Operator has a status of Delayed or Delinquent, the Exit Bus Oracle will assume that the Node Operator is unresponsive and re-route new incoming validator exit requests to operators that are not considered delinquent or delayed. Due to the re-routing of validator exit requests, the DAO should consider (via an ad-hoc vote) overriding the total limit of active validators for the relevant Node Operator such that if/when they resume a status of in good standing, they are not benefiting at the expense of Node Operators who took over the processing of the re-routed exits requests.
    • Once a Delinquent Node Operator has processed all signalled validator exit requests (and thus their number of Delinquent validators in next Accounting Oracle report is updated to 0), they will recommence receiving validator exit requests. Their status shall revert to “In good standing” after 5 days (i.e. provided any newly received validator exit requests are processed timely). During this 5 day “cooldown period” they will continue to not receive new stake and receive halved rewards.

Penalties

A node operator may be penalized if they have strictly more “stuck” validators than “refunded” validators (see reimbursement section of policy). While this condition is met, the Node Operator receives only half of the rewards and does not receive any new stake. Once the Node Operator manages to either withdraw the required number of validators or compensates for the lost validators and increases the “refunded” count through DAO voting, the node operator is considered under penalty for STUCK_PENALTY_DELAY seconds and then returns to the normal state. Rewards are automatically restored to normal, but in order to start to receive new stake, the Node Operator (or anyone else) have to call the permissionless method clearNodeOperatorPenalty.

đź“„
Lido Validator Exits Policy Status: Ratified (1.0) Validator Exits Policy 1.0 (IPFS), HackMD (current) Discussion thread

NO Tooling

  • Key API Service - a Node Operator-hosted API that loads the operator’s public validator keys, calculates the next keys to be exited, and requests via other tooling (e.g. eth-do, a custom script for web3 signer, etc) signed VEMs to be stored (as pre-signed messages) + made available to the Validator Ejector.
  • Validator Ejector - a daemon that listens for Ejector Oracle reports, retrieves pre-signed exit messages for exiting validators, and processes the validator exit requests (i.e. submits the VEMs to the CL for broadcast).
  • Lido-CLI - a CLI to interact with various aspects of the Lido on Ethereum protocol. Useful for Node Operators to grab data about protocol activity and status, and numerous details about validator exits statuses.
đź“–
Please refer to the below guide for details and an overview of a reference ansible implementation Link: NOTE (this replace the previous doc in notion)
What's Changing for Node Operators in Lido V2 - HackMD

# What's Changing for Node Operators in Lido V2 ## Intro [Lido V2](https://blog.lido.fi/introducin

hackmd.io

What's Changing for Node Operators in Lido V2 - HackMD

Other tooling / scripts / resources

  • Some Node Operators are additional running Oracles and/or DSM Guardian Daemons — they’re not relevant to everyone, so if you’ve never heard of them before you can ignore them.
🎥
See below for recordings of various deep-dive sessions!

Resources

Name
Description
Link
Validator Exits policy
The Lido Policy regarding exits
https://hackmd.io/zHYFZr4eRGm3Ju9_vkcSgQ#B3-Policy-Statement
Validator exits discussion
Governance forum post with all relevant discussions leading up to the policy
https://research.lido.fi/t/lido-validator-exits-policy-draft-for-discussion/3864
Goerli V2 PSA
Segment in NOM Community Call #4, introducing the steps necessary for the upcoming Goerli hardfork
https://www.youtube.com/watch?v=hms1Cln1Z0E&t=152s
Ejector Oracle Specs
Technical specs on how the Ejector Oracle will function
https://hackmd.io/PbUr-orDTAqH63F4-n8-pw?view#Ejector-oracle
New tooling works
Information on the works necessary to set up the suggested tooling
https://hackmd.io/jXgXjSIbQ1a_fUJeyUyL2A
Exits Deep-Dive #1
Recording of one of the 2 technical deep-dives into exits. Aimed at Node operators.
https://drive.google.com/file/d/153KSfnWxH0x7V_xH961N7N38NICtcUPf/view?usp=share_link
Exits Deep-Dive #2
Recording of one of the 2 technical deep-dives into exits. Aimed at Node operators.
https://drive.google.com/file/d/1RgNZEpNpwur5wYlv5TAzKQ4Q9CpsYctl/view?usp=share_link
Zheijiang exit walkthrough #1
Recording of one of the 2 technical deep-dives showing validator exit flow.
Zoom recording (online) (passcode .K0V#wj9) Google drive (video, chat transcript)
Zheijiang exit walkthrough #2
Recording of one of the 2 technical deep-dives showing validator exit flow.
Zoom Recording (online) (passcode 5A9@i#=5) Google drive (video, chat transcript)
Lido keys api repo
KAPI (Key API) repository
https://github.com/lidofinance/lido-keys-api
Lido Validator Ejector repo
Validator Ejector (eventlistening daemon) repository
https://github.com/lidofinance/validator-ejector
Lido CLI repo
https://github.com/lidofinance/lido-cli
image

Follow & Contribute

https://twitter.com/LidoFinance
https://discord.com/invite/lido
https://github.com/lidofinance

‣
Hide
Logo

Terms of Use

Operators

Ethereum Node Operators

Node Operators Database

Modules

Curated

Simple DVT

Community Staking

Other

Operators Stats & Metrics

Community Lifeguards Initiative

ValSet Monthly Updates

XDiscordGitHub