From 013d0e7edeb8fd6cc9245637968cb17a2ad7e5f1 Mon Sep 17 00:00:00 2001 From: Joe Date: Thu, 12 Oct 2023 15:20:20 +0200 Subject: [PATCH 01/30] feature-gate-threshold-automation init --- .../0066-feature-gate-threshold-automation.md | 245 ++++++++++++++++++ 1 file changed, 245 insertions(+) create mode 100644 proposals/0066-feature-gate-threshold-automation.md diff --git a/proposals/0066-feature-gate-threshold-automation.md b/proposals/0066-feature-gate-threshold-automation.md new file mode 100644 index 000000000..f68bc8014 --- /dev/null +++ b/proposals/0066-feature-gate-threshold-automation.md @@ -0,0 +1,245 @@ +--- +simd: '0066' +title: Feature Gate Threshold Automation +authors: + - Tyera Eulburg + - Joe Caulfield +category: Standard +type: Core +status: Draft +created: 2023-10-11 +feature: (fill in with feature tracking issues once accepted) +--- + +## Summary + +This SIMD outlines a proposal for automating the feature activation process +based on a stake-weighted support threshold, rather than manual human action. + +## Motivation + +Feature gates wrap new cluster functionality, and typically change the rules of +consensus. As such, a feature gate needs to be supported by a strong majority +of cluster stake when it is activated, or else it risks partitioning the +network. The current feature-gate system comprises two steps: + +1. An individual key-holder queues a feature gate for activation +2. The runtime automatically activates the feature on the next epoch boundary + +The key-holder is the one who (with the help of the solana-cli) assesses the +amount of stake that recognizes the feature and decides whether it is safe to +activate. This is obviously brittle and subject to human error. + +This SIMD proposes that the runtime itself replaces the human to assess the +amount of stake that supports a particular queued feature and only +automatically activates it when a preset threshold is met. + +## New Terminology + +- Feature Gate program: The BPF program that will own all feature accounts and +possess the capability to revoke them. +- Feature-Request account: The PDA under the Feature Gate program used to track +features requested for activation. +- Feature-Queue account: The PDA under the Feature Gate program used to track +features queued for activation that must have support signaled by nodes. + +## Detailed Design + +Before this SIMD is published, we will deploy a BPF program to manage feature +activations, which will allow us to revoke pending feature activations +(previously not possible). +The feature activation process will otherwise be unchanged, so no SIMD is +required. +This program will be upgradeable; the authority and process through which it +will be upgraded is outside the scope of this SIMD. Since this SIMD requires an +update to the program, the upgrade details will be determined in parallel. + +The initial Feature Gate Program can be found in this pull request to the +Solana Program Library: +. +The runtime changes and feature activation that will enable this program can be +found in this feature tracking issue: +. + +### Overview + +Consider two arbitrary versions of the Solana runtime, where the newer version +has more features in its feature set. + +| v1.14.25 | v1.16.0 | +| --------- | --------- | +| Feature A | Feature A | +| | Feature B | +| Feature C | Feature C | +| | Feature D | + +The runtime should activate each of these features when a strong majority of +the network supports it. “Support” for a feature means that a node is running a +version of the Solana runtime where the code required for the feature is +present. For instance, on a cluster with 50% of stake running v1.14.25 and 50% +of stake running v1.16.0, only Features A and C should be eligible for +activation. + +This SIMD proposes 3 steps over the span of one epoch to determine which +feature gates are queued for activation and assess stake support: + +1. On an epoch boundary, the runtime generates a list of feature-gates queued +for activation. +2. Validators signal which of the queued feature-gates they support. +3. On the next epoch boundary, the runtime activates the feature-gates that +have the necessary stake support + +### Representing the Current Feature Set + +As mentioned above, a BPF program is being deployed that will replace the +non-existent account `Feature111111111111111111111111111111111111`, which is +the owner of all feature accounts. + +The Feature Gate program will utilize two PDAs to track the authoritative +feature set queued for activation. One PDA (the Feature-Request PDA) will +store newly requested feature activations and the other (the Feature-Queue +PDA) will store the previous epoch’s requested activations. + +When a new epoch begins, the Feature-Request PDA will be an empty list. When +key-holders submit the CLI command to activate a feature, the Feature Gate +Program appends the feature ID to the Feature-Request set at the same time as +creating the feature account . Revoked features are removed from the +Feature-Request list. At the end of the epoch, the Feature-Request set is +written to the feature queue PDA, where it becomes immutable. The +Feature-Request PDA is then reset to an empty list again. + +The newly created immutable Feature-Queue can then be used by nodes to signal +support for potential activation in the next epoch. + +Note that for any given epoch, the epoch will begin with an immutable set of +previously requested feature activations, and end with the activation of those +with enough support. This process takes one epoch. Therefore, it will take at +least one more epoch than it currently does from the time a key-holder submits +a feature for activation via CLI to when it is actually activated. + +### Signaling Support for Features + +With an on-chain reference point to determine the features queued for +activation, nodes can signal support for specific features in the queue. + +A node signals its support for a feature set by sending a transaction to the +Feature Gate program, signed by their authorized voter, to store a record of +their supported feature set. +This supported feature set would be a bit mask of the list of queued features, +and it would be stored in a PDA derived from their vote address. +Nodes simply examine the Feature-Queue list, and mark a 1 for any features they +have in their feature set. The rest are marked 0. + +In the previous example, a node running v1.14.25 would signal the following bit +mask: + +``` +1 0 1 0 +``` + +Similarly, a node running v1.16.0 would signal: + +``` +1 1 1 1 +``` + +Validators send these transactions signaling their support for a queued set of +features on the first slot of every epoch and on startup after any reboot. + +Note: If a feature is revoked before it is moved to the immutable Feature-Queue +list, it is simply removed from the Feature-Request list and never presented to +nodes. However, if a feature is revoked after it becomes part of the immutable +Feature-Queue list, it remains in the list, and nodes still signal their +support if the feature ID is present in their software. However, the runtime +won’t activate this feature if its corresponding feature account no longer +exists on-chain. +This is how the runtime currently handles features and can be observed here: +. + +### Activating Supported Features + +Once the epoch concludes, the runtime uses the validator support signals to +activate the proper features. + +To do this, the runtime walks all of the PDAs derived from the vote accounts, +examines their bit-masks, and sums the stake supporting each feature-gate to +determine which feature gates meet the desired threshold of supported stake. + +For the initial implementation, the threshold will be hard-coded to 95% of +stake, but future iterations on the Feature Program could allow feature +key-holders to set a custom threshold for each feature gate +(including 0%, ie. –yolo). + +Once a list of stake-supported features is generated, the runtime rechecks the +Feature accounts in case any have been revoked. Then the runtime processes the +activations using the existing feature logic. + +At this point, the runtime clears the existing validator support PDA accounts +and returns to step 1 to generate the next list of queued features. + +### Conclusion + +The current familiar process of activating a feature with +`solana feature activate` will now merely suggest this feature be considered +for activation by the aforementioned mechanism. In other words, it’s no longer +a guarantee that your feature will be activated when you run this command. + +Instead, the runtime will decide through stake whether or not a feature should +be activated. This will be especially useful in a world where two separate +clients will aim to push features and want to agree on their activation. + +## Alternatives Considered + +### Representing the Current Feature Set (Alternative) + +An alternate approach to the Feature Gate program’s on-chain storage is to +instead use one single PDA to store the queue and reset it every epoch. This +would allow us to use one less account and one less epoch, but it would make +things trickier when it comes to the timing of validators signaling support for +features. + +For example, a key-holder might submit a feature for activation at a point late +in the epoch, so validators would need to be able to signal support for this +feature whenever it is submitted. This means validators would have to +consistently check the feature queue for changes, and possibly issue more +signals in the epoch. + +This approach may increase the total required work for a node in a given epoch. + +### Signaling Support for Features (Alternative) + +Rather than sending a transaction to the Feature Gate program, a node could +instead signal the bit mask in their block header whenever they are the leader. + +This would introduce a change to the block header and remove this information +from being publicly and transparently on-chain. + +## Impact + +This new process for activating features directly impacts core contributors and +validators. + +Core contributors will no longer bear the responsibility of ensuring the proper +stake supports their feature activation. However, it will change the process +by which they can override this stake requirement to push a feature through. + +Validators will be responsible for signaling their vote using a transaction +which they've previously not included in their process. They also will have a +more significant impact on feature activations if they neglect to upgrade their +software version. + +## Security Considerations + +This proposal increases security for feature activations by removing the human +element from ensuring the proper stake supports a feature. + +However, it also adds a dependency on a BPF program and the data within its +owned accounts to perform feature activations, which previously has not been +the case. This may introduce some security concerns around this relationship +between the runtime and the Feature Gate program. + +This proposal could also potentially extend the length of time required for +integrating feature-gated changes, which may include security fixes. However, +the feature-gate process is relatively slow in its current state, and neither +the current process or this proposed process would have any implications for +critical, time-sensitive issues. From 9b601c14072f3a1cf588f08534696867be456b22 Mon Sep 17 00:00:00 2001 From: Joe Date: Thu, 19 Oct 2023 11:54:57 +0200 Subject: [PATCH 02/30] bump to 0072 --- ...-automation.md => 0072-feature-gate-threshold-automation.md} | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) rename proposals/{0066-feature-gate-threshold-automation.md => 0072-feature-gate-threshold-automation.md} (99%) diff --git a/proposals/0066-feature-gate-threshold-automation.md b/proposals/0072-feature-gate-threshold-automation.md similarity index 99% rename from proposals/0066-feature-gate-threshold-automation.md rename to proposals/0072-feature-gate-threshold-automation.md index f68bc8014..efab74b6f 100644 --- a/proposals/0066-feature-gate-threshold-automation.md +++ b/proposals/0072-feature-gate-threshold-automation.md @@ -1,5 +1,5 @@ --- -simd: '0066' +simd: '0072' title: Feature Gate Threshold Automation authors: - Tyera Eulburg From 24bc2c5133804df23e24f112d13292711c7d5e24 Mon Sep 17 00:00:00 2001 From: Joe Date: Thu, 19 Oct 2023 12:31:26 +0200 Subject: [PATCH 03/30] address minor corrections --- .../0072-feature-gate-threshold-automation.md | 24 ++++++++++--------- 1 file changed, 13 insertions(+), 11 deletions(-) diff --git a/proposals/0072-feature-gate-threshold-automation.md b/proposals/0072-feature-gate-threshold-automation.md index efab74b6f..8788e3545 100644 --- a/proposals/0072-feature-gate-threshold-automation.md +++ b/proposals/0072-feature-gate-threshold-automation.md @@ -36,8 +36,7 @@ automatically activates it when a preset threshold is met. ## New Terminology -- Feature Gate program: The BPF program that will own all feature accounts and -possess the capability to revoke them. +- Feature Gate program: The BPF program that will own all feature accounts. - Feature-Request account: The PDA under the Feature Gate program used to track features requested for activation. - Feature-Queue account: The PDA under the Feature Gate program used to track @@ -74,11 +73,13 @@ has more features in its feature set. | | Feature D | The runtime should activate each of these features when a strong majority of -the network supports it. “Support” for a feature means that a node is running a -version of the Solana runtime where the code required for the feature is -present. For instance, on a cluster with 50% of stake running v1.14.25 and 50% -of stake running v1.16.0, only Features A and C should be eligible for -activation. +the network supports it. +"Strong majority" in this context is defined by the current feature activation +process as 95% of stake. +“Support” for a feature means that a node is running a version of the Solana +runtime where the code required for the feature is present. For instance, on a +cluster with 50% of stake running v1.14.25 and 50% of stake running v1.16.0, +only Features A and C should be eligible for activation. This SIMD proposes 3 steps over the span of one epoch to determine which feature gates are queued for activation and assess stake support: @@ -105,8 +106,9 @@ key-holders submit the CLI command to activate a feature, the Feature Gate Program appends the feature ID to the Feature-Request set at the same time as creating the feature account . Revoked features are removed from the Feature-Request list. At the end of the epoch, the Feature-Request set is -written to the feature queue PDA, where it becomes immutable. The -Feature-Request PDA is then reset to an empty list again. +written to the feature queue PDA by the runtime, where it becomes immutable. +The Feature-Request PDA is then reset to an empty list again, also by the +runtime. The newly created immutable Feature-Queue can then be used by nodes to signal support for potential activation in the next epoch. @@ -122,7 +124,7 @@ a feature for activation via CLI to when it is actually activated. With an on-chain reference point to determine the features queued for activation, nodes can signal support for specific features in the queue. -A node signals its support for a feature set by sending a transaction to the +A node signals its support for a feature set by sending an instruction to the Feature Gate program, signed by their authorized voter, to store a record of their supported feature set. This supported feature set would be a bit mask of the list of queued features, @@ -162,7 +164,7 @@ Once the epoch concludes, the runtime uses the validator support signals to activate the proper features. To do this, the runtime walks all of the PDAs derived from the vote accounts, -examines their bit-masks, and sums the stake supporting each feature-gate to +examines their bitmasks, and sums the stake supporting each feature-gate to determine which feature gates meet the desired threshold of supported stake. For the initial implementation, the threshold will be hard-coded to 95% of From fa1f218c7ee059576346ab937f53c309e2ed23f7 Mon Sep 17 00:00:00 2001 From: Joe Date: Thu, 14 Dec 2023 21:36:56 -0500 Subject: [PATCH 04/30] revise based on related SIMDs --- .../0072-feature-gate-threshold-automation.md | 77 ++++++++----------- 1 file changed, 32 insertions(+), 45 deletions(-) diff --git a/proposals/0072-feature-gate-threshold-automation.md b/proposals/0072-feature-gate-threshold-automation.md index 8788e3545..96f78c220 100644 --- a/proposals/0072-feature-gate-threshold-automation.md +++ b/proposals/0072-feature-gate-threshold-automation.md @@ -36,7 +36,9 @@ automatically activates it when a preset threshold is met. ## New Terminology -- Feature Gate program: The BPF program that will own all feature accounts. +- Feature Gate program: The Core BPF program introduced in + [SIMD 0089](https://github.com/solana-foundation/solana-improvement-documents/pull/89) + that will own all feature accounts. - Feature-Request account: The PDA under the Feature Gate program used to track features requested for activation. - Feature-Queue account: The PDA under the Feature Gate program used to track @@ -44,24 +46,6 @@ features queued for activation that must have support signaled by nodes. ## Detailed Design -Before this SIMD is published, we will deploy a BPF program to manage feature -activations, which will allow us to revoke pending feature activations -(previously not possible). -The feature activation process will otherwise be unchanged, so no SIMD is -required. -This program will be upgradeable; the authority and process through which it -will be upgraded is outside the scope of this SIMD. Since this SIMD requires an -update to the program, the upgrade details will be determined in parallel. - -The initial Feature Gate Program can be found in this pull request to the -Solana Program Library: -. -The runtime changes and feature activation that will enable this program can be -found in this feature tracking issue: -. - -### Overview - Consider two arbitrary versions of the Solana runtime, where the newer version has more features in its feature set. @@ -81,34 +65,38 @@ runtime where the code required for the feature is present. For instance, on a cluster with 50% of stake running v1.14.25 and 50% of stake running v1.16.0, only Features A and C should be eligible for activation. -This SIMD proposes 3 steps over the span of one epoch to determine which -feature gates are queued for activation and assess stake support: +This SIMD proposes the following steps for activating features: -1. On an epoch boundary, the runtime generates a list of feature-gates queued -for activation. -2. Validators signal which of the queued feature-gates they support. -3. On the next epoch boundary, the runtime activates the feature-gates that -have the necessary stake support +1. In some epoch 0, keyholders submit features for activation using the Solana + CLI. +2. On the epoch boundary, the runtime generates a list of feature-gates queued + for activation. +3. During epoch 1, validators signal which of the queued feature-gates they + support in their software. +4. On the next epoch boundary, the runtime activates the feature-gates that have + the necessary stake support. ### Representing the Current Feature Set -As mentioned above, a BPF program is being deployed that will replace the -non-existent account `Feature111111111111111111111111111111111111`, which is -the owner of all feature accounts. +As mentioned above, the Feature Gate program at address +`Feature111111111111111111111111111111111111` is the owner of all feature +accounts. The Feature Gate program shall utilize two PDAs to keep track of all +feature-gates: -The Feature Gate program will utilize two PDAs to track the authoritative -feature set queued for activation. One PDA (the Feature-Request PDA) will -store newly requested feature activations and the other (the Feature-Queue -PDA) will store the previous epoch’s requested activations. +- **Feature-Request PDA**: Stores a *mutable* list of newly requested feature + activations submitted during the current epoch. +- **Feature-Queue PDA**: Stores an *immutable* list of the previous epoch's + requested activations. When a new epoch begins, the Feature-Request PDA will be an empty list. When key-holders submit the CLI command to activate a feature, the Feature Gate Program appends the feature ID to the Feature-Request set at the same time as creating the feature account . Revoked features are removed from the -Feature-Request list. At the end of the epoch, the Feature-Request set is -written to the feature queue PDA by the runtime, where it becomes immutable. -The Feature-Request PDA is then reset to an empty list again, also by the -runtime. +Feature-Request list. + +At the end of the epoch, the Feature-Request set is written to the Feature-Queue +PDA by the runtime, where it becomes immutable. The Feature-Request PDA is then +reset to an empty list again, also by the runtime. The newly created immutable Feature-Queue can then be used by nodes to signal support for potential activation in the next epoch. @@ -167,7 +155,7 @@ To do this, the runtime walks all of the PDAs derived from the vote accounts, examines their bitmasks, and sums the stake supporting each feature-gate to determine which feature gates meet the desired threshold of supported stake. -For the initial implementation, the threshold will be hard-coded to 95% of +For the initial implementation, the threshold shall be hard-coded to 95% of stake, but future iterations on the Feature Program could allow feature key-holders to set a custom threshold for each feature gate (including 0%, ie. –yolo). @@ -176,8 +164,12 @@ Once a list of stake-supported features is generated, the runtime rechecks the Feature accounts in case any have been revoked. Then the runtime processes the activations using the existing feature logic. -At this point, the runtime clears the existing validator support PDA accounts -and returns to step 1 to generate the next list of queued features. +At this point, the runtime performs a garbage collection process and returns to +step 1 to generate the next list of queued features. More on this process below. + +### Garbage Collection + +TODO ### Conclusion @@ -235,11 +227,6 @@ software version. This proposal increases security for feature activations by removing the human element from ensuring the proper stake supports a feature. -However, it also adds a dependency on a BPF program and the data within its -owned accounts to perform feature activations, which previously has not been -the case. This may introduce some security concerns around this relationship -between the runtime and the Feature Gate program. - This proposal could also potentially extend the length of time required for integrating feature-gated changes, which may include security fixes. However, the feature-gate process is relatively slow in its current state, and neither From 4a6530372ac8bec2b2276a46b103b2356c26e9d9 Mon Sep 17 00:00:00 2001 From: Joe Date: Thu, 14 Dec 2023 21:44:25 -0500 Subject: [PATCH 05/30] add garbage collection process --- .../0072-feature-gate-threshold-automation.md | 38 +++++++++++++++++-- 1 file changed, 34 insertions(+), 4 deletions(-) diff --git a/proposals/0072-feature-gate-threshold-automation.md b/proposals/0072-feature-gate-threshold-automation.md index 96f78c220..0632862e9 100644 --- a/proposals/0072-feature-gate-threshold-automation.md +++ b/proposals/0072-feature-gate-threshold-automation.md @@ -36,13 +36,20 @@ automatically activates it when a preset threshold is met. ## New Terminology -- Feature Gate program: The Core BPF program introduced in +- **Feature Gate program**: The Core BPF program introduced in [SIMD 0089](https://github.com/solana-foundation/solana-improvement-documents/pull/89) that will own all feature accounts. -- Feature-Request account: The PDA under the Feature Gate program used to track +- **Feature-Request PDA**: The PDA under the Feature Gate program used to track features requested for activation. -- Feature-Queue account: The PDA under the Feature Gate program used to track +- **Feature-Queue PDA**: The PDA under the Feature Gate program used to track features queued for activation that must have support signaled by nodes. +- **Validator Support-Signal PDA**: The PDAs under the Feature Gate program + created by validator support signal transactions to track a vote account's + list of features supported. +- **Feature Gate Tombstone PDA**: The PDA under the Feature Gate program used to + assign as the owner of accounts no longer needed for the feature activation + process, effectively removing them from the Feature Gate program's owned + accounts list. ## Detailed Design @@ -169,7 +176,30 @@ step 1 to generate the next list of queued features. More on this process below. ### Garbage Collection -TODO +Since submitting features for activation, creating the Feature-Request and +Feature-Queue PDAs, and signaling support for features through PDAs will create +many accounts per epoch, a garbage collection process is required. + +This garbage collection process will involve the use of the Feature Gate +Tombstone account - an empty PDA of the Feature Gate program used to assign as +the owner of accounts no longer required by the feature activation process. + +On epoch rollover, the following accounts will be assigned to the Feature Gate +Tombstone PDA by the Feature Gate program, effectively removing them from its +list of owned accounts: + +- The previous epoch's Feature-Request and Feature-Queue PDAs +- The previous epoch's Validator Support-Signal PDAs +- Any other feature accounts unused when the current epoch's Feature-Queue PDA + is created + +Note: The tombstone shall be a PDA under the Feature Gate program to preserve +the accounts in case a record of any feature activation is necessary to be +queried in the future. + +With this garbage collection in place, the runtime can efficiently minimize the +scope of accounts that are required to be loaded to perform activations on epoch +rollover. ### Conclusion From a7d6a49b72b1a65e6fc7fa6a12b46d1531778358 Mon Sep 17 00:00:00 2001 From: Joe Date: Thu, 14 Dec 2023 21:53:41 -0500 Subject: [PATCH 06/30] add PDA details --- .../0072-feature-gate-threshold-automation.md | 46 ++++++++++++++----- 1 file changed, 35 insertions(+), 11 deletions(-) diff --git a/proposals/0072-feature-gate-threshold-automation.md b/proposals/0072-feature-gate-threshold-automation.md index 0632862e9..c5df56ddf 100644 --- a/proposals/0072-feature-gate-threshold-automation.md +++ b/proposals/0072-feature-gate-threshold-automation.md @@ -95,15 +95,17 @@ feature-gates: - **Feature-Queue PDA**: Stores an *immutable* list of the previous epoch's requested activations. -When a new epoch begins, the Feature-Request PDA will be an empty list. When -key-holders submit the CLI command to activate a feature, the Feature Gate -Program appends the feature ID to the Feature-Request set at the same time as -creating the feature account . Revoked features are removed from the -Feature-Request list. - -At the end of the epoch, the Feature-Request set is written to the Feature-Queue -PDA by the runtime, where it becomes immutable. The Feature-Request PDA is then -reset to an empty list again, also by the runtime. +When a new epoch begins, a new Feature-Request PDA will be created, thus +initializing an empty list. When key-holders submit the CLI command to activate +a feature, the Feature Gate program appends the feature ID to the +Feature-Request set at the same time as creating the feature account. Revoked +features are removed from the Feature-Request list when the Feature Gate +program's `RevokePendingActivation` instruction is invoked. + +At the end of the epoch, the Feature-Request set is written to a new +Feature-Queue PDA (for the next epoch) by the runtime, where it becomes +immutable. A new empty-list Feature-Request PDA is then created for the next +epoch, also by the runtime. The newly created immutable Feature-Queue can then be used by nodes to signal support for potential activation in the next epoch. @@ -114,6 +116,18 @@ with enough support. This process takes one epoch. Therefore, it will take at least one more epoch than it currently does from the time a key-holder submits a feature for activation via CLI to when it is actually activated. +Proposed Program-Derived Address for Feature-Request PDA: + +``` +"feature_request" + +``` + +Proposed Program-Derived Address for Feature-Queue PDA: + +``` +"feature_queue" + +``` + ### Signaling Support for Features With an on-chain reference point to determine the features queued for @@ -140,8 +154,12 @@ Similarly, a node running v1.16.0 would signal: 1 1 1 1 ``` -Validators send these transactions signaling their support for a queued set of -features on the first slot of every epoch and on startup after any reboot. +Validators shall send transactions containing the Feature Gate program's +`SignalSupportForFeatureSet` instruction, which would contain their bit mask, on +the first slot of every epoch and on startup after any reboot. When this +instruction is invoked, the Feature Gate program stores the submitted bit mask +in a PDA derived from the validator's vote account. This effectively signals a +node's support for a queued set of features. Note: If a feature is revoked before it is moved to the immutable Feature-Queue list, it is simply removed from the Feature-Request list and never presented to @@ -153,6 +171,12 @@ exists on-chain. This is how the runtime currently handles features and can be observed here: . +Proposed Program-Derived Address for Validator Support-Signal PDA: + +``` +"validator_support" + + +``` + ### Activating Supported Features Once the epoch concludes, the runtime uses the validator support signals to From 843c83e296a70412e49ec2b25e4244250ffb92f3 Mon Sep 17 00:00:00 2001 From: Joe Date: Thu, 14 Dec 2023 22:02:38 -0500 Subject: [PATCH 07/30] revise signal cadence --- .../0072-feature-gate-threshold-automation.md | 21 ++++++++++++++----- 1 file changed, 16 insertions(+), 5 deletions(-) diff --git a/proposals/0072-feature-gate-threshold-automation.md b/proposals/0072-feature-gate-threshold-automation.md index c5df56ddf..c349b12eb 100644 --- a/proposals/0072-feature-gate-threshold-automation.md +++ b/proposals/0072-feature-gate-threshold-automation.md @@ -155,11 +155,22 @@ Similarly, a node running v1.16.0 would signal: ``` Validators shall send transactions containing the Feature Gate program's -`SignalSupportForFeatureSet` instruction, which would contain their bit mask, on -the first slot of every epoch and on startup after any reboot. When this -instruction is invoked, the Feature Gate program stores the submitted bit mask -in a PDA derived from the validator's vote account. This effectively signals a -node's support for a queued set of features. +`SignalSupportForFeatureSet` instruction, which would contain their bit mask, at +some aritrary point during the epoch and on startup after any reboot. + +Note that a validator client's particular implementation may use their own +discretion to determine the appropriate cadence at which to send this +transaction during the course of an epoch, with the following restrictions: + +- The signaling transaction **must** be submitted after any reboot. +- No signaling transactions will be considered after **128 slots before the + epoch boundary**. This cut-off is to ensure proper caching. +- The **last signal submitted** before the 128-slot cut-off will be the final + signal considered in the activation phase. + +When the `SignalSupportForFeatureSet` instruction is invoked, the Feature Gate +program stores the submitted bit mask in a PDA derived from the validator's vote +account. This effectively signals a node's support for a queued set of features. Note: If a feature is revoked before it is moved to the immutable Feature-Queue list, it is simply removed from the Feature-Request list and never presented to From e513e828369df335249f23c12cebe4a7318bc5e6 Mon Sep 17 00:00:00 2001 From: Joe Date: Thu, 14 Dec 2023 22:14:38 -0500 Subject: [PATCH 08/30] add instruction layout example --- .../0072-feature-gate-threshold-automation.md | 31 +++++++++++++++++++ 1 file changed, 31 insertions(+) diff --git a/proposals/0072-feature-gate-threshold-automation.md b/proposals/0072-feature-gate-threshold-automation.md index c349b12eb..4b0c32464 100644 --- a/proposals/0072-feature-gate-threshold-automation.md +++ b/proposals/0072-feature-gate-threshold-automation.md @@ -158,6 +158,37 @@ Validators shall send transactions containing the Feature Gate program's `SignalSupportForFeatureSet` instruction, which would contain their bit mask, at some aritrary point during the epoch and on startup after any reboot. +Consider the instruction as it may appear in the Feature Gate program: + +```rust +pub enum FeatureGateInstruction { + /// Signal support for a feature set. + /// + /// This instruction submits a bit mask representing the current epoch's + /// feature queue. + /// + /// Validators submit these bit masks to describe feature-gates supported + /// by their current client software version. + /// + /// A `1` value represents support for the feature at that index of the + /// bit mask, while a `0` represents a lack of support (or rejection). + /// A "pending" feature activation is a feature account that has been + /// allocated and assigned, but hasn't yet been updated by the runtime + /// with an `activation_slot`. + /// + /// Accounts expected by this instruction: + /// + /// 0. `[w]` Validator Support-Signal PDA + /// 1. `[s]` Vote account + SignalSupportForFeatureSet { + bit_mask: Vec, + }, +} +``` + +Validators are also required to prepend an instruction to fund this account with +rent-exempt lamports if it does not exist prior to signaling. + Note that a validator client's particular implementation may use their own discretion to determine the appropriate cadence at which to send this transaction during the course of an epoch, with the following restrictions: From d705e54e3352a7004c1b7b56f845053073fcd5a8 Mon Sep 17 00:00:00 2001 From: Joe Date: Thu, 14 Dec 2023 22:31:08 -0500 Subject: [PATCH 09/30] add state layout examples --- .../0072-feature-gate-threshold-automation.md | 39 ++++++++++++++++--- 1 file changed, 33 insertions(+), 6 deletions(-) diff --git a/proposals/0072-feature-gate-threshold-automation.md b/proposals/0072-feature-gate-threshold-automation.md index 4b0c32464..9b6513830 100644 --- a/proposals/0072-feature-gate-threshold-automation.md +++ b/proposals/0072-feature-gate-threshold-automation.md @@ -122,12 +122,29 @@ Proposed Program-Derived Address for Feature-Request PDA: "feature_request" + ``` +Proposed state for Feature-Request PDA: + +```rust +struct FeatureRequest { + requested_features: Vec, +} +``` + Proposed Program-Derived Address for Feature-Queue PDA: ``` "feature_queue" + ``` +Proposed state for Feature-Queue PDA: + +```rust +struct FeatureQueue { + feature_queue: Vec, +} +``` + + ### Signaling Support for Features With an on-chain reference point to determine the features queued for @@ -203,6 +220,22 @@ When the `SignalSupportForFeatureSet` instruction is invoked, the Feature Gate program stores the submitted bit mask in a PDA derived from the validator's vote account. This effectively signals a node's support for a queued set of features. + +Proposed Program-Derived Address for Validator Support-Signal PDA: + +``` +"validator_support" + + +``` + +Proposed state for Validator Support-Signal PDA: + +```rust +struct ValidatorSupportSignal { + bit_mask: Vec, +} +``` + + Note: If a feature is revoked before it is moved to the immutable Feature-Queue list, it is simply removed from the Feature-Request list and never presented to nodes. However, if a feature is revoked after it becomes part of the immutable @@ -213,12 +246,6 @@ exists on-chain. This is how the runtime currently handles features and can be observed here: . -Proposed Program-Derived Address for Validator Support-Signal PDA: - -``` -"validator_support" + + -``` - ### Activating Supported Features Once the epoch concludes, the runtime uses the validator support signals to From a63abbbcaa9f17a1e49fb781b0f34e527060a076 Mon Sep 17 00:00:00 2001 From: Joe Date: Thu, 14 Dec 2023 22:47:09 -0500 Subject: [PATCH 10/30] add multi-sig activation gate --- .../0072-feature-gate-threshold-automation.md | 48 ++++++++++++++++++- 1 file changed, 46 insertions(+), 2 deletions(-) diff --git a/proposals/0072-feature-gate-threshold-automation.md b/proposals/0072-feature-gate-threshold-automation.md index 9b6513830..30e19285f 100644 --- a/proposals/0072-feature-gate-threshold-automation.md +++ b/proposals/0072-feature-gate-threshold-automation.md @@ -74,8 +74,8 @@ only Features A and C should be eligible for activation. This SIMD proposes the following steps for activating features: -1. In some epoch 0, keyholders submit features for activation using the Solana - CLI. +1. In some epoch 0, authorized contributors submit features for activation using + the Solana CLI. 2. On the epoch boundary, the runtime generates a list of feature-gates queued for activation. 3. During epoch 1, validators signal which of the queued feature-gates they @@ -83,6 +83,50 @@ This SIMD proposes the following steps for activating features: 4. On the next epoch boundary, the runtime activates the feature-gates that have the necessary stake support. +### Authorizing Feature Submissions + +Currently, anyone can create a feature account using the Solana CLI with the +command `solana feature activate`. This command will send a transaction with the +necessary instructions to allocate the proper space for a feature account and +assign it to `Feature111111111111111111111111111111111111`. + +In order to protect against spam, this command shall now require a signature from +a multi-signature authority, with keyholders from each validator client: Solana +Labs, Jito, and Jump. + +This command shall now send a transaction with the following instructions: + +- Fund the feature account with rent-exempt lamports for feature state +- Allocate the proper space for feature state +- Assign the account to `Feature111111111111111111111111111111111111` +- Invoke the Feature Gate program's `SubmitFeatureForActivation` instruction + +When the `SubmitFeatureForActivation` instruction is invoked, the Feature Gate +program will write the proper feature state into the account. + +Consider the instruction as it may appear in the Feature Gate program: + +```rust +pub enum FeatureGateInstruction { + /// Submit a feature for activation. + /// + /// Features may only be submitted for activation by the multi-signature + /// authority that presides over the feature activation process. + /// + /// Accounts expected by this instruction: + /// + /// 0. `[w]` Feature account + /// 1. `[s]` Multi-signature authority + SubmitFeatureForActivation, +} +``` + +Accounts owned by `Feature111111111111111111111111111111111111` will not be +recognized in the feature activation process without the proper state. + +The address of the multi-sig shall be hard-coded into the Feature Gate program +to validate the signature. + ### Representing the Current Feature Set As mentioned above, the Feature Gate program at address From 7a6c3fcc0773d17713ebd19d598290e04d95a8b3 Mon Sep 17 00:00:00 2001 From: Joe Date: Thu, 25 Jan 2024 17:19:44 -0600 Subject: [PATCH 11/30] add additional elaboration --- .../0072-feature-gate-threshold-automation.md | 47 ++++++++++++------- 1 file changed, 29 insertions(+), 18 deletions(-) diff --git a/proposals/0072-feature-gate-threshold-automation.md b/proposals/0072-feature-gate-threshold-automation.md index 30e19285f..8030ec16a 100644 --- a/proposals/0072-feature-gate-threshold-automation.md +++ b/proposals/0072-feature-gate-threshold-automation.md @@ -26,13 +26,13 @@ network. The current feature-gate system comprises two steps: 1. An individual key-holder queues a feature gate for activation 2. The runtime automatically activates the feature on the next epoch boundary -The key-holder is the one who (with the help of the solana-cli) assesses the -amount of stake that recognizes the feature and decides whether it is safe to -activate. This is obviously brittle and subject to human error. +The key-holder is the one who *manually* (with the help of the solana-cli) +assesses the amount of stake that recognizes the feature and decides whether +it is safe to activate. This is obviously brittle and subject to human error. -This SIMD proposes that the runtime itself replaces the human to assess the -amount of stake that supports a particular queued feature and only -automatically activates it when a preset threshold is met. +This SIMD proposes that the runtime itself replaces the human with an automated +process to assess the amount of stake that supports a particular queued feature +and only activate it when a preset threshold is met. ## New Terminology @@ -40,13 +40,13 @@ automatically activates it when a preset threshold is met. [SIMD 0089](https://github.com/solana-foundation/solana-improvement-documents/pull/89) that will own all feature accounts. - **Feature-Request PDA**: The PDA under the Feature Gate program used to track -features requested for activation. + features requested for activation. - **Feature-Queue PDA**: The PDA under the Feature Gate program used to track -features queued for activation that must have support signaled by nodes. + features queued for activation that must have support signaled by nodes. - **Validator Support-Signal PDA**: The PDAs under the Feature Gate program created by validator support signal transactions to track a vote account's list of features supported. -- **Feature Gate Tombstone PDA**: The PDA under the Feature Gate program used to +- **Feature Tombstone PDA**: The PDA under the Feature Gate program used to assign as the owner of accounts no longer needed for the feature activation process, effectively removing them from the Feature Gate program's owned accounts list. @@ -91,12 +91,12 @@ necessary instructions to allocate the proper space for a feature account and assign it to `Feature111111111111111111111111111111111111`. In order to protect against spam, this command shall now require a signature from -a multi-signature authority, with keyholders from each validator client: Solana -Labs, Jito, and Jump. +a multi-signature authority, with keyholders from Solana Labs and potentially +other validator clients. This command shall now send a transaction with the following instructions: -- Fund the feature account with rent-exempt lamports for feature state +- Fund the feature account with the required lamports deposit minimum - Allocate the proper space for feature state - Assign the account to `Feature111111111111111111111111111111111111` - Invoke the Feature Gate program's `SubmitFeatureForActivation` instruction @@ -121,11 +121,22 @@ pub enum FeatureGateInstruction { } ``` -Accounts owned by `Feature111111111111111111111111111111111111` will not be -recognized in the feature activation process without the proper state. +Since anyone can create accounts and assign them to +`Feature111111111111111111111111111111111111`, the Feature Gate program's +`SubmitFeatureForActivation` instruction provides a necessary step to protect +against spam. -The address of the multi-sig shall be hard-coded into the Feature Gate program -to validate the signature. +This instruction will: +- Check that the account's lamports balance meets the required minimum +- Check that the account has allocated enough space for the `Feature` state +- Check that the account is owned by + `Feature111111111111111111111111111111111111` +- Serialize an un-initialized `Feature` layout to the account data + +Only accounts which have been successfully processed by the Feature Gate +program's `SubmitFeatureForActivation` will have serialized data, therefore only +accounts owned by `Feature111111111111111111111111111111111111` with serialized +data will be recognized for feature activation. ### Representing the Current Feature Set @@ -372,8 +383,8 @@ This approach may increase the total required work for a node in a given epoch. Rather than sending a transaction to the Feature Gate program, a node could instead signal the bit mask in their block header whenever they are the leader. -This would introduce a change to the block header and remove this information -from being publicly and transparently on-chain. +This would introduce a change to the block header, which may prove to be a +difficult change to integrate. ## Impact From 23f1b24be5c4474c7630455c983370c4c582eb70 Mon Sep 17 00:00:00 2001 From: Joe Date: Mon, 5 Feb 2024 15:51:44 -0600 Subject: [PATCH 12/30] init new version of 0072 --- .../0072-feature-gate-threshold-automation.md | 422 +++++------------- 1 file changed, 105 insertions(+), 317 deletions(-) diff --git a/proposals/0072-feature-gate-threshold-automation.md b/proposals/0072-feature-gate-threshold-automation.md index 8030ec16a..3860b0186 100644 --- a/proposals/0072-feature-gate-threshold-automation.md +++ b/proposals/0072-feature-gate-threshold-automation.md @@ -7,7 +7,7 @@ authors: category: Standard type: Core status: Draft -created: 2023-10-11 +created: 2024-01-25 feature: (fill in with feature tracking issues once accepted) --- @@ -16,384 +16,171 @@ feature: (fill in with feature tracking issues once accepted) This SIMD outlines a proposal for automating the feature activation process based on a stake-weighted support threshold, rather than manual human action. +With this new process, contributors no longer have to assess stake support for a +feature before activation. Instead, the assessment is done by the runtime. + ## Motivation Feature gates wrap new cluster functionality, and typically change the rules of consensus. As such, a feature gate needs to be supported by a strong majority of cluster stake when it is activated, or else it risks partitioning the -network. The current feature-gate system comprises two steps: +network. The current feature-gate process involves two steps: -1. An individual key-holder queues a feature gate for activation +1. An individual key-holder stages a feature gate for activation 2. The runtime automatically activates the feature on the next epoch boundary The key-holder is the one who *manually* (with the help of the solana-cli) assesses the amount of stake that recognizes the feature and decides whether it is safe to activate. This is obviously brittle and subject to human error. -This SIMD proposes that the runtime itself replaces the human with an automated -process to assess the amount of stake that supports a particular queued feature -and only activate it when a preset threshold is met. +If instead the runtime was to assess this stake support for activating a +feature, this would eliminate the key-holder's responsibility to asses stake +support, reducing the risk of human error. + +In a world where multiple clients will aim to push features and seek to agree on +their activation, a more automated and secure process will be extremely +beneficial. ## New Terminology -- **Feature Gate program**: The Core BPF program introduced in +- **Feature Gate program:** The Core BPF program introduced in [SIMD 0089](https://github.com/solana-foundation/solana-improvement-documents/pull/89) that will own all feature accounts. -- **Feature-Request PDA**: The PDA under the Feature Gate program used to track - features requested for activation. -- **Feature-Queue PDA**: The PDA under the Feature Gate program used to track - features queued for activation that must have support signaled by nodes. -- **Validator Support-Signal PDA**: The PDAs under the Feature Gate program - created by validator support signal transactions to track a vote account's - list of features supported. -- **Feature Tombstone PDA**: The PDA under the Feature Gate program used to - assign as the owner of accounts no longer needed for the feature activation - process, effectively removing them from the Feature Gate program's owned - accounts list. +- **Staged Features PDA:** The PDA under the Feature Gate program used to track + features submitted for activation for a particular epoch. +- **Support Signal PDA:** The PDA under the Feature Gate program used to store + a bit mask of the staged features a node supports. +- **Feature Tombstone:** A static address with no on-chain account for assigning + accounts under, effectively "archiving" them and removing them from the + Feature Gate program's owned accounts. ## Detailed Design -Consider two arbitrary versions of the Solana runtime, where the newer version -has more features in its feature set. - -| v1.14.25 | v1.16.0 | -| --------- | --------- | -| Feature A | Feature A | -| | Feature B | -| Feature C | Feature C | -| | Feature D | - -The runtime should activate each of these features when a strong majority of -the network supports it. -"Strong majority" in this context is defined by the current feature activation -process as 95% of stake. -“Support” for a feature means that a node is running a version of the Solana -runtime where the code required for the feature is present. For instance, on a -cluster with 50% of stake running v1.14.25 and 50% of stake running v1.16.0, -only Features A and C should be eligible for activation. - -This SIMD proposes the following steps for activating features: - -1. In some epoch 0, authorized contributors submit features for activation using - the Solana CLI. -2. On the epoch boundary, the runtime generates a list of feature-gates queued - for activation. -3. During epoch 1, validators signal which of the queued feature-gates they - support in their software. -4. On the next epoch boundary, the runtime activates the feature-gates that have - the necessary stake support. - -### Authorizing Feature Submissions - -Currently, anyone can create a feature account using the Solana CLI with the -command `solana feature activate`. This command will send a transaction with the -necessary instructions to allocate the proper space for a feature account and -assign it to `Feature111111111111111111111111111111111111`. - -In order to protect against spam, this command shall now require a signature from -a multi-signature authority, with keyholders from Solana Labs and potentially -other validator clients. - -This command shall now send a transaction with the following instructions: - -- Fund the feature account with the required lamports deposit minimum -- Allocate the proper space for feature state -- Assign the account to `Feature111111111111111111111111111111111111` -- Invoke the Feature Gate program's `SubmitFeatureForActivation` instruction - -When the `SubmitFeatureForActivation` instruction is invoked, the Feature Gate -program will write the proper feature state into the account. - -Consider the instruction as it may appear in the Feature Gate program: - -```rust -pub enum FeatureGateInstruction { - /// Submit a feature for activation. - /// - /// Features may only be submitted for activation by the multi-signature - /// authority that presides over the feature activation process. - /// - /// Accounts expected by this instruction: - /// - /// 0. `[w]` Feature account - /// 1. `[s]` Multi-signature authority - SubmitFeatureForActivation, -} -``` - -Since anyone can create accounts and assign them to -`Feature111111111111111111111111111111111111`, the Feature Gate program's -`SubmitFeatureForActivation` instruction provides a necessary step to protect -against spam. - -This instruction will: -- Check that the account's lamports balance meets the required minimum -- Check that the account has allocated enough space for the `Feature` state -- Check that the account is owned by - `Feature111111111111111111111111111111111111` -- Serialize an un-initialized `Feature` layout to the account data - -Only accounts which have been successfully processed by the Feature Gate -program's `SubmitFeatureForActivation` will have serialized data, therefore only -accounts owned by `Feature111111111111111111111111111111111111` with serialized -data will be recognized for feature activation. - -### Representing the Current Feature Set - -As mentioned above, the Feature Gate program at address -`Feature111111111111111111111111111111111111` is the owner of all feature -accounts. The Feature Gate program shall utilize two PDAs to keep track of all -feature-gates: - -- **Feature-Request PDA**: Stores a *mutable* list of newly requested feature - activations submitted during the current epoch. -- **Feature-Queue PDA**: Stores an *immutable* list of the previous epoch's - requested activations. - -When a new epoch begins, a new Feature-Request PDA will be created, thus -initializing an empty list. When key-holders submit the CLI command to activate -a feature, the Feature Gate program appends the feature ID to the -Feature-Request set at the same time as creating the feature account. Revoked -features are removed from the Feature-Request list when the Feature Gate -program's `RevokePendingActivation` instruction is invoked. - -At the end of the epoch, the Feature-Request set is written to a new -Feature-Queue PDA (for the next epoch) by the runtime, where it becomes -immutable. A new empty-list Feature-Request PDA is then created for the next -epoch, also by the runtime. - -The newly created immutable Feature-Queue can then be used by nodes to signal -support for potential activation in the next epoch. - -Note that for any given epoch, the epoch will begin with an immutable set of -previously requested feature activations, and end with the activation of those -with enough support. This process takes one epoch. Therefore, it will take at -least one more epoch than it currently does from the time a key-holder submits -a feature for activation via CLI to when it is actually activated. - -Proposed Program-Derived Address for Feature-Request PDA: - -``` -"feature_request" + -``` - -Proposed state for Feature-Request PDA: - -```rust -struct FeatureRequest { - requested_features: Vec, -} -``` +The new process is comprised of the following steps: -Proposed Program-Derived Address for Feature-Queue PDA: +1. **Feature Creation:** Contributors create feature accounts as they do now. +2. **Staging Features for Activation:** In some epoch N, a multi-signature + authority stages for activation some or all of the features created in step + 1. +3. **Signaling Support for Staged Features:** During the next epoch (epoch N+1), + validators signal which of the staged feature-gates they support in their + software. +4. **Activation and Garbage Collection:** At the end of epoch N+1, the runtime + activates the feature-gates that have the necessary stake support. At the + same time, the runtime also archives activated feature accounts and PDAs no + longer required by the process. -``` -"feature_queue" + -``` +### Step 1: Feature Creation -Proposed state for Feature-Queue PDA: +The first step is creation of a feature account, done by submitting a +transaction containing System instructions to fund, allocate, and assign the +feature account to `Feature111111111111111111111111111111111111`. -```rust -struct FeatureQueue { - feature_queue: Vec, -} -``` +### Step 2: Staging Features for Activation +A multi-signature authority, comprised of key-holders from Solana Labs and +possibly other validator client teams in the future, will have the authority to +stage created features for activation. +In the future, this authority could be replaced by validator governance. -### Signaling Support for Features +The multi-signature authority stages a feature for activation by invoking the +Feature Gate program's `StageFeatureForActivation` instruction. This instruction +expects: -With an on-chain reference point to determine the features queued for -activation, nodes can signal support for specific features in the queue. +- Data: The feature ID +- Accounts: + - Staged Features PDA: writable + - Multi-signature authority: signer -A node signals its support for a feature set by sending an instruction to the -Feature Gate program, signed by their authorized voter, to store a record of -their supported feature set. -This supported feature set would be a bit mask of the list of queued features, -and it would be stored in a PDA derived from their vote address. -Nodes simply examine the Feature-Queue list, and mark a 1 for any features they -have in their feature set. The rest are marked 0. +`StageFeatureForActivation` will add the provided feature ID to the **next +epoch's** Staged Features PDA. -In the previous example, a node running v1.14.25 would signal the following bit -mask: +The Staged Features PDA for a given epoch stores a list of all feature IDs that +were staged for activation during the previous epoch. This list shall have a +maximum length of 8 (ie. `[Pubkey; 8]`). -``` -1 0 1 0 -``` - -Similarly, a node running v1.16.0 would signal: +The proposed seeds for deriving the Staged Features PDA are provided below, +where the `` is the epoch during which the contained feature IDs will be +assessed for stake support and considered for activation. ``` -1 1 1 1 -``` - -Validators shall send transactions containing the Feature Gate program's -`SignalSupportForFeatureSet` instruction, which would contain their bit mask, at -some aritrary point during the epoch and on startup after any reboot. - -Consider the instruction as it may appear in the Feature Gate program: - -```rust -pub enum FeatureGateInstruction { - /// Signal support for a feature set. - /// - /// This instruction submits a bit mask representing the current epoch's - /// feature queue. - /// - /// Validators submit these bit masks to describe feature-gates supported - /// by their current client software version. - /// - /// A `1` value represents support for the feature at that index of the - /// bit mask, while a `0` represents a lack of support (or rejection). - /// A "pending" feature activation is a feature account that has been - /// allocated and assigned, but hasn't yet been updated by the runtime - /// with an `activation_slot`. - /// - /// Accounts expected by this instruction: - /// - /// 0. `[w]` Validator Support-Signal PDA - /// 1. `[s]` Vote account - SignalSupportForFeatureSet { - bit_mask: Vec, - }, -} +"staged_features" + ``` -Validators are also required to prepend an instruction to fund this account with -rent-exempt lamports if it does not exist prior to signaling. +### Step 3: Signaling Support for Staged Features -Note that a validator client's particular implementation may use their own -discretion to determine the appropriate cadence at which to send this -transaction during the course of an epoch, with the following restrictions: +With an on-chain reference point to determine the features staged for activation +for a particular epoch, nodes can signal their support for the staged features +supported by their software. -- The signaling transaction **must** be submitted after any reboot. -- No signaling transactions will be considered after **128 slots before the - epoch boundary**. This cut-off is to ensure proper caching. -- The **last signal submitted** before the 128-slot cut-off will be the final - signal considered in the activation phase. +A node signals its support for staged features by invoking the Feature Gate +program's `SignalSupportForStagedFeatures` instruction. This instruction +expects: -When the `SignalSupportForFeatureSet` instruction is invoked, the Feature Gate -program stores the submitted bit mask in a PDA derived from the validator's vote -account. This effectively signals a node's support for a queued set of features. +- Data: A `u8` bit mask of the staged features. +- Accounts: + - Support Signal PDA: writable + - Vote account: signer +A `1` bit represents support for a feature. For example, for staged features +`[A, B, C, D, E, F, G, H]`, if a node wishes to signal support for all features +except `E` and `H`, their `u8` value would be 246, or `11110110`. -Proposed Program-Derived Address for Validator Support-Signal PDA: +A node's submitted bit mask is then stored in a Support Signal PDA derived from +that node's vote address. The proposed seeds are defined below. ``` -"validator_support" + + +"support_signal" + ``` -Proposed state for Validator Support-Signal PDA: +Nodes should send a transaction containing this instruction at some arbitrary +point during the epoch at least 128 slots before the end of the epoch and on +startup after any reboot. -```rust -struct ValidatorSupportSignal { - bit_mask: Vec, -} -``` - - -Note: If a feature is revoked before it is moved to the immutable Feature-Queue -list, it is simply removed from the Feature-Request list and never presented to -nodes. However, if a feature is revoked after it becomes part of the immutable -Feature-Queue list, it remains in the list, and nodes still signal their -support if the feature ID is present in their software. However, the runtime -won’t activate this feature if its corresponding feature account no longer -exists on-chain. -This is how the runtime currently handles features and can be observed here: -. - -### Activating Supported Features - -Once the epoch concludes, the runtime uses the validator support signals to -activate the proper features. - -To do this, the runtime walks all of the PDAs derived from the vote accounts, -examines their bitmasks, and sums the stake supporting each feature-gate to -determine which feature gates meet the desired threshold of supported stake. - -For the initial implementation, the threshold shall be hard-coded to 95% of -stake, but future iterations on the Feature Program could allow feature -key-holders to set a custom threshold for each feature gate -(including 0%, ie. –yolo). - -Once a list of stake-supported features is generated, the runtime rechecks the -Feature accounts in case any have been revoked. Then the runtime processes the -activations using the existing feature logic. - -At this point, the runtime performs a garbage collection process and returns to -step 1 to generate the next list of queued features. More on this process below. +Note: If a feature is revoked, the list of staged features will not change, and +nodes may still signal support for this feature. However, the runtime will not +activate this feature if its corresponding feature account no longer exists +on-chain. -### Garbage Collection +### Step 4: Activation and Garbage Collection -Since submitting features for activation, creating the Feature-Request and -Feature-Queue PDAs, and signaling support for features through PDAs will create -many accounts per epoch, a garbage collection process is required. +During the epoch rollover, the runtime uses the validator support signals to +determine which staged features to activate. -This garbage collection process will involve the use of the Feature Gate -Tombstone account - an empty PDA of the Feature Gate program used to assign as -the owner of accounts no longer required by the feature activation process. +To do this, the runtime walks all of the vote accounts, derives their Support +Signal PDA to read their bit mask, and tallies up the total stake support for +each staged feature. The runtime will also zero-out each bit mask, resetting +each Support Signal PDA for the next epoch. -On epoch rollover, the following accounts will be assigned to the Feature Gate -Tombstone PDA by the Feature Gate program, effectively removing them from its -list of owned accounts: +Only features whose stake support meets the required threshold are activated. +This threshold shall be set to 95% initially, but future iterations on the +process could allow feature key-holders to set a custom threshold per-feature. -- The previous epoch's Feature-Request and Feature-Queue PDAs -- The previous epoch's Validator Support-Signal PDAs -- Any other feature accounts unused when the current epoch's Feature-Queue PDA - is created +If a feature is not activated, either because it has been revoked or it did not +meet the required stake support, it must be resubmitted according to Step 2. -Note: The tombstone shall be a PDA under the Feature Gate program to preserve -the accounts in case a record of any feature activation is necessary to be -queried in the future. +To ensure this new process doesn't overload the Feature Gate program's owned +accounts, during the activation stage, garbage collection will: -With this garbage collection in place, the runtime can efficiently minimize the -scope of accounts that are required to be loaded to perform activations on epoch -rollover. +- Archive any activated feature accounts +- Archive this epoch's Staged Features PDA -### Conclusion +The runtime "archives" an account by assigning it to the Feature Tombstone. -The current familiar process of activating a feature with -`solana feature activate` will now merely suggest this feature be considered -for activation by the aforementioned mechanism. In other words, it’s no longer -a guarantee that your feature will be activated when you run this command. - -Instead, the runtime will decide through stake whether or not a feature should -be activated. This will be especially useful in a world where two separate -clients will aim to push features and want to agree on their activation. +Created features that were not staged for activation or did not meet the +required stake support will not be garbage collected. ## Alternatives Considered -### Representing the Current Feature Set (Alternative) - -An alternate approach to the Feature Gate program’s on-chain storage is to -instead use one single PDA to store the queue and reset it every epoch. This -would allow us to use one less account and one less epoch, but it would make -things trickier when it comes to the timing of validators signaling support for -features. - -For example, a key-holder might submit a feature for activation at a point late -in the epoch, so validators would need to be able to signal support for this -feature whenever it is submitted. This means validators would have to -consistently check the feature queue for changes, and possibly issue more -signals in the epoch. - -This approach may increase the total required work for a node in a given epoch. - -### Signaling Support for Features (Alternative) - -Rather than sending a transaction to the Feature Gate program, a node could -instead signal the bit mask in their block header whenever they are the leader. - -This would introduce a change to the block header, which may prove to be a -difficult change to integrate. - ## Impact This new process for activating features directly impacts core contributors and validators. Core contributors will no longer bear the responsibility of ensuring the proper -stake supports their feature activation. However, it will change the process -by which they can override this stake requirement to push a feature through. +stake supports their feature activation. However, this proposal does not include +a mechanism for overriding or customizing the stake requirement. This capability +should be proposed separately. Validators will be responsible for signaling their vote using a transaction which they've previously not included in their process. They also will have a @@ -410,3 +197,4 @@ integrating feature-gated changes, which may include security fixes. However, the feature-gate process is relatively slow in its current state, and neither the current process or this proposed process would have any implications for critical, time-sensitive issues. + From f83033898c0735aa03d8beff897aed9978707ee8 Mon Sep 17 00:00:00 2001 From: Joe C Date: Wed, 13 Mar 2024 19:29:20 -0500 Subject: [PATCH 13/30] add reusable support signal PDA --- .../0072-feature-gate-threshold-automation.md | 49 ++++++++++--------- 1 file changed, 25 insertions(+), 24 deletions(-) diff --git a/proposals/0072-feature-gate-threshold-automation.md b/proposals/0072-feature-gate-threshold-automation.md index 3860b0186..14b192eaf 100644 --- a/proposals/0072-feature-gate-threshold-automation.md +++ b/proposals/0072-feature-gate-threshold-automation.md @@ -47,12 +47,9 @@ beneficial. [SIMD 0089](https://github.com/solana-foundation/solana-improvement-documents/pull/89) that will own all feature accounts. - **Staged Features PDA:** The PDA under the Feature Gate program used to track - features submitted for activation for a particular epoch. + features submitted for activation. - **Support Signal PDA:** The PDA under the Feature Gate program used to store a bit mask of the staged features a node supports. -- **Feature Tombstone:** A static address with no on-chain account for assigning - accounts under, effectively "archiving" them and removing them from the - Feature Gate program's owned accounts. ## Detailed Design @@ -61,14 +58,12 @@ The new process is comprised of the following steps: 1. **Feature Creation:** Contributors create feature accounts as they do now. 2. **Staging Features for Activation:** In some epoch N, a multi-signature authority stages for activation some or all of the features created in step - 1. + 1, to be activated at the end of the *next epoch*. 3. **Signaling Support for Staged Features:** During the next epoch (epoch N+1), validators signal which of the staged feature-gates they support in their software. -4. **Activation and Garbage Collection:** At the end of epoch N+1, the runtime - activates the feature-gates that have the necessary stake support. At the - same time, the runtime also archives activated feature accounts and PDAs no - longer required by the process. +4. **Feature Activation:** At the end of epoch N+1, the runtime activates the + feature-gates that have the required stake support. ### Step 1: Feature Creation @@ -76,6 +71,8 @@ The first step is creation of a feature account, done by submitting a transaction containing System instructions to fund, allocate, and assign the feature account to `Feature111111111111111111111111111111111111`. +This step is unchanged from its original procedure. + ### Step 2: Staging Features for Activation A multi-signature authority, comprised of key-holders from Solana Labs and @@ -92,20 +89,26 @@ expects: - Staged Features PDA: writable - Multi-signature authority: signer +One single PDA will be used to store two lists of features: + +- The list of features being considered for activation at the end of the + *current epoch*. +- The list of features newly staged for activation, to be considered for + activation in the *next epoch*. + +Each list will be prefixed by the `u64` value of its corresponding epoch, +and each list will have a maximum length of 8 (ie. `[Pubkey; 8]`). + `StageFeatureForActivation` will add the provided feature ID to the **next -epoch's** Staged Features PDA. +epoch's** set of staged features. -The Staged Features PDA for a given epoch stores a list of all feature IDs that -were staged for activation during the previous epoch. This list shall have a -maximum length of 8 (ie. `[Pubkey; 8]`). +An example of how this account's state might work to coordinate both lists can +be found here: -The proposed seeds for deriving the Staged Features PDA are provided below, -where the `` is the epoch during which the contained feature IDs will be -assessed for stake support and considered for activation. + -``` -"staged_features" + -``` +The Staged Features PDA could be derived simply from one literal seed: +`"staged_features"`. ### Step 3: Signaling Support for Staged Features @@ -142,7 +145,7 @@ nodes may still signal support for this feature. However, the runtime will not activate this feature if its corresponding feature account no longer exists on-chain. -### Step 4: Activation and Garbage Collection +### Step 4: Feature Activation During the epoch rollover, the runtime uses the validator support signals to determine which staged features to activate. @@ -160,10 +163,8 @@ If a feature is not activated, either because it has been revoked or it did not meet the required stake support, it must be resubmitted according to Step 2. To ensure this new process doesn't overload the Feature Gate program's owned -accounts, during the activation stage, garbage collection will: - -- Archive any activated feature accounts -- Archive this epoch's Staged Features PDA +accounts, during the activation stage, garbage collection will archive any +activated feature accounts. The runtime "archives" an account by assigning it to the Feature Tombstone. From 2fb901b8eda738a8973fcd4552ef83f099ca2dc0 Mon Sep 17 00:00:00 2001 From: Joe C Date: Wed, 13 Mar 2024 19:51:10 -0500 Subject: [PATCH 14/30] update to use validator epoch stake syscall --- .../0072-feature-gate-threshold-automation.md | 59 +++++++++---------- 1 file changed, 27 insertions(+), 32 deletions(-) diff --git a/proposals/0072-feature-gate-threshold-automation.md b/proposals/0072-feature-gate-threshold-automation.md index 14b192eaf..c9a73ed95 100644 --- a/proposals/0072-feature-gate-threshold-automation.md +++ b/proposals/0072-feature-gate-threshold-automation.md @@ -19,6 +19,10 @@ based on a stake-weighted support threshold, rather than manual human action. With this new process, contributors no longer have to assess stake support for a feature before activation. Instead, the assessment is done by the runtime. +> Note: This SIMD is subject to change based on the results of the SIMD to +> introduce the Validator Epoch Stake Syscall, as defined below, to which a link +> will be provided in this proposal when available. + ## Motivation Feature gates wrap new cluster functionality, and typically change the rules of @@ -48,8 +52,8 @@ beneficial. that will own all feature accounts. - **Staged Features PDA:** The PDA under the Feature Gate program used to track features submitted for activation. -- **Support Signal PDA:** The PDA under the Feature Gate program used to store - a bit mask of the staged features a node supports. +- **Validator Epoch Stake Syscall:** The new syscall that returns the current + epoch stake for a given vote account address. ## Detailed Design @@ -89,15 +93,17 @@ expects: - Staged Features PDA: writable - Multi-signature authority: signer -One single PDA will be used to store two lists of features: - -- The list of features being considered for activation at the end of the - *current epoch*. -- The list of features newly staged for activation, to be considered for - activation in the *next epoch*. +One single PDA will be used to store two lists of features. Its data will be +structured as follows: -Each list will be prefixed by the `u64` value of its corresponding epoch, -and each list will have a maximum length of 8 (ie. `[Pubkey; 8]`). +- 8 bytes for the current epoch. +- A fixed-size list of size 8 containing 32-byte feature IDs staged for + activation at the end of the *current epoch*. +- A fixed-size list of size 8 containing the `u64` value of total stake support + for each feature ID in the prior list. +- 8 bytes for the next epoch. +- A fixed-size list of size 8 containing 32-byte feature IDs staged for + activation at the end of the *next epoch*. `StageFeatureForActivation` will add the provided feature ID to the **next epoch's** set of staged features. @@ -122,19 +128,16 @@ expects: - Data: A `u8` bit mask of the staged features. - Accounts: - - Support Signal PDA: writable + - Staged Features PDA: writable - Vote account: signer A `1` bit represents support for a feature. For example, for staged features `[A, B, C, D, E, F, G, H]`, if a node wishes to signal support for all features except `E` and `H`, their `u8` value would be 246, or `11110110`. -A node's submitted bit mask is then stored in a Support Signal PDA derived from -that node's vote address. The proposed seeds are defined below. - -``` -"support_signal" + -``` +A node's submitted bit mask is then used to add that node's current epoch stake +to the Staged Features PDA for each feature for which it has signaled support. +This is done by using the Validator Epoch Stake Syscall. Nodes should send a transaction containing this instruction at some arbitrary point during the epoch at least 128 slots before the end of the epoch and on @@ -147,13 +150,9 @@ on-chain. ### Step 4: Feature Activation -During the epoch rollover, the runtime uses the validator support signals to -determine which staged features to activate. - -To do this, the runtime walks all of the vote accounts, derives their Support -Signal PDA to read their bit mask, and tallies up the total stake support for -each staged feature. The runtime will also zero-out each bit mask, resetting -each Support Signal PDA for the next epoch. +During the epoch rollover, the runtime can simply load the Staged Features PDA +and calculate the stake - as a percentage of the total epoch stake - in support +for each feature ID to determine which staged features to activate. Only features whose stake support meets the required threshold are activated. This threshold shall be set to 95% initially, but future iterations on the @@ -162,14 +161,10 @@ process could allow feature key-holders to set a custom threshold per-feature. If a feature is not activated, either because it has been revoked or it did not meet the required stake support, it must be resubmitted according to Step 2. -To ensure this new process doesn't overload the Feature Gate program's owned -accounts, during the activation stage, garbage collection will archive any -activated feature accounts. - -The runtime "archives" an account by assigning it to the Feature Tombstone. - -Created features that were not staged for activation or did not meet the -required stake support will not be garbage collected. +Once the epoch rollover is complete, as soon as the Feature Gate program is +invoked again - either via `StageFeatureForActivation` or +`SignalSupportForStagedFeature` - the Staged Features PDA's account state is +updated and the process begins again. ## Alternatives Considered From ab9574074f8d6762486e1797ae30483945a92f29 Mon Sep 17 00:00:00 2001 From: Joe Caulfield Date: Thu, 9 May 2024 08:47:07 -0500 Subject: [PATCH 15/30] add updates from SIMD 0133 --- proposals/0072-feature-gate-threshold-automation.md | 11 ++++------- 1 file changed, 4 insertions(+), 7 deletions(-) diff --git a/proposals/0072-feature-gate-threshold-automation.md b/proposals/0072-feature-gate-threshold-automation.md index c9a73ed95..8aa8f089d 100644 --- a/proposals/0072-feature-gate-threshold-automation.md +++ b/proposals/0072-feature-gate-threshold-automation.md @@ -19,10 +19,6 @@ based on a stake-weighted support threshold, rather than manual human action. With this new process, contributors no longer have to assess stake support for a feature before activation. Instead, the assessment is done by the runtime. -> Note: This SIMD is subject to change based on the results of the SIMD to -> introduce the Validator Epoch Stake Syscall, as defined below, to which a link -> will be provided in this proposal when available. - ## Motivation Feature gates wrap new cluster functionality, and typically change the rules of @@ -52,8 +48,9 @@ beneficial. that will own all feature accounts. - **Staged Features PDA:** The PDA under the Feature Gate program used to track features submitted for activation. -- **Validator Epoch Stake Syscall:** The new syscall that returns the current - epoch stake for a given vote account address. +- **Get Epoch Stake Syscall:** The new syscall introduced in + [SIMD 0133](https://github.com/solana-foundation/solana-improvement-documents/pull/133) + that returns the current epoch stake for a given vote account address. ## Detailed Design @@ -137,7 +134,7 @@ except `E` and `H`, their `u8` value would be 246, or `11110110`. A node's submitted bit mask is then used to add that node's current epoch stake to the Staged Features PDA for each feature for which it has signaled support. -This is done by using the Validator Epoch Stake Syscall. +This is done by using the Get Epoch Stake Syscall. Nodes should send a transaction containing this instruction at some arbitrary point during the epoch at least 128 slots before the end of the epoch and on From 1a8f21a5951383f868ea3c4b0d2821a3e324a1b7 Mon Sep 17 00:00:00 2001 From: Joe Caulfield Date: Mon, 13 May 2024 09:12:29 -0500 Subject: [PATCH 16/30] add some context and wording --- .../0072-feature-gate-threshold-automation.md | 31 ++++++++++++++++--- 1 file changed, 26 insertions(+), 5 deletions(-) diff --git a/proposals/0072-feature-gate-threshold-automation.md b/proposals/0072-feature-gate-threshold-automation.md index 8aa8f089d..44cff9e83 100644 --- a/proposals/0072-feature-gate-threshold-automation.md +++ b/proposals/0072-feature-gate-threshold-automation.md @@ -110,13 +110,13 @@ be found here: -The Staged Features PDA could be derived simply from one literal seed: +The Staged Features PDA will be derived simply from one literal seed: `"staged_features"`. ### Step 3: Signaling Support for Staged Features With an on-chain reference point to determine the features staged for activation -for a particular epoch, nodes can signal their support for the staged features +for a particular epoch, nodes will signal their support for the staged features supported by their software. A node signals its support for staged features by invoking the Feature Gate @@ -128,6 +128,17 @@ expects: - Staged Features PDA: writable - Vote account: signer +A bit mask is used as a compressed ordered list of indices. This has two main +benefits: + +- Minimizes transaction size for nodes, requiring only one byte to describe + 256 bytes (8 * 32) of data. The alternative would be sending `n` number of + 32-byte addresses in each instruction. +- Reduce compute required to search the list of staged features for a matching + address. The bitmask's format provides the Feature Gate program with the + indices it needs to store supported stake without searching through the staged + features for a match. + A `1` bit represents support for a feature. For example, for staged features `[A, B, C, D, E, F, G, H]`, if a node wishes to signal support for all features except `E` and `H`, their `u8` value would be 246, or `11110110`. @@ -138,10 +149,16 @@ This is done by using the Get Epoch Stake Syscall. Nodes should send a transaction containing this instruction at some arbitrary point during the epoch at least 128 slots before the end of the epoch and on -startup after any reboot. +startup after any reboot. Transactions sent too late (< 128 slots before epoch +end) will be rejected by the Feature Gate program. -Note: If a feature is revoked, the list of staged features will not change, and -nodes may still signal support for this feature. However, the runtime will not +If a node does not send this transaction, or sends it too late in the epoch (> +128 slots before the end), their stake is not tallied. This is analogous to a +node sending this transaction in a valid point in the epoch signalling support +for zero features. + +If a feature is revoked, the list of staged features will not change, and nodes +may still signal support for this feature. However, the runtime will not activate this feature if its corresponding feature account no longer exists on-chain. @@ -155,6 +172,10 @@ Only features whose stake support meets the required threshold are activated. This threshold shall be set to 95% initially, but future iterations on the process could allow feature key-holders to set a custom threshold per-feature. +As mentioned previously, if a feature was revoked, it will no longer exist +on-chain, and therefore will be not activated by the runtime, regardless of +calculated stake support. + If a feature is not activated, either because it has been revoked or it did not meet the required stake support, it must be resubmitted according to Step 2. From ea129bacfa0a42fbc81ce41760996569162eef3c Mon Sep 17 00:00:00 2001 From: Joe Caulfield Date: Mon, 13 May 2024 09:31:24 -0500 Subject: [PATCH 17/30] add c struct for PDA layout --- .../0072-feature-gate-threshold-automation.md | 40 +++++++++++++++---- 1 file changed, 32 insertions(+), 8 deletions(-) diff --git a/proposals/0072-feature-gate-threshold-automation.md b/proposals/0072-feature-gate-threshold-automation.md index 44cff9e83..46e9a13f6 100644 --- a/proposals/0072-feature-gate-threshold-automation.md +++ b/proposals/0072-feature-gate-threshold-automation.md @@ -93,14 +93,38 @@ expects: One single PDA will be used to store two lists of features. Its data will be structured as follows: -- 8 bytes for the current epoch. -- A fixed-size list of size 8 containing 32-byte feature IDs staged for - activation at the end of the *current epoch*. -- A fixed-size list of size 8 containing the `u64` value of total stake support - for each feature ID in the prior list. -- 8 bytes for the next epoch. -- A fixed-size list of size 8 containing 32-byte feature IDs staged for - activation at the end of the *next epoch*. +```c +#define FEATURE_ID_SIZE 32 +#define MAX_FEATURES 8 + +/** + * A Feature ID and its corresponding stake support, as signalled by validators. + */ +typedef struct { + /** Feature identifier (32 bytes). */ + uint8_t feature_id[FEATURE_ID_SIZE]; + /** Stake support (little-endian u64). */ + uint8_t stake[8]; +} FeatureStake; + +/** + * Staged features for activation. + */ +typedef struct { + /** The current epoch (little-endian u64). */ + uint8_t current_epoch[8]; + /** + * Features staged for activation at the end of the current epoch, with + * their corresponding signalled stake support. + */ + FeatureStake current_feature_stakes[MAX_FEATURES]; + + /** The next epoch (little-endian u64). */ + uint8_t next_epoch[8]; + /** Feature IDs staged for the _next_ epoch. */ + uint8_t next_features[MAX_FEATURES][FEATURE_ID_SIZE]; +} StagedFeatures; +``` `StageFeatureForActivation` will add the provided feature ID to the **next epoch's** set of staged features. From bb1c79786ea2e67d3f5fb9393684d248d88f4976 Mon Sep 17 00:00:00 2001 From: Joe Caulfield Date: Thu, 16 May 2024 00:59:29 -0500 Subject: [PATCH 18/30] hard-coded threshold --- proposals/0072-feature-gate-threshold-automation.md | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/proposals/0072-feature-gate-threshold-automation.md b/proposals/0072-feature-gate-threshold-automation.md index 46e9a13f6..91c1b961d 100644 --- a/proposals/0072-feature-gate-threshold-automation.md +++ b/proposals/0072-feature-gate-threshold-automation.md @@ -193,8 +193,9 @@ and calculate the stake - as a percentage of the total epoch stake - in support for each feature ID to determine which staged features to activate. Only features whose stake support meets the required threshold are activated. -This threshold shall be set to 95% initially, but future iterations on the -process could allow feature key-holders to set a custom threshold per-feature. +This threshold will be hard-coded in the runtime to 95% initially, but future +iterations on the process could allow feature key-holders to set a custom +threshold per-feature. As mentioned previously, if a feature was revoked, it will no longer exist on-chain, and therefore will be not activated by the runtime, regardless of From b85755d8c370589cb178486fe9b14669123fce5c Mon Sep 17 00:00:00 2001 From: Joe Caulfield Date: Thu, 16 May 2024 14:16:08 -0500 Subject: [PATCH 19/30] change deadline to 4500 slots --- .../0072-feature-gate-threshold-automation.md | 20 ++++++++++--------- 1 file changed, 11 insertions(+), 9 deletions(-) diff --git a/proposals/0072-feature-gate-threshold-automation.md b/proposals/0072-feature-gate-threshold-automation.md index 91c1b961d..a797c0fc8 100644 --- a/proposals/0072-feature-gate-threshold-automation.md +++ b/proposals/0072-feature-gate-threshold-automation.md @@ -171,15 +171,17 @@ A node's submitted bit mask is then used to add that node's current epoch stake to the Staged Features PDA for each feature for which it has signaled support. This is done by using the Get Epoch Stake Syscall. -Nodes should send a transaction containing this instruction at some arbitrary -point during the epoch at least 128 slots before the end of the epoch and on -startup after any reboot. Transactions sent too late (< 128 slots before epoch -end) will be rejected by the Feature Gate program. - -If a node does not send this transaction, or sends it too late in the epoch (> -128 slots before the end), their stake is not tallied. This is analogous to a -node sending this transaction in a valid point in the epoch signalling support -for zero features. +Nodes are required to submit a transaction containing this instruction: + +- Any time at least 4500 slots before the end of the epoch. +- During startup after any reboot. + +Transactions sent too late (< 4500 slots before the epoch end) will be rejected +by the Feature Gate program. + +If a node does not send this transaction or it is rejected, their stake is not +tallied. This is analogous to a node sending this transaction in a valid point +in the epoch signalling support for zero features. If a feature is revoked, the list of staged features will not change, and nodes may still signal support for this feature. However, the runtime will not From 66e63710f10c8e1a31f5b98551d76cfc09e8ae06 Mon Sep 17 00:00:00 2001 From: Joe Caulfield Date: Wed, 22 May 2024 10:43:31 -0500 Subject: [PATCH 20/30] clarity suggestions --- .../0072-feature-gate-threshold-automation.md | 27 ++++++++----------- 1 file changed, 11 insertions(+), 16 deletions(-) diff --git a/proposals/0072-feature-gate-threshold-automation.md b/proposals/0072-feature-gate-threshold-automation.md index a797c0fc8..2cbf1fd60 100644 --- a/proposals/0072-feature-gate-threshold-automation.md +++ b/proposals/0072-feature-gate-threshold-automation.md @@ -2,7 +2,7 @@ simd: '0072' title: Feature Gate Threshold Automation authors: - - Tyera Eulburg + - Tyera Eulberg - Joe Caulfield category: Standard type: Core @@ -44,12 +44,12 @@ beneficial. ## New Terminology - **Feature Gate program:** The Core BPF program introduced in - [SIMD 0089](https://github.com/solana-foundation/solana-improvement-documents/pull/89) + [SIMD 0089](./0089-programify-feature-gate-program.md) that will own all feature accounts. - **Staged Features PDA:** The PDA under the Feature Gate program used to track features submitted for activation. - **Get Epoch Stake Syscall:** The new syscall introduced in - [SIMD 0133](https://github.com/solana-foundation/solana-improvement-documents/pull/133) + [SIMD 0133](./0133-syscall-get-epoch-stake.md) that returns the current epoch stake for a given vote account address. ## Detailed Design @@ -76,7 +76,7 @@ This step is unchanged from its original procedure. ### Step 2: Staging Features for Activation -A multi-signature authority, comprised of key-holders from Solana Labs and +A multi-signature authority, comprised of key-holders from Anza and possibly other validator client teams in the future, will have the authority to stage created features for activation. In the future, this authority could be replaced by validator governance. @@ -129,11 +129,6 @@ typedef struct { `StageFeatureForActivation` will add the provided feature ID to the **next epoch's** set of staged features. -An example of how this account's state might work to coordinate both lists can -be found here: - - - The Staged Features PDA will be derived simply from one literal seed: `"staged_features"`. @@ -171,7 +166,7 @@ A node's submitted bit mask is then used to add that node's current epoch stake to the Staged Features PDA for each feature for which it has signaled support. This is done by using the Get Epoch Stake Syscall. -Nodes are required to submit a transaction containing this instruction: +Nodes should submit a transaction containing this instruction: - Any time at least 4500 slots before the end of the epoch. - During startup after any reboot. @@ -180,7 +175,7 @@ Transactions sent too late (< 4500 slots before the epoch end) will be rejected by the Feature Gate program. If a node does not send this transaction or it is rejected, their stake is not -tallied. This is analogous to a node sending this transaction in a valid point +tallied. This is analogous to a node sending this transaction at a valid slot in the epoch signalling support for zero features. If a feature is revoked, the list of staged features will not change, and nodes @@ -190,14 +185,14 @@ on-chain. ### Step 4: Feature Activation -During the epoch rollover, the runtime can simply load the Staged Features PDA +During the epoch rollover, the runtime must load the Staged Features PDA and calculate the stake - as a percentage of the total epoch stake - in support for each feature ID to determine which staged features to activate. -Only features whose stake support meets the required threshold are activated. -This threshold will be hard-coded in the runtime to 95% initially, but future -iterations on the process could allow feature key-holders to set a custom -threshold per-feature. +Every feature whose stake support meets the required threshold must be +activated. This threshold will be hard-coded in the runtime to 95% initially, +but future iterations on the process could allow feature key-holders to set a +custom threshold per-feature. As mentioned previously, if a feature was revoked, it will no longer exist on-chain, and therefore will be not activated by the runtime, regardless of From 58cb43658c04f8a5827b0122f4647a7bc9f3c330 Mon Sep 17 00:00:00 2001 From: Joe Caulfield Date: Wed, 22 May 2024 13:31:44 -0500 Subject: [PATCH 21/30] add more program specification --- .../0072-feature-gate-threshold-automation.md | 47 ++++++++++++++----- 1 file changed, 36 insertions(+), 11 deletions(-) diff --git a/proposals/0072-feature-gate-threshold-automation.md b/proposals/0072-feature-gate-threshold-automation.md index 2cbf1fd60..344c2fa0c 100644 --- a/proposals/0072-feature-gate-threshold-automation.md +++ b/proposals/0072-feature-gate-threshold-automation.md @@ -54,6 +54,18 @@ beneficial. ## Detailed Design +This proposal outlines a new feature activation process. The new process +includes changes to the runtime's feature activation process as well as the +Core BPF Feature Gate program proposed in +[SIMD 0089](./0089-programify-feature-gate-program.md). + +The new process will utilize the Feature Gate program to enable the runtime to +activate staged features that meet the necessary stake support while preventing +the activation of those that do not. + +Two new instructions, as well as two types of PDAs, will be added to the +Feature Gate program. They are detailed in this proposal. + The new process is comprised of the following steps: 1. **Feature Creation:** Contributors create feature accounts as they do now. @@ -81,8 +93,8 @@ possibly other validator client teams in the future, will have the authority to stage created features for activation. In the future, this authority could be replaced by validator governance. -The multi-signature authority stages a feature for activation by invoking the -Feature Gate program's `StageFeatureForActivation` instruction. This instruction +The multi-signature authority stages a feature for activation by invoking a new +Feature Gate program instruction: `StageFeatureForActivation`. This instruction expects: - Data: The feature ID @@ -138,8 +150,8 @@ With an on-chain reference point to determine the features staged for activation for a particular epoch, nodes will signal their support for the staged features supported by their software. -A node signals its support for staged features by invoking the Feature Gate -program's `SignalSupportForStagedFeatures` instruction. This instruction +A node signals its support for staged features by invoking another new Feature +Gate program instruction: `SignalSupportForStagedFeatures`. This instruction expects: - Data: A `u8` bit mask of the staged features. @@ -162,9 +174,11 @@ A `1` bit represents support for a feature. For example, for staged features `[A, B, C, D, E, F, G, H]`, if a node wishes to signal support for all features except `E` and `H`, their `u8` value would be 246, or `11110110`. -A node's submitted bit mask is then used to add that node's current epoch stake -to the Staged Features PDA for each feature for which it has signaled support. -This is done by using the Get Epoch Stake Syscall. +The `SignalSupportForStagedFeatures` instruction processor will provide the +vote account's address to the `GetEpochStake` syscall to retrieve the stake +delegated to that vote account for the epoch. Then, using the `1` values +provided in the bitmask, the processor will add this stake value to each +corresponding feature ID's stake support in the Staged Features PDA. Nodes should submit a transaction containing this instruction: @@ -201,10 +215,21 @@ calculated stake support. If a feature is not activated, either because it has been revoked or it did not meet the required stake support, it must be resubmitted according to Step 2. -Once the epoch rollover is complete, as soon as the Feature Gate program is -invoked again - either via `StageFeatureForActivation` or -`SignalSupportForStagedFeature` - the Staged Features PDA's account state is -updated and the process begins again. +Once the epoch rollover is complete, the Feature Gate program will reset the +Staged Features PDA for the new epoch the very first time it is invoked +without error in the new epoch. Both instructions - `StageFeatureForActivation` +and `SignalSupportForStagedFeature` - will check the value for `next_epoch` +against the *current epoch* value in the `Clock` sysvar before executing the +steps outlined previously in this proposal. If a match is detected, the account +state is reset for the new epoch as follows: + +1. The `current_feature_stakes` list is reset to an empty list. +2. The `current_epoch` is set to the value of `next_epoch`. +3. The feature IDs from the `next_features` list are written into the + `current_feature_stakes` list with stake support initialized to zero. +4. The `next_epoch` value is set to the *next* upcoming epoch number + (`current_epoch` + 1). +5. The `next_features` list is reset to an empty list. ## Alternatives Considered From da132f7d003b4ad800c75e46dc9d54e79ca6287d Mon Sep 17 00:00:00 2001 From: Joe Caulfield Date: Mon, 3 Jun 2024 14:03:20 -0500 Subject: [PATCH 22/30] one PDA per epoch --- .../0072-feature-gate-threshold-automation.md | 70 ++++++++++--------- 1 file changed, 37 insertions(+), 33 deletions(-) diff --git a/proposals/0072-feature-gate-threshold-automation.md b/proposals/0072-feature-gate-threshold-automation.md index 344c2fa0c..b102f0906 100644 --- a/proposals/0072-feature-gate-threshold-automation.md +++ b/proposals/0072-feature-gate-threshold-automation.md @@ -69,13 +69,13 @@ Feature Gate program. They are detailed in this proposal. The new process is comprised of the following steps: 1. **Feature Creation:** Contributors create feature accounts as they do now. -2. **Staging Features for Activation:** In some epoch N, a multi-signature +2. **Staging Features for Activation:** In some epoch `N-1`, a multi-signature authority stages for activation some or all of the features created in step - 1, to be activated at the end of the *next epoch*. -3. **Signaling Support for Staged Features:** During the next epoch (epoch N+1), + 1, to be activated at the end of the *next epoch* (epoch `N`). +3. **Signaling Support for Staged Features:** During the next epoch (epoch `N`), validators signal which of the staged feature-gates they support in their software. -4. **Feature Activation:** At the end of epoch N+1, the runtime activates the +4. **Feature Activation:** At the end of epoch `N`, the runtime activates the feature-gates that have the required stake support. ### Step 1: Feature Creation @@ -102,8 +102,24 @@ expects: - Staged Features PDA: writable - Multi-signature authority: signer -One single PDA will be used to store two lists of features. Its data will be -structured as follows: +A PDA will be created for each epoch in which features are staged to be +activated. If no features are staged for a given epoch, that epoch's +corresponding PDA is not created. + +These PDAs will not be garbage collected and can be referenced for historical +purposes. + +When the first feature for an epoch is staged, the PDA is created. The +`StageFeatureForActivation` processor will debit from the multisig enough +lamports to allocate the new Staged Features PDA. + +The address of the Staged Features PDA for a given epoch is derived as follows: + +``` +"staged_features" + < epoch number > +``` + +The data for the Staged Features PDA will be structured as follows: ```c #define FEATURE_ID_SIZE 32 @@ -123,26 +139,26 @@ typedef struct { * Staged features for activation. */ typedef struct { - /** The current epoch (little-endian u64). */ - uint8_t current_epoch[8]; /** * Features staged for activation at the end of the current epoch, with * their corresponding signalled stake support. */ FeatureStake current_feature_stakes[MAX_FEATURES]; - - /** The next epoch (little-endian u64). */ - uint8_t next_epoch[8]; - /** Feature IDs staged for the _next_ epoch. */ - uint8_t next_features[MAX_FEATURES][FEATURE_ID_SIZE]; } StagedFeatures; ``` -`StageFeatureForActivation` will add the provided feature ID to the **next -epoch's** set of staged features. +`StageFeatureForActivation` may only be invoked during epoch `N-1`, where `N` is +the epoch number used to derive the Staged Features program-derived address. +This is checked by the Feature Gate program using the Clock sysvar. + +Features staged during epoch `N-1` are staged to be activated at the end of +epoch `N`. -The Staged Features PDA will be derived simply from one literal seed: -`"staged_features"`. +`StageFeatureForActivation` will add the provided feature ID to the list with +stake support initialized to `0`. + +As depicted in the above layout, a maximum of 8 features can be staged for a +given epoch. ### Step 3: Signaling Support for Staged Features @@ -180,6 +196,10 @@ delegated to that vote account for the epoch. Then, using the `1` values provided in the bitmask, the processor will add this stake value to each corresponding feature ID's stake support in the Staged Features PDA. +Just like the `StageFeatureForActivation` instruction, the Clock sysvar will be +used to ensure the Staged Features PDA corresponding to the *current* epoch was +provided. + Nodes should submit a transaction containing this instruction: - Any time at least 4500 slots before the end of the epoch. @@ -215,22 +235,6 @@ calculated stake support. If a feature is not activated, either because it has been revoked or it did not meet the required stake support, it must be resubmitted according to Step 2. -Once the epoch rollover is complete, the Feature Gate program will reset the -Staged Features PDA for the new epoch the very first time it is invoked -without error in the new epoch. Both instructions - `StageFeatureForActivation` -and `SignalSupportForStagedFeature` - will check the value for `next_epoch` -against the *current epoch* value in the `Clock` sysvar before executing the -steps outlined previously in this proposal. If a match is detected, the account -state is reset for the new epoch as follows: - -1. The `current_feature_stakes` list is reset to an empty list. -2. The `current_epoch` is set to the value of `next_epoch`. -3. The feature IDs from the `next_features` list are written into the - `current_feature_stakes` list with stake support initialized to zero. -4. The `next_epoch` value is set to the *next* upcoming epoch number - (`current_epoch` + 1). -5. The `next_features` list is reset to an empty list. - ## Alternatives Considered ## Impact From 4de5abda25c26a11f0ac3bb67b71fd7cb5e68e72 Mon Sep 17 00:00:00 2001 From: Joe Caulfield Date: Mon, 3 Jun 2024 14:18:02 -0500 Subject: [PATCH 23/30] remove slots remaining constraint --- .../0072-feature-gate-threshold-automation.md | 14 +++----------- 1 file changed, 3 insertions(+), 11 deletions(-) diff --git a/proposals/0072-feature-gate-threshold-automation.md b/proposals/0072-feature-gate-threshold-automation.md index b102f0906..998e9c3ad 100644 --- a/proposals/0072-feature-gate-threshold-automation.md +++ b/proposals/0072-feature-gate-threshold-automation.md @@ -200,17 +200,9 @@ Just like the `StageFeatureForActivation` instruction, the Clock sysvar will be used to ensure the Staged Features PDA corresponding to the *current* epoch was provided. -Nodes should submit a transaction containing this instruction: - -- Any time at least 4500 slots before the end of the epoch. -- During startup after any reboot. - -Transactions sent too late (< 4500 slots before the epoch end) will be rejected -by the Feature Gate program. - -If a node does not send this transaction or it is rejected, their stake is not -tallied. This is analogous to a node sending this transaction at a valid slot -in the epoch signalling support for zero features. +If a node does not send this transaction successfully during the current epoch, +their stake is not tallied. This is analogous to a node signalling support for +zero features. If a feature is revoked, the list of staged features will not change, and nodes may still signal support for this feature. However, the runtime will not From 072344f3111d96d5fa1ac3b9ef99a4bc6ccdea59 Mon Sep 17 00:00:00 2001 From: Joe C Date: Mon, 3 Jun 2024 14:20:06 -0500 Subject: [PATCH 24/30] Update proposals/0072-feature-gate-threshold-automation.md Co-authored-by: Tyera --- proposals/0072-feature-gate-threshold-automation.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/proposals/0072-feature-gate-threshold-automation.md b/proposals/0072-feature-gate-threshold-automation.md index 998e9c3ad..045533bf6 100644 --- a/proposals/0072-feature-gate-threshold-automation.md +++ b/proposals/0072-feature-gate-threshold-automation.md @@ -196,8 +196,8 @@ delegated to that vote account for the epoch. Then, using the `1` values provided in the bitmask, the processor will add this stake value to each corresponding feature ID's stake support in the Staged Features PDA. -Just like the `StageFeatureForActivation` instruction, the Clock sysvar will be -used to ensure the Staged Features PDA corresponding to the *current* epoch was +Similar to the `StageFeatureForActivation` instruction, the Clock sysvar will be +used to ensure the Staged Features PDA corresponding to the *current* epoch `N` was provided. If a node does not send this transaction successfully during the current epoch, From 929606120363728e701fea587f9b7f85e31f416e Mon Sep 17 00:00:00 2001 From: Joe Caulfield Date: Mon, 3 Jun 2024 17:44:16 -0500 Subject: [PATCH 25/30] instruction clarity --- proposals/0072-feature-gate-threshold-automation.md | 11 +++++++---- 1 file changed, 7 insertions(+), 4 deletions(-) diff --git a/proposals/0072-feature-gate-threshold-automation.md b/proposals/0072-feature-gate-threshold-automation.md index 045533bf6..6aca472fb 100644 --- a/proposals/0072-feature-gate-threshold-automation.md +++ b/proposals/0072-feature-gate-threshold-automation.md @@ -100,7 +100,8 @@ expects: - Data: The feature ID - Accounts: - Staged Features PDA: writable - - Multi-signature authority: signer + - Multi-signature authority: signer + - Payer: optional signer A PDA will be created for each epoch in which features are staged to be activated. If no features are staged for a given epoch, that epoch's @@ -110,10 +111,12 @@ These PDAs will not be garbage collected and can be referenced for historical purposes. When the first feature for an epoch is staged, the PDA is created. The -`StageFeatureForActivation` processor will debit from the multisig enough -lamports to allocate the new Staged Features PDA. +`StageFeatureForActivation` processor will debit from the payer account enough +lamports to allocate the new Staged Features PDA. A payer account must be +provided for the first staged feature. -The address of the Staged Features PDA for a given epoch is derived as follows: +The address of the Staged Features PDA for a given epoch is derived as follows, +where `epoch number` is a little-endian `u64`: ``` "staged_features" + < epoch number > From 458b3ace2de0c737344abbfe10c988cc971813f0 Mon Sep 17 00:00:00 2001 From: Joe Caulfield Date: Wed, 14 Aug 2024 09:57:17 -0400 Subject: [PATCH 26/30] update `StageFeatureForActivation` instruction --- .../0072-feature-gate-threshold-automation.md | 65 ++++++++++--------- 1 file changed, 33 insertions(+), 32 deletions(-) diff --git a/proposals/0072-feature-gate-threshold-automation.md b/proposals/0072-feature-gate-threshold-automation.md index 6aca472fb..fc01bb354 100644 --- a/proposals/0072-feature-gate-threshold-automation.md +++ b/proposals/0072-feature-gate-threshold-automation.md @@ -46,8 +46,8 @@ beneficial. - **Feature Gate program:** The Core BPF program introduced in [SIMD 0089](./0089-programify-feature-gate-program.md) that will own all feature accounts. -- **Staged Features PDA:** The PDA under the Feature Gate program used to track - features submitted for activation. +- **Staged Features PDA:** A PDA under the Feature Gate program used to track + features submitted for activation per epoch. - **Get Epoch Stake Syscall:** The new syscall introduced in [SIMD 0133](./0133-syscall-get-epoch-stake.md) that returns the current epoch stake for a given vote account address. @@ -63,8 +63,8 @@ The new process will utilize the Feature Gate program to enable the runtime to activate staged features that meet the necessary stake support while preventing the activation of those that do not. -Two new instructions, as well as two types of PDAs, will be added to the -Feature Gate program. They are detailed in this proposal. +Two new instructions and one new type of PDA will be added to the Feature Gate +program. They are detailed in this proposal. The new process is comprised of the following steps: @@ -95,31 +95,42 @@ In the future, this authority could be replaced by validator governance. The multi-signature authority stages a feature for activation by invoking a new Feature Gate program instruction: `StageFeatureForActivation`. This instruction -expects: +expects the Staged Features PDA (defined below) to either exist or be funded +with enough rent-exempt lamports to initialize state. The processor will +allocate, assign, and initialize the account if it does not exist. + +The `StageFeatureForActivation` instruction is structured as follows: -- Data: The feature ID +- Data: None - Accounts: + - Feature account - Staged Features PDA: writable - Multi-signature authority: signer - - Payer: optional signer + - System program (required only for initializing) + +Note that features can only be staged in the epoch prior to the target +activation epoch. This means a feature staged by invoking +`StageFeatureForActivation` in epoch `N-1` is scheduled for activation (through +the process below) at the end of epoch `N`. This is checked by the `Clock` +sysvar. + +When a feature is staged for activation, initial stake support is zero. + +#### Staged Features PDA State -A PDA will be created for each epoch in which features are staged to be -activated. If no features are staged for a given epoch, that epoch's -corresponding PDA is not created. +A Staged Features PDA will be created for each epoch in which features are +staged to be activated. If no features are staged for a given epoch, that +epoch's corresponding Staged Features PDA will never be initialized and thus +will not exist. These PDAs will not be garbage collected and can be referenced for historical purposes. -When the first feature for an epoch is staged, the PDA is created. The -`StageFeatureForActivation` processor will debit from the payer account enough -lamports to allocate the new Staged Features PDA. A payer account must be -provided for the first staged feature. - The address of the Staged Features PDA for a given epoch is derived as follows, -where `epoch number` is a little-endian `u64`: +where `epoch` is a `u64` serialized to eight little-endian bytes: ``` -"staged_features" + < epoch number > +"staged_features" + ``` The data for the Staged Features PDA will be structured as follows: @@ -132,10 +143,10 @@ The data for the Staged Features PDA will be structured as follows: * A Feature ID and its corresponding stake support, as signalled by validators. */ typedef struct { - /** Feature identifier (32 bytes). */ + /** Feature identifier (32 bytes for public key). */ uint8_t feature_id[FEATURE_ID_SIZE]; - /** Stake support (little-endian u64). */ - uint8_t stake[8]; + /** Stake support (u64 serialized to little-endian). */ + uint8_t stake_support[8]; } FeatureStake; /** @@ -146,20 +157,10 @@ typedef struct { * Features staged for activation at the end of the current epoch, with * their corresponding signalled stake support. */ - FeatureStake current_feature_stakes[MAX_FEATURES]; + FeatureStake features[MAX_FEATURES]; } StagedFeatures; ``` -`StageFeatureForActivation` may only be invoked during epoch `N-1`, where `N` is -the epoch number used to derive the Staged Features program-derived address. -This is checked by the Feature Gate program using the Clock sysvar. - -Features staged during epoch `N-1` are staged to be activated at the end of -epoch `N`. - -`StageFeatureForActivation` will add the provided feature ID to the list with -stake support initialized to `0`. - As depicted in the above layout, a maximum of 8 features can be staged for a given epoch. @@ -203,7 +204,7 @@ Similar to the `StageFeatureForActivation` instruction, the Clock sysvar will be used to ensure the Staged Features PDA corresponding to the *current* epoch `N` was provided. -If a node does not send this transaction successfully during the current epoch, +If a node does not send this instruction successfully during the current epoch, their stake is not tallied. This is analogous to a node signalling support for zero features. From d9f399d087b6cd75048f0c2ee1107afde14fb70f Mon Sep 17 00:00:00 2001 From: Joe Caulfield Date: Wed, 14 Aug 2024 10:04:54 -0400 Subject: [PATCH 27/30] update `SignalSupportForStagedFeatures` instruction --- .../0072-feature-gate-threshold-automation.md | 111 +++++++++++++++--- 1 file changed, 94 insertions(+), 17 deletions(-) diff --git a/proposals/0072-feature-gate-threshold-automation.md b/proposals/0072-feature-gate-threshold-automation.md index fc01bb354..22170fb9d 100644 --- a/proposals/0072-feature-gate-threshold-automation.md +++ b/proposals/0072-feature-gate-threshold-automation.md @@ -48,6 +48,9 @@ beneficial. that will own all feature accounts. - **Staged Features PDA:** A PDA under the Feature Gate program used to track features submitted for activation per epoch. +- **Validator Support Signal PDA:** A PDA under the Feature Gate program used to + track a validator's support signal bitmask, which signals which features they + support. - **Get Epoch Stake Syscall:** The new syscall introduced in [SIMD 0133](./0133-syscall-get-epoch-stake.md) that returns the current epoch stake for a given vote account address. @@ -172,12 +175,49 @@ supported by their software. A node signals its support for staged features by invoking another new Feature Gate program instruction: `SignalSupportForStagedFeatures`. This instruction -expects: +expects the Validator Support Signal PDA (defined below) to either exist or be +funded with enough rent-exempt lamports to initialize state. The processor will +allocate, assign, and initialize the account if it does not exist. + +The `SignalSupportForStagedFeatures` instruction is structured as follows: - Data: A `u8` bit mask of the staged features. - Accounts: - Staged Features PDA: writable - - Vote account: signer + - Validator Support Signal PDA: writable + - Vote account + - Authorized voter: signer + - System program (required only for initializing) + +The authorized voter signer must match the authorized voter stored in the vote +account's state. + +The `SignalSupportForStagedFeatures` instruction processor will provide the +vote account's address to the `GetEpochStake` syscall to retrieve the stake +delegated to that vote account for the epoch. Then, using the `1` values +provided in the bitmask (defined below), the processor will add this stake value +to each corresponding feature ID's stake support in the Staged Features PDA. + +A node's stake support for features is always accounted for using their most +recently sent bitmask. Each time a node sends a bitmask, if a previous bitmask +was already sent for that node, their previous bitmask is first used to deduct +stake support before adding stake support for the features signalled by the new +bitmask. + +Similar to the `StageFeatureForActivation` instruction, the Clock sysvar will be +used to ensure the Staged Features PDA corresponding to the *current* epoch `N` was +provided. + +If a node does not send this instruction successfully during the current epoch, +their stake is not tallied. This is analogous to a node signalling support for +zero features. + +If a feature is revoked, the list of staged features will not change, and nodes +may still signal support for this feature. However, the runtime will not +activate this feature if its corresponding feature account no longer exists +on-chain. + +#### Signal Bitmask A bit mask is used as a compressed ordered list of indices. This has two main benefits: @@ -194,24 +234,61 @@ A `1` bit represents support for a feature. For example, for staged features `[A, B, C, D, E, F, G, H]`, if a node wishes to signal support for all features except `E` and `H`, their `u8` value would be 246, or `11110110`. -The `SignalSupportForStagedFeatures` instruction processor will provide the -vote account's address to the `GetEpochStake` syscall to retrieve the stake -delegated to that vote account for the epoch. Then, using the `1` values -provided in the bitmask, the processor will add this stake value to each -corresponding feature ID's stake support in the Staged Features PDA. +#### Validator Support Signal PDA State -Similar to the `StageFeatureForActivation` instruction, the Clock sysvar will be -used to ensure the Staged Features PDA corresponding to the *current* epoch `N` was -provided. +A Validator Support Signal PDA will be created for each vote account. It will +store the node's submitted bitmasks. -If a node does not send this instruction successfully during the current epoch, -their stake is not tallied. This is analogous to a node signalling support for -zero features. +As mentioned previously, a node's most recently submitted bitmask is considered +their signal for the epoch. Validator Support Signal state allows one bitmask +per epoch, but can store multiple epochs of historical bitmasks. This is useful +for querying stake support post-epoch. When a new bitmask is submitted for an +epoch with an existing entry in the account state, it is overwritten. -If a feature is revoked, the list of staged features will not change, and nodes -may still signal support for this feature. However, the runtime will not -activate this feature if its corresponding feature account no longer exists -on-chain. +These accounts are never garbage collected since they are reused every epoch. +This means nodes are only required to pay for rent-exemption once. + +The address of the Validator Support Signal PDA (for a given epoch?) is derived +as follows, where `vote_address` is the 32 bytes of the validator's vote address. + +``` +"support_signal" +``` + +The data for the Validator Support Signal PDA will be structured as follows: + +```c +#define MAX_SIGNALS 4 + +/** + * A validator's support signal bitmask along with the epoch the signal + * corresponds to. + */ +typedef struct { + /** + * The epoch the support signal corresponds to (u64 serialized to + * little-endian). + */ + uint8_t epoch[8]; + /** The support signal bitmask. */ + u8 signal; + /** Padding for 8-byte alignment. */ + uint8_t _padding[7]; +} SupportSignalWithEpoch; + +/** + * A validator's support signal bitmasks with their corresponding epochs. + */ +typedef struct { + /** + * The support signal bitmasks with their corresponding epochs. + */ + SupportSignalWithEpoch signals[MAX_SIGNALS]; +} ValidatorSupportSignal; +``` + +As depicted in the above layout, a validator's signal can be stored (and +queried) for up to 4 epochs. ### Step 4: Feature Activation From b87cadb83c6d551a5e0fdd8df1935a733f3c88e9 Mon Sep 17 00:00:00 2001 From: Joe Caulfield Date: Wed, 14 Aug 2024 10:46:24 -0400 Subject: [PATCH 28/30] update runtime step --- .../0072-feature-gate-threshold-automation.md | 31 +++++++++---------- 1 file changed, 15 insertions(+), 16 deletions(-) diff --git a/proposals/0072-feature-gate-threshold-automation.md b/proposals/0072-feature-gate-threshold-automation.md index 22170fb9d..d68f507cd 100644 --- a/proposals/0072-feature-gate-threshold-automation.md +++ b/proposals/0072-feature-gate-threshold-automation.md @@ -212,11 +212,6 @@ If a node does not send this instruction successfully during the current epoch, their stake is not tallied. This is analogous to a node signalling support for zero features. -If a feature is revoked, the list of staged features will not change, and nodes -may still signal support for this feature. However, the runtime will not -activate this feature if its corresponding feature account no longer exists -on-chain. - #### Signal Bitmask A bit mask is used as a compressed ordered list of indices. This has two main @@ -292,21 +287,25 @@ queried) for up to 4 epochs. ### Step 4: Feature Activation -During the epoch rollover, the runtime must load the Staged Features PDA -and calculate the stake - as a percentage of the total epoch stake - in support -for each feature ID to determine which staged features to activate. +At the end of the epoch, the runtime loads the Staged Features PDA for the +current epoch and calculates the stake - as a percentage of the total epoch +stake - in support of each feature to determine which staged features to +activate. Every feature whose stake support meets the required threshold must be activated. This threshold will be hard-coded in the runtime to 95% initially, -but future iterations on the process could allow feature key-holders to set a -custom threshold per-feature. - -As mentioned previously, if a feature was revoked, it will no longer exist -on-chain, and therefore will be not activated by the runtime, regardless of -calculated stake support. +but future iterations on the process could make this threshold configurable. + +Features can be revoked at any point up until this step (staged or unstaged). If +a feature is revoked, nodes may still have signalled support for it, but the +runtime will not activate the feature since the account will not exist on-chain. +For more information, see the +[Feature Gate Program's](https://github.com/solana-program/feature-gate) +`RevokePendingActivation` instruction, as proposed in +[SIMD 0089](./0089-programify-feature-gate-program.md). -If a feature is not activated, either because it has been revoked or it did not -meet the required stake support, it must be resubmitted according to Step 2. +If a feature is not activated, it must be resubmitted according to Step 2. If it +is revoked, it must be resubmitted according to Step 1. ## Alternatives Considered From 6e74a91149dc513195f128d06d7b799a45229418 Mon Sep 17 00:00:00 2001 From: Joe Caulfield Date: Mon, 9 Dec 2024 18:06:45 +0900 Subject: [PATCH 29/30] update state init requirement --- proposals/0072-feature-gate-threshold-automation.md | 12 +++++------- 1 file changed, 5 insertions(+), 7 deletions(-) diff --git a/proposals/0072-feature-gate-threshold-automation.md b/proposals/0072-feature-gate-threshold-automation.md index d68f507cd..85fc2e9d9 100644 --- a/proposals/0072-feature-gate-threshold-automation.md +++ b/proposals/0072-feature-gate-threshold-automation.md @@ -98,9 +98,9 @@ In the future, this authority could be replaced by validator governance. The multi-signature authority stages a feature for activation by invoking a new Feature Gate program instruction: `StageFeatureForActivation`. This instruction -expects the Staged Features PDA (defined below) to either exist or be funded -with enough rent-exempt lamports to initialize state. The processor will -allocate, assign, and initialize the account if it does not exist. +expects the Staged Features PDA (defined below) to either exist or be allocated +with sufficient space and owned by the Feature Gate program, in order to +initialize state. The `StageFeatureForActivation` instruction is structured as follows: @@ -109,7 +109,6 @@ The `StageFeatureForActivation` instruction is structured as follows: - Feature account - Staged Features PDA: writable - Multi-signature authority: signer - - System program (required only for initializing) Note that features can only be staged in the epoch prior to the target activation epoch. This means a feature staged by invoking @@ -176,8 +175,8 @@ supported by their software. A node signals its support for staged features by invoking another new Feature Gate program instruction: `SignalSupportForStagedFeatures`. This instruction expects the Validator Support Signal PDA (defined below) to either exist or be -funded with enough rent-exempt lamports to initialize state. The processor will -allocate, assign, and initialize the account if it does not exist. +allocated with sufficient space and owned by the Feature Gate program, in order +to initialize state. The `SignalSupportForStagedFeatures` instruction is structured as follows: @@ -187,7 +186,6 @@ The `SignalSupportForStagedFeatures` instruction is structured as follows: - Validator Support Signal PDA: writable - Vote account - Authorized voter: signer - - System program (required only for initializing) The authorized voter signer must match the authorized voter stored in the vote account's state. From 4591f5525dd971145bef67739f7ada9bbf520368 Mon Sep 17 00:00:00 2001 From: Joe Caulfield Date: Mon, 9 Dec 2024 19:42:11 +0900 Subject: [PATCH 30/30] mark as idea --- proposals/0072-feature-gate-threshold-automation.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/proposals/0072-feature-gate-threshold-automation.md b/proposals/0072-feature-gate-threshold-automation.md index 85fc2e9d9..eb969642f 100644 --- a/proposals/0072-feature-gate-threshold-automation.md +++ b/proposals/0072-feature-gate-threshold-automation.md @@ -6,7 +6,7 @@ authors: - Joe Caulfield category: Standard type: Core -status: Draft +status: Idea created: 2024-01-25 feature: (fill in with feature tracking issues once accepted) ---