Skip to content

Commit

Permalink
[readme] provide detailed information about bonds processing (#146)
Browse files Browse the repository at this point in the history
* [readme] provide detailed information about bonds processing

* [readme] adding info on delegated rpc query

* [review] fixes from review
  • Loading branch information
ochaloup authored Dec 10, 2024
1 parent 1ef8d28 commit 793c6ed
Show file tree
Hide file tree
Showing 4 changed files with 220 additions and 33 deletions.
116 changes: 113 additions & 3 deletions packages/validator-bonds-cli/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -478,22 +478,126 @@ Configuration parameters:
validator-bonds -um show-config vbMaRfmTCg92HWGzmd53APkMNpPnGVGZTUHwUJQkXAU
```

## Details on Bond Processing

Bond calculation and settlement occur with a one-epoch delay.
Funds are charged at the start of epoch X+1 based on data from epoch X.

### Auction

The auction for epoch X determines the effective bid (`auctionEffectiveBidPmpe`)
for each validator for that epoch.
This value, calculated from bids across participating validators,
defines the SOL cost per 10,000 SOL staked.

**Example:**
If `auctionEffectiveBidPmpe` = `0.123` and a validator is delegated 100K SOL by Marinade,
the payment is:
`0.123 * 100,000 / 10,000 = 1.23 SOL`.

**Access Data:**
- The results of the auction are stored within the pipeline results
https://github.com/marinade-finance/ds-sam-pipeline/tree/main/auctions
(see the JSON file `<epoch>/outputs/results.json`)
- The data are loaded to API and are available at
https://scoring.marinade.finance/api/v1/scores/sam?epoch=X
- The data is displayed at dashboard https://psr.marinade.finance/

#### Settlement Creation

Using delegated stake and auction results from epoch X, Bonds processing creates on-chain
`Settlement` data at the start of epoch X+1. These are funded from the validator's Bond
based on the auction outcome.
The processing runs at the start of epoch X+1 for epoch X, as it is only then clear how many
SOLs Marinade delegated to each validator. The data is sourced from the Solana snapshot
taken at the end of epoch X.

Settlements can be claimed by stakers for 4 epochs.
Unclaimed funds are returned to the validator's Bond.

- **Access Data:**
- Discord: [PSR feed channel](https://discord.com/channels/823564092379627520/1223330302890348754).
- Historical data: [Google Cloud storage](https://console.cloud.google.com/storage/browser/marinade-validator-bonds-mainnet).

### PSR Events

Bonds can also be charged for [PSR events](https://marinade.finance/how-it-works/psr).

**Note:**
The term "uptime" refers to "voting uptime," i.e., the number of
[vote credits](https://docs.anza.xyz/proposals/timely-vote-credits) earned.
Bond calculations ensure validators earn inflation rewards equal to or above the network average.
Validators below the standard are charged to cover the shortfall,
and a Settlement is created for this purpose.

### Verifying Charged Amounts

Validators can verify charged amounts and funded SOLs on-chain.

**Options:**
- **Current State:** Use the [CLI show command](#show-the-bond-account) to see the current on-chain Bond state
- _NOTE:_ data from `show-bond` represents current on-chain data not data used
for bonds calculation of particular epoch
- **Historical Data:**
- Dashboard: [PSR Bonds Dashboard](https://psr.marinade.finance/).
- Auction data: [Auction scores API](https://scoring.marinade.finance/api/v1/scores/sam?epoch=X).
- Settlement data: [Google Cloud storage](https://console.cloud.google.com/storage/browser/marinade-validator-bonds-mainnet).

For advanced on-chain queries, refer to the [on-chain analysis documentation](../../programs/validator-bonds/ON_CHAIN_ANALYSIS.md).


## Searching Bonds funded stake accounts

Bond program assigns the funded stake accounts with `withdrawal` authority of address
`7cgg6KhPd1G8oaoB48RyPDWu7uZs51jUpDYB3eq4VebH`. To query the all stake accounts
one may use the RPC call of `getProgramAccounts`.
`7cgg6KhPd1G8oaoB48RyPDWu7uZs51jUpDYB3eq4VebH`.

Technical details of the stake account layout can be found in Solana source code [for staker and withdrawer](https://github.com/solana-labs/solana/blob/v1.17.15/sdk/program/src/stake/state.rs#L60)
and for [voter pubkey](https://github.com/solana-labs/solana/blob/v1.17.15/sdk/program/src/stake/state.rs#L414).

To query all the stake accounts
one may use the RPC call of `getProgramAccounts`.

```sh
RPC_URL='https://api.mainnet-beta.solana.com'
curl $RPC_URL -X POST -H "Content-Type: application/json" -d '
{
"jsonrpc": "2.0",
"id": 1,
"method": "getProgramAccounts",
"params": [
"Stake11111111111111111111111111111111111111",
{
"encoding": "base64",
"dataSlice": {
"offset": 0,
"length": 0
},
"filters": [
{
"memcmp": {
"offset": 44,
"bytes": "7cgg6KhPd1G8oaoB48RyPDWu7uZs51jUpDYB3eq4VebH"
}
}
]
}
]
}
' | jq '.'
```

To query by parameters one needs to add an offset of the data.
For all stake accounts assigned under Bond and delegated to a validator
one uses voter key.

```
STAKER_OFFSET = 12 // 4 for enum, 8 rent exempt reserve
WITHDRAWER_OFFSET = 44 // 4 + 8 + staker pubkey
// to whom the stake is delegated
VOTER_PUBKEY_OFFSET = 124 // 4 for enum + 120 for Meta
```

```sh
```
RPC_URL='https://api.mainnet-beta.solana.com'
curl $RPC_URL -X POST -H "Content-Type: application/json" -d '
{
Expand All @@ -514,6 +618,12 @@ curl $RPC_URL -X POST -H "Content-Type: application/json" -d '
"offset": 44,
"bytes": "7cgg6KhPd1G8oaoB48RyPDWu7uZs51jUpDYB3eq4VebH"
}
},
{
"memcmp": {
"offset": 124,
"bytes": "<<vote account address>>"
}
}
]
}
Expand Down
73 changes: 43 additions & 30 deletions programs/validator-bonds/ON_CHAIN_ANALYSIS.md
Original file line number Diff line number Diff line change
Expand Up @@ -115,48 +115,42 @@ WHERE 1=1
ORDER BY floor(block_id/432000) ASC
```


## Query Validator Bonds instructions event data

One can query the Anchor instructions for events as they are emitted in the contract code,
see e.g., [FundBond event](https://github.com/marinade-finance/validator-bonds/blob/contract-v2.0.0/programs/validator-bonds/src/instructions/bond/fund_bond.rs#L127).

```sql
select block_timestamp, ixs.value:data
from solana.core.fact_events fe
inner JOIN solana.core.fact_transactions ft USING(block_timestamp, tx_id, succeeded)
,LATERAL FLATTEN(input => fe.inner_instruction:instructions) ixs,
where 1=1
and fe.succeeded
and fe.program_id = 'vBoNdEvzMrSai7is21XgVYik65mqtaKXuSdMBJ1xkW4'
and fe.block_timestamp > current_date - 7
-- and fe.block_timestamp < current_date - 7
and array_contains('Program log: Instruction: FundBond'::variant, ft.log_messages)
and array_contains('<<bond-account-pubkey>>'::variant, fe.instruction:accounts)
-- filter the list of inner instructions for only the emit log CPI events
-- this instruction is always emitted by the same Anchor CPI PDA defined for the bond program
and array_contains('j6cZKhHTFuWsiCgPT5wriQpZWqWWUSQqjDJ8S2YDvDL'::variant, ixs.value:accounts)
ORDER BY block_timestamp ASC
```

```sql
-- FundSettlement
select block_timestamp, ixs.value:data
from solana.core.fact_events fe
inner JOIN solana.core.fact_transactions ft USING(block_timestamp, tx_id, succeeded)
,LATERAL FLATTEN(input => fe.inner_instruction:instructions) ixs,
where
fe.succeeded
SELECT
-- floor(block_id/432000) as epoch,
-- tx_id,
block_timestamp,
block_id,
ixs.value:data
FROM solana.core.fact_events fe
INNER JOIN
solana.core.fact_transactions ft USING(block_timestamp, tx_id, succeeded),
LATERAL FLATTEN(input => fe.inner_instruction:instructions) ixs
WHERE fe.succeeded
-- and fe.block_id >= 700*432000
and fe.program_id = 'vBoNdEvzMrSai7is21XgVYik65mqtaKXuSdMBJ1xkW4'

-- Type of INSTRUCTION searching for
-- and array_contains('Program log: Instruction: FundBond'::variant, ft.log_messages)
and array_contains('Program log: Instruction: FundSettlement'::variant, ft.log_messages)
-- vote account: CaraHZBReeNNYAJ326DFsvy41M2p1KWTEoBAwBL6bmWZ -> bond account: 3zVyxrxkR2a3oWoBku7CaAG7UW6JS65vqyoTdG1Djig9
and array_contains('3zVyxrxkR2a3oWoBku7CaAG7UW6JS65vqyoTdG1Djig9'::variant, fe.instruction:accounts)
-- and array_contains('Program log: Instruction: ClaimWithdrawRequest'::variant, ft.log_messages)

-- filter instructions by Bond pubkey
and array_contains('<<bond-account-pubkey>>'::variant, fe.instruction:accounts)

-- from the list of inner instructions getting only those that contains the CPI event data
-- the CPI PDA call address is always the same for bond program
and array_contains('j6cZKhHTFuWsiCgPT5wriQpZWqWWUSQqjDJ8S2YDvDL'::variant, ixs.value:accounts)
ORDER BY block_timestamp ASC
order by block_timestamp ASC;
```

### Investigate the event data

The string found in the `ixs.value:data` column can be decrypted using the
[Validator Bonds CLI](../../packages/validator-bonds-cli/README.md)
`show-event` command.
Expand All @@ -165,4 +159,23 @@ Run it like this:

```sh
pnpm --silent cli show-event -f json <<base58-format-cpi-event-data>>
```
```
or one can use the script [`parse-flipside-event-csv`](../../scripts/parse-flipside-event-csv.sh).
See following guideline:
1. Run the FlipSide query with Bond account defined within
https://flipsidecrypto.xyz/studio/queries
2. Download the `Results` CSV file
![Download the `Results` CSV file](../../resources/onchain/howto-download-results.png)
3. Get running the parsing script to list the funded amounts per transaction
```
./scripts/parse-flipside-event-csv.sh ~/Downloads/download-query-results-37ee1ecb-3e1b-438d-b410-5a1d617ccbe3.csv
./scripts/parse-flipside-event-csv.sh: parsing file 'Downloads/download-query-results-37ee1ecb-3e1b-438d-b410-5a1d617ccbe3.csv'
Skipping 'BLOCK_TIMESTAMP'
2024-11-1600:13:35.000;698;7.335386511
2024-11-1722:27:48.000;699;0.501340858
2024-11-1722:59:21.000;699;17.833468698
2024-11-2008:40:44.000;700;9.445008408
Total lamports: 35.115204475
```
Binary file added resources/onchain/howto-download-results.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
64 changes: 64 additions & 0 deletions scripts/parse-flipside-event-csv.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,64 @@
#!/bin/bash

SLOTS_PER_EPOCH=432000

solsdecimal() {
N="$@"
if [ "$N" = "null" ]; then
echo "unknown"
return 1
fi
DECIMALS=9
[ "x$N" == "x" ] && N=`xsel -p -o`
[ "x$N" == "x" ] && echo "No input" && return 1
if [ ${#N} -lt $DECIMALS ]; then
FILLING_ZEROS=$(printf "%0.s0" $(seq 1 $((9-${#N}))))
echo "0.${FILLING_ZEROS}${N}"
else
SOLS="${N::-$DECIMALS}"
echo "${SOLS:-0}.${N:${#SOLS}}"
fi
}

parse_csv_line() {
local input_file="$(realpath "$1")"

SUM_LAMPORTS=0
cd ~/marinade/validator-bonds/
# set -x
# expecting a csv file with the following format: timestamp,blockid,data
while IFS=, read -r timestamp blockid data; do
# Remove any trailing whitespace
timestamp=$(echo "$timestamp" | tr -d '[:space:]')
blockid=$(echo "$blockid" | tr -d '[:space:]')
data=$(echo "$data" | tr -d '[:space:]')

EVENT=$(pnpm run --silent -- cli show-event "$data" -f json)
[[ $? -ne 0 ]] && echo "Skipping '$timestamp'" && continue >&2

LAMPORTS=$(echo "$EVENT" | jq '.data.fundingAmount')
LAMPORTS_FUNDED=$(echo "$EVENT" | jq '.data.lamportsFunded')
if [ "$LAMPORTS" = "null" ]; then
LAMPORTS=$(echo "$EVENT" | jq '.data.depositedAmount')
fi
SETTLEMENT=$(echo "$EVENT" | jq '.data.settlement')
if [ "$LAMPORTS" = "null" ]; then
echo "$EVENT"
break
fi

SUM_LAMPORTS=$((SUM_LAMPORTS+LAMPORTS))
SOLS=$(solsdecimal $LAMPORTS)
EPOCH=$((blockid/$SLOTS_PER_EPOCH))
echo -n "$timestamp;$EPOCH;$SOLS"
# [ "$SETTLEMENT" != "null" ] && echo -n ";settlement: $SETTLEMENT"
# [ "$LAMPORTS_FUNDED" != "null" ] && echo -n ";sumFunded:$(solsdecimal $LAMPORTS_FUNDED)"
echo
done < "$input_file"
echo "Total lamports: $(solsdecimal $SUM_LAMPORTS)"

cd -
}

echo "$0: parsing file '$1'"
parse_csv_line "$1"

0 comments on commit 793c6ed

Please sign in to comment.