feat(share/availability/full): Introduce Q4 trimming for archival nodes #4028
Conversation
Codecov ReportAttention: Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #4028 +/- ##
==========================================
+ Coverage 44.83% 45.80% +0.97%
==========================================
Files 265 307 +42
Lines 14620 22073 +7453
==========================================
+ Hits 6555 10111 +3556
- Misses 7313 10894 +3581
- Partials 752 1068 +316 ☔ View full report in Codecov by Sentry. |
| @@ -10,4 +10,5 @@ import ( | |||
| // from the node's datastore. | |||
| type Pruner interface { | |||
| Prune(context.Context, *header.ExtendedHeader) error | |||
| Kind() string | |||
There was a problem hiding this comment.
Okay so i hate this
we can implicitly do this by checking for the presence of a historical block (without its .q4 file) but I think that could be much more invasive and annoying to sort out.
There was a problem hiding this comment.
I think this is the cleanest option as of right now.
| @@ -42,51 +34,55 @@ func ConstructModule(tp node.Type, cfg *Config) fx.Option { | |||
| // This is necessary to invoke the pruner service as independent thanks to a | |||
| // quirk in FX. | |||
| fx.Invoke(func(_ *pruner.Service) {}), | |||
| fx.Invoke(func(ctx context.Context, ds datastore.Batching, p pruner.Pruner) error { | |||
| return pruner.DetectPreviousRun(ctx, ds, p.Kind()) | |||
There was a problem hiding this comment.
This can now run on all node types bc the check will be ignored if previous pruner kind == current pruner kind.
|
|
||
| if cp.PrunerKind != expectedKind { | ||
| // do not allow reversion back to archival mode | ||
| if cp.PrunerKind == "full" { |
There was a problem hiding this comment.
happy to extract into consts (archival, full, light)
renaynay
changed the title
renaynay
force-pushed
the
archival-trim
branch
from
69d8bbe to
2b062ba
Compare
This PR introduces trimming of
.q4files outside the sampling window for archival nodes.It also updates the pruner checkpoint such that it's possible to convert from an archival node --> full pruning node but not the other way around (the node runner would need to resync the node in this case).
All deletion logic now lives in either
share/availability/lightorshare/availability/fullbut the pruner service is still contained in a separate package and now runs by default as all node types will have some form of pruning happening.