Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

style!: issue15 - cleaner polynomial interface #21

Merged
merged 22 commits into from
Jan 8, 2025
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
22 commits
Select commit Hold shift + click to select a range
5d3ec00
style!: issue15 - cleaner polynomial interface
samlaf Jan 4, 2025
b07e0a0
refactor!: change kzg commit functions to take a specific form
samlaf Jan 6, 2025
339d309
test: fix tests after new breaking api changes
samlaf Jan 6, 2025
e8ad64e
style!: rename commit_coef_form -> commit_coeff_form
samlaf Jan 6, 2025
98139b4
test: fix benchmarks following api changes
samlaf Jan 6, 2025
d88faf5
style: fix lint issues
samlaf Jan 6, 2025
f61a433
fix: removed polymorphism on commit function
samlaf Jan 6, 2025
965fea8
docs: add README section about kzg commitment
samlaf Jan 6, 2025
5933b01
style: cargo fmt
samlaf Jan 6, 2025
0ed4496
docs: fix commitment explanation in readme
samlaf Jan 7, 2025
09444ff
docs: add doc comment for fn convert_by_padding_empty_byte
samlaf Jan 7, 2025
464ac34
docs: make read_file_chunks comment more precise
samlaf Jan 7, 2025
88eb5ff
docs: add "monomial" prefix to srs points in comment
samlaf Jan 7, 2025
c4c2e91
style: rename function to_coef_form -> to_coeff_form for consistency
samlaf Jan 7, 2025
123578b
docs: clarify some comments
samlaf Jan 7, 2025
5ee952e
docs: clean up and make much more precise data type documentation + r…
samlaf Jan 7, 2025
c3653a8
style: switch poly eval and coeff forms to have eval at top of file
samlaf Jan 7, 2025
e61ddcf
test: add test to check length of polynomials during conversions
samlaf Jan 7, 2025
194b32b
docs: move readme docs to lib.rs rust docs
samlaf Jan 7, 2025
8ca7620
docs: fix doc code that was failing
samlaf Jan 7, 2025
54626b1
style: cargo fmt
samlaf Jan 8, 2025
5c8e0ac
test: fix cargo doc test
samlaf Jan 8, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 7 additions & 9 deletions Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -8,17 +8,16 @@ description = "This library offers a set of functions for generating and interac
readme = "README.md"
repository = "https://github.com/Layr-Labs/rust-kzg-bn254"
license-file = "LICENSE"
exclude = [
"tests/*",
"benches/*",
]
exclude = ["tests/*", "benches/*"]
# TODO: is this needed for the image to show up in the rust docs?
include = ["./kzg_commitment_diagram.png"]

[dependencies]
ark-bn254 = "0.5.0"
ark-ec = {version = "0.5.0", features = ["parallel"]}
ark-ff = {version = "0.5.0", features = ["parallel"]}
ark-ec = { version = "0.5.0", features = ["parallel"] }
ark-ff = { version = "0.5.0", features = ["parallel"] }
ark-serialize = "0.5.0"
ark-std = {version = "0.5.0", features = ["parallel"]}
ark-std = { version = "0.5.0", features = ["parallel"] }
directories = "5.0.1"
hex-literal = "0.4.1"
rand = "0.8.5"
Expand All @@ -28,7 +27,7 @@ num-bigint = "0.4"
rayon = "^1.5"
num-traits = "0.2"
byteorder = "1.4"
ark-poly = {version = "0.5.0", features = ["parallel"]}
ark-poly = { version = "0.5.0", features = ["parallel"] }
crossbeam-channel = "0.5"
num_cpus = "1.13.0"
sys-info = "0.9"
Expand Down Expand Up @@ -109,4 +108,3 @@ panic = 'unwind'
incremental = false
codegen-units = 16
rpath = false

47 changes: 9 additions & 38 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,56 +1,27 @@
# rust-kzg-bn254

## Description
[![Docs](https://docs.rs/rust-kzg-bn254/badge.svg)](https://docs.rs/rust-kzg-bn254/latest/rust_kzg_bn254/)
[![Crate](https://img.shields.io/crates/v/rust-kzg-bn254.svg)](https://crates.io/crates/rust-kzg-bn254)

This library offers a set of functions for generating and interacting with bn254 KZG commitments and proofs in rust, with the motivation of supporting fraud and validity proof logic in EigenDA rollup integrations.

## Warning & Disclaimer

This code is unaudited and under construction. This is experimental software and is provided on an "as is" and "as available" basis and may not work at all. It should not be used in production.

## Setup for testing

1. To test, please download the provided G1 and G2 points from [DA Resources](https://github.com/Layr-Labs/eigenda/tree/master/inabox/resources/kzg),
2. Specify these files in the `kzg.setup()` function, leave the `g2_power_of2_path` empty, and specify `srs_order` to be 3000.

## Configuring with the EigenDA KZG trusted setup

1. Follow the setup instructions to download the G1 and G2 powers of 2 points from the [Operator Setup Guide](https://github.com/Layr-Labs/eigenda-operator-setup)
2. Specify the files in `kzg.setup()` function, leave the `g2_points` empty, and specify the `srs_order` per the guide.
3. Note that this is process will take a few minutes to load since it is a bit intensive.

## Clippy
Linting can be triggered via running `cargo clippy --all --manifest-path Cargo.toml -- -D warnings`.

## Quick Start

See the `test_compute_kzg_proof` function in [./tests/kzg_test.rs](./tests/kzg_test.rs#) for an end to end usage of the library.

## Requirements

1. SRS points required are in the same format as provided by EigenDA.
2. Committing is performed in Lagrange format. The required IFFT is done within the function and is not required to be performed separately.
3. For proof generation, the data is treated as evaluation of polynomial. The required (I)FFT is performed by the compute function and is not required to be performed separately.
See the `test_compute_kzg_proof` function in [./tests/kzg_test.rs](./tests/kzg_test.rs) for an end to end usage of the library.

## Function Reference
Also make sure to check out the examples in our [docs](https://docs.rs/rust-kzg-bn254/latest/rust_kzg_bn254/).

### `from_bytes_and_pad()`

The `Blob` is loaded with `from_bytes_and_pad` which accepts bytes and "pads" it so that the data fits within the requirements of Eigen DA functioning. It also keeps track of the blob length after padding.

### `to_polynomial()`

From the `Blob`, a polynomial can be obtained via calling the `to_polynomial()` function. This converts the Blob to Field elements, then calculates the next power of 2 from this length of field elements and appends `zero` value elements for the remaining length.

### `data_setup_custom` and `data_setup_mins` parameters

The `data_setup_custom` (for testing) or `data_setup_mins` should be used to specify the number of chunks and chunk length. These parameters are used to calculate the FFT params required for FFT operations.

### `commit()`

The `commit` function takes in a `polynomial`. It is computed over `lagrange` basis by performing the (i)FFT depending on the `polynomial` form specified.
## Setup for testing

1. To test, please download the provided G1 and G2 points from [DA Resources](https://github.com/Layr-Labs/eigenda/tree/master/inabox/resources/kzg),
2. Specify these files in the `kzg.setup()` function, leave the `g2_power_of2_path` empty, and specify `srs_order` to be 3000.

### `compute_kzg_proof_with_roots_of_unity()`
## Warning & Disclaimer

The `compute_kzg_proof_with_roots_of_unity` takes in a `Polynomial` and an `index` at which it needs to be computed.
This code is unaudited and under construction. This is experimental software and is provided on an "as is" and "as available" basis and may not work at all. It should not be used in production.
4 changes: 2 additions & 2 deletions benches/bench_g1_ifft.rs
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
use criterion::{black_box, criterion_group, criterion_main, Criterion};
use rust_kzg_bn254::kzg::Kzg;
use rust_kzg_bn254::kzg::KZG;
use std::time::Duration;

fn generate_powers_of_2(limit: u64) -> Vec<usize> {
Expand All @@ -16,7 +16,7 @@ fn generate_powers_of_2(limit: u64) -> Vec<usize> {

fn bench_g1_ifft(c: &mut Criterion) {
c.bench_function("bench_g1_ifft", |b| {
let kzg = Kzg::setup(
let kzg = KZG::setup(
"tests/test-files/mainnet-data/g1.131072.point",
"",
"tests/test-files/mainnet-data/g2.point.powerOf2",
Expand Down
22 changes: 8 additions & 14 deletions benches/bench_kzg_commit.rs
Original file line number Diff line number Diff line change
@@ -1,11 +1,11 @@
use criterion::{criterion_group, criterion_main, Criterion};
use rand::Rng;
use rust_kzg_bn254::{blob::Blob, kzg::Kzg, polynomial::PolynomialFormat};
use rust_kzg_bn254::{blob::Blob, kzg::KZG};
use std::time::Duration;

fn bench_kzg_commit(c: &mut Criterion) {
let mut rng = rand::thread_rng();
let mut kzg = Kzg::setup(
let mut kzg = KZG::setup(
"tests/test-files/mainnet-data/g1.131072.point",
"",
"tests/test-files/mainnet-data/g2.point.powerOf2",
Expand All @@ -17,34 +17,28 @@ fn bench_kzg_commit(c: &mut Criterion) {
c.bench_function("bench_kzg_commit_10000", |b| {
let random_blob: Vec<u8> = (0..10000).map(|_| rng.gen_range(32..=126) as u8).collect();
let input = Blob::from_raw_data(&random_blob);
let input_poly = input
.to_polynomial(PolynomialFormat::InCoefficientForm)
.unwrap();
let input_poly = input.to_polynomial_coeff_form();
kzg.data_setup_custom(1, input.len().try_into().unwrap())
.unwrap();
b.iter(|| kzg.commit(&input_poly).unwrap());
b.iter(|| kzg.commit_coeff_form(&input_poly).unwrap());
});

c.bench_function("bench_kzg_commit_30000", |b| {
let random_blob: Vec<u8> = (0..30000).map(|_| rng.gen_range(32..=126) as u8).collect();
let input = Blob::from_raw_data(&random_blob);
let input_poly = input
.to_polynomial(PolynomialFormat::InCoefficientForm)
.unwrap();
let input_poly = input.to_polynomial_coeff_form();
kzg.data_setup_custom(1, input.len().try_into().unwrap())
.unwrap();
b.iter(|| kzg.commit(&input_poly).unwrap());
b.iter(|| kzg.commit_coeff_form(&input_poly).unwrap());
});

c.bench_function("bench_kzg_commit_50000", |b| {
let random_blob: Vec<u8> = (0..50000).map(|_| rng.gen_range(32..=126) as u8).collect();
let input = Blob::from_raw_data(&random_blob);
let input_poly = input
.to_polynomial(PolynomialFormat::InCoefficientForm)
.unwrap();
let input_poly = input.to_polynomial_coeff_form();
kzg.data_setup_custom(1, input.len().try_into().unwrap())
.unwrap();
b.iter(|| kzg.commit(&input_poly).unwrap());
b.iter(|| kzg.commit_coeff_form(&input_poly).unwrap());
});
}

Expand Down
16 changes: 6 additions & 10 deletions benches/bench_kzg_commit_large_blobs.rs
Original file line number Diff line number Diff line change
@@ -1,11 +1,11 @@
use criterion::{criterion_group, criterion_main, Criterion};
use rand::Rng;
use rust_kzg_bn254::{blob::Blob, kzg::Kzg, polynomial::PolynomialFormat};
use rust_kzg_bn254::{blob::Blob, kzg::KZG};
use std::time::Duration;

fn bench_kzg_commit(c: &mut Criterion) {
let mut rng = rand::thread_rng();
let mut kzg = Kzg::setup(
let mut kzg = KZG::setup(
"tests/test-files/mainnet-data/g1.32mb.point",
"",
"tests/test-files/mainnet-data/g2.point.powerOf2",
Expand All @@ -19,25 +19,21 @@ fn bench_kzg_commit(c: &mut Criterion) {
.map(|_| rng.gen_range(32..=126) as u8)
.collect();
let input = Blob::from_raw_data(&random_blob);
let input_poly = input
.to_polynomial(PolynomialFormat::InCoefficientForm)
.unwrap();
let input_poly = input.to_polynomial_coeff_form();
kzg.data_setup_custom(1, input.len().try_into().unwrap())
.unwrap();
b.iter(|| kzg.commit(&input_poly).unwrap());
b.iter(|| kzg.commit_coeff_form(&input_poly).unwrap());
});

c.bench_function("bench_kzg_commit_16mb", |b| {
let random_blob: Vec<u8> = (0..16_252_000)
.map(|_| rng.gen_range(32..=126) as u8)
.collect();
let input = Blob::from_raw_data(&random_blob);
let input_poly = input
.to_polynomial(PolynomialFormat::InCoefficientForm)
.unwrap();
let input_poly = input.to_polynomial_coeff_form();
kzg.data_setup_custom(1, input.len().try_into().unwrap())
.unwrap();
b.iter(|| kzg.commit(&input_poly).unwrap());
b.iter(|| kzg.commit_coeff_form(&input_poly).unwrap());
});
}

Expand Down
28 changes: 11 additions & 17 deletions benches/bench_kzg_proof.rs
Original file line number Diff line number Diff line change
@@ -1,11 +1,11 @@
use criterion::{criterion_group, criterion_main, Criterion};
use rand::Rng;
use rust_kzg_bn254::{blob::Blob, kzg::Kzg, polynomial::PolynomialFormat};
use rust_kzg_bn254::{blob::Blob, kzg::KZG};
use std::time::Duration;

fn bench_kzg_proof(c: &mut Criterion) {
let mut rng = rand::thread_rng();
let mut kzg = Kzg::setup(
let mut kzg = KZG::setup(
"tests/test-files/mainnet-data/g1.131072.point",
"",
"tests/test-files/mainnet-data/g2.point.powerOf2",
Expand All @@ -17,47 +17,41 @@ fn bench_kzg_proof(c: &mut Criterion) {
c.bench_function("bench_kzg_proof_10000", |b| {
let random_blob: Vec<u8> = (0..10000).map(|_| rng.gen_range(32..=126) as u8).collect();
let input = Blob::from_raw_data(&random_blob);
let input_poly = input
.to_polynomial(PolynomialFormat::InCoefficientForm)
.unwrap();
let input_poly = input.to_polynomial_eval_form();
kzg.data_setup_custom(1, input.len().try_into().unwrap())
.unwrap();
let index =
rand::thread_rng().gen_range(0..input_poly.get_length_of_padded_blob_as_fr_vector());
rand::thread_rng().gen_range(0..input_poly.len_underlying_blob_field_elements());
b.iter(|| {
kzg.compute_kzg_proof_with_roots_of_unity(&input_poly, index.try_into().unwrap())
kzg.compute_proof_with_roots_of_unity(&input_poly, index.try_into().unwrap())
.unwrap()
});
});

c.bench_function("bench_kzg_proof_30000", |b| {
let random_blob: Vec<u8> = (0..30000).map(|_| rng.gen_range(32..=126) as u8).collect();
let input = Blob::from_raw_data(&random_blob);
let input_poly = input
.to_polynomial(PolynomialFormat::InCoefficientForm)
.unwrap();
let input_poly = input.to_polynomial_eval_form();
kzg.data_setup_custom(1, input.len().try_into().unwrap())
.unwrap();
let index =
rand::thread_rng().gen_range(0..input_poly.get_length_of_padded_blob_as_fr_vector());
rand::thread_rng().gen_range(0..input_poly.len_underlying_blob_field_elements());
b.iter(|| {
kzg.compute_kzg_proof_with_roots_of_unity(&input_poly, index.try_into().unwrap())
kzg.compute_proof_with_roots_of_unity(&input_poly, index.try_into().unwrap())
.unwrap()
});
});

c.bench_function("bench_kzg_proof_50000", |b| {
let random_blob: Vec<u8> = (0..50000).map(|_| rng.gen_range(32..=126) as u8).collect();
let input = Blob::from_raw_data(&random_blob);
let input_poly = input
.to_polynomial(PolynomialFormat::InCoefficientForm)
.unwrap();
let input_poly = input.to_polynomial_eval_form();
kzg.data_setup_custom(1, input.len().try_into().unwrap())
.unwrap();
let index =
rand::thread_rng().gen_range(0..input_poly.get_length_of_padded_blob_as_fr_vector());
rand::thread_rng().gen_range(0..input_poly.len_underlying_blob_field_elements());
b.iter(|| {
kzg.compute_kzg_proof_with_roots_of_unity(&input_poly, index.try_into().unwrap())
kzg.compute_proof_with_roots_of_unity(&input_poly, index.try_into().unwrap())
.unwrap()
});
});
Expand Down
6 changes: 3 additions & 3 deletions benches/bench_kzg_setup.rs
Original file line number Diff line number Diff line change
@@ -1,11 +1,11 @@
use criterion::{criterion_group, criterion_main, Criterion};
use rust_kzg_bn254::kzg::Kzg;
use rust_kzg_bn254::kzg::KZG;
use std::time::Duration;

fn bench_kzg_setup(c: &mut Criterion) {
c.bench_function("bench_kzg_setup", |b| {
b.iter(|| {
Kzg::setup(
KZG::setup(
"tests/test-files/g1.point",
"tests/test-files/g2.point",
"tests/test-files/g2.point.powerOf2",
Expand All @@ -16,7 +16,7 @@ fn bench_kzg_setup(c: &mut Criterion) {
});

b.iter(|| {
Kzg::setup(
KZG::setup(
"tests/test-files/mainnet-data/g1.131072.point",
"",
"tests/test-files/mainnet-data/g2.point.powerOf2",
Expand Down
Loading
Loading