Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

style!: issue15 - cleaner polynomial interface #21

Merged
merged 22 commits into from
Jan 8, 2025
Merged
Show file tree
Hide file tree
Changes from 9 commits
Commits
Show all changes
22 commits
Select commit Hold shift + click to select a range
5d3ec00
style!: issue15 - cleaner polynomial interface
samlaf Jan 4, 2025
b07e0a0
refactor!: change kzg commit functions to take a specific form
samlaf Jan 6, 2025
339d309
test: fix tests after new breaking api changes
samlaf Jan 6, 2025
e8ad64e
style!: rename commit_coef_form -> commit_coeff_form
samlaf Jan 6, 2025
98139b4
test: fix benchmarks following api changes
samlaf Jan 6, 2025
d88faf5
style: fix lint issues
samlaf Jan 6, 2025
f61a433
fix: removed polymorphism on commit function
samlaf Jan 6, 2025
965fea8
docs: add README section about kzg commitment
samlaf Jan 6, 2025
5933b01
style: cargo fmt
samlaf Jan 6, 2025
0ed4496
docs: fix commitment explanation in readme
samlaf Jan 7, 2025
09444ff
docs: add doc comment for fn convert_by_padding_empty_byte
samlaf Jan 7, 2025
464ac34
docs: make read_file_chunks comment more precise
samlaf Jan 7, 2025
88eb5ff
docs: add "monomial" prefix to srs points in comment
samlaf Jan 7, 2025
c4c2e91
style: rename function to_coef_form -> to_coeff_form for consistency
samlaf Jan 7, 2025
123578b
docs: clarify some comments
samlaf Jan 7, 2025
5ee952e
docs: clean up and make much more precise data type documentation + r…
samlaf Jan 7, 2025
c3653a8
style: switch poly eval and coeff forms to have eval at top of file
samlaf Jan 7, 2025
e61ddcf
test: add test to check length of polynomials during conversions
samlaf Jan 7, 2025
194b32b
docs: move readme docs to lib.rs rust docs
samlaf Jan 7, 2025
8ca7620
docs: fix doc code that was failing
samlaf Jan 7, 2025
54626b1
style: cargo fmt
samlaf Jan 8, 2025
5c8e0ac
test: fix cargo doc test
samlaf Jan 8, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
19 changes: 12 additions & 7 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -34,11 +34,7 @@ See the `test_compute_kzg_proof` function in [./tests/kzg_test.rs](./tests/kzg_t

## Function Reference

### `from_bytes_and_pad()`

The `Blob` is loaded with `from_bytes_and_pad` which accepts bytes and "pads" it so that the data fits within the requirements of Eigen DA functioning. It also keeps track of the blob length after padding.

### `to_polynomial()`
### `to_polynomial_coeff_form()` / `to_polynomial_eval_form()`

From the `Blob`, a polynomial can be obtained via calling the `to_polynomial()` function. This converts the Blob to Field elements, then calculates the next power of 2 from this length of field elements and appends `zero` value elements for the remaining length.

Expand All @@ -51,6 +47,15 @@ The `data_setup_custom` (for testing) or `data_setup_mins` should be used to spe
The `commit` function takes in a `polynomial`. It is computed over `lagrange` basis by performing the (i)FFT depending on the `polynomial` form specified.


### `compute_kzg_proof_with_roots_of_unity()`
### `compute_proof_with_roots_of_unity()`

The `compute_proof_with_roots_of_unity` takes in a `Polynomial` and an `index` at which it needs to be computed.

## KZG Commitments

Below diagram explains the difference types involved between polynomials, SRS points, and kzg commitments.
A KZG commitment can be taken by an inner product between (poly_eval, srs_monomial) or (poly_coeff, srs_lagrange). FFT and IFFT operations can be performed to convert between these forms.
samlaf marked this conversation as resolved.
Show resolved Hide resolved

![KZG Commitments](./kzg_commitment_diagram.png)

The `compute_kzg_proof_with_roots_of_unity` takes in a `Polynomial` and an `index` at which it needs to be computed.
Our current codebase has the types PolynomialEvalForm and PolynomialCoeffForm to represent the polynomial in evaluation and coefficient form respectively. However, we do not have types to represent the two forms of srs points. They are implicitly assumed to be in monomial form when loaded, and an IFFT is performed before taking the inner product with the polynomial in evaluation form.
bxue-l2 marked this conversation as resolved.
Show resolved Hide resolved
20 changes: 7 additions & 13 deletions benches/bench_kzg_commit.rs
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
use criterion::{criterion_group, criterion_main, Criterion};
use rand::Rng;
use rust_kzg_bn254::{blob::Blob, kzg::Kzg, polynomial::PolynomialFormat};
use rust_kzg_bn254::{blob::Blob, kzg::Kzg};
use std::time::Duration;

fn bench_kzg_commit(c: &mut Criterion) {
Expand All @@ -17,34 +17,28 @@ fn bench_kzg_commit(c: &mut Criterion) {
c.bench_function("bench_kzg_commit_10000", |b| {
let random_blob: Vec<u8> = (0..10000).map(|_| rng.gen_range(32..=126) as u8).collect();
let input = Blob::from_raw_data(&random_blob);
let input_poly = input
.to_polynomial(PolynomialFormat::InCoefficientForm)
.unwrap();
let input_poly = input.to_polynomial_coeff_form();
kzg.data_setup_custom(1, input.len().try_into().unwrap())
.unwrap();
b.iter(|| kzg.commit(&input_poly).unwrap());
b.iter(|| kzg.commit_coeff_form(&input_poly).unwrap());
});

c.bench_function("bench_kzg_commit_30000", |b| {
let random_blob: Vec<u8> = (0..30000).map(|_| rng.gen_range(32..=126) as u8).collect();
let input = Blob::from_raw_data(&random_blob);
let input_poly = input
.to_polynomial(PolynomialFormat::InCoefficientForm)
.unwrap();
let input_poly = input.to_polynomial_coeff_form();
kzg.data_setup_custom(1, input.len().try_into().unwrap())
.unwrap();
b.iter(|| kzg.commit(&input_poly).unwrap());
b.iter(|| kzg.commit_coeff_form(&input_poly).unwrap());
});

c.bench_function("bench_kzg_commit_50000", |b| {
let random_blob: Vec<u8> = (0..50000).map(|_| rng.gen_range(32..=126) as u8).collect();
let input = Blob::from_raw_data(&random_blob);
let input_poly = input
.to_polynomial(PolynomialFormat::InCoefficientForm)
.unwrap();
let input_poly = input.to_polynomial_coeff_form();
kzg.data_setup_custom(1, input.len().try_into().unwrap())
.unwrap();
b.iter(|| kzg.commit(&input_poly).unwrap());
b.iter(|| kzg.commit_coeff_form(&input_poly).unwrap());
});
}

Expand Down
14 changes: 5 additions & 9 deletions benches/bench_kzg_commit_large_blobs.rs
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
use criterion::{criterion_group, criterion_main, Criterion};
use rand::Rng;
use rust_kzg_bn254::{blob::Blob, kzg::Kzg, polynomial::PolynomialFormat};
use rust_kzg_bn254::{blob::Blob, kzg::Kzg};
use std::time::Duration;

fn bench_kzg_commit(c: &mut Criterion) {
Expand All @@ -19,25 +19,21 @@ fn bench_kzg_commit(c: &mut Criterion) {
.map(|_| rng.gen_range(32..=126) as u8)
.collect();
let input = Blob::from_raw_data(&random_blob);
let input_poly = input
.to_polynomial(PolynomialFormat::InCoefficientForm)
.unwrap();
let input_poly = input.to_polynomial_coeff_form();
kzg.data_setup_custom(1, input.len().try_into().unwrap())
.unwrap();
b.iter(|| kzg.commit(&input_poly).unwrap());
b.iter(|| kzg.commit_coeff_form(&input_poly).unwrap());
});

c.bench_function("bench_kzg_commit_16mb", |b| {
let random_blob: Vec<u8> = (0..16_252_000)
.map(|_| rng.gen_range(32..=126) as u8)
.collect();
let input = Blob::from_raw_data(&random_blob);
let input_poly = input
.to_polynomial(PolynomialFormat::InCoefficientForm)
.unwrap();
let input_poly = input.to_polynomial_coeff_form();
kzg.data_setup_custom(1, input.len().try_into().unwrap())
.unwrap();
b.iter(|| kzg.commit(&input_poly).unwrap());
b.iter(|| kzg.commit_coeff_form(&input_poly).unwrap());
});
}

Expand Down
26 changes: 10 additions & 16 deletions benches/bench_kzg_proof.rs
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
use criterion::{criterion_group, criterion_main, Criterion};
use rand::Rng;
use rust_kzg_bn254::{blob::Blob, kzg::Kzg, polynomial::PolynomialFormat};
use rust_kzg_bn254::{blob::Blob, kzg::Kzg};
use std::time::Duration;

fn bench_kzg_proof(c: &mut Criterion) {
Expand All @@ -17,47 +17,41 @@ fn bench_kzg_proof(c: &mut Criterion) {
c.bench_function("bench_kzg_proof_10000", |b| {
let random_blob: Vec<u8> = (0..10000).map(|_| rng.gen_range(32..=126) as u8).collect();
let input = Blob::from_raw_data(&random_blob);
let input_poly = input
.to_polynomial(PolynomialFormat::InCoefficientForm)
.unwrap();
let input_poly = input.to_polynomial_eval_form();
kzg.data_setup_custom(1, input.len().try_into().unwrap())
.unwrap();
let index =
rand::thread_rng().gen_range(0..input_poly.get_length_of_padded_blob_as_fr_vector());
rand::thread_rng().gen_range(0..input_poly.len_underlying_blob_field_elements());
b.iter(|| {
kzg.compute_kzg_proof_with_roots_of_unity(&input_poly, index.try_into().unwrap())
kzg.compute_proof_with_roots_of_unity(&input_poly, index.try_into().unwrap())
.unwrap()
});
});

c.bench_function("bench_kzg_proof_30000", |b| {
let random_blob: Vec<u8> = (0..30000).map(|_| rng.gen_range(32..=126) as u8).collect();
let input = Blob::from_raw_data(&random_blob);
let input_poly = input
.to_polynomial(PolynomialFormat::InCoefficientForm)
.unwrap();
let input_poly = input.to_polynomial_eval_form();
kzg.data_setup_custom(1, input.len().try_into().unwrap())
.unwrap();
let index =
rand::thread_rng().gen_range(0..input_poly.get_length_of_padded_blob_as_fr_vector());
rand::thread_rng().gen_range(0..input_poly.len_underlying_blob_field_elements());
b.iter(|| {
kzg.compute_kzg_proof_with_roots_of_unity(&input_poly, index.try_into().unwrap())
kzg.compute_proof_with_roots_of_unity(&input_poly, index.try_into().unwrap())
.unwrap()
});
});

c.bench_function("bench_kzg_proof_50000", |b| {
let random_blob: Vec<u8> = (0..50000).map(|_| rng.gen_range(32..=126) as u8).collect();
let input = Blob::from_raw_data(&random_blob);
let input_poly = input
.to_polynomial(PolynomialFormat::InCoefficientForm)
.unwrap();
let input_poly = input.to_polynomial_eval_form();
kzg.data_setup_custom(1, input.len().try_into().unwrap())
.unwrap();
let index =
rand::thread_rng().gen_range(0..input_poly.get_length_of_padded_blob_as_fr_vector());
rand::thread_rng().gen_range(0..input_poly.len_underlying_blob_field_elements());
b.iter(|| {
kzg.compute_kzg_proof_with_roots_of_unity(&input_poly, index.try_into().unwrap())
kzg.compute_proof_with_roots_of_unity(&input_poly, index.try_into().unwrap())
.unwrap()
});
});
Expand Down
38 changes: 16 additions & 22 deletions benches/bench_kzg_verify.rs
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
use criterion::{criterion_group, criterion_main, Criterion};
use rand::Rng;
use rust_kzg_bn254::{blob::Blob, kzg::Kzg, polynomial::PolynomialFormat};
use rust_kzg_bn254::{blob::Blob, kzg::Kzg};
use std::time::Duration;

fn bench_kzg_verify(c: &mut Criterion) {
Expand All @@ -17,58 +17,52 @@ fn bench_kzg_verify(c: &mut Criterion) {
c.bench_function("bench_kzg_verify_10000", |b| {
let random_blob: Vec<u8> = (0..10000).map(|_| rng.gen_range(32..=126) as u8).collect();
let input = Blob::from_raw_data(&random_blob);
let input_poly = input
.to_polynomial(PolynomialFormat::InCoefficientForm)
.unwrap();
let input_poly = input.to_polynomial_eval_form();
kzg.data_setup_custom(1, input.len().try_into().unwrap())
.unwrap();
let index =
rand::thread_rng().gen_range(0..input_poly.get_length_of_padded_blob_as_fr_vector());
let commitment = kzg.commit(&input_poly.clone()).unwrap();
rand::thread_rng().gen_range(0..input_poly.len_underlying_blob_field_elements());
let commitment = kzg.commit_eval_form(&input_poly).unwrap();
let proof = kzg
.compute_kzg_proof_with_roots_of_unity(&input_poly, index.try_into().unwrap())
.compute_proof_with_roots_of_unity(&input_poly, index.try_into().unwrap())
.unwrap();
let value_fr = input_poly.get_at_index(index).unwrap();
let z_fr = kzg.get_nth_root_of_unity(index).unwrap();
b.iter(|| kzg.verify_kzg_proof(commitment, proof, *value_fr, *z_fr));
b.iter(|| kzg.verify_proof(commitment, proof, *value_fr, *z_fr));
});

c.bench_function("bench_kzg_verify_30000", |b| {
let random_blob: Vec<u8> = (0..30000).map(|_| rng.gen_range(32..=126) as u8).collect();
let input = Blob::from_raw_data(&random_blob);
let input_poly = input
.to_polynomial(PolynomialFormat::InCoefficientForm)
.unwrap();
let input_poly = input.to_polynomial_eval_form();
kzg.data_setup_custom(1, input.len().try_into().unwrap())
.unwrap();
let index =
rand::thread_rng().gen_range(0..input_poly.get_length_of_padded_blob_as_fr_vector());
let commitment = kzg.commit(&input_poly.clone()).unwrap();
rand::thread_rng().gen_range(0..input_poly.len_underlying_blob_field_elements());
let commitment = kzg.commit_eval_form(&input_poly).unwrap();
let proof = kzg
.compute_kzg_proof_with_roots_of_unity(&input_poly, index.try_into().unwrap())
.compute_proof_with_roots_of_unity(&input_poly, index.try_into().unwrap())
.unwrap();
let value_fr = input_poly.get_at_index(index).unwrap();
let z_fr = kzg.get_nth_root_of_unity(index).unwrap();
b.iter(|| kzg.verify_kzg_proof(commitment, proof, *value_fr, *z_fr));
b.iter(|| kzg.verify_proof(commitment, proof, *value_fr, *z_fr));
});

c.bench_function("bench_kzg_verify_50000", |b| {
let random_blob: Vec<u8> = (0..50000).map(|_| rng.gen_range(32..=126) as u8).collect();
let input = Blob::from_raw_data(&random_blob);
let input_poly = input
.to_polynomial(PolynomialFormat::InCoefficientForm)
.unwrap();
let input_poly = input.to_polynomial_eval_form();
kzg.data_setup_custom(1, input.len().try_into().unwrap())
.unwrap();
let index =
rand::thread_rng().gen_range(0..input_poly.get_length_of_padded_blob_as_fr_vector());
let commitment = kzg.commit(&input_poly.clone()).unwrap();
rand::thread_rng().gen_range(0..input_poly.len_underlying_blob_field_elements());
let commitment = kzg.commit_eval_form(&input_poly).unwrap();
let proof = kzg
.compute_kzg_proof_with_roots_of_unity(&input_poly, index.try_into().unwrap())
.compute_proof_with_roots_of_unity(&input_poly, index.try_into().unwrap())
.unwrap();
let value_fr = input_poly.get_at_index(index).unwrap();
let z_fr = kzg.get_nth_root_of_unity(index).unwrap();
b.iter(|| kzg.verify_kzg_proof(commitment, proof, *value_fr, *z_fr));
b.iter(|| kzg.verify_proof(commitment, proof, *value_fr, *z_fr));
});
}

Expand Down
Binary file added kzg_commitment_diagram.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
32 changes: 19 additions & 13 deletions src/blob.rs
Original file line number Diff line number Diff line change
@@ -1,7 +1,6 @@
use crate::{
errors::BlobError,
helpers,
polynomial::{Polynomial, PolynomialFormat},
polynomial::{PolynomialCoeffForm, PolynomialEvalForm},
};

/// A blob which is Eigen DA spec aligned.
Expand All @@ -15,28 +14,31 @@ pub struct Blob {
impl Blob {
/// Creates a new `Blob` from the given blob_data.
/// blob_data should already be padded according to DA specs, meaning
/// that it contains bn254 field elements. Otherwise, use [`Blob::from_raw_data`].
/// that it contains bn254 field elements. Otherwise, use
/// [`Blob::from_raw_data`].
///
/// WARNING: This function does not check if the bytes are modulo bn254
/// if the data has 32 byte segments exceeding the modulo of the field
/// then the bytes will be modded by the order of the field and the data
/// will be transformed incorrectly.
/// TODO: we should check that the bytes are correct and return an error instead of
/// relying on the users reading this documentation.
/// TODO: we should check that the bytes are correct and return an error
/// instead of relying on the users reading this documentation.
pub fn new(blob_data: &[u8]) -> Self {
Blob {
blob_data: blob_data.to_vec(),
}
}

/// Creates a new `Blob` from the provided raw_data byte slice and pads it according
/// to DA specs. If the data is already padded, use [`Blob::new`] instead.
/// Creates a new `Blob` from the provided raw_data byte slice and pads it
/// according to DA specs. If the data is already padded, use
/// [`Blob::new`] instead.
bxue-l2 marked this conversation as resolved.
Show resolved Hide resolved
pub fn from_raw_data(raw_data: &[u8]) -> Self {
let blob_data = helpers::convert_by_padding_empty_byte(raw_data);
Blob { blob_data }
}

/// Returns the raw data of the blob, removing any padding added by [`Blob::from_raw_data`].
/// Returns the raw data of the blob, removing any padding added by
/// [`Blob::from_raw_data`].
pub fn to_raw_data(&self) -> Vec<u8> {
helpers::remove_empty_byte_from_padded_bytes_unchecked(&self.blob_data)
}
Expand All @@ -56,12 +58,16 @@ impl Blob {
self.blob_data.is_empty()
}

/// Converts the blob data to a `Polynomial` if the data is padded.
pub fn to_polynomial(&self, form: PolynomialFormat) -> Result<Polynomial, BlobError> {
/// Convert the blob data to a `PolynomialEvalForm`.
pub fn to_polynomial_eval_form(&self) -> PolynomialEvalForm {
let fr_vec = helpers::to_fr_array(&self.blob_data);
let poly = Polynomial::new(&fr_vec, self.len(), form)
.map_err(|err| BlobError::GenericError(err.to_string()))?;
Ok(poly)
PolynomialEvalForm::new(fr_vec)
}

/// Convert the blob data to a `PolynomialCoefForm`.
pub fn to_polynomial_coeff_form(&self) -> PolynomialCoeffForm {
let fr_vec = helpers::to_fr_array(&self.blob_data);
PolynomialCoeffForm::new(fr_vec)
}
}

Expand Down
Loading
Loading