Skip to content

Commit

Permalink
Merge branch 'master' into master
Browse files Browse the repository at this point in the history
  • Loading branch information
smagdali authored Nov 1, 2024
2 parents 05574b4 + 11169e5 commit e7b86c5
Show file tree
Hide file tree
Showing 57 changed files with 3,738 additions and 75 deletions.
2 changes: 1 addition & 1 deletion .fleek.json
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
{
"build": {
"image": "hugomrdias/puppeteer",
"image": "node:20.2",
"command": "npm install && npm run build",
"publicDir": "public"
}
Expand Down
3 changes: 2 additions & 1 deletion .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,8 @@ yarn.lock
package-lock.json
node_modules
resources
static/_gen
data/toc.json
api/dist
api/worker
.hugo_build.lock
assets/jsconfig.json
1 change: 1 addition & 0 deletions .nvmrc
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
v20.2.0
42 changes: 31 additions & 11 deletions api/index.js
Original file line number Diff line number Diff line change
Expand Up @@ -31,21 +31,41 @@ async function handleRequest(event) {
async function cov(event) {
const url = new URL(event.request.url)
// https://github.com/filecoin-project/lotus
const repo = url.searchParams.get('repo').split('/').slice(3).join('/')
const data = await get(event, {
url: `https://codecov.io/api/gh/${repo}`,
const [owner, repo] = url.searchParams.get('repo').split('/').slice(3)
const headers = {
'User-Agent': 'ianconsolata',
Accept: 'application/json',
Authorization: `Bearer ${CODECOV_TOKEN}`,
}
const repo_resp = await get(event, {
url: `https://api.codecov.io/api/v2/github/${owner}/repos/${repo}/`,
transform: (data) => {
const out = {
cov: dlv(data, 'commit.totals.c', 0),
ci: dlv(data, 'commit.ci_passed', false),
repo: dlv(data, 'repo.name', 'N/A'),
org: dlv(data, 'owner.username', 'N/A'),
lang: dlv(data, 'repo.language', 'N/A'),
branch: dlv(data, 'branch', 'master'),
lang: dlv(data, 'language', 'N/A'),
}
return out
},
headers,
})
return data
const repo_data = await repo_resp.json()

const cov_data = await get(event, {
url: `https://api.codecov.io/api/v2/github/${owner}/repos/${repo}/branches/${repo_data.branch}/`,
transform: (data) => {
const out = {
cov: dlv(data, 'head_commit.totals.coverage', 0),
ci: dlv(data, 'head_commit.ci_passed', false),
repo: repo,
org: owner,
lang: repo_data.lang,
branch: repo_data.branch,
}
return out
},
headers,
})
return cov_data
}

async function github(event) {
Expand All @@ -56,7 +76,7 @@ async function github(event) {
const path = file.slice(7).join('/')
const ref = file[6]
const headers = {
'User-Agent': 'hugomrdias',
'User-Agent': 'ianconsolata',
Authorization: `token ${GITHUB_TOKEN}`,
}

Expand All @@ -82,7 +102,7 @@ async function github(event) {

async function releases(event) {
const headers = {
'User-Agent': 'hugomrdias',
'User-Agent': 'ianconsolata',
Authorization: `token ${GITHUB_TOKEN}`,
}
const rsp = await get(event, {
Expand Down
3 changes: 2 additions & 1 deletion api/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,7 @@
"dlv": "^1.1.3",
"merge-options": "^3.0.3",
"nanoid": "^3.1.16",
"p-map": "^4.0.0"
"p-map": "^4.0.0",
"wrangler": "^2.17.0"
}
}
8 changes: 3 additions & 5 deletions api/wrangler.toml
Original file line number Diff line number Diff line change
@@ -1,7 +1,5 @@
name = "specs-api"
type = "webpack"
account_id = "fffa4b4363a7e5250af8357087263b3a"
account_id = "e85b6fdda17f8c464c343dc84d1373e8"
workers_dev = true
route = ""
zone_id = ""
webpack_config = "webpack.config.js"
main = "index.js"
compatibility_date = "2023-04-27"
6 changes: 5 additions & 1 deletion config.toml
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,10 @@ canonifyurls = false
theme = "book"
# relativeURLs = true

# allow building even if getting remote JSON fails
# needed because rust-fil-proofs has no coverage
ignoreErrors = ["error-remote-getjson"]

# Book configuration
disablePathToLower = true
enableGitInfo = true
Expand Down Expand Up @@ -41,7 +45,7 @@ enableGitInfo = true
# weight = 10

[params]
API = 'https://specs-api.protocol-labs.workers.dev'
API = 'https://specs-api.filecoin-foundation.workers.dev'
# (Optional, default true) Controls table of contents visibility on right side of pages.
# Start and end levels can be controlled with markup.tableOfContents setting.
# You can also specify this parameter per page in front matter.
Expand Down
10 changes: 9 additions & 1 deletion content/algorithms/crypto/poseidon.md
Original file line number Diff line number Diff line change
Expand Up @@ -356,7 +356,15 @@ while providing security against known attacks (statistical, interpolation, and

<br/>

{{<plain>}}$\constb R_F, R_P = \texttt{calc\_round\_numbers}(p, M, t, \alpha)${{</plain>}}\
{{<plain>}}$\constb R_F, R_P = \texttt{calc\_round\_numbers}(p, M, t, c_{\alpha}})${{</plain>}}\
where the S-box case, {{<plain>}}$c_{\alpha}${{</plain>}}, is given by
{{<plain>}}
$c_{\alpha} = \begin{cases}
0 & \if \alpha = 3 \cr
1 & \if \alpha = 5 \cr
2 & \if \alpha = \neg 1
\end{cases}
{{</plain>}}\
The number of full and partial rounds, both are positive integers $R_F, R_P \typecolon \mathbb{Z}_{>0}$ and $R_F$ is even.

$R_F$ and $R_P$ are calculated using either the Python script [`calc_round_numbers.py`](https://extgit.iaik.tugraz.at/krypto/hadeshash/-/blob/9d80ec0473ad7cde5a12f3aac46439ad0da68c0a/code/scripts/calc_round_numbers.py) or the [`neptune`](https://github.com/filecoin-project/neptune) Rust library, denoted {{<plain>}}$\texttt{calc\_round\_numbers}${{</plain>}}. Both methods calculate the round numbers via brute-force; by iterating over all reasonable values for $R_F$ and $R_P$ and choosing the pair that satisfies the security inequalities (provided below) while minimizing the number of S-boxes.
Expand Down
10 changes: 5 additions & 5 deletions content/algorithms/expected_consensus/_index.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ dashboardTests: 0

## Algorithm

Expected Consensus (EC) is a probabilistic Byzantine fault-tolerant consensus protocol. At a high level, it operates by running a leader election every epoch in which, on expectation, a set number of participants may be eligible to submit a block. EC guarantees that these winners will be anonymous until they reveal themselves by submitting a proof that they have been elected, the `ElectionProof`. Each winning miner can submit one such proof per round and will be rewarded proportionally to its power. From this point on, each wining miner also creates a proof of storage (aka Winning PoSt). Each proof can be derived from a [properly formatted beacon entry](randomness), as described below.
Expected Consensus (EC) is a probabilistic Byzantine fault-tolerant consensus protocol. At a high level, it operates by running a leader election every epoch in which, on expectation, a set number of participants may be eligible to submit a block. EC guarantees that these winners will be anonymous until they reveal themselves by submitting a proof that they have been elected, the `ElectionProof`. Each winning miner can submit one such proof per round and will be rewarded proportionally to its power. From this point on, each winning miner also creates a proof of storage (aka Winning PoSt). Each proof can be derived from a [properly formatted beacon entry](randomness), as described below.

All valid blocks submitted in a given round form a `Tipset`. Every block in a Tipset adds weight to its chain. The 'best' chain is the one with the highest weight, which is to say that the fork choice rule is to choose the heaviest known chain. For more details on how to select the heaviest chain, see [Chain Selection](expected_consensus#chain-selection). While on expectation at least one block will be generated at every round, in cases where no one finds a block in a given round, a miner can simply run leader election again for the next epoch with the appropriate random seed, thereby ensuring liveness in the protocol.

Expand Down Expand Up @@ -173,9 +173,9 @@ WinCount guarantees that a lucky single block will earn the same reward as the r

**Alternative Options for the Distribution/Sortition**

Bernouli, Binomial and Poisson distributions have been considered for the _WinCount_ of a miner with power `p` out of a total network power of `N`. There are the following options:
Bernoulli, Binomial and Poisson distributions have been considered for the _WinCount_ of a miner with power `p` out of a total network power of `N`. There are the following options:

- Option 1: WinCount(p,N) ~ Bernouli(pE/N)
- Option 1: WinCount(p,N) ~ Bernoulli(pE/N)
- Option 2: WinCount(p,N) ~ Binomial(E, p/N)
- Option 3: WinCount(p,N) ~ Binomial(p, E/N)
- Option 4: WinCount(p,N) ~ Binomial(p/M, ME/N)
Expand All @@ -189,7 +189,7 @@ In Option 5 (the one used in Filecoin Leader Election), the ticket targets for e

**Rationale for the Poisson Sortition choice**

- Option 1 - Bernouli(pE/N): this option is easy to implement, but comes with a drawback: if the miner's power exceeds 1/E, the miner's WinCount is always 1, but never higher than 1.
- Option 1 - Bernoulli(pE/N): this option is easy to implement, but comes with a drawback: if the miner's power exceeds 1/E, the miner's WinCount is always 1, but never higher than 1.
- Option 2 - Binomial(E, p/N): the expectation of WinCount stays the same irrespectively of whether the miner splits their power into more than one Sybil nodes, but the variance increases if they choose to Sybil. Risk-seeking miners will prefer to Sybil, while risk-averse miners will prefer to pool, none of which is a behaviour the protocol should encourage. This option is not computationally-expensive as it would involve calculation of factorials and fixed-point multiplications (or small integer exponents) only.
- Option 3 - Binomial(p, E/N): this option is computationally inefficient. It involves very large integer exponents.
- Option 4 - Binomial(p/M, ME/N): the complexity of this option depends on the value of M. A small M results in high computational cost, similarly to Option 3. A large M, on the other hand, leads to a situation similar to that of Option 2, where a risk-seeking miner is incentivized to Sybil. Clearly none of these are desirable properties.
Expand Down Expand Up @@ -342,7 +342,7 @@ The exact value for these parameters remain to be determined, but for testing pu
- `log2b(X) = floor(log2(x)) = (binary length of X) - 1` and `log2b(0) = 0`. Note that that special case should never be used (given it would mean an empty power table).

{{< hint warning >}}
**Note that if your implementation does not allow for rounding to the fourth decimal**, miners should apply the [tie-breaker below](selecting-between-tipsets-with-equal-weight). Weight changes will be on the order of single digit numbers on expectation, so this should not have an outsized impact on chain consensus across implementations.
**Note that if your implementation does not allow for rounding to the fourth decimal**, miners should apply the [tie-breaker below](#section-algorithms.expected_consensus.selecting-between-tipsets-with-equal-weight). Weight changes will be on the order of single digit numbers on expectation, so this should not have an outsized impact on chain consensus across implementations.
{{< /hint >}}

`ParentWeight` is the aggregate chain weight of a given block's parent set. It is calculated as
Expand Down
2 changes: 1 addition & 1 deletion content/algorithms/pos/post.md
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,7 @@ Before continuing into more details of the _WinnningPoSt_ and _WindowPoSt_ algor

## WinningPoSt

At the beginning of each epoch, a small number of storage miners (up to five, currently) are elected to mine new blocks, by Filecoin's [Expected Consensus](algorithms#expected_consensus) algorithm. Recall that the Filecoin blockchain operates on the basis of _tipsets_, which are groups of up to five blocks. This means that multiple blocks (up to five) can be mined at the same height.
At the beginning of each epoch, a small number of storage miners are elected to mine new blocks, by Filecoin's [Expected Consensus](algorithms#expected_consensus) algorithm. Recall that the Filecoin blockchain operates on the basis of _tipsets_, therefore multiple blocks can be mined at the same height.

Each of the miners that are elected to mine a block have to submit a proof that they keep a sealed copy of the data which they have included in their proposed block, before the end of the current epoch. Successful submission of this proof is the _WinningPoSt_, which in turn grants the miner the Filecoin Block Reward, as well as the opportunity to charge other nodes fees in order to include their messages in the block. If a miner misses the epoch-end deadline, then the miner misses the opportunity to mine a block and get a Block Reward. No penalty is incurred in this case.

Expand Down
4 changes: 2 additions & 2 deletions content/algorithms/sdr/_index.md
Original file line number Diff line number Diff line change
Expand Up @@ -1156,8 +1156,8 @@ $\line{11}{}{\return \Sha{254}(\preimage) \as \Fq}$

**Implementation:**

- [`storage_proofs::porep::stacked::circuit::proof::StackedCircuit::synthesize()`](https://github.com/filecoin-project/rust-fil-proofs/blob/master/storage-proofs/porep/src/stacked/circuit/proof.rs#L77)
- [`storage_proofs::porep::stacked::circuit::params::Proof::synthesize()`](https://github.com/filecoin-project/rust-fil-proofs/blob/master/storage-proofs/porep/src/stacked/circuit/params.rs#L77)
- [`storage_proofs::porep::stacked::circuit::proof::StackedCircuit::synthesize()`](https://github.com/filecoin-project/rust-fil-proofs/blob/108517271f8166cabc23e1966d8e8f397ec75ae0/storage-proofs-porep/src/stacked/circuit/proof.rs#L64)
- [`storage_proofs::porep::stacked::circuit::params::Proof::synthesize()`](https://github.com/filecoin-project/rust-fil-proofs/blob/108517271f8166cabc23e1966d8e8f397ec75ae0/storage-proofs-porep/src/stacked/circuit/proof.rs#L92)

**Additional Notation:**

Expand Down
30 changes: 30 additions & 0 deletions content/appendix/audit_reports.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,17 @@ dashboardAudit: n/a

Security is a critical component in ensuring Filecoin can fulfill its mission to be the storage network for humanity. In addition to robust secure development processes, trainings, theory audits, and investing in external security research, the Filecoin project has engaged reputable third party auditing specialists to ensure that the theory behind the protocol and its implementation delivers the intended value, enabling Filecoin to be a safe and secure network. This section covers a selection of audit reports that have been published on Filecoin's theory and implementation.

## Filecoin Virtual Machine

### `2023-03-09` Filecoin EVM (FEVM)

- Report: [Filecoin EVM Audit](<https://github.com/oak-security/audit-reports/blob/master/Filecoin%20Foundation/2023-03-09%20Audit%20Report%20-%20Filecoin%20EVM%20(FEVM)%20v1.1.pdf>)
- Audit conducted by **Oak Security**

The audit covers the implementation of:

- FEVM's [builtin actors](https://github.com/filecoin-project/builtin-actors/tree/1b11df4b399550753a4105f45f58bc07015af2a3/actors/evm) out of which only [actors/evm](https://github.com/filecoin-project/builtin-actors/tree/1b11df4b399550753a4105f45f58bc07015af2a3/actors/evm) and [actors/eam](https://github.com/filecoin-project/builtin-actors/tree/1b11df4b399550753a4105f45f58bc07015af2a3/actors/eam) were included in scope along with code base of [ref-fvm](https://github.com/filecoin-project/ref-fvm). The report included auditing EVM runtime action and implementation, correctness of EVM opcodes, including Ethereum Address Manager(EAM). The report also included issues and enhancements methods for gas model and F4 addresses. The audit team also reviewed the message execution flow and kernel setup, WASM integration and FVM logs. All the valid issues raised by the audit were resolved and acknowledged including a few informational issues. More details on these issues are available in the report.

## Lotus

### `2020-10-20` Lotus Mainnet Ready Security Audit
Expand All @@ -22,6 +33,17 @@ The scope of this audit covered:
- The Lotus Daemon: Core component responsible for handling the Blockchain node logic by handling peer- to-peer networking, chain syncing, block validation, data retrieval and transfer, etc.
- The Lotus Storage Miner: Mining component used to manage a single storage miner by contributing to the network through Sector commitments and Proofs-of-Spacetime data proving it is storing the sectors it has committed to. This component communicates with the Lotus daemon via JSON-RPC API calls.

## Venus

### `2021-06-29` Venus Security Audit

- Report: [Venus Security Assessment](https://leastauthority.com/static/publications/LeastAuthority_Filecoin_Foundation_Venus_Final_Audit_Report.pdf)
- Audit conducted by: **Least Authority**

The scope of this audit covered:

- The Venus Daemon: Core component responsible for handling the Filecoin node logic by handling peer-to-peer networking, chain syncing, block validation, etc.

## Actors

### `2020-10-19` Actors Mainnet Ready Security Audit
Expand All @@ -33,6 +55,14 @@ This audit covers the implementation of Filecoin's builtin Actors, focusing on t

## Proofs

### `2021-05-31` SnarkPack audit

An audit was conducted on the cryptographic part of [SnarkPack](https://eprint.iacr.org/2021/529.pdf), that is used in the [FIP0009](https://github.com/filecoin-project/FIPs/blob/master/FIPS/fip-0009.md):

- [Report](https://hackmd.io/@LIRa8YONSwKxiRz3cficng/B105no8w_) from Matteo Campanelli, a well known cryptography [researcher](https://www.binarywhales.com/)

One major issue was found in the report by Campanelli where the challenges of each prove commits were not tied to the aggregated proof; this could have led up to malicious miner forge valid aggregated proofs without the individual prove commits. The rest of the issues were of medium to informal severity.

### `2020-10-20` Filecoin Bellman and BLS Signatures

- Report: [**Filecoin Bellman/BLS Signatures Cryptography Review**](https://research.nccgroup.com/wp-content/uploads/2020/10/NCC_Group_ProtocolLabs_PRLB007_Report_2020-10-20_v1.0.pdf)
Expand Down
2 changes: 1 addition & 1 deletion content/glossary/_index.md
Original file line number Diff line number Diff line change
Expand Up @@ -377,7 +377,7 @@ Every mined block has a computed `weight`, also called its `WinCount`. Together,

[_Window Proof-of-Spacetime_ (WindowPoSt)](post#windowpost) is the mechanism by which the commitments made by [storage miners](glossary#storage-miner-actor) are audited. It sees each 24-hour period broken down into a series of windows. Correspondingly, each storage miner's set of pledged [sectors](glossary#sector) is partitioned into subsets, one subset for each window. Within a given window, each storage miner must submit a [Proof-of-Spacetime](glossary#proof-of-spacetime-post) for each sector in their respective subset. This requires ready access to each of the challenged sectors, and will result in a [zk-SNARK-compressed](glossary#zksnark) proof published to the Filecoin [blockchain](glossary#blockchain) as a [message](glossary#message) in a [block](glossary#block). In this way, every sector of [pledged storage](glossary#pledged-storage) is audited at least once in any 24-hour period, and a permanent, verifiable, and public record attesting to each storage miner's continued commitment is kept.

The Filecoin network expects constant availability of stored data. Failing to submit WindowPoSt for a sector will result in a [fault](glossary#fault), and the storage miner supplying the sector will be [slashed](glossary#slash).
The Filecoin network expects constant availability of stored data. Failing to submit WindowPoSt for a sector will result in a [fault](glossary#fault), and the storage miner supplying the sector will be [slashed](glossary#slashing).

## Winning Proof-of-Spacetime (WinningPoSt)

Expand Down
2 changes: 1 addition & 1 deletion content/implementations/_index.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ Filecoin is targeting [multiple implementations](https://filecoin.io/blog/announ

- [Lotus](https://lotu.sh): the Go-based implementation, supported by [Protocol Labs](https://protocol.ai),
- [Forest](https://github.com/ChainSafe/forest): the Rust-based implementation, supported by [ChainSafe](https://chainsafe.io),
- [Fuhon](https://github.com/filecoin-project/cpp-filecoin): the C++-based implementation, supported by [Soramitsu](https://soramitsu.co.jp),
- [Fuhon (deprecated)](https://github.com/filecoin-project/cpp-filecoin): the C++-based implementation, supported by [Soramitsu](https://soramitsu.co.jp),
- [Venus](https://github.com/filecoin-project/go-filecoin): a second Go-based implementation of Filecoin, previously called `go-filecoin`, which is maintained by the [IPFS-Force Community](https://github.com/ipfs-force-community).

{{<dashboard-impl>}}
4 changes: 3 additions & 1 deletion content/implementations/fuhon.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
---
title: Fuhon
title: Fuhon (deprecated)
weight: 4
dashboardWeight: 1
dashboardState: reliable
Expand All @@ -15,3 +15,5 @@ Fuhon is the C++ implementation of Filecoin. The implementation uses Rust librar
You can find the Fuhon codebase [here](https://github.com/filecoin-project/cpp-filecoin).

The Fuhon implementation of Filecoin is supported by [Soramitsu](https://soramitsu.co.jp/).

Since May 2022, this implementation has been deprecated and out of support. The existing code repositories will remain public.
Loading

0 comments on commit e7b86c5

Please sign in to comment.