feature(mining): Enable mining RPCs by default in production builds (#…

…7740)

* Enable getblocktemplate-rpcs in production builds

* Update mining instructions

* Consistently use ZF funding stream addresses in examples

* Simplify skip_getblocktemplate()

* Set the test miner address based on the network

* Correctly skip checks in lightwalletd send transactions test

* Wait longer for transaction query

* Wait longer before re-checking the mempool

* Skip mempool checks if it wasn't queried by lightwalletd

* rustfmt

* clippy and rustfmt

* Fix attribute issue

* Fix typo

Co-authored-by: Arya <aryasolhi@gmail.com>

---------

Co-authored-by: Arya <aryasolhi@gmail.com>

README.md

@@ -128,11 +128,15 @@ In future releases, the `progress_bar = "summary"` config will show a few key me
 and the "detailed" config will show all available metrics. Please let us know which metrics are
 important to you!
 
+##### Configuring Mining
+
+Zebra can be configured for mining by passing a `MINER_ADDRESS` and port mapping to Docker.
+See the [mining support docs](https://zebra.zfnd.org/user/mining-docker.html) for more details.
+
 ##### Custom Build Features
 
 You can also build Zebra with additional [Cargo features](https://doc.rust-lang.org/cargo/reference/features.html#command-line-feature-options):
 
-- `getblocktemplate-rpcs` for [mining support](https://zebra.zfnd.org/user/mining.html)
 - `prometheus` for [Prometheus metrics](https://zebra.zfnd.org/user/metrics.html)
 - `sentry` for [Sentry monitoring](https://zebra.zfnd.org/user/tracing.html#sentry-production-monitoring)
 - `elasticsearch` for [experimental Elasticsearch support](https://zebra.zfnd.org/user/elasticsearch.html)

book/src/user/mining-docker.md

@@ -1,20 +1,21 @@
 # Mining with Zebra in Docker
 
-Some of our published [Docker images](https://hub.docker.com/r/zfnd/zebra/tags)
-have the `.experimental` suffix in their name. We compile these images with the
-`getblocktemplate-rpcs` feature, and you can use them for your mining
-operations. For example, executing
+Zebra's [Docker images](https://hub.docker.com/r/zfnd/zebra/tags) can be used for your mining
+operations. If you don't have Docker, see the
+[manual configuration instructions](https://zebra.zfnd.org/user/mining.html).
+
+Using docker, you can start mining by running:
 
 ```bash
-docker run -e MINER_ADDRESS="t1XhG6pT9xRqRQn3BHP7heUou1RuYrbcrCc" -p 8232:8232 zfnd/zebra:v1.1.0.experimental
+docker run -e MINER_ADDRESS="t3dvVE3SQEi7kqNzwrfNePxZ1d4hUyztBA1" -p 8232:8232 zfnd/zebra:latest
 ```
 
-will start a container on Mainnet and bind port 8232 on your Docker host. If you
+This command starts a container on Mainnet and binds port 8232 on your Docker host. If you
 want to start generating blocks, you need to let Zebra sync first.
 
 Note that you must pass the address for your mining rewards via the
 `MINER_ADDRESS` environment variable when you are starting the container, as we
-did in the example above. The address we used starts with the prefix `t1`,
+did with the ZF funding stream address above. The address we used starts with the prefix `t1`,
 meaning it is a Mainnet P2PKH address. Please remember to set your own address
 for the rewards.
 
@@ -35,7 +36,7 @@ variable to `Testnet` and use a Testnet address for the rewards. For example,
 running
 
 ```bash
-docker run -e NETWORK="Testnet" -e MINER_ADDRESS="t27eWDgjFYJGVXmzrXeVjnb5J3uXDM9xH9v" -p 18232:18232 zfnd/zebra:v1.1.0.experimental
+docker run -e NETWORK="Testnet" -e MINER_ADDRESS="t27eWDgjFYJGVXmzrXeVjnb5J3uXDM9xH9v" -p 18232:18232 zfnd/zebra:latest
 ```
 
 will start a container on Testnet and bind port 18232 on your Docker host, which

book/src/user/mining-testnet-s-nomp.md

@@ -82,7 +82,7 @@ These fixes disable mining pool operator payments and miner payments: they just
 ## Install `s-nomp`
 
 <details><summary>General instructions with Debian/Ubuntu examples</summary>
-    
+
 #### Install dependencies
 
 1. Install `redis` and run it on the default port: <https://redis.io/docs/getting-started/>
@@ -136,7 +136,7 @@ These fixes disable mining pool operator payments and miner payments: they just
 </details>
 
 <details><summary>Arch-specific instructions</summary>
-    
+
 #### Install dependencies
 
 1. Install [`redis`](https://redis.io/docs/getting-started/) and run it on the default port:

book/src/user/mining.md

@@ -1,29 +1,24 @@
 # Mining Zcash with zebra
 
-Zebra's RPC methods support miners and mining pools. 
+Zebra's RPC methods support miners and mining pools.
 
 Contents:
 
-- [Download and build Zebra](#download-and-build-zebra)
+- [Download Zebra](#download-and-build-zebra)
 - [Configure zebra for mining](#configure-zebra-for-mining)
   - [Miner address](#miner-address)
   - [RPC section](#rpc-section)
 - [Running zebra](#running-zebra)
 - [Testing the setup](#testing-the-setup)
 - [Run a mining pool](#run-a-mining-pool)
 
-## Download and build Zebra for mining
+## Download Zebra
 [#download-and-build-zebra]: #download-and-build-zebra
 
-Before installing please make sure you have the [Zebra dependencies](https://github.com/ZcashFoundation/zebra#build-instructions) in your OS.
+The easiest way to run Zebra for mining is with [our docker images](https://zebra.zfnd.org/user/mining-docker.html).
 
-```console
-cargo install --locked --features getblocktemplate-rpcs --git https://github.com/ZcashFoundation/zebra zebrad
-```
-
-Zebra binary will be at ` ~/.cargo/bin/zebrad`.
-
-It is important to install with the `getblocktemplate-rpcs` feature so the final binary will contain mining support.
+If you have [installed Zebra another way](https://zebra.zfnd.org/user/install.html), follow the
+instructions below to start mining:
 
 ## Configure zebra for mining
 [#configure-zebra-for-mining]: #configure-zebra-for-mining
@@ -46,10 +41,10 @@ Node miner address is required. At the moment zebra only allows `p2pkh` or `p2sh
 
 ```
 [mining]
-miner_address = 't3Vz22vK5z2LcKEdg16Yv4FFneEL1zg9ojd'
+miner_address = 't3dvVE3SQEi7kqNzwrfNePxZ1d4hUyztBA1'
 ```
 
-The above address is just the first address of the [Founders' Reward](https://zips.z.cash/protocol/protocol.pdf#foundersreward) section of the Zcash protocol. It's a Mainnet address and it is used here purely as an example.
+The above address is the ZF Mainnet funding stream address. It is used here purely as an example.
 
 ### RPC section
 [#rpc-section]: #rpc-section

zebra-rpc/src/config/mining.rs

@@ -46,9 +46,6 @@ impl Config {
     /// This is used to ignore the mining section of the configuration if the feature is not
     /// enabled, allowing us to log a warning when the config found is different from the default.
     pub fn skip_getblocktemplate(&self) -> bool {
-        #[cfg(feature = "getblocktemplate-rpcs")]
-        return false;
-        #[cfg(not(feature = "getblocktemplate-rpcs"))]
-        return true;
+        !cfg!(feature = "getblocktemplate-rpcs")
     }
 }

zebrad/Cargo.toml

@@ -52,14 +52,14 @@ features = [
 
 [features]
 # In release builds, don't compile debug logging code, to improve performance.
-default = ["release_max_level_info", "progress-bar"]
+default = ["release_max_level_info", "progress-bar", "getblocktemplate-rpcs"]
 
 # Default features for official ZF binary release builds
 default-release-binaries = ["default", "sentry"]
 
 # Production features that activate extra dependencies, or extra features in dependencies
 
-# Experimental mining RPC support
+# Mining RPC support
 getblocktemplate-rpcs = [
     "zebra-rpc/getblocktemplate-rpcs",
     "zebra-consensus/getblocktemplate-rpcs",
@@ -68,12 +68,13 @@ getblocktemplate-rpcs = [
     "zebra-chain/getblocktemplate-rpcs",
 ]
 
+# Experimental elasticsearch indexing
 elasticsearch = [
     "zebra-state/elasticsearch",
 ]
 
+# Tracing and monitoring
 sentry = ["dep:sentry"]
-flamegraph = ["tracing-flame", "inferno"]
 journald = ["tracing-journald"]
 filter-reload = ["hyper"]
 
@@ -105,6 +106,8 @@ max_level_info = ["tracing/max_level_info", "log/max_level_info"]
 max_level_debug = ["tracing/max_level_debug", "log/max_level_debug"]
 
 # Testing features that activate extra dependencies
+flamegraph = ["tracing-flame", "inferno"]
+
 proptest-impl = [
     "proptest",
     "proptest-derive",

zebrad/src/lib.rs

@@ -57,7 +57,7 @@
 //!
 //! ### JSON-RPC
 //!
-//! * `getblocktemplate-rpcs`: Experimental mining pool RPC support (currently incomplete)
+//! * `getblocktemplate-rpcs`: Mining pool RPC support, enabled by default in production builds.
 //!
 //! ### Metrics
 //!

zebrad/tests/acceptance.rs

@@ -207,7 +207,7 @@ fn generate_no_args() -> Result<()> {
     let _init_guard = zebra_test::init();
 
     let child = testdir()?
-        .with_config(&mut default_test_config()?)?
+        .with_config(&mut default_test_config(Mainnet)?)?
         .spawn_child(args!["generate"])?;
 
     let output = child.wait_with_output()?;
@@ -269,7 +269,7 @@ fn generate_args() -> Result<()> {
 fn help_no_args() -> Result<()> {
     let _init_guard = zebra_test::init();
 
-    let testdir = testdir()?.with_config(&mut default_test_config()?)?;
+    let testdir = testdir()?.with_config(&mut default_test_config(Mainnet)?)?;
 
     let child = testdir.spawn_child(args!["help"])?;
     let output = child.wait_with_output()?;
@@ -314,7 +314,7 @@ fn start_no_args() -> Result<()> {
     let _init_guard = zebra_test::init();
 
     // start caches state, so run one of the start tests with persistent state
-    let testdir = testdir()?.with_config(&mut persistent_test_config()?)?;
+    let testdir = testdir()?.with_config(&mut persistent_test_config(Mainnet)?)?;
 
     let mut child = testdir.spawn_child(args!["-v", "start"])?;
 
@@ -341,7 +341,7 @@ fn start_no_args() -> Result<()> {
 fn start_args() -> Result<()> {
     let _init_guard = zebra_test::init();
 
-    let testdir = testdir()?.with_config(&mut default_test_config()?)?;
+    let testdir = testdir()?.with_config(&mut default_test_config(Mainnet)?)?;
     let testdir = &testdir;
 
     let mut child = testdir.spawn_child(args!["start"])?;
@@ -366,7 +366,7 @@ fn start_args() -> Result<()> {
 #[tokio::test]
 async fn db_init_outside_future_executor() -> Result<()> {
     let _init_guard = zebra_test::init();
-    let config = default_test_config()?;
+    let config = default_test_config(Mainnet)?;
 
     let start = Instant::now();
 
@@ -392,7 +392,7 @@ async fn db_init_outside_future_executor() -> Result<()> {
 fn persistent_mode() -> Result<()> {
     let _init_guard = zebra_test::init();
 
-    let testdir = testdir()?.with_config(&mut persistent_test_config()?)?;
+    let testdir = testdir()?.with_config(&mut persistent_test_config(Mainnet)?)?;
     let testdir = &testdir;
 
     let mut child = testdir.spawn_child(args!["-v", "start"])?;
@@ -457,7 +457,7 @@ fn ephemeral(cache_dir_config: EphemeralConfig, cache_dir_check: EphemeralCheck)
 
     let _init_guard = zebra_test::init();
 
-    let mut config = default_test_config()?;
+    let mut config = default_test_config(Mainnet)?;
     let run_dir = testdir()?;
 
     let ignored_cache_dir = run_dir.path().join("state");
@@ -547,7 +547,7 @@ fn ephemeral(cache_dir_config: EphemeralConfig, cache_dir_check: EphemeralCheck)
 fn version_no_args() -> Result<()> {
     let _init_guard = zebra_test::init();
 
-    let testdir = testdir()?.with_config(&mut default_test_config()?)?;
+    let testdir = testdir()?.with_config(&mut default_test_config(Mainnet)?)?;
 
     let child = testdir.spawn_child(args!["--version"])?;
     let output = child.wait_with_output()?;
@@ -568,7 +568,7 @@ fn version_no_args() -> Result<()> {
 fn version_args() -> Result<()> {
     let _init_guard = zebra_test::init();
 
-    let testdir = testdir()?.with_config(&mut default_test_config()?)?;
+    let testdir = testdir()?.with_config(&mut default_test_config(Mainnet)?)?;
     let testdir = &testdir;
 
     // unrecognized option `-f`
@@ -622,7 +622,7 @@ fn app_no_args() -> Result<()> {
     let _init_guard = zebra_test::init();
 
     // start caches state, so run one of the start tests with persistent state
-    let testdir = testdir()?.with_config(&mut persistent_test_config()?)?;
+    let testdir = testdir()?.with_config(&mut persistent_test_config(Mainnet)?)?;
 
     tracing::info!(?testdir, "running zebrad with no config (default settings)");
 
@@ -1319,7 +1319,7 @@ async fn metrics_endpoint() -> Result<()> {
     let url = format!("http://{endpoint}");
 
     // Write a configuration that has metrics endpoint_addr set
-    let mut config = default_test_config()?;
+    let mut config = default_test_config(Mainnet)?;
     config.metrics.endpoint_addr = Some(endpoint.parse().unwrap());
 
     let dir = testdir()?.with_config(&mut config)?;
@@ -1376,7 +1376,7 @@ async fn tracing_endpoint() -> Result<()> {
     let url_filter = format!("{url_default}/filter");
 
     // Write a configuration that has tracing endpoint_addr option set
-    let mut config = default_test_config()?;
+    let mut config = default_test_config(Mainnet)?;
     config.tracing.endpoint_addr = Some(endpoint.parse().unwrap());
 
     let dir = testdir()?.with_config(&mut config)?;
@@ -1483,7 +1483,7 @@ async fn rpc_endpoint(parallel_cpu_threads: bool) -> Result<()> {
 
     // Write a configuration that has RPC listen_addr set
     // [Note on port conflict](#Note on port conflict)
-    let mut config = random_known_rpc_port_config(parallel_cpu_threads)?;
+    let mut config = random_known_rpc_port_config(parallel_cpu_threads, Mainnet)?;
 
     let dir = testdir()?.with_config(&mut config)?;
     let mut child = dir.spawn_child(args!["start"])?;
@@ -1542,7 +1542,7 @@ async fn rpc_endpoint_client_content_type() -> Result<()> {
 
     // Write a configuration that has RPC listen_addr set
     // [Note on port conflict](#Note on port conflict)
-    let mut config = random_known_rpc_port_config(true)?;
+    let mut config = random_known_rpc_port_config(true, Mainnet)?;
 
     let dir = testdir()?.with_config(&mut config)?;
     let mut child = dir.spawn_child(args!["start"])?;
@@ -1628,7 +1628,7 @@ fn non_blocking_logger() -> Result<()> {
 
         // Write a configuration that has RPC listen_addr set
         // [Note on port conflict](#Note on port conflict)
-        let mut config = random_known_rpc_port_config(false)?;
+        let mut config = random_known_rpc_port_config(false, Mainnet)?;
         config.tracing.filter = Some("trace".to_string());
         config.tracing.buffer_limit = 100;
         let zebra_rpc_address = config.rpc.listen_addr.unwrap();
@@ -2049,7 +2049,7 @@ fn zebra_zcash_listener_conflict() -> Result<()> {
     let listen_addr = format!("127.0.0.1:{port}");
 
     // Write a configuration that has our created network listen_addr
-    let mut config = default_test_config()?;
+    let mut config = default_test_config(Mainnet)?;
     config.network.listen_addr = listen_addr.parse().unwrap();
     let dir1 = testdir()?.with_config(&mut config)?;
     let regex1 = regex::escape(&format!("Opened Zcash protocol endpoint at {listen_addr}"));
@@ -2078,7 +2078,7 @@ fn zebra_metrics_conflict() -> Result<()> {
     let listen_addr = format!("127.0.0.1:{port}");
 
     // Write a configuration that has our created metrics endpoint_addr
-    let mut config = default_test_config()?;
+    let mut config = default_test_config(Mainnet)?;
     config.metrics.endpoint_addr = Some(listen_addr.parse().unwrap());
     let dir1 = testdir()?.with_config(&mut config)?;
     let regex1 = regex::escape(&format!(r"Opened metrics endpoint at {listen_addr}"));
@@ -2107,7 +2107,7 @@ fn zebra_tracing_conflict() -> Result<()> {
     let listen_addr = format!("127.0.0.1:{port}");
 
     // Write a configuration that has our created tracing endpoint_addr
-    let mut config = default_test_config()?;
+    let mut config = default_test_config(Mainnet)?;
     config.tracing.endpoint_addr = Some(listen_addr.parse().unwrap());
     let dir1 = testdir()?.with_config(&mut config)?;
     let regex1 = regex::escape(&format!(r"Opened tracing endpoint at {listen_addr}"));
@@ -2141,7 +2141,7 @@ fn zebra_rpc_conflict() -> Result<()> {
     // [Note on port conflict](#Note on port conflict)
     //
     // This is the required setting to detect port conflicts.
-    let mut config = random_known_rpc_port_config(false)?;
+    let mut config = random_known_rpc_port_config(false, Mainnet)?;
 
     let dir1 = testdir()?.with_config(&mut config)?;
     let regex1 = regex::escape(&format!(
@@ -2168,7 +2168,7 @@ fn zebra_state_conflict() -> Result<()> {
 
     // A persistent config has a fixed temp state directory, but asks the OS to
     // automatically choose an unused port
-    let mut config = persistent_test_config()?;
+    let mut config = persistent_test_config(Mainnet)?;
     let dir_conflict = testdir()?.with_config(&mut config)?;
 
     // Windows problems with this match will be worked on at #1654
@@ -2333,7 +2333,7 @@ async fn delete_old_databases() -> Result<()> {
         return Ok(());
     }
 
-    let mut config = default_test_config()?;
+    let mut config = default_test_config(Mainnet)?;
     let run_dir = testdir()?;
     let cache_dir = run_dir.path().join("state");
 
@@ -2449,7 +2449,7 @@ async fn submit_block() -> Result<()> {
 #[test]
 fn end_of_support_is_checked_at_start() -> Result<()> {
     let _init_guard = zebra_test::init();
-    let testdir = testdir()?.with_config(&mut default_test_config()?)?;
+    let testdir = testdir()?.with_config(&mut default_test_config(Mainnet)?)?;
     let mut child = testdir.spawn_child(args!["start"])?;
 
     // Give enough time to start up the eos task.
@@ -2608,10 +2608,9 @@ async fn state_format_test(
         let test_name = &format!("{base_test_name}/apply_fake_version/{fake_version}");
         tracing::info!(?network, "running {test_name} using zebra-state");
 
-        let mut config = UseAnyState
-            .zebrad_config(test_name, false, Some(dir.path()))
+        let config = UseAnyState
+            .zebrad_config(test_name, false, Some(dir.path()), network)
             .expect("already checked config")?;
-        config.network.network = network;
 
         zebra_state::write_database_format_version_to_disk(fake_version, &config.state, network)
             .expect("can't write fake database version to disk");