Merge from v0.8.23

CMakeLists.txt

@@ -21,7 +21,7 @@ include(EthPolicy)
 eth_policy()
 
 # project name and version should be set after cmake_policy CMP0048
-set(PROJECT_VERSION "0.8.22")
+set(PROJECT_VERSION "0.8.23")
 # OSX target needed in order to support std::visit
 set(CMAKE_OSX_DEPLOYMENT_TARGET "10.14")
 project(solidity VERSION ${PROJECT_VERSION} LANGUAGES C CXX)

Changelog.md

@@ -1,3 +1,14 @@
+### 0.8.23 (2023-11-08)
+
+Important Bugfixes:
+ * Optimizer: Fix block deduplicator bug which led to blocks which are identical apart from the contents of ``verbatim`` instructions to be treated as equivalent and thus collapsed into a single one.
+
+
+Compiler Features:
+ * Commandline Interface: An empty ``--yul-optimizations`` sequence can now be always provided.
+ * Standard JSON Interface: An empty ``optimizerSteps`` sequence can now always be provided.
+
+
 ### 0.8.22 (2023-10-25)
 
 Language Features:

README.md

@@ -3,10 +3,10 @@
 [![Matrix Chat](https://img.shields.io/badge/Matrix%20-chat-brightgreen?style=plastic&logo=matrix)](https://matrix.to/#/#ethereum_solidity:gitter.im)
 [![Gitter Chat](https://img.shields.io/badge/Gitter%20-chat-brightgreen?style=plastic&logo=gitter)](https://gitter.im/ethereum/solidity)
 [![Solidity Forum](https://img.shields.io/badge/Solidity_Forum%20-discuss-brightgreen?style=plastic&logo=discourse)](https://forum.soliditylang.org/)
-[![Twitter Follow](https://img.shields.io/twitter/follow/solidity_lang?style=plastic&logo=twitter)](https://twitter.com/solidity_lang)
+[![X Follow](https://img.shields.io/twitter/follow/solidity_lang?style=plastic&logo=x)](https://X.com/solidity_lang)
 [![Mastodon Follow](https://img.shields.io/mastodon/follow/000335908?domain=https%3A%2F%2Ffosstodon.org%2F&logo=mastodon&style=plastic)](https://fosstodon.org/@solidity)
 
-You can talk to us on Gitter and Matrix, tweet at us on Twitter or create a new topic in the Solidity forum. Questions, feedback, and suggestions are welcome!
+You can talk to us on Gitter and Matrix, tweet at us on X (previously Twitter) or create a new topic in the Solidity forum. Questions, feedback, and suggestions are welcome!
 
 Solidity is a statically typed, contract-oriented, high-level language for implementing smart contracts on the Tron platform.
 

ReleaseChecklist.md

@@ -2,8 +2,7 @@
 
 ### Requirements
  - [ ] GitHub account with access to [solidity](https://github.com/ethereum/solidity), [solc-js](https://github.com/ethereum/solc-js),
-       [solc-bin](https://github.com/ethereum/solc-bin), [homebrew-ethereum](https://github.com/ethereum/homebrew-ethereum),
-       [solidity-website](https://github.com/ethereum/solidity-website).
+       [solc-bin](https://github.com/ethereum/solc-bin), [solidity-website](https://github.com/ethereum/solidity-website).
  - [ ] DockerHub account with push rights to the [``solc`` image](https://hub.docker.com/r/ethereum/solc).
  - [ ] Launchpad (Ubuntu One) account with a membership in the ["Ethereum" team](https://launchpad.net/~ethereum) and
        a gnupg key for your email in the ``ethereum.org`` domain (has to be version 1, gpg2 won't work).
@@ -21,7 +20,6 @@ At least a day before the release:
  - [ ] Rerun CI on the top commits of main branches in all repositories that do not have daily activity by creating a test branch or PR:
       - [ ] ``solc-js``
       - [ ] ``solc-bin`` (make sure the bytecode comparison check did run)
-      - [ ] ``homebrew-ethereum``
  - [ ] (Optional) Create a prerelease in our Ubuntu PPA by following the steps in the PPA section below on ``develop`` rather than on a tag.
        This is recommended especially when dealing with PPA for the first time, when we add a new Ubuntu version or when the PPA scripts were modified in this release cycle.
  - [ ] Verify that the release tarball of ``solc-js`` works.
@@ -58,6 +56,7 @@ At least a day before the release:
        Remove different variants of the same name manually before using the output.
  - [ ] Check that all tests on the latest commit in ``develop`` are green.
  - [ ] Click the ``Publish release`` button on the release page, creating the tag.
+       **Important: Must not be done before all the PRs, including changelog cleanup and date, are merged.**
  - [ ] Wait for the CI runs on the tag itself.
 
 ### Upload Release Artifacts and Publish Binaries
@@ -73,7 +72,6 @@ At least a day before the release:
 
 ### Homebrew and MacOS
  - [ ] Update the version and the hash (``sha256sum solidity_$VERSION.tar.gz``) in the [``solidity`` formula in Homebrew core repository](https://github.com/Homebrew/homebrew-core/blob/master/Formula/solidity.rb).
- - [ ] Update the version and the hash (``sha256sum solidity_$VERSION.tar.gz``) in [our custom ``solidity`` Homebrew formula](https://github.com/ethereum/homebrew-ethereum/blob/master/solidity.rb).
 
 ### Docker
  - [ ] Run ``./scripts/docker_deploy_manual.sh v$VERSION``.

docs/abi-spec.rst

@@ -252,7 +252,21 @@ Given the contract:
     }
 
 
-Thus, for our ``Foo`` example if we wanted to call ``baz`` with the parameters ``69`` and
+Thus, for our ``Foo`` example, if we wanted to call ``bar`` with the argument ``["abc", "def"]``, we would pass 68 bytes total, broken down into:
+
+- ``0xfce353f6``: the Method ID. This is derived from the signature ``bar(bytes3[2])``.
+- ``0x6162630000000000000000000000000000000000000000000000000000000000``: the first part of the first
+  parameter, a ``bytes3`` value ``"abc"`` (left-aligned).
+- ``0x6465660000000000000000000000000000000000000000000000000000000000``: the second part of the first
+  parameter, a ``bytes3`` value ``"def"`` (left-aligned).
+
+In total:
+
+.. code-block:: none
+
+    0xfce353f661626300000000000000000000000000000000000000000000000000000000006465660000000000000000000000000000000000000000000000000000000000
+
+If we wanted to call ``baz`` with the parameters ``69`` and
 ``true``, we would pass 68 bytes total, which can be broken down into:
 
 - ``0xcdcd77c0``: the Method ID. This is derived as the first 4 bytes of the Keccak hash of
@@ -271,20 +285,6 @@ In total:
 It returns a single ``bool``. If, for example, it were to return ``false``, its output would be
 the single byte array ``0x0000000000000000000000000000000000000000000000000000000000000000``, a single bool.
 
-If we wanted to call ``bar`` with the argument ``["abc", "def"]``, we would pass 68 bytes total, broken down into:
-
-- ``0xfce353f6``: the Method ID. This is derived from the signature ``bar(bytes3[2])``.
-- ``0x6162630000000000000000000000000000000000000000000000000000000000``: the first part of the first
-  parameter, a ``bytes3`` value ``"abc"`` (left-aligned).
-- ``0x6465660000000000000000000000000000000000000000000000000000000000``: the second part of the first
-  parameter, a ``bytes3`` value ``"def"`` (left-aligned).
-
-In total:
-
-.. code-block:: none
-
-    0xfce353f661626300000000000000000000000000000000000000000000000000000000006465660000000000000000000000000000000000000000000000000000000000
-
 If we wanted to call ``sam`` with the arguments ``"dave"``, ``true`` and ``[1,2,3]``, we would
 pass 292 bytes total, broken down into:
 

docs/bugs.json

@@ -1,4 +1,14 @@
 [
+    {
+        "uid": "SOL-2023-3",
+        "name": "VerbatimInvalidDeduplication",
+        "summary": "All ``verbatim`` blocks are considered identical by deduplicator and can incorrectly be unified when surrounded by identical opcodes.",
+        "description": "The block deduplicator is a step of the opcode-based optimizer which identifies equivalent assembly blocks and merges them into a single one. However, when blocks contained ``verbatim``, their comparison was performed incorrectly, leading to the collapse of assembly blocks which are identical except for the contents of the ``verbatim`` items. Since ``verbatim`` is only available in Yul, compilation of Solidity sources is not affected.",
+        "link": "https://blog.soliditylang.org/2023/11/08/verbatim-invalid-deduplication-bug/",
+        "introduced": "0.8.5",
+        "fixed": "0.8.23",
+        "severity": "low"
+    },
     {
         "uid": "SOL-2023-2",
         "name": "FullInlinerNonExpressionSplitArgumentEvaluationOrder",

docs/bugs_by_version.json

@@ -1743,6 +1743,7 @@
     },
     "0.8.10": {
         "bugs": [
+            "VerbatimInvalidDeduplication",
             "FullInlinerNonExpressionSplitArgumentEvaluationOrder",
             "MissingSideEffectsOnSelectorAccess",
             "AbiReencodingHeadOverflowWithStaticArrayCleanup",
@@ -1754,6 +1755,7 @@
     },
     "0.8.11": {
         "bugs": [
+            "VerbatimInvalidDeduplication",
             "FullInlinerNonExpressionSplitArgumentEvaluationOrder",
             "MissingSideEffectsOnSelectorAccess",
             "AbiReencodingHeadOverflowWithStaticArrayCleanup",
@@ -1766,6 +1768,7 @@
     },
     "0.8.12": {
         "bugs": [
+            "VerbatimInvalidDeduplication",
             "FullInlinerNonExpressionSplitArgumentEvaluationOrder",
             "MissingSideEffectsOnSelectorAccess",
             "AbiReencodingHeadOverflowWithStaticArrayCleanup",
@@ -1778,6 +1781,7 @@
     },
     "0.8.13": {
         "bugs": [
+            "VerbatimInvalidDeduplication",
             "FullInlinerNonExpressionSplitArgumentEvaluationOrder",
             "MissingSideEffectsOnSelectorAccess",
             "StorageWriteRemovalBeforeConditionalTermination",
@@ -1791,6 +1795,7 @@
     },
     "0.8.14": {
         "bugs": [
+            "VerbatimInvalidDeduplication",
             "FullInlinerNonExpressionSplitArgumentEvaluationOrder",
             "MissingSideEffectsOnSelectorAccess",
             "StorageWriteRemovalBeforeConditionalTermination",
@@ -1802,6 +1807,7 @@
     },
     "0.8.15": {
         "bugs": [
+            "VerbatimInvalidDeduplication",
             "FullInlinerNonExpressionSplitArgumentEvaluationOrder",
             "MissingSideEffectsOnSelectorAccess",
             "StorageWriteRemovalBeforeConditionalTermination",
@@ -1811,6 +1817,7 @@
     },
     "0.8.16": {
         "bugs": [
+            "VerbatimInvalidDeduplication",
             "FullInlinerNonExpressionSplitArgumentEvaluationOrder",
             "MissingSideEffectsOnSelectorAccess",
             "StorageWriteRemovalBeforeConditionalTermination"
@@ -1819,20 +1826,23 @@
     },
     "0.8.17": {
         "bugs": [
+            "VerbatimInvalidDeduplication",
             "FullInlinerNonExpressionSplitArgumentEvaluationOrder",
             "MissingSideEffectsOnSelectorAccess"
         ],
         "released": "2022-09-08"
     },
     "0.8.18": {
         "bugs": [
+            "VerbatimInvalidDeduplication",
             "FullInlinerNonExpressionSplitArgumentEvaluationOrder",
             "MissingSideEffectsOnSelectorAccess"
         ],
         "released": "2023-02-01"
     },
     "0.8.19": {
         "bugs": [
+            "VerbatimInvalidDeduplication",
             "FullInlinerNonExpressionSplitArgumentEvaluationOrder",
             "MissingSideEffectsOnSelectorAccess"
         ],
@@ -1854,19 +1864,28 @@
     },
     "0.8.20": {
         "bugs": [
+            "VerbatimInvalidDeduplication",
             "FullInlinerNonExpressionSplitArgumentEvaluationOrder",
             "MissingSideEffectsOnSelectorAccess"
         ],
         "released": "2023-05-10"
     },
     "0.8.21": {
-        "bugs": [],
+        "bugs": [
+            "VerbatimInvalidDeduplication"
+        ],
         "released": "2023-07-19"
     },
     "0.8.22": {
-        "bugs": [],
+        "bugs": [
+            "VerbatimInvalidDeduplication"
+        ],
         "released": "2023-10-25"
     },
+    "0.8.23": {
+        "bugs": [],
+        "released": "2023-11-08"
+    },
     "0.8.3": {
         "bugs": [
             "FullInlinerNonExpressionSplitArgumentEvaluationOrder",
@@ -1894,6 +1913,7 @@
     },
     "0.8.5": {
         "bugs": [
+            "VerbatimInvalidDeduplication",
             "FullInlinerNonExpressionSplitArgumentEvaluationOrder",
             "MissingSideEffectsOnSelectorAccess",
             "AbiReencodingHeadOverflowWithStaticArrayCleanup",
@@ -1906,6 +1926,7 @@
     },
     "0.8.6": {
         "bugs": [
+            "VerbatimInvalidDeduplication",
             "FullInlinerNonExpressionSplitArgumentEvaluationOrder",
             "MissingSideEffectsOnSelectorAccess",
             "AbiReencodingHeadOverflowWithStaticArrayCleanup",
@@ -1918,6 +1939,7 @@
     },
     "0.8.7": {
         "bugs": [
+            "VerbatimInvalidDeduplication",
             "FullInlinerNonExpressionSplitArgumentEvaluationOrder",
             "MissingSideEffectsOnSelectorAccess",
             "AbiReencodingHeadOverflowWithStaticArrayCleanup",
@@ -1930,6 +1952,7 @@
     },
     "0.8.8": {
         "bugs": [
+            "VerbatimInvalidDeduplication",
             "FullInlinerNonExpressionSplitArgumentEvaluationOrder",
             "MissingSideEffectsOnSelectorAccess",
             "AbiReencodingHeadOverflowWithStaticArrayCleanup",
@@ -1943,6 +1966,7 @@
     },
     "0.8.9": {
         "bugs": [
+            "VerbatimInvalidDeduplication",
             "FullInlinerNonExpressionSplitArgumentEvaluationOrder",
             "MissingSideEffectsOnSelectorAccess",
             "AbiReencodingHeadOverflowWithStaticArrayCleanup",

docs/internals/optimizer.rst

@@ -30,6 +30,11 @@ for a stand-alone Yul mode.
     The `peephole optimizer <https://en.wikipedia.org/wiki/Peephole_optimization>`_ is always
     enabled by default and can only be turned off via the :ref:`Standard JSON <compiler-api>`.
 
+.. note::
+    An empty optimizer sequence is accepted even without ``--optimize`` in order to fully disable
+    the user-supplied portion of the Yul :ref:`optimizer sequence <selecting-optimizations>`, as by default,
+    even when the optimizer is not turned on, the :ref:`unused pruner <unused-pruner>` step will be run.
+
 You can find more details on both optimizer modules and their optimization steps below.
 
 Benefits of Optimizing Solidity Code
@@ -329,6 +334,7 @@ Abbreviation Full name
 Some steps depend on properties ensured by ``BlockFlattener``, ``FunctionGrouper``, ``ForLoopInitRewriter``.
 For this reason the Yul optimizer always applies them before applying any steps supplied by the user.
 
+.. _selecting-optimizations:
 
 Selecting Optimizations
 -----------------------

libevmasm/AssemblyItem.h

@@ -140,7 +140,7 @@ class AssemblyItem
 		else if (type() == Operation)
 			return instruction() < _other.instruction();
 		else if (type() == VerbatimBytecode)
-			return *m_verbatimBytecode == *_other.m_verbatimBytecode;
+			return *m_verbatimBytecode < *_other.m_verbatimBytecode;
 		else
 			return data() < _other.data();
 	}

libsolidity/interface/CompilerStack.cpp

@@ -69,6 +69,7 @@
 #include <libyul/YulStack.h>
 #include <libyul/AST.h>
 #include <libyul/AsmParser.h>
+#include <libyul/optimiser/Suite.h>
 
 #include <liblangutil/Scanner.h>
 #include <liblangutil/SemVerHandler.h>
@@ -99,6 +100,7 @@ using namespace solidity;
 using namespace solidity::langutil;
 using namespace solidity::frontend;
 using namespace solidity::stdlib;
+using namespace solidity::yul;
 using namespace std::string_literals;
 
 using solidity::util::errinfo_comment;
@@ -1672,6 +1674,21 @@ std::string CompilerStack::createMetadata(Contract const& _contract, bool _forIR
 			details["yulDetails"]["stackAllocation"] = m_optimiserSettings.optimizeStackAllocation;
 			details["yulDetails"]["optimizerSteps"] = m_optimiserSettings.yulOptimiserSteps + ":" + m_optimiserSettings.yulOptimiserCleanupSteps;
 		}
+		else if (
+			OptimiserSuite::isEmptyOptimizerSequence(m_optimiserSettings.yulOptimiserSteps) &&
+			OptimiserSuite::isEmptyOptimizerSequence(m_optimiserSettings.yulOptimiserCleanupSteps)
+		)
+		{
+			solAssert(m_optimiserSettings.optimizeStackAllocation == false);
+			details["yulDetails"] = Json::objectValue;
+			details["yulDetails"]["optimizerSteps"] = ":";
+		}
+		else
+		{
+			solAssert(m_optimiserSettings.optimizeStackAllocation == false);
+			solAssert(m_optimiserSettings.yulOptimiserSteps == OptimiserSettings::DefaultYulOptimiserSteps);
+			solAssert(m_optimiserSettings.yulOptimiserCleanupSteps == OptimiserSettings::DefaultYulOptimiserCleanupSteps);
+		}
 
 		meta["settings"]["optimizer"]["details"] = std::move(details);
 	}