add removal of nodes / branches v2 (#348)

* add 'delete' element for node/branch removal

Signed-off-by: Niclas Wesemann <niclas.wesemann@motius.de>

docs/vspec2x_arch.md

@@ -1,7 +1,5 @@
 # vspec2x Design Decisions, Architecture and Limitations
 
-
-
 ## Overlays
 
 Each overlay is treated as a separate tree and merged (after expansion) on the existing tree.
@@ -11,7 +9,6 @@ This means that descriptions, comments, datatype, unit and similar attributes ge
 
 Expansion is the process where a branch with instances is replaced with multiple branches.
 
-
 This will result in that e.g. `Vehicle.Cabin.Door` is expanded to the following branches
 
 * `Vehicle.Cabin.Door.Row1.Left`
@@ -22,42 +19,118 @@ This will result in that e.g. `Vehicle.Cabin.Door` is expanded to the following
 For some exporters expansion can be suppressed by using the `--no_expand` option.
 Then instance information will be represented by other means in the resulting output.
 
+## Deletion / Node removal
+
+Nodes can be removed from the tree by using the `delete` element in the overlay.
+This is useful when a signal is not used in a next version of the specification or if you
+simply want to delete a node or branch from the specification, e.g. you are not using a signal of the base
+specification.
+Please note, that the deletion for branches will delete all nodes that are connected to that branch (which is
+desired behavior). Also, if a branch node is deleted, all nodes that are connected to that branch will be deleted
+irrespective of what their delete element value is.
+
+We chose three examples to show what you can do with the delete element. Let's say we have the following excerpt from
+the base vehicle signal specification:
+
+```yaml
+Vehicle.Service:
+  description: Service data.
+  type: branch
+
+Vehicle.Service.DistanceToService:
+  datatype: float
+  description: Remaining distance to service (of any kind). Negative values indicate service overdue.
+  type: sensor
+  unit: km
+
+Vehicle.Service.IsServiceDue:
+  datatype: boolean
+  description: Indicates if vehicle needs service (of any kind). True = Service needed now or in the near future. False = No known need for service.
+  type: sensor
+
+Vehicle.Service.TimeToService:
+  datatype: int32
+  description: Remaining time to service (of any kind). Negative values indicate service overdue.
+  type: sensor
+  unit: s
+```
+
+Now if you want to delete the `Vehicle.Service.TimeToService` node from the specification, you can do this by adding the
+delete element to your overlay like this:
+
+```yaml
+Vehicle.Service.TimeToService:
+  type: sensor
+  datatype: int32
+  delete: true
+```
+
+Let's say you now want to delete the whole branch `Vehicle.Service` from the specification. You can do this by adding:
+
+```yaml
+Vehicle.Service:
+  type: branch
+  delete: true
+```
+
+Also, the delement element can be used on instances after they have been expanded. If you want to delete a node or
+branch that has been expanded using instances you can add the `delete` element to the overlay, let's say the vehicle
+only has two doors in the front. In this case we would like to delete the signals for the rear doors:
+
+```yaml
+Vehicle.Cabin.Door.Row2:
+  type: branch
+  delete: true
+```
+
+By adding the `delete: true` to a node or branch all nodes and branches connected to it are deleted by vss-tools
+when converting to a different format.
+
+Please note that for branches you need to provide at least the `type` element to
+the overlay. For nodes you at least have to provide the `type` and `datatype` elements. Currently, the elements provided
+do not have to match the previously given elements in the base specification.
+
 ## Expansion and Overlays
 
 Sometimes an overlay only refers to a signal in a specific branch, like:
 
-```
+```yaml
 Vehicle.Cabin.Door.Row2.Right.NewSignal:
   datatype: int8
   type: sensor
   unit: km
   description: A new signal for just one door.
 ```
 
-We do not want this signal expanded, and the tooling prevents expansion by taking the instance `Row2` and comparing with instances declared for `Door`.
+We do not want this signal expanded, and the tooling prevents expansion by taking the instance `Row2` and comparing with
+instances declared for `Door`.
 
-```
+```yaml
 Door:
   type: branch
   instances:
     - Row[1,2]
-    - ["Left","Right"]
+    - [ "Left","Right" ]
   description: d1
   comment: c1
 ```
 
-It will do this by taking the instances on first level (`Row1` and `Row2`) and if comparing with current branch (`Row2`).
-If they match it will repeat the check for `Right`and finally merge `NewSignal` with other signals in `Vehicle.Cabin.Door`.
+It will do this by taking the instances on first level (`Row1` and `Row2`) and if comparing with current
+branch (`Row2`).
+If they match it will repeat the check for `Right`and finally merge `NewSignal` with other signals
+in `Vehicle.Cabin.Door`.
 
-Description, comments and other data defined for a specific instance (like `Vehicle.Cabin.Door.Row2.Right.NewSignal` above) have precedence
+Description, comments and other data defined for a specific instance (like `Vehicle.Cabin.Door.Row2.Right.NewSignal`
+above) have precedence
 over data defined for the unexpanded signal `Vehicle.Cabin.Door.NewSignal`.
 
-The merge algorithm tries to be smart, so that if you use `Row*` it assume it is an instantiated branch if branch has an instance declaration of type `Row[m,n]`,
-even if the the value of `Row*` is outside the range. It will however in that case not inherit values from the base branch.
+The merge algorithm tries to be smart, so that if you use `Row*` it assume it is an instantiated branch if branch has an
+instance declaration of type `Row[m,n]`,
+even if the the value of `Row*` is outside the range. It will however in that case not inherit values from the base
+branch.
 
 ## Linters and Static Code Checkers
 
-
 ### MyPy
 
 [Mypy](https://mypy-lang.org/) is used for static type checking of code.
@@ -80,7 +153,6 @@ Suppressed error categories include:
   and can thus not be analyzed.
 * Mypy does not like "method variables" like `load_flat_model.include_index`
 
-
 ### Flake8
 
 Flake8 is used as linter. It is also configured to be used as pre-commit hook.

tests/vspec/.gitignore

@@ -2,3 +2,4 @@
 **/out.json
 **/out.txt
 **/out.vspec
+**/out.yaml

tests/vspec/test_node_removal/test_files/test.vspec

@@ -0,0 +1,71 @@
+# Copyright (c) 2024 Contributors to COVESA
+#
+# This program and the accompanying materials are made available under the
+# terms of the Mozilla Public License 2.0, which is available at
+# https://www.mozilla.org/en-US/MPL/2.0/
+#
+# SPDX-License-Identifier: MPL-2.0
+
+A:
+  type: branch
+  description: A is a test node
+A.Float:
+  datatype: float
+  type: actuator
+  unit: mm
+  description: A.Float is a leaf of A of datatype float.
+A.Int16:
+  datatype: int16
+  type: sensor
+  unit: rpm
+  description: A.Int16 is a leaf of A of datatype int16.
+A.String:
+  datatype: string
+  type: sensor
+  description: A.String is a leaf of A of datatype string.
+  deprecation: This is test deprecation, let's say it used to be called Str instead String.
+A.StringArray:
+  datatype: string[]
+  type: sensor
+  description: A.StringArray is a leaf of A of datatype string array.
+A.B:
+  type: branch
+  description: B is a branch of A.
+A.B.Int32:
+  datatype: int32
+  type: sensor
+  unit: rpm
+  description: A.B.Int32 is a leaf of A.B of datatype int32.
+A.B.NewName:
+  datatype: uint32
+  type: sensor
+  unit: mm
+  description: A.B.NewName's old name is 'OldName'. And its even older name is 'OlderName'.
+  fka: ['A.B.OlderName', 'A.B.OldName']
+A.B.IsLeaf:
+  datatype: string
+  type: actuator
+  allowed: ["YES", "NO"]
+  description: This node is a leaf of the tree and it has allowed values (aka an enum).
+A.B.Min:
+  datatype: uint8
+  type: sensor
+  min: 10
+  unit: percent
+  description: A leaf that uses a minimum value.
+A.B.Max:
+  datatype: uint8
+  type: sensor
+  unit: percent
+  min: 0
+  max: 100
+  description: A leaf that uses a maximum value.
+A.C:
+  type: branch
+  description: C is a branch of A.
+  instances: Instance[1,2]
+A.C.Test:
+  datatype: uint32
+  type: sensor
+  unit: mm
+  description: A.C.Test is a leaf of A.C of datatype uint32.

tests/vspec/test_node_removal/test_files/test_del_branch_overlay.vspec

@@ -0,0 +1,11 @@
+# Copyright (c) 2024 Contributors to COVESA
+#
+# This program and the accompanying materials are made available under the
+# terms of the Mozilla Public License 2.0, which is available at
+# https://www.mozilla.org/en-US/MPL/2.0/
+#
+# SPDX-License-Identifier: MPL-2.0
+
+A.B:
+  type: branch
+  delete: true

tests/vspec/test_node_removal/test_files/test_del_instance_overlay.vspec

@@ -0,0 +1,11 @@
+# Copyright (c) 2024 Contributors to COVESA
+#
+# This program and the accompanying materials are made available under the
+# terms of the Mozilla Public License 2.0, which is available at
+# https://www.mozilla.org/en-US/MPL/2.0/
+#
+# SPDX-License-Identifier: MPL-2.0
+
+A.C.Instance2:
+  type: branch
+  delete: true

tests/vspec/test_node_removal/test_files/test_del_node_overlay.vspec

@@ -0,0 +1,12 @@
+# Copyright (c) 2024 Contributors to COVESA
+#
+# This program and the accompanying materials are made available under the
+# terms of the Mozilla Public License 2.0, which is available at
+# https://www.mozilla.org/en-US/MPL/2.0/
+#
+# SPDX-License-Identifier: MPL-2.0
+
+A.B.Int32:
+  type: sensor
+  datatype: int32
+  delete: true

tests/vspec/test_node_removal/test_files/test_del_wrong_instance_overlay.vspec

@@ -0,0 +1,11 @@
+# Copyright (c) 2024 Contributors to COVESA
+#
+# This program and the accompanying materials are made available under the
+# terms of the Mozilla Public License 2.0, which is available at
+# https://www.mozilla.org/en-US/MPL/2.0/
+#
+# SPDX-License-Identifier: MPL-2.0
+
+A.C.Instance3:
+  type: branch
+  delete: true

tests/vspec/test_node_removal/test_files/test_deleted_branch.vspec

@@ -0,0 +1,72 @@
+# Copyright (c) 2024 Contributors to COVESA
+#
+# This program and the accompanying materials are made available under the
+# terms of the Mozilla Public License 2.0, which is available at
+# https://www.mozilla.org/en-US/MPL/2.0/
+#
+# SPDX-License-Identifier: MPL-2.0
+
+A:
+  type: branch
+  description: A is a test node
+A.Float:
+  datatype: float
+  type: actuator
+  unit: mm
+  description: A.Float is a leaf of A of datatype float.
+A.Int16:
+  datatype: int16
+  type: sensor
+  unit: rpm
+  description: A.Int16 is a leaf of A of datatype int16.
+A.String:
+  datatype: string
+  type: sensor
+  description: A.String is a leaf of A of datatype string.
+  deprecation: This is test deprecation, let's say it used to be called Str instead String.
+A.StringArray:
+  datatype: string[]
+  type: sensor
+  description: A.StringArray is a leaf of A of datatype string array.
+A.B:
+  type: branch
+  description: B is a branch of A.
+  delete: true
+A.B.Int32:
+  datatype: int32
+  type: sensor
+  unit: rpm
+  description: A.B.Int32 is a leaf of A.B of datatype int32.
+A.B.NewName:
+  datatype: uint32
+  type: sensor
+  unit: mm
+  description: A.B.NewName's old name is 'OldName'. And its even older name is 'OlderName'.
+  fka: ['A.B.OlderName', 'A.B.OldName']
+A.B.IsLeaf:
+  datatype: string
+  type: actuator
+  allowed: ["YES", "NO"]
+  description: This node is a leaf of the tree and it has allowed values (aka an enum).
+A.B.Min:
+  datatype: uint8
+  type: sensor
+  min: 10
+  unit: percent
+  description: A leaf that uses a minimum value.
+A.B.Max:
+  datatype: uint8
+  type: sensor
+  unit: percent
+  min: 0
+  max: 100
+  description: A leaf that uses a maximum value.
+A.C:
+  type: branch
+  description: C is a branch of A.
+  instances: Instance[1,2]
+A.C.Test:
+  datatype: uint32
+  type: sensor
+  unit: mm
+  description: A.C.Test is a leaf of A.C of datatype uint32.