docs: how to use layers

docs/how-to/index.md

@@ -24,6 +24,7 @@ As your needs grow, you may want to orchestrate multiple services.
 :maxdepth: 1
 
 Manage service dependencies <service-dependencies>
+Use layers <use-layers>
 ```
 
 

docs/how-to/use-layers.md

@@ -0,0 +1,347 @@
+# How to use layers
+
+Orchestrating multiple services in remote systems is a common requirement in modern software. Consider a typical web application, which might involve:
+
+- a web server (like Nginx)
+- an application server (like uWSGI)
+- a database (like PostgreSQL)
+- a caching service (like Redis)
+
+Each of these services needs to be configured, started, and stopped in a coordinated manner.
+
+As the system scales, the operational overhead grows exponentially because multiple groups of applications need to be managed, and they need to be deployed in different environments with slightly different configurations. Without proper service orchestration tooling, managing dependencies and ensuring consistent behaviour across multiple services in multiple environments is complex and error-prone. 
+
+With Pebble, it's possible to reduce this operational overhead by splitting service configurations into different layers, and depending on how we make use of this feature, it can offer great advantages, especially when the system scales out:
+
+- We can organize services into logical groups, which greatly improves the readability and maintainability of the configurations. With this declarative, layered approach, it is easier to understand the overall system.
+- We can use a declarative and idempotent approach where we define one set of configurations per environment. In this way, we know exactly what's running in a given environment and we don't have to calculate values and patches to know what's in there.
+- While idempotency is generally desirable, layered configurations can also accommodate imperative overrides when necessary. For example, we can define base layers, environment-specific override layers, and temporary patch layers, so that we can apply the base layers across all environments, apply a temporary patch to a specific service, or customize a service for a particular environment.
+
+---
+
+## Pebble layers
+
+A layer is a configuration file that defines the desired state of services running on a system. 
+
+Layers are organized within a `layers/` subdirectory in the `$PEBBLE` directory, and their filenames are similar to `001-base-layer.yaml`, where the numerically prefixed filenames ensure a specific order of the layers, and the labels after the prefix uniquely identifies the layer. For example, `001-base-layer.yaml`, `002-override-layer.yaml`.
+
+A layer specifies service properties like the command to execute, startup behaviour, dependencies on other services, and environment variables. For example, here is an example of the current configuration format:
+
+```yaml
+summary: Simple layer
+
+description: |
+    A better description for a simple layer.
+
+services:
+    srv1:
+        override: replace
+        summary: Service summary
+        command: cmd arg1 "arg2a arg2b"
+        startup: enabled
+        after:
+            - srv2
+        before:
+            - srv3
+        requires:
+            - srv2
+            - srv3
+        environment:
+            VAR1: val1
+            VAR2: val2
+            VAR3: val3
+
+    srv2:
+        override: replace
+        startup: enabled
+        command: cmd
+        before:
+            - srv3
+
+    srv3:
+        override: replace
+        command: cmd
+```
+
+For full details of all fields, see the [complete layer specification](../reference/layer-specification).
+
+## Layer override
+
+Each layer can define new services or modify existing ones defined in preceding layers. Crucially, the mandatory `override` field in each service definition determines how the layer's configuration interacts with the previously defined service of the same name (if any):
+
+- `override: replace` completely replaces the previous definition of a service.
+- `override: merge` combines the current layer's settings with the existing ones, allowing for incremental modifications.
+
+Any of the fields can be replaced individually in a merged service configuration. To illustrate, here is a sample override layer that might sit on top of the one defined in the previous section:
+
+```yaml
+summary: Simple override layer
+
+services:
+    srv1:
+        override: merge
+        environment:
+            VAR3: val3
+        after:
+            - srv4
+        before:
+            - srv5
+
+    srv2:
+        override: replace
+        summary: Replaced service
+        startup: disabled
+        command: cmd
+
+    srv4:
+        override: replace
+        command: cmd
+        startup: enabled
+
+    srv5:
+        override: replace
+        command: cmd
+```
+
+See the [full layer specification](../reference/layer-specification) for more details.
+
+---
+
+## Use layers as logical groups
+
+If we are to orchestrate multiple applications, we can group related ones into the same layer.
+
+For example, if we have an Nginx webserver serving two applications, each with a frontend, a backend and a database:
+
+```
+nginx ──┬──> app2-frontend -> app2->backend -> app2-db
+        └──> app2-frontend -> app2->backend -> app2-db
+```
+
+We can group them by applications, for example:
+
+`001-nginx.yaml`:
+
+```yaml
+summary: Nginx layer
+
+description: |
+    An Nginx layer.
+
+services:
+    nginx:
+        override: replace
+        summary: Nginx
+        command: foo
+        startup: enabled
+```
+
+`002-app1.yaml`:
+
+```yaml
+summary: Layer for app1
+
+description: |
+    Layer for app1.
+
+services:
+    app1-database:
+        override: replace
+        summary: database for app1
+        command: foo
+        startup: enabled
+    app1-backend:
+        override: replace
+        summary: backend for app1
+        command: foo
+        after:
+            - app1-database
+    app1-frontend:
+        override: replace
+        summary: frontend for app1
+        command: foo
+        startup: enabled
+        after:
+            - app1-backend
+```
+
+`003-app2.yaml`:
+
+```yaml
+summary: Layer for app2
+
+description: |
+    Layer for app2.
+
+services:
+    app2-database:
+        override: replace
+        summary: database for app2
+        command: foo
+        startup: enabled
+    app2-backend:
+        override: replace
+        summary: backend for app2
+        command: foo
+        after:
+            - app2-database
+    app2-frontend:
+        override: replace
+        summary: frontend for app2
+        command: foo
+        startup: enabled
+        after:
+            - app2-backend
+```
+
+Alternatively, we can also group them by functionalities:
+
+`001-webserver.yaml`:
+
+```yaml
+summary: Nginx layer
+
+description: |
+    An Nginx layer.
+
+services:
+    nginx:
+        override: replace
+        summary: Nginx
+        command: foo
+        startup: enabled
+```
+
+`002-database.yaml`:
+
+```yaml
+summary: Layer for database
+
+description: |
+    Layer for database.
+
+services:
+    app1-database:
+        override: replace
+        summary: database for app1
+        command: foo
+        startup: enabled
+    app2-database:
+        override: replace
+        summary: database for app2
+        command: foo
+        startup: enabled
+```
+
+`003-backend.yaml`:
+
+```yaml
+summary: Layer for backend
+
+description: |
+    Layer for backend.
+
+services:
+    app1-backend:
+        override: replace
+        summary: backend for app1
+        command: foo
+        startup: enabled
+    app2-backend:
+        override: replace
+        summary: backend for app2
+        command: foo
+        startup: enabled
+```
+
+`004-frontend.yaml`:
+
+```yaml
+summary: Layer for frontend
+
+description: |
+    Layer for frontend.
+
+services:
+    app1-frontend:
+        override: replace
+        summary: frontend for app1
+        command: foo
+        startup: enabled
+    app2-frontend:
+        override: replace
+        summary: frontend for app2
+        command: foo
+        startup: enabled
+```
+---
+
+## Use override
+
+If we are to run the same set of services in multiple environments with slightly different configurations, we can use a base layer/environment override structure.
+
+For example, if we have a service `app1` running in both `dev` and `test` environments with slightly different configurations (for example, an environment variable named `ENV` with values as `dev` or `test), we can define a base layer, then two override layers, each for one environment:
+
+`001-base-layer.yaml`:
+
+```yaml
+summary: Simple layer
+
+description: |
+    A better description for a simple layer.
+
+services:
+    app1:
+        override: replace
+        summary: Service summary
+        command: cmd arg1 "arg2a arg2b"
+        startup: enabled
+```
+
+`002-env-override-dev.yaml`:
+
+```yaml
+summary: Override layer for the dev environment.
+
+services:
+    app1:
+        override: merge
+        environment:
+            ENV: dev
+```
+
+`002-env-override-test.yaml`:
+
+```yaml
+summary: Override layer for the test environment.
+
+services:
+    app1:
+        override: merge
+        environment:
+            ENV: test
+```
+
+In this way, when we use Pebble to run this service in different environments we can put the `001-base-layer.yaml` file into all environments. Then, in the dev environment, we also put the `002-env-override-dev.yaml` file there, and in the test environment, only ``002-env-override-test.yaml`:
+
+Dev env:
+
+```bash
+.
+└── layers
+    ├── 001-base-layer.yaml
+    └── 002-env-override-dev.yaml
+```
+
+Test env:
+
+```bash
+.
+└── layers
+    ├── 001-base-layer.yaml
+    └── 002-env-override-test.yaml
+```
+
+## See more
+
+- [Layer specification](/reference/layer-specification.md)

docs/reference/index.md

@@ -16,7 +16,6 @@ Changes and tasks <changes-and-tasks>
 CLI commands <cli-commands>
 Health checks <health-checks>
 Identities <identities>
-Layers <layers>
 Layer specification <layer-specification>
 Log forwarding <log-forwarding>
 Notices <notices>