elastic · kilfoyle · Dec 6, 2023 · Dec 6, 2023 · Dec 7, 2023 · Jan 15, 2024
@@ -10,6 +10,7 @@ Include these sample settings in your standalone {agent} `elastic-agent.yml` con
 * <<config-file-example-apache-logs>>
 * <<config-file-example-apache-metrics>>
 
+[discrete]
 [[config-file-example-apache-logs]]
 == Apache HTTP Server logs
 
@@ -68,7 +69,7 @@ inputs: <7>
 <1> For available output settings, refer to <<elastic-agent-output-configuration,Configure outputs for standalone {agents}>>.
 <2> For settings specific to the {es} output, refer to <<elasticsearch-output,Configure the {es} output>>.
 <3> The URL of the Elasticsearch cluster where output should be sent, including the port number. For example `https://12345ab6789cd12345ab6789cd.us-central1.gcp.cloud.es.io:443`.
-<4> An {kibana-ref}/api-keys.html[API key] used to authenticate with the {es} cluster.
+<4> An <<create-api-key-standalone-agent,API key>> used to authenticate with the {es} cluster.
 <5> For available download settings, refer to <<elastic-agent-standalone-download,Configure download settings for standalone Elastic Agent upgrades>>.
 <6> For available monitoring settings, refer to <<elastic-agent-monitoring-configuration,Configure monitoring for standalone {agents}>>.
 <7> For available input settings, refer to <<elastic-agent-input-configuration,Configure inputs for standalone {agents}>>.
@@ -79,6 +80,7 @@ inputs: <7>
 <12> Refer to {integrations-docs}/apache#logs[Logs] in the Apache HTTP Server integration documentation for the logs available to ingest and exported fields.
 <13> Path to the log files to be monitored.
 
+[discrete]
 [[config-file-example-apache-metrics]]
 == Apache HTTP Server metrics
 
@@ -120,7 +122,7 @@ inputs: <7>
 <1> For available output settings, refer to <<elastic-agent-output-configuration,Configure outputs for standalone {agents}>>.
 <2> For settings specific to the {es} output, refer to <<elasticsearch-output,Configure the {es} output>>.
 <3> The URL of the Elasticsearch cluster where output should be sent, including the port number. For example `https://12345ab6789cd12345ab6789cd.us-central1.gcp.cloud.es.io:443`.
-<4> An {kibana-ref}/api-keys.html[API key] used to authenticate with the {es} cluster.
+<4> An <<create-api-key-standalone-agent,API key>> used to authenticate with the {es} cluster.
 <5> For available download settings, refer to <<elastic-agent-standalone-download,Configure download settings for standalone Elastic Agent upgrades>>.
 <6> For available monitoring settings, refer to <<elastic-agent-monitoring-configuration,Configure monitoring for standalone {agents}>>.
 <7> For available input settings, refer to <<elastic-agent-input-configuration,Configure inputs for standalone {agents}>>.

@@ -0,0 +1,160 @@
+[[config-file-example-aws-s3]]
+= Config file example: AWS S3
+
+++++
+<titleabbrev>AWS S3</titleabbrev>
+++++
+
+Include these sample settings in your standalone {agent} `elastic-agent.yml` configuration file to ingest data from from AWS S3.
+
+* <<config-file-example-aws-s3-logs>>
+* <<config-file-example-aws-s3-metrics>>
+
+[discrete]
+[[config-file-example-aws-s3-logs]]
+== AWS S3 access logs
+
+["source","yaml"]
+----
+outputs: <1>
+  default:
+    type: elasticsearch <2>
+    hosts:
+      - '{elasticsearch-host-url}' <3>
+    api_key: "my_api_key" <4>
+agent:
+  download: <5>
+    sourceURI: 'https://artifacts.elastic.co/downloads/'
+  monitoring: <6>
+    enabled: true
+    use_output: default
+    namespace: default
+    logs: true
+    metrics: true
+inputs: <7>
+  - id: "insert a unique identifier here" <8>
+    name: aws-1
+    revision: 1
+    type: aws-s3 <9>
+    use_output: default
+    data_stream: <10>
+      namespace: default
+    streams:
+      - id: "insert a unique identifier here" <11>
+        data_stream: <10>
+          dataset: aws.s3access <12>
+          type: logs <9>
+        queue_url: 'http://queue.url'
+        max_number_of_messages: 5
+        tags:
+          - forwarded
+          - aws-s3access
+        publisher_pipeline.disable_host: true
+----
+
+<1> For available output settings, refer to <<elastic-agent-output-configuration,Configure outputs for standalone {agents}>>.
+<2> For settings specific to the {es} output, refer to <<elasticsearch-output,Configure the {es} output>>.
+<3> The URL of the {es} cluster where output should be sent, including the port number. For example `https://12345ab6789cd12345ab6789cd.us-central1.gcp.cloud.es.io:443`.
+<4> An <<create-api-key-standalone-agent,API key>> used to authenticate with the {es} cluster.
+<5> For available download settings, refer to <<elastic-agent-standalone-download,Configure download settings for standalone Elastic Agent upgrades>>.
+<6> For available monitoring settings, refer to <<elastic-agent-monitoring-configuration,Configure monitoring for standalone {agents}>>.
+<7> For available input settings, refer to <<elastic-agent-input-configuration,Configure inputs for standalone {agents}>>.
+<8> A user-defined ID to uniquely identify the input stream.
+<9> For available input types, refer to <<elastic-agent-inputs-list>>.
+<10> Learn about <<data-streams>> for time series data.
+<11> Specify a unique ID for each individual input stream. Naming the ID by appending the associated `data_stream` dataset (for example `{user-defined-unique-id}-aws.s3access`) is a recommended practice, but any unique ID will work.
+<12> Refer to the {integrations-docs}/aws#reference[reference section] in the AWS integration documentation for the type of metrics collected and exported fields.
+
+[discrete]
+[[config-file-example-aws-s3-metrics]]
+== AWS S3 daily storage and request metrics
+
+["source","yaml"]
+----
+outputs: <1>
+  default:
+    type: elasticsearch <2>
+    hosts:
+      - '{elasticsearch-host-url}' <3>
+    api_key: "my_api_key" <4>
+agent:
+  download: <5>
+    sourceURI: 'https://artifacts.elastic.co/downloads/'
+  monitoring: <6>
+    enabled: true
+    use_output: default
+    namespace: default
+    logs: true
+    metrics: true
+inputs: <7>
+  - id: "insert a unique identifier here" <8>
+    name: aws-2
+    revision: 1
+    type: aws/metrics <9>
+    use_output: default
+    data_stream: <10>
+      namespace: default
+    streams:
+      - id: "insert a unique identifier here" <11>
+        data_stream: <10>
+          dataset: aws.s3_daily_storage <12>
+          type: metrics
+        metricsets: <13>
+          - cloudwatch
+        period: 24h
+        metrics:
+          - name:
+              - BucketSizeBytes
+              - NumberOfObjects
+            namespace: AWS/S3
+            statistic:
+              - Average
+      - id: "insert a unique identifier here" <8>
+        data_stream: <10>
+          dataset: aws.s3_request <12>
+          type: metrics
+        metricsets: <13>
+          - cloudwatch
+        period: 1m
+        metrics:
+          - name:
+              - SelectScannedBytes
+              - SelectReturnedBytes
+              - BytesDownloaded
+              - BytesUploaded
+              - 4xxErrors
+              - 5xxErrors
+              - FirstByteLatency
+              - TotalRequestLatency
+            namespace: AWS/S3
+            statistic:
+              - Average
+          - name:
+              - AllRequests
+              - GetRequests
+              - PutRequests
+              - DeleteRequests
+              - HeadRequests
+              - PostRequests
+              - SelectRequests
+              - ListRequests
+              - BytesDownloaded
+              - BytesUploaded
+            namespace: AWS/S3
+            statistic:
+              - Sum
+----
+
+<1> For available output settings, refer to <<elastic-agent-output-configuration,Configure outputs for standalone {agents}>>.
+<2> For settings specific to the {es} output, refer to <<elasticsearch-output,Configure the {es} output>>.
+<3> The URL of the Elasticsearch cluster where output should be sent, including the port number. For example `https://12345ab6789cd12345ab6789cd.us-central1.gcp.cloud.es.io:443`.
+<4> An <<create-api-key-standalone-agent,API key>> used to authenticate with the {es} cluster.
+<5> For available download settings, refer to <<elastic-agent-standalone-download,Configure download settings for standalone Elastic Agent upgrades>>.
+<6> For available monitoring settings, refer to <<elastic-agent-monitoring-configuration,Configure monitoring for standalone {agents}>>.
+<7> For available input settings, refer to <<elastic-agent-input-configuration,Configure inputs for standalone {agents}>>.
+<8> A user-defined ID to uniquely identify the input stream.
+<9> For available input types, refer to <<elastic-agent-inputs-list>>.
+<10> Learn about <<data-streams>> for time series data.
+<11> Specify a unique ID for each individual input stream. Naming the ID by appending the associated `data_stream` dataset (for example `{user-defined-unique-id}-aws.cloudwatch`) is a recommended practice, but any unique ID will work.
+<12> A user-defined dataset. You can specify anything that makes sense to signify the source of the data.
+<13> Refer to the {integrations-docs}/aws#reference[reference section] in the AWS integration documentation for the type of metrics collected and exported fields.
@@ -0,0 +1,36 @@
+[[config-file-example-ingest-file]]
+= Config file example: Ingest from a file
+
+++++
+<titleabbrev>Ingest from a file</titleabbrev>
+++++
+
+The following example shows a minimal configuration to ingest data from a single file.
+
+["source","yaml"]
+----
+outputs: <1>
+  default:
+    type: elasticsearch <2>
+    hosts:
+      - '{elasticsearch-host-url}' <3>
+    api_key: "my_api_key" <4>
+agent:
+  download: <5>
+    sourceURI: 'https://artifacts.elastic.co/downloads/'
+inputs: <6>
+  - type: filestream <7>
+    id: "insert a unique identifier here" <8>
+    paths: <9>
+      - /var/log/my-application/log-file.log
+----
+
+<1> For available output settings, refer to <<elastic-agent-output-configuration,Configure outputs for standalone {agents}>>.
+<2> For settings specific to the {es} output, refer to <<elasticsearch-output,Configure the {es} output>>.
+<3> The URL of the {es} cluster where output should be sent, including the port number. For example `https://12345ab6789cd12345ab6789cd.us-central1.gcp.cloud.es.io:443`.
+<4> An <<create-api-key-standalone-agent,API key>> used to authenticate with the {es} cluster.
+<5> For available download settings, refer to <<elastic-agent-standalone-download,Configure download settings for standalone Elastic Agent upgrades>>.
+<6> For available input settings, refer to <<elastic-agent-input-configuration,Configure inputs for standalone {agents}>>.
+<7> For available input types, refer to <<elastic-agent-inputs-list>>.
+<8> A user-defined ID to uniquely identify the input stream.
+<9> Path to the log files to be monitored.
@@ -0,0 +1,52 @@
+[[config-file-example-ingest-http-metrics]]
+= Config file example: Ingest metrics from an HTTP endpoint
+
+++++
+<titleabbrev>Ingest HTTP metrics</titleabbrev>
+++++
+
+The following example shows a minimal configuration to ingest data from an HTTP endpoint.
+
+["source","yaml"]
+----
+outputs: <1>
+  default:
+    type: elasticsearch <2>
+    hosts:
+      - '{elasticsearch-host-url}' <3>
+    api_key: "my_api_key" <4>
+agent:
+  download: <5>
+    sourceURI: 'https://artifacts.elastic.co/downloads/'
+inputs: <6>
+  - id: "insert a unique identifier here" <7>
+    type: {metric type} <8>
+    use_output: default
+    data_stream: <9>
+      namespace: default
+    streams:
+      - id: "insert a unique identifier here" <10>
+        data_stream: <9>
+          dataset: "insert a name here" <11>
+          type: metrics
+        metricsets: <12>
+          - {data-stream name}
+        hosts:
+          - '{host-url-and-port}' <13>
+        period: 10s
+        server_status_path: {path-to-metrics-data}
+----
+
+<1> For available output settings, refer to <<elastic-agent-output-configuration,Configure outputs for standalone {agents}>>.
+<2> For settings specific to the {es} output, refer to <<elasticsearch-output,Configure the {es} output>>.
+<3> The URL of the Elasticsearch cluster where output should be sent, including the port number. For example `https://12345ab6789cd12345ab6789cd.us-central1.gcp.cloud.es.io:443`.
+<4> An <<create-api-key-standalone-agent,API key>> used to authenticate with the {es} cluster.
+<5> For available download settings, refer to <<elastic-agent-standalone-download,Configure download settings for standalone Elastic Agent upgrades>>.
+<6> For available input settings, refer to <<elastic-agent-input-configuration,Configure inputs for standalone {agents}>>.
+<7> A user-defined ID to uniquely identify the input stream.
+<8> For available input types, refer to <<elastic-agent-inputs-list>>.
+<9> Learn about <<data-streams>> for time series data.
+<10> Specify a unique ID for each individual input stream. Naming the ID by appending the associated `data_stream` dataset (for example `{user-defined-unique-id}-nginx.stubstatus`) is a recommended practice, but any unique ID will work.
+<11> A user-defined dataset. You can specify anything that makes sense to signify the source of the data.
+<12> Refer to `Metrics reference` in the documentation for any {integrations-docs}[integrations] that support {agent} metrics ingestion, to find the type of metrics that can be collected and the exported fields. For example, to collect Nginx metrics you can specify the `/nginx_status` path.
+<13> The URL and port number from where the metrics will be collected. For example `http://127.0.0.1:80`.
@@ -0,0 +1,46 @@
+[[config-file-example-multiple-outputs]]
+= Config file example: Configure different outputs for each input
+
+++++
+<titleabbrev>Configure different outputs for each input</titleabbrev>
+++++
+
+The following example shows a minimal configuration to assign different outputs for each input. Here, the {es} output is the default, and a secondary {ls} output is defined. Two filestream inputs are configured, each using a different output.
+
+["source","yaml"]
+----
+outputs: <1>
+  default:
+    type: elasticsearch <2>
+    hosts:
+      - '{elasticsearch-host-url}' <3>
+    api_key: "my_api_key" <4>
+  "unique-output-identifier": <5>
+    type:  logstash <6>
+    hosts: 
+      - '{logstash-host-url}' <7>
+inputs: <8>
+  - type: filestream <9>
+    id: "insert a unique identifier here" <10>
+    use_output: 'default' <11>
+    paths: <12>
+      - /var/log/my-application/log-file.log
+  - type: filestream <9>
+    id: "insert a unique identifier here" <10>
+    use_output: "unique-output-identifier" <11>
+    paths: <12>
+      - /var/log/my-application/another-log-file.log
+----
+
+<1> For available output settings, refer to <<elastic-agent-output-configuration,Configure outputs for standalone {agents}>>.
+<2> For settings specific to the {es} output, refer to <<elasticsearch-output,Configure the {es} output>>.
+<3> The URL of the {es} cluster where output should be sent, including the port number. For example `https://12345ab6789cd12345ab6789cd.us-central1.gcp.cloud.es.io:443`.
+<4> An <<create-api-key-standalone-agent,API key>> used to authenticate with the {es} cluster.
+<5> A unique name for the output. The name should also be referenced by the `use-output` setting for the input that will use this output.
+<6> For settings specific to the {ls} output, refer to <<logstash-output,{ls} output>>.
+<7> The URL of the {ls} instance where output should be sent, including the port number. For example `127.0.0.1:5044`.
+<8> For available input settings, refer to <<elastic-agent-input-configuration,Configure inputs for standalone {agents}>>.
+<9> For available input types, refer to <<elastic-agent-inputs-list>>.
+<10> A user-defined ID to uniquely identify each input stream.
+<11> The unique identifier of the output where data from this input should be sent. This should match an ID specified in the `outputs` section of the configuration.
+<12> Path to the log files to be monitored.
@@ -10,6 +10,7 @@ Include these sample settings in your standalone {agent} `elastic-agent.yml` con
 * <<config-file-example-nginx-logs>>
 * <<config-file-example-nginx-metrics>>
 
+[discrete]
 [[config-file-example-nginx-logs]]
 == Nginx HTTP Server logs
 
@@ -73,7 +74,7 @@ inputs: <7>
 <1> For available output settings, refer to <<elastic-agent-output-configuration,Configure outputs for standalone {agents}>>.
 <2> For settings specific to the {es} output, refer to <<elasticsearch-output,Configure the {es} output>>.
 <3> The URL of the {es} cluster where output should be sent, including the port number. For example `https://12345ab6789cd12345ab6789cd.us-central1.gcp.cloud.es.io:443`.
-<4> An {kibana-ref}/api-keys.html[API key] used to authenticate with the {es} cluster.
+<4> An <<create-api-key-standalone-agent,API key>> used to authenticate with the {es} cluster.
 <5> For available download settings, refer to <<elastic-agent-standalone-download,Configure download settings for standalone Elastic Agent upgrades>>.
 <6> For available monitoring settings, refer to <<elastic-agent-monitoring-configuration,Configure monitoring for standalone {agents}>>.
 <7> For available input settings, refer to <<elastic-agent-input-configuration,Configure inputs for standalone {agents}>>.
@@ -127,7 +128,7 @@ inputs: <7>
 <1> For available output settings, refer to <<elastic-agent-output-configuration,Configure outputs for standalone {agents}>>.
 <2> For settings specific to the {es} output, refer to <<elasticsearch-output,Configure the {es} output>>.
 <3> The URL of the Elasticsearch cluster where output should be sent, including the port number. For example `https://12345ab6789cd12345ab6789cd.us-central1.gcp.cloud.es.io:443`.
-<4> An {kibana-ref}/api-keys.html[API key] used to authenticate with the {es} cluster.
+<4> An <<create-api-key-standalone-agent,API key>> used to authenticate with the {es} cluster.
 <5> For available download settings, refer to <<elastic-agent-standalone-download,Configure download settings for standalone Elastic Agent upgrades>>.
 <6> For available monitoring settings, refer to <<elastic-agent-monitoring-configuration,Configure monitoring for standalone {agents}>>.
 <7> For available input settings, refer to <<elastic-agent-input-configuration,Configure inputs for standalone {agents}>>.