[release-1.4] RHIDP-4807: Third-party plugin installation in RHDH (#793)

* RHIDP-4807: Third-party plugin installation in RHDH

* fixed minor issues

* Incorporated review suggestions

* incorporated review suggestions

* fixed links and added context to install plugin assembly

* review suggestions incorporated

* review suggestions incorporated

* review suggestions incorporated

* review suggestions incorporated

* review suggestions incorporated

---------

Co-authored-by: Heena Manwani <hmanwani@redhat.com>

assemblies/dynamic-plugins/assembly-install-third-party-plugins-rhdh.adoc

@@ -0,0 +1,32 @@
+[id="assembly-install-third-party-plugins-rhdh"]
+= Installing third-party plugins in {product}
+:context: assembly-install-third-party-plugins-rhdh
+
+You can install a third-party plugins in {product} without rebuilding the {product-very-short} application.
+
+The location of the `dynamic-plugin-config.yaml` file depends on the deployment method. For more details, refer to xref:proc-config-dynamic-plugins-rhdh-operator_rhdh-installing-rhdh-plugins[Installing dynamic plugins with the {product} Operator] and xref:con-install-dynamic-plugin-helm_rhdh-installing-rhdh-plugins[Installing dynamic plugins using the Helm chart].
+
+Plugins are defined in the `plugins` array within the `dynamic-plugin-config.yaml` file. Each plugin is represented as an object with the following properties:
+
+* `package`: The plugin's package definition, which can be an OCI image, a TGZ file, a JavaScript package, or a directory path.
+* `disabled`: A boolean value indicating whether the plugin is enabled or disabled.
+* `integrity`: The integrity hash of the package, required for TGZ file and JavaScript packages.
+* `pluginConfig`: The plugin's configuration. For backend plugins, this is optional; for frontend plugins, it is required. The `pluginConfig` is a fragment of the `app-config.yaml` file, and any added properties are merged with the {product-very-short} `app-config.yaml` file.
+
+[NOTE]
+====
+You can also load dynamic plugins from another directory, though this is intended for development or testing purposes and is not recommended for production, except for plugins included in the {product-very-short} container image.
+====
+
+//OCI image
+include::../modules/dynamic-plugins/proc-load-plugin-oci-image.adoc[leveloffset=+2]
+
+//TGZ file
+include::../modules/dynamic-plugins/proc-load-plugin-tgz-file.adoc[leveloffset=+2]
+
+//JavaScript package
+include::../modules/dynamic-plugins/proc-load-plugin-js-package.adoc[leveloffset=+2]
+
+//example third-party plugin installation
+include::../modules/dynamic-plugins/ref-example-third-party-plugin-installation.adoc[leveloffset=+2]
+

assemblies/dynamic-plugins/assembly-installing-rhdh-plugins.adoc

@@ -1,25 +1,23 @@
 [id="rhdh-installing-rhdh-plugins_{context}"]
 = Installing dynamic plugins in {product}
+:context: rhdh-installing-rhdh-plugins
 
 The dynamic plugin support is based on the backend plugin manager package, which is a service that scans a configured root directory (`dynamicPlugins.rootDirectory` in the `app-config.yaml` file) for dynamic plugin packages and loads them dynamically.
 
 You can use the dynamic plugins that come preinstalled with {product} or install external dynamic plugins from a public NPM registry.
 
 // Operator installation
-include::../modules/dynamic-plugins/proc-config-dynamic-plugins-rhdh-operator.adoc[leveloffset=+1]
+include::../modules/dynamic-plugins/proc-config-dynamic-plugins-rhdh-operator.adoc[leveloffset=+2]
 
 // Helm installation
-include::../modules/dynamic-plugins/con-install-dynamic-plugin-helm.adoc[leveloffset=+1]
-include::../modules/dynamic-plugins/proc-obtaining-integrity-checksum.adoc[leveloffset=+2]
-include::../modules/dynamic-plugins/ref-example-dynamic-plugin-helm-installations.adoc[leveloffset=+2]
-include::../modules/dynamic-plugins/proc-rhdh-example-external-dynamic-plugins.adoc[leveloffset=+2]
+include::../modules/dynamic-plugins/con-install-dynamic-plugin-helm.adoc[leveloffset=+2]
+//include::../modules/dynamic-plugins/proc-obtaining-integrity-checksum.adoc[leveloffset=+3] //from tkral:This is documented in a better way in "= Installing third-party plugins in {product}" we can remove this module. It is not even placed correctly here as this is not specific to helm chart in anyway
+include::../modules/dynamic-plugins/ref-example-dynamic-plugin-helm-installations.adoc[leveloffset=+3]
+//include::../modules/dynamic-plugins/proc-rhdh-example-external-dynamic-plugins.adoc[leveloffset=+3] assemblies/dynamic-plugins/assembly-third-party-plugins-installation.adoc replaces this
 
 // Air gapped environment
 //include::../modules/dynamic-plugins/proc-rhdh-installing-external-dynamic-plugins-airgapped.adoc[leveloffset=+1]
-include::../modules/dynamic-plugins/proc-using-custom-npm-registry.adoc[leveloffset=+1]
-
-// Viewing installed plugins
-include::../modules/dynamic-plugins/proc-viewing-installed-plugins.adoc[leveloffset=+1]
+include::../modules/dynamic-plugins/proc-install-plugins-using-custom-npm-registry.adoc[leveloffset=+2]
 
 //basic plugin configuration
 //include::../modules/dynamic-plugins/con-basic-config-dynamic-plugins.adoc[leveloffset=+1]

assemblies/dynamic-plugins/assembly-package-publish-third-party-dynamic-plugin.adoc

@@ -0,0 +1,23 @@
+[id="assembly-package-publish-third-party-dynamic-plugin"]
+= Packaging and publishing third-party plugins as dynamic plugins
+:context: assembly-package-publish-third-party-dynamic-plugin
+
+After xref:proc-export-third-party-plugins-rhdh_assembly-third-party-plugins[exporting a third-party plugin], you can package the derived package into one of the following supported formats:
+
+* Open Container Initiative (OCI) image (recommended)
+* TGZ file
+* JavaScript package
++
+[IMPORTANT]
+====
+Exported dynamic plugin packages must only be published to private NPM registries.
+====
+
+//OCI image
+include::../modules/dynamic-plugins/proc-create-plugin-oci-image.adoc[leveloffset=+2]
+
+//TGZ file
+include::../modules/dynamic-plugins/proc-create-plugin-tgz-file.adoc[leveloffset=+2]
+
+//JavaScript package
+include::../modules/dynamic-plugins/proc-create-plugin-js-package.adoc[leveloffset=+2]

assemblies/dynamic-plugins/assembly-third-party-plugins.adoc

@@ -0,0 +1,26 @@
+[id="assembly-third-party-plugins"]
+= Third-party plugins in {product}
+:context: assembly-third-party-plugins
+
+You can integrate third-party dynamic plugins into {product} to enhance its functionality without modifying its source code or rebuilding it. To add these plugins, export them as derived packages.
+
+While exporting the plugin package, you must ensure that dependencies are correctly bundled or marked as shared, depending on their relationship to the {product-short} environment.
+
+To integrate a third-party plugin into {product-short}:
+
+. First, obtain the plugin's source code.
+. Export the plugin as a dynamic plugin package. See xref:proc-export-third-party-plugins-rhdh_assembly-third-party-plugins[].
+. Package and publish the dynamic plugin. See xref:assembly-package-publish-third-party-dynamic-plugin[].
+. Install the plugin in the {product-short} environment. See xref:assembly-install-third-party-plugins-rhdh[].
+
+//Export third-party plugins
+include::../modules/dynamic-plugins/proc-export-third-party-plugins-rhdh.adoc[leveloffset=+2]
+
+//package and publish third-party plugins
+include::assembly-package-publish-third-party-dynamic-plugin.adoc[leveloffset=+1]
+
+//Install third-party plugin
+include::assembly-install-third-party-plugins-rhdh.adoc[leveloffset=+1]
+
+
+

modules/dynamic-plugins/proc-create-plugin-js-package.adoc

@@ -0,0 +1,39 @@
+[id="proc-create-plugin-js-package_{context}"]
+= Creating a JavaScript package with dynamic packages
+
+[WARNING]
+====
+The derived dynamic plugin JavaScript packages must not be published to the public NPM registry. If you must publish to the NPM registry, use a private registry.
+====
+
+.Prerequisites
+* You have exported a third-party dynamic plugin package. For more information, see xref:proc-export-third-party-plugins-rhdh_assembly-third-party-plugins[].
+
+.Procedure
+. Navigate to the `dist-dynamic` directory.
+. Run the following command to publish the package to your private NPM registry:
++
+--
+.Example command to publish a plugin package to an NPM registry
+[source,terminal]
+----
+npm publish --registry <npm_registry_url>
+----
+
+[TIP]
+====
+You can add the following to your `package.json` file before running the `export` command:
+
+.Example `package.json` file
+[source,json]
+----
+{
+  "publishConfig": {
+    "registry": "<npm_registry_url>"
+  }
+}
+----
+
+If you modify `publishConfig` after exporting the dynamic plugin, re-run the `export-dynamic-plugin` command to ensure the correct configuration is included.
+====
+--

modules/dynamic-plugins/proc-create-plugin-oci-image.adoc

@@ -0,0 +1,37 @@
+[id="proc-create-plugin-oci-image_{context}"]
+= Creating an OCI image with dynamic packages
+
+.Prerequisites
+* You have installed `podman` or `docker`.
+* You have exported a third-party dynamic plugin package. For more information, see xref:proc-export-third-party-plugins-rhdh_assembly-third-party-plugins[].
+
+.Procedure
+. Navigate to the plugin's root directory (not the `dist-dynamic` directory).
+. Run the following command to package the plugin into an OCI image:
++
+--
+.Example command to package an exported third-party plugin
+[source,terminal]
+----
+npx @janus-idp/cli@latest package package-dynamic-plugins --tag quay.io/example/image:v0.0.1
+----
+
+In the previous command, the `--tag` argument specifies the image name and tag.
+--
+. Run one of the following commands to push the image to a registry:
++
+--
+.Example command to push an image to a registry using podman
+[source,terminal]
+----
+podman push quay.io/example/image:v0.0.1
+----
+
+.Example command to push an image to a registry using docker
+[source,terminal]
+----
+docker push quay.io/example/image:v0.0.1
+----
+
+The output of the `package-dynamic-plugins` command provides the plugin's path for use in the `dynamic-plugin-config.yaml` file.
+--

modules/dynamic-plugins/proc-create-plugin-tgz-file.adoc

@@ -0,0 +1,70 @@
+[id="proc-create-plugin-tgz-file_{context}"]
+= Creating a TGZ file with dynamic packages
+
+.Prerequisites
+* You have exported a third-party dynamic plugin package. For more information, see xref:proc-export-third-party-plugins-rhdh_assembly-third-party-plugins[].
+
+.Procedure
+. Navigate to the `dist-dynamic` directory.
+. Run the following command to create a `tgz` archive:
++
+--
+.Example command to create a `tgz` archive
+[source,terminal]
+----
+npm pack
+----
+You can obtain the integrity hash from the output of the `npm pack` command by using the `--json` flag as follows:
+
+.Example command to obtain the integrity hash of a `tgz` archive
+[source,terminal]
+----
+npm pack --json | head -n 10
+----
+--
+
+. Host the archive on a web server accessible to your {product-very-short} instance, and reference its URL in the `dynamic-plugin-config.yaml` file as follows:
++
+--
+.Example `dynamic-plugin-config.yaml` file
+[source,yaml]
+----
+plugins:
+  - package: https://example.com/backstage-plugin-myplugin-1.0.0.tgz
+    integrity: sha512-<hash>
+----
+--
+. Run the following command to package the plugins:
++
+--
+.Example command to package a dynamic plugin
+[source,terminal]
+----
+npm pack --pack-destination ~/test/dynamic-plugins-root/
+----
+
+[TIP]
+====
+To create a plugin registry using HTTP server on {ocp-short}, run the following commands:
+
+.Example commands to build and deploy an HTTP server in {ocp-short}
+[source,terminal]
+----
+oc project rhdh
+oc new-build httpd --name=plugin-registry --binary
+oc start-build plugin-registry --from-dir=dynamic-plugins-root --wait
+oc new-app --image-stream=plugin-registry
+----
+====
+--
+
+. Configure your {product-very-short} to use plugins from the HTTP server by editing the `dynamic-plugin-config.yaml` file:
++
+--
+.Example configuration to use packaged plugins in {product-very-short} 
+[source,yaml]
+----
+plugins:
+  - package: http://plugin-registry:8080/backstage-plugin-myplugin-1.9.6.tgz
+----
+--

modules/dynamic-plugins/proc-export-third-party-plugins-rhdh.adoc

@@ -0,0 +1,124 @@
+[id="proc-export-third-party-plugins-rhdh_{context}"]
+= Exporting third-party plugins in {product}
+
+To use plugins in {product}, you can export plugins as derived dynamic plugin packages. These packages contain the plugin code and dependencies, ready for dynamic plugin integration into {product-short}.
+
+.Prerequisites
+* The `@janus-idp/cli` package is installed. Use the latest version (`@latest` tag) for compatibility with the most recent features and fixes.
+* Node.js and NPM is installed and configured.
+* The third-party plugin is compatible with your {product} version. For more information, see link:https://github.com/janus-idp/backstage-showcase/blob/main/docs/dynamic-plugins/versions.md[Version compatibility matrix].
+* The third-party plugin must have a valid `package.json` file in its root directory, containing all required metadata and dependencies.
++
+--
+Backend plugins::
++
+To ensure compatibility with the dynamic plugin support and enable their use as dynamic plugins, existing backend plugins must be compatible with the new Backstage backend system. Additionally, these plugins must be rebuilt using a dedicated CLI command.
++
+The new Backstage backend system entry point (created using `createBackendPlugin()` or `createBackendModule()`)  must be exported as the default export from either the main package or an `alpha` package (if the plugin instance support is still provided using `alpha` APIs). This doesn't add any additional requirement on top of the standard plugin development guidelines of the plugin instance.
++
+The dynamic export mechanism identifies private dependencies and sets the `bundleDependencies` field in the `package.json` file. This export mechanism ensures that the dynamic plugin package is published as a self-contained package, with its private dependencies bundled in a private `node_modules` folder.
++
+Certain plugin dependencies require specific handling in the derived packages, such as:
++
+* *Shared dependencies* are provided by the {product-very-short} application and listed as `peerDependencies` in `package.json` file, not bundled in the dynamic plugin package. For example, by default, all `@backstage` scoped packages are shared.
++
+You can use the `--shared-package` flag to specify shared dependencies, that are expected to be provided by {product} application and not bundled in the dynamic plugin package.
++
+To treat a `@backstage` package as private, use the negation prefix (`!`). For example, when a plugin depends on the package in `@backstage` that is not provided by the {product} application. 
+
+* *Embedded dependencies* are bundled into the dynamic plugin package with their dependencies hoisted to the top level. By default, packages with `-node` or `-common` suffixes are embedded.
++
+You can use the `--embed-package` flag to specify additional embedded packages. For example, packages from the same workspace that do not follow the default naming convention.
++
+The following is an example of exporting a dynamic plugin with shared and embedded packages:
++
+.Example dynamic plugin export with shared and embedded packages
+[source,terminal]
+----
+npx @janus-idp/cli@latest export-dynamic-plugin --shared-package '!/@backstage/plugin-notifications/' <1> --embed-package @backstage/plugin-notifications-backend <2>
+----
++
+<1> `@backstage/plugin-notifications` package is treated as a private dependency and is bundled in the dynamic plugin package, despite being in the `@backstage` scope.
+<2> `@backstage/plugin-notifications-backend` package is marked as an embedded dependency and is bundled in the dynamic plugin package.
+
+Front-end plugins::
++
+Front-end plugins can use `scalprum` for configuration, which the CLI can generate automatically during the export process. The generated default configuration is logged when running the following command:
++
+.Example command to log the default configuration
+[source,terminal]
+----
+npx @janus-idp/cli@latest export-dynamic
+----
++
+The following is an example of default `scalprum` configuration:
++
+.Default `scalprum` configuration
+[source,json]
+----
+"scalprum": {
+  "name": "<package_name>",  // The Webpack container name matches the NPM package name, with "@" replaced by "." and "/" removed.
+  "exposedModules": {
+    "PluginRoot": "./src/index.ts"  // The default module name is "PluginRoot" and doesn't need explicit specification in the app-config.yaml file.
+  }
+}
+----
++
+You can add a `scalprum` section to the `package.json` file. For example:
++
+.Example `scalprum` customization
+[source,json]
+----
+"scalprum": {
+  "name": "custom-package-name",
+  "exposedModules": {
+    "FooModuleName": "./src/foo.ts",
+    "BarModuleName": "./src/bar.ts"
+    // Define multiple modules here, with each exposed as a separate entry point in the Webpack container.
+  }
+}
+----
++
+Dynamic plugins might need adjustments for {product-short} needs, such as static JSX for mountpoints or dynamic routes. These changes are optional but might be incompatible with static plugins.
++
+To include static JSX, define an additional export and use it as the dynamic plugin's `importName`. For example:
++
+.Example static and dynamic plugin export
+[source,tsx]
+----
+// For a static plugin
+export const EntityTechdocsContent = () => {...}
+
+// For a dynamic plugin
+export const DynamicEntityTechdocsContent = {
+  element: EntityTechdocsContent,
+  staticJSXContent: (
+    <TechDocsAddons>
+      <ReportIssue />
+    </TechDocsAddons>
+  ),
+};
+----
+--
+
+.Procedure
+* Use the `package export-dynamic-plugin` command from the `@janus-idp/cli` package to export the plugin:
++
+--
+.Example command to export a third-party plugin
+[source,terminal]
+----
+npx @janus-idp/cli@latest package export-dynamic-plugin
+----
+
+Ensure that you execute the previous command in the root directory of the plugin's JavaScript package (containing `package.json` file).
+
+The resulting derived package will be located in the `dist-dynamic` subfolder. The exported package name consists of the original plugin name with `-dynamic` appended.
+
+[WARNING]
+====
+The derived dynamic plugin JavaScript packages must not be published to the public NPM registry. For more appropriate packaging options, see xref:assembly-package-publish-third-party-dynamic-plugin[]. If you must publish to the NPM registry, use a private registry.
+====
+--
+
+