README.adoc.jam

:imagesdir: images
:jamal_home: https://github.com/verhas/jamal/
//{%@rot13 QB ABG RQVG GUVF SVYR; GUVF VF TRARENGRQ SEBZ ERNQZR.nqbp.wnz%}
{%@import res:jamal.jim%}
{%@import RELEASES.jim%}
{%@import version.jim%}
{%@define output=[source]
----
{%#trimLines trimVertical
{%@try!
{%!#output {%lastCode%}%}%}%}
----
%}
{%@snip:xml pom=pom.xml%}
{%#define VERSION={%pom /project/version/text()%}%}
{%@define pu$folder=images%}
{%@define pu$template=%}
{%@snip:collect from=./jamal-maven-plugin/src/main/java/com/javax0/jamal/maven/JamalMojo.java%}
{%@snip:collect from=./jamal-api/src/main/java%}
{%@snip:collect from=./jamal-engine/src/main/java%}
{%@import environmentvariables.jim%}
{%@references file=README.jrf comment%}

= image:logo.svg[width=100px] Jamal Macro Language

Jamal is a tool for automating documentation maintenance.
It will copy the information from your documented system into the documentation automatically and repeat it when the system is updated.
It reduces the risk of outdated documentation by automating its maintenance.

Jamal can be used with AsciiDoc, Markdown, XML, JSON, YAML, Javadoc, and any other markup languages.
The application integrates seamlessly into the document processing tools' pipeline in the least intrusive, non-invasive way.

Technically, Jamal is a meta-markup language enhancing the features of the original markup language.
The conversion is done as a preprocessing step prior to the markup processing.
It can be inserted into any toolchain without risk, as it can also be removed at any time.

TOC

{%@ref TOC%}
{%TOC%}

[cols="a,a,a", frame=none, grid=none]
|===
|image::jetbrains.svg[Supported,link="https://www.jetbrains.com",width=50px, alt="JetBrains"] Developed using https://www.jetbrains.com/community/opensource/#support[JetBrains OSS License]
|image::https://javadoc.io/badge/com.javax0.jamal/jamal-parent.svg[link=https://javadoc.io/doc/com.javax0.jamal]
|image:{jamal_home}actions/workflows/jamal-build.yml/badge.svg[Java CI with Maven,link={jamal_home}actions/workflows/jamal-build.yml]
|===

{%#do
if VERSION.string:contains("text=SNAPSHOT") then
<< "NOTE: "
<< """This is the latest development documentation.
To see the documentation of a release, you have to follow the link in the documentation link:{%@file RELEASES.adoc%}[`RELEASES.adoc`].
If you want to read the latest release, please visit link:{jamal_home}/tree/v"""+LAST_RELEASE+"[Jamal"+LAST_RELEASE+"]"
end if
%}

{%@define :TOC=%}
{%@define section($id,$label)=
== $label [[$id]]
{%#define :TOC={%:TOC%}
. <<$id>>%}
%}

{%section /Installation/Installation and Usage%}

Jamal is implemented as a Java library and is embedded in different applications.
It can be used as a command-line tool, as a Maven plugin, as a Maven extension, as a JavaDoc doclet, as a JShell script, as a JBang script, as a Java library, and as a JSR223 script engine.
The installation may be as simple as starting it from the command line (JBang) when the embedding application is already installed.

In the following sections, we will discuss how to install and use Jamal in the different applications it is embedded in.

. <<Starting Command Line Version, Starting Command Line Version>>
. <<MavenPlugin, Maven Plugin>>
. <<MavenExtension, Maven Extension>>
. <<Starting with JBang, Starting with JBang>>
. <<ASCIIDOC, IntelliJ, AsciidocFx>>

==== Starting Command Line Version

To install Jamal and use it on Linux, macOS, or Windows, you can use the installer kit.
The installer kit is attached to the release starting with version 2.8.1.
The installer kit contains the Jamal code as well as a minimal Java environment to run it.
All you need to do is download the installer kit and execute it.
The installation is straightforward and does not require any special knowledge.

Currently, the installation kits are available for

* Linux Intel 64-bit architecture,
* macOS Intel 64-bit and `Mx` architecture, and
* Windows Intel 64-bit architecture.

Installation kits come along with a tailored version of the Java 21 JDK temurin distribution.
The Java source is compatible with Java 17, and the compiled JAR files are compatible with Java 11.

.Supported platform for installation kits
[NOTE]
====
The above kits are created on GitHub using the GitHub actions.
The project for creating the kits is separate from the Jamal development repository `https://github.com/verhas/jamal_packaging`.
Upon request, I can provide Linux arm 64-bit installation kit, but that is not automatically built.
I also plan to support that platform as soon as GitHub will support arm64 linux and windows workflows.
====

.Windows PATH setting
[NOTE]
====
The Windows installation kit does not modify the path of the operating system.
It means that you need to

* start Jamal from the command line using the full path to the `jamal` executable, or

* modify the user or system level PATH to include the directory where the `jamal` executable is located.

The typical installation location for Jamal on Windows is `C:\Program Files\jamal`.
====
If you have Java installed on your system and do not want to install another Java environment, you can download the ZIP file from the Maven repository.

The repository contains the `jamal.sh` shell script that can be used to start Jamal from the command line.
It assumes that the libraries are already installed in the local Maven repository.
After that, you can type:

  ./jamal.sh

that will show you a short usage message:

[source,text]
----
{%#snip:define usage_snippet={%@include [verbatim] usage.txt%}%}{%@snip usage_snippet%}
----

For more information about the command line version, read the link:jamal-cmd/README.adoc[documentation].


[[MavenPlugin]]
==== Maven Plugin

You can start Jamal from Maven as a plugin.
The plugin is available in the Maven Central repository.

To do that, you must have Maven installed.
Having that, you can issue the command:

[source]
----
mvn com.javax0.jamal:jamal-maven-plugin:{%VERSION%}:jamal
----

if you have a `pom.xml` file in your directory.

If you do not have one, then read the documentation of the Jamal Maven plugin at
link:jamal-maven-plugin/README.adoc[Jamal Maven Plugin README].
It is short and straightforward.

[[MavenExtension]]
==== Maven Extension

You can use Jamal macros to maintain your Maven POM files.
Move the content of the POM XML into the file `pom.jam` and extend it freely with Jamal macros.
Create a `.mvn` directory with an `extensions.xml` file in your project root.

[source,xml]
----
<?xml version="1.0" encoding="UTF-8"?>
<extensions>
    <extension>
        <groupId>com.javax0.jamal</groupId>
        <artifactId>jamal-maven-extension</artifactId>
        <version>{%VERSION%}</version>
    </extension>
</extensions>
----

Next time you start Maven, it will include Jamal in the processing chain.
It will also generate the `pom.xml` files from the `pom.jam` files, so your IDE and other tools that depend on the XML format keep working.

For more information about the Maven extension, read the link:jamal-maven-extension/README.adoc[documentation].

==== Starting with JBang

JBang (https://www.jbang.dev) is a popular command-line tool that eases the startup of Java applications.
Jamal can be started using JBang.
This is the recommended way to run Java from the command line if you have limited experience with Java.
When running Jamal using JBang, JBang will install everything that is needed to execute Jamal in a clean and non-intrusive way.

JBang installation is described on the documentation link:https://www.jbang.dev/documentation/guide/latest/installation.html[page] of JBang.

To start Jamal when you have J

Bang installed on your machine, type:

[source,bash]
----
jbang jamal@verhas ... options ...
----

This command will invoke the command line version automatically.
The syntax and meaning of the options are the same as in the case of the command line version.
This startup also loads all the safe Jamal extensions, including `snippet`, and `debug` and some others.

If you want to see the exact list of the modules this startup loads, have a look at the link:jbangstarter.java[starter] file.

[NOTE]
====
If you have used Jamal with JBang before, then JBang will store its catalog file in the local cache.
When you start Jamal using `jbang jamal@verhas ...` and you see an old version starting, then delete the file

```
~/.jbang/cache/urls/d917b991facb86b9860fa179df2c804fc2090cc76a83fb15b49f47cc2e885f7c/jbangstarter.java
```

After that, you can start JBang again.
It will download the new catalog, always pointing to the latest release.
You will find the command that deletes this file in the root of the project in the shell script `jbang-cache-evict`.
====

[[ASCIIDOC]]
=== IntelliJ, AsciidocFx

Using Jamal in IntelliJ together with the Asciidoctor plugin is fairly easy.
All you have to do is download a ZIP file from the Maven repository and explode it into a directory.
The details are described in the link:jamal-asciidoc/README.adoc[documentation].

When the installation is done, all you need to do is start IntelliJ and open the project.
You can edit your AsciiDoc files, and the plugin will automatically invoke Jamal to process the macros.

If you want to use AsciidocFX, the same package should be used.
The installation is similar, downloading the ZIP file and extracting it to a directory.
The detailed documentation is in the link:jamal-asciidoc/README.adoc[documentation].

{%section |GS|Getting Started / Quick Start%}

You can start using Jamal in five minutes as described in link:documentation/TUTORIAL-5min.adoc[the tutorial], __"{%#string:substring (begin=1 end=-1) {%@include [lines=1]documentation/TUTORIAL-5min.adoc.jam%}%}"__.

{%section /Configuration/Configuration%}

Jamal has many configuration parameters, but each of these has reasonable default values.
It means that you do not need to configure Jamal before using it.
Configuration is needed only when you want to change some of the default values or use a macro package that, without configuration, could pose a security risk.

Configuration values can be set in the following ways:

. Using system properties
. Using environment variables
. Using a configuration file in the user's home directory (`~/.jamal`)

The configuration values are searched for in this order.
Different macros use different configuration keys.
They are documented along with the macro documentation.

The environment variables and their meanings are documented in their
link:./documentation/ENVIRONMENT_VARIABLES.adoc[documentation].

{%section /Features/Features%}

Jamal is a meta markup language that extends existing markup languages in a transparent way.
The language is designed so that it will not interfere with any existing or future markup.

image::text2text.png[align=center]

The original markup, for example, AsciiDoc or Markdown, is responsible for formatting and semantic definition of the text.
Jamal will do the extra task, which is not or in some cases only partially supported by the document markup.
Without Jamal or some other similar tools, these tasks are performed manually.

Jamal can

* collect information from source code and other non-document files,

* transform the collected information to fit

** the document markup,
** the document format, and
** the document semantics.

Jamal can include other files, parts of files, number the included lines, filter lines, replace parts of the lines, reorder lines, and many other things as needed.

.Philosophy
> When information exists in the documented system or in the documentation, it must not be manually copied.
The copy and the transformation of the information must be automated.

Jamal is implemented in Java.
You can write user-defined macros in Jamal itself and built-in macros in Java, Kotlin, or other JVM languages.

You can execute Jamal from Maven, Javadoc, CLI, AsciiDocFX, IntelliJ, and other applications.
Jamal is extensible with multiple different SPIs.
One such SPI is the debugging interface.
The library includes a debugger that you can use via a React.js web client to debug the macro evaluations step-by-step.

The library comes with more than 200 macros for different purposes.
The macros are grouped into modules.
The largest module is the document maintenance module (snippet macros), but there are modules to handle

* JSON,

* YAML,

* XML, and other data formats.

The use of Jamal makes it possible to include automatically generated images, for example, from PlantUML, Graphviz, or other tools into any markup-formatted document.
You can also include programmatic formatting and content calculation using Groovy, Ruby, and other languages.

{%section /Contributing/Contributing%}

Jamal is an open-source project, and the developers welcome any contribution.
We treat all suggestions, requests, comments, or any other contribution with respect.

=== Use and Feedback

First and foremost, you can contribute by using Jamal and giving feedback.
Start using it and tell us what you like and what you do not like.
A program without users is not a program.
If you use Jamal, you are a contributor, and if you wish, we will include you as a reference in the documentation.

=== Documentation

You can contribute to Jamal by reading the documentation.
If you find a typo, a mistake, or something that is not clear, please tell us.
The best way is to fork the project, fix the documentation, and send us a pull request.
Even a single character correction is welcome as a full-blown pull request.

You can also write documentation.
Writing documentation is a huge task, and we are happy to accept any help.
We are gravely missing, for example, "How to" tutorials.
Why?
Because as developers, we develop Jamal first and our use is limited to the use cases we have.
We are not using Jamal in the same way as you do.

Every use is different, write about it.

Jamal supports the JSR223 standard.
That way, Jamal can be used in any JSR223 compliant application that may need scripting.
The macro opening and closing strings are `{` and `}` in this case unless the script attributes `open` and `close` are set.
Script bindings are put into Jamal macros and are loaded from Jamal macros after execution.
We would love a tutorial describing this feature.

=== Articles

If you find Jamal fascinating, you can write an article about it.
We have experience writing and publishing articles, and we can help you.
We also write articles, but we cannot write your article.

Help us spread the word.

=== Conference

Talk about Jamal at conferences.
Give a talk or just mention it in your presentation.
Or just mention it at the coffee break or other social events.

=== Find bugs

If something does not work as you expect, please

 tell us.
It may be a code bug, or it may be a documentation bug.

It is NEVER a user error.
If it works as _we_ expect and not as _you_ expect, then it is a bug in the documentation.

Use the GitHub issue tracker to report bugs.
If you can locate the bug in the code and have a suggestion to fix it, then you can also send us a pull request.

=== Suggest Features

If you miss a feature, please tell us.
We will consider it, and if it is a good idea, we will implement it.
If you can implement it, then send us a pull request.

=== Create Macro Libraries

Jamal can be extended with new macros.
Create your own macros for your own use.
If you think that your macros are useful for others, then create a new module and publish it.
We will be happy to reference it in the Jamal project.

{%section /Documentation/Documentation%}

Since Jamal is a complex program, the documentation is split into several parts.
The modules, each has its documentation in the form of a README file in the module directory.
Jamal is eating its own dog food, so the documentation is written in Jamal and AsciiDoc.

Here we will link the different documentation parts.

=== Core Macros

The core built-in macros are part of the core package.
They contain those essential macros that are vital for the working and use of Jamal.
These macros are documented in their separate documentation each.

They are
{%#for $doc in ({%@listDir (format="$simpleName" pattern=".*.adoc$" maxDepth=1) documentation/macros%})=
* link:documentation/macros/$doc[`$doc`]
%}

There are two special user-defined macros, `output:writable` and `output:charset`.
These can control the output file creation.
Note that this is not a core feature of Jamal, but most current embeddings (AsciiDoc, Maven plugin, and command line) support these macros.

By default, the generated file is not writable.
This is to prevent accidental editing of the generated files.
Many times the generated files are stored along with the Jamal files, and it is an easy mistake to edit the generated file.
To prevent this, the generated file is read-only by default.

There are cases when the output has to be writable.
An example is the live template XML file that cannot be handled properly by IntelliJ if it is read-only.
If the value of the macro `output:writable` is `true`, then the generated file will be writable.

The macro `output:charset` can be used to set the character set of the generated file.
The default value is UTF-8.

Note that even if you set the character set to `UTF-16LE`, the generated file will not contain the BOM.

=== Applications, Embedding

* link:./jamal-asciidoc/README.adoc[Jamal AsciiDoc Documentation], How to configure and use Jamal to edit AsciiDoc files using the IntelliJ editor in a WYSIWYG way, or the AsciiDocFX editor.
* link:./jamal-doclet/README.adoc[Jamal Doclet Documentation], How to use Jamal in Javadoc.
* link:./jamal-maven-plugin/README.adoc[Jamal Maven Plugin README], How to use Jamal as a Maven plugin.
* link:./jamal-maven-extension/README.adoc[Jamal Maven Extension README], How to use Jamal as a Maven extension.
* link:./documentation/JAMAL_API.adoc[Jamal API Documentation], How to use Jamal as a Java library.

=== Debugger

The debugger is a web-based, interactive tool using React.js.
There is no separate documentation describing where to click and how to use it.
The existing documentation describes the debugging architecture and how to start Jamal in debug mode.

Anyway, here is a screenshot of the debugger in action:

image::jamaldebugger2559x1089.png[]
* link:./jamal-debug/README.adoc#[Debugger Technical Documentation]

=== Programming Language Modules

{%@define LANGUAGE(Ruby)=* link:./jamal-{%@case:lower Ruby%}/README.adoc#[Ruby Module README], How to use Ruby code in your Jamal source%}
{%LANGUAGE Ruby%}
{%LANGUAGE Groovy%}
{%LANGUAGE ScriptBasic%}
{%LANGUAGE Prog%} is a simple BASIC-like language tightly integrated with Jamal.

=== Other External Modules

* link:./jamal-io/README.adoc[Io Module README], How to read and write external files from Jamal macros
* link:./jamal-jamal/README.adoc[Jamal Jamal Module README], How to use Jamal inside Jamal as an embedded language
* link:./jamal-markdown/README.adoc[Jamal Markdown Module README], Convert markdown to HTML, mainly usable together with the Jamal Doclet to have Markdown in Javadoc
* link:./jamal-mock/README.adoc[Jamal Mock Module README], Mock built-in macros to test macros that are to run in a specific environment
* link:./jamal-snippet/README.adoc[Jamal Snippet Module README], Use snippets to compile your documentation
* link:./jamal-yaml/README.adoc[Jamal Yaml Module README], Use data from Yaml files in your macros and use macros in your Yaml files
* link:./jamal-json/README.adoc[Jamal JSON Module README], Use data from JSON files in your macros and use macros in your JSON files
* link:./jamal-assertions/README.adoc[Jamal Assertions Module README], contains macros to make assertions to ensure the consistency of your documentation
* link:./jamal-word/README.adoc[DOCX Word Processing README], describes the Jamal Microsoft Word Processing module and the macros that are specific to DOCX processing

=== Test Support

* link:./jamal-testsupport/README.adoc[Jamal Test Module README], Use this module to test your own Java or Kotlin implemented macros.

{%section /License/License%}

Jamal is distributed under the Apache 2.0 license.

{%section /Changelog/Changelog%}

Jamal uses GitHub.
The changelog is maintained online on the link:https://github.com/verhas/jamal/releases[GitHub releases page].
There is also a link:RELEASES.adoc[local copy] of the release notes.

{%section /Roadmap/Roadmap%}

The roadmap is maintained in the document: link:ROADMAP.adoc[ROADMAP].
It is more like a collection of ideas and plans than a strict roadmap.

{%section /Support/Support%}

Jamal is an open-source project and is currently not backed by any company.

{%section /FAQ/FAQ%}

See the separate document: link:FAQ.adoc[FAQ].

{%section /Maintenance/Maintenance of this document%}

The documents of this project are formatted as AsciiDoc documents with Jamal meta markup.