Add asciidoc guide

[bookwar]

This change converts the guide from docbook to asciidoc format via several steps:

• With convert.py script we use pandoc to convert existing xml pages to the asciidoc format.
  Most of the pages from the guise can be converted automatically except the index page (cockpit-guide.xml)
• Unfortunately pandoc doesn't convert man pages nicely, so they require manual changes after the initial conversion.
• We then add a cockpit-guide.adoc file as a new index page for the guide, which contains proper include directives for all other pages.
• We add Makefile to build html and man pages from asciidoc sources

To see the local copy you can run

$ cd ./doc/guide/asciidoc
$ make
$ firefox output/html/cockpit-guide.html

Questions to consider:

• Is it ok to use single page HTML document for the guide or do we want to preserver a multipage structure of the current guide?
• Is it ok to change formatting of the man pages (section titles, layout) to a more typical asciidoc format, see example in: https://docs.asciidoctor.org/asciidoctor/latest/manpage-backend/ with Name and Synopsis sections.
• Do we have specific preferences for CSS/styling?

[jelly]

I've tried this out, apparently I use asciidoctor locally. Giving probably known man page errors:

asciidoctor: ERROR: cockpit-bridge.adoc: line 1: non-conforming manpage title
asciidoctor: ERROR: cockpit-bridge.adoc: line 1: name section expected
asciidoctor: ERROR: cockpit-tls.adoc: line 1: non-conforming manpage title
asciidoctor: ERROR: cockpit-tls.adoc: line 1: name section expected
asciidoctor: ERROR: cockpit-ws.adoc: line 1: non-conforming manpage title
asciidoctor: ERROR: cockpit-ws.adoc: line 1: name section expected
asciidoctor: ERROR: pam_ssh_add.adoc: line 1: non-conforming manpage title
asciidoctor: ERROR: pam_ssh_add.adoc: line 1: name section expected

Few things I noticed:

• Only cockpit.conf(5) is a link, the other man pages aren't. (But this seems to be already the case with the old docs)
• The Link to API Listing is now less useful as there is no inline TOC. Now links to file:///home/jelle/projects/cockpit/asciidoc/doc/guide/asciidoc/output/html/cockpit-guide.html#development I can't our Cockpit API pages at all like https://cockpit-project.org/guide/latest/cockpit-file.html Found the pages in source code but they are simply not referenced anywhere, so cockpit-channel.adoc, cockpit-file.adoc etc. needs to be referenced in api-base1.adoc
• The man pages are indeed broken for cockpit-ws/cockpit-tls SYNOPSIS and NAME seem to be wrongly converted
• There are Description sections in the TOC (all the way at the bottom of the sidebar) which seem out of place:
  

The most notable change is that instead of an inline TOC, we now have a sidebar.

Regarding your question about CSS, @garrett theme'd the documentation on our website to be more in line with the general website look and feel. The CSS overrides for that are here https://github.com/cockpit-project/cockpit-project.github.io/blob/main/assets/site/guide.scss

(We sync the documentation from Cockpit to our website every release)

[bookwar]

* Only `cockpit.conf(5)` is a link, the other man pages aren't. (But this seems to be already the case with the old docs)
* The man pages are indeed broken for cockpit-ws/cockpit-tls SYNOPSIS and NAME seem to be wrongly converted

Since man pages need to be adjusted manually, I wanted to clarify the chosen direction first before editing each of them. So the cockpit.conf(5) is the example, and I will adjust the rest of the links in the same way. Same for Name/Synopsis, i only edited the cockpit.adoc and cockpit-desktop.adoc so far.

* The Link to `API Listing` is now less useful as there is no inline TOC. Now links to file:///home/jelle/projects/cockpit/asciidoc/doc/guide/asciidoc/output/html/cockpit-guide.html#development I can't our Cockpit API pages at all like https://cockpit-project.org/guide/latest/cockpit-file.html Found the pages in source code but they are simply not referenced anywhere, so cockpit-channel.adoc, cockpit-file.adoc etc. needs to be referenced in api-base1.adoc
* There are Description sections in the TOC (all the way at the bottom of the sidebar) which seem out of place:

This is one issue, i think. The api doc files are included, but they do not have a top-level header inside their own file, so the first header they show is "Description".

---

As we agreed at the call to proceed with asciidoc forward, I am going to add the remaining formatting fixes. And then we should decide on single vs multipage html approach, sidebar layout and css.

[martinpitt]

Thanks @bookwar ! Wrt. your questions:

is it ok to use single page HTML document for the guide or do we want to preserver a multipage structure of the current guide?

We embed these links a lot: git grep http.*cockpit.*guide. Much of it is easy to change, like the internal doc/i18n.md or the ws container README. But we need at least a redirect or other compatibility for these two:

pkg/metrics/metrics.jsx: <Button component="a" variant="link" href="https://cockpit-project.org/guide/latest/feature-pcp.html"
pkg/shell/hosts_dialog.jsx:

Not sure how much control we have over the file names, but if multi-page can spit out these file names, that'd be a good reason to stay with multi-page.

Is it ok to change formatting of the man pages (section titles, layout) to a more typical asciidoc format, see example in: https://docs.asciidoctor.org/asciidoctor/latest/manpage-backend/ with Name and Synopsis sections.

Yes, please.

Do we have specific preferences for CSS/styling?

This is more of a @garrett question, but not from my side -- to be honest, I'd really like to avoid CSS customizations unless there is a really compelling reason. These are always a long-time commitment to maintaining and checking them, and we have no automated way to alert us of regressions.

[martinpitt]

Thanks! I didn't dive into any details yet, as this is a PoC.

[martinpitt]

OOI, why is this necessary? The .xml files will go away completely, no?

[martinpitt]

This looks really strange -- This should get some more structure, like "NAME", "SYNOPSIS", and proper verbatim formatting and such. But I suppose that's what you meant with "man pages need work".

[garrett]

Regarding the CSS overrides:

They were made so we could integrate the docs into the site, and therefore have the docs searchable on the site. This is useful for people who are looking for information in our documentation when they are on our website. People have been able to find the doc by browsing to the documentation and also searching for the documentation, and it meant that we could have a cohesive docs section with the documentation from anywhere (exclusively on the website for some, others from the docbook guide, and still others from various files in git repos, and even from the GitHub wiki too).

Of all the documentation we have, it is all handled by the website natively via Markdown, except for the docbook-generated HTML guide. The guide is the only thing that has needed special handling (build docbook to HTML first) and it is the only one with custom CSS, as it had really awful inline CSS in the (horrific) docbook-generated HTML that needed to be overridden.

As for the "maintenance burden" for the docs CSS... in the past half decade, it was effectively 0. The CSS was never updated by itself once I did the overrides initially, aside from very minor fixes around 6 years ago. The docs transformation from docbook was consistent; the tooling and conversion did not change. The only changes in the past several years were global changes across the site; we had nothing doc-specific for many years. And doc-specific stuff was a couple of commits containing a few lines in 2020 and 2019 prior, and they were small enhancements (like improving tables on mobile).

You can see that in the history:
https://github.com/cockpit-project/cockpit-project.github.io/commits/main/assets/site/guide.scss

And the SASS prior:
https://github.com/cockpit-project/cockpit-project.github.io/commits/main/assets/site/guide.sass

In other words, the benefits (having docs integrated into our site, being searchable) outweigh the cost (basically no real specific maintenance needed since 2018, aside from 2 extremely minor mobile tweaks in 2019, which was 6 years ago).

If we converted the AsciiDoc to Docbook for doc generation for integration on the website, nothing should really change. If we use AsciiDoc directly for the website, then we probably don't have to have any special CSS (or very minimal CSS), as AsciiDoc in Jekyll would work like Markdown in Jekyll. HOWEVER (and this is a big however), GitHub Pages does not support AsciiDoc. See https://pages.github.com/versions/ and note that AsciiDoc support is not there. We'd need to change the way we build the website to be able to use the plugin to have native AsciiDoc support.

So, that all said, we could implement this in three passes, with minimal disruption during each:

1. Have AsciiDoc translated to Docbook for now, and build the guides as-is for the website to keep things going. This could even be done in the doc integration script behind the scenes.
2. Switch the website to build with the non-GitHub-Pages method, which means we'd use GitHub Actions instead. This would let us use any plugin and even the latest version of Jekyll. However, it means our testing forks probably won't be possible or at least as easy to do. Jekyll has a page on this @ https://jekyllrb.com/docs/continuous-integration/github-actions/
3. Once we have the GitHub Actions version of our site up and working, then we could include the AsciiDoc plugin to Jekyll and the documentation would be natively compiled in. We would need a step to sync it over before build time, or perhaps even use a submodule. But we probably won't need custom CSS (or would only need a very tiny bit, instead of having to override compiled in inline CSS, like we have to do with docbook-generated HTML). It's this plugin: https://github.com/asciidoctor/jekyll-asciidoc
4. Update the doc sync script to pull in the AsciiDoc files instead of the messing with docbook-generated HTML.

We could even be able to skip step 1, merge 2 and 3 (as step 3 is basically adding a module line to the Gemfile), and do this in 2 PRs, if we defer on merging this in until we're ready with a GitHub Actions migration and make sure that works. That'd let us have the latest version of Jekyll with "native" AsciiDoc support using the AsciiDoc plugin.

---

Summary:

1. Additional CSS wasn't ever a burden.
2. We have a pretty clear plan: Switch from GitHub Pages to GitHub Actions, include "native" AsciiDoc support.
3. Integrated AsciiDoc support likely wouldn't require custom CSS anyway. (If it does, it should be extremely minimal. And wouldn't need to be updated. Not that the existing one was ever updated, really.)

[garrett]

Note, there's an example AsciiDoc-powered website via GitHub Actions starter @ https://github.com/asciidoctor/jekyll-asciidoc-quickstart with probably the most relevant part to us @ https://github.com/asciidoctor/jekyll-asciidoc-quickstart/blob/main/.github/workflows/publish.yml

I don't know how this would impact our fork preview method, if that is even still possible via GitHub Actions. Hopefully. And I hope it's easy and transparent for us to do that still too. It's very useful for us to each have our own staging site like we have had.