Enforce strict docs compilation and add Documentation best practices guidelines

docs/Project.toml

@@ -2,8 +2,10 @@
 CSV = "336ed68f-0bac-5ca0-87d4-7b16caf5d00b"
 DataStructures = "864edb3b-99cc-5e75-8d2d-829cb0a9cfe8"
 Documenter = "e30172f5-a6a5-5a46-863b-614d45cd2de4"
+DocumenterInterLinks = "d12716ef-a0f6-4df4-a9f1-a5a34e75c656"
 InfrastructureSystems = "2cd47ed4-ca9b-11e9-27f2-ab636a7671f1"
 Literate = "98b081ad-f1c9-55d3-8b20-4c87d4299306"
 
 [compat]
 InfrastructureSystems = "2"
+Documenter = "1"

docs/make.jl

@@ -1,6 +1,13 @@
 using Documenter, InfrastructureSystems
 import DataStructures: OrderedDict
 using Literate
+using DocumenterInterLinks
+
+links = InterLinks(
+    "Documenter" => "https://documenter.juliadocs.org/stable/",
+    "Julia" => "https://docs.julialang.org/en/v1/",
+    "JuliaFormatter" => "https://domluna.github.io/JuliaFormatter.jl/stable/",
+)
 
 if haskey(ENV, "GITHUB_ACTIONS")
     ENV["JULIA_DEBUG"] = "Documenter"
@@ -17,6 +24,19 @@ pages = OrderedDict(
             "dev_guide/logging.md",
         ],
         "Style Guide" => "style.md",
+        "Documentation Best Practices" => Any[
+            "Explanation" => "docs_best_practices/explanation.md",
+            "How to..." => Any[
+                "Complete Basic Requirements Checklist" => "docs_best_practices/how-to/requirements_checklist.md",
+                "Clean Up General Formatting" => "docs_best_practices/how-to/general_formatting.md",
+                "Write a Tutorial" => "docs_best_practices/how-to/write_a_tutorial.md",
+                "Write a How-To Guide" => "docs_best_practices/how-to/write_a_how-to.md",
+                "Organize APIs and Write Docstrings" => "docs_best_practices/how-to/write_docstrings_org_api.md",
+                "Troubleshoot Common Errors" => "docs_best_practices/how-to/troubleshoot.md",
+                "Compile and View Documentation Locally" => "docs_best_practices/how-to/compile.md",
+                "View Draft Documentation on Github" => "docs_best_practices/how-to/view_github.md",],
+            "Reference" => Any["docs_best_practices/reference/useful_links.md",],
+        ],
         "API" => "InfrastructureSystems.md"
 )
 
@@ -51,7 +71,7 @@ makedocs(
         size_threshold = nothing),
     sitename = "InfrastructureSystems.jl",
     pages = Any[p for p in pages],
-    warnonly = Documenter.except(),
+    plugins = [links],
 )
 
 

docs/src/InfrastructureSystems.md

@@ -1,5 +1,7 @@
 # [InfrastructureSystems API Reference](@id API_ref)
 
 ```@autodocs
-Modules = [InfrastructureSystems]
+Modules = [InfrastructureSystems,
+    InfrastructureSystems.Simulation,
+    InfrastructureSystems.Optimization]
 ```

docs/src/dev_guide/logging.md

@@ -1,4 +1,4 @@
-# Logging
+# [Logging](@id log)
 
 `InfrastructureSystems.jl` provides a `MultiLogger` object that allows customized
 logging to console and file. Refer to the [logging

docs/src/docs_best_practices/explanation.md

@@ -0,0 +1,44 @@
+# Explanation
+
+## History and Motivation
+
+During the first phase of Sienna's (previously SIIP's) development, Sienna used a 3-part
+documentation organization, based on the expected needs of different user personas:
+
+  - **Modeler**: Users that want to use existing functionality
+  - **Model Developer**: Users that want to develop custom structs, components, models, and/or workflows
+  - **Code Base Developers**: Users that want to add new core functionalities or fix bugs in the core capabilities
+
+However, as Sienna's user base has expanded, it has become apparent that this previous
+organization is no longer serving. As of 2024, a new effort is underway to clean up and
+re-organize the Sienna documentation according to the 4-part [Diataxis](https://diataxis.fr/)
+framework, a well-established, systematic approach to technical documentation split up into:
+
+  - [Tutorials](https://diataxis.fr/tutorials/)
+  - [How-to guides](https://diataxis.fr/how-to-guides/)
+  - [Reference](https://diataxis.fr/reference/)
+  - [Explanation](https://diataxis.fr/explanation/)
+
+In addition, the current documentation has multiple quality issues, including misformatted
+text, broken reference links, and documentation that has been written but is not visible to
+users in the API ("missing docstrings"). While the [style guide](@ref style_guide)
+has been available, the guide focuses primarily on the style of code itself, without
+providing clear guidelines and best practices for other parts of the documentation besides
+docstrings. In addition, the first stage of Sienna's development coincided with the initial
+development of the [`Documenter.jl`](https://documenter.juliadocs.org/stable/) package.
+Early versions of Sienna's packages were documented requiring `Documenter.jl` v0.27, and in
+the meantime, `Documenter.jl` has released its v1.0 and onwards, which contain much
+more rigorous checks for documentation quality. Sienna's packages have not kept up with
+these improvements.
+
+## Purpose of the Documentation Best Practices
+
+We aim to remedy the historical issues above through a concerted clean up and
+re-organization effort and compliance with `Documenter.jl` >v1.0's quality control checks,
+following these best practice guidelines. These guidelines are not intended to reiterate
+[Diataxis](https://diataxis.fr/), beyond regularly reminding contributers to refer to them
+-- and contributers should read the [Diataxis](https://diataxis.fr/) website in its entirety
+before getting started. Instead, the best practices are intended to bridge the gap where
+there are Julia- or Sienna-specific recommendations, either to consistently implement the
+Diataxis framework or to highlight common documentation issues throughout the Sienna
+packages that need to be addressed as part of our concerted clean-up effort.

docs/src/docs_best_practices/how-to/compile.md

@@ -0,0 +1,70 @@
+# Compile and View Documentation Locally
+
+## Pre-Step a: Update Docs Environment (First Time)
+
+The first time you compile documentation for a package, make sure the `docs/` environment
+(i.e., `docs/Manifest.toml`, not the main `Manifest.toml` in the root of the repository)
+is pointing to your local version of the package, so it compiles your local changes:
+
+```
+julia --project=docs
+using Pkg
+Pkg.develop(path = "..")
+```
+
+## Pre-Step b: Auto-Generate Structs (If Needed)
+
+Most documentation changes are made directly to markdown (.md) files, but if you changed one
+of Sienna's .json descriptor files, you must first
+[follow the instructions here](@ref "Auto-Generation of Component Structs") to auto-generate
+the structs from the .json to have your changes propagated into the markdown files used for
+documentation.
+
+**Example**: You updated `PowerSystems.jl`'
+[`power_system_structs.json`](https://github.com/NREL-Sienna/PowerSystems.jl/blob/main/src/descriptors/power_system_structs.json)
+file.
+
+From a terminal at the root of the repository (i.e., `PowerSystems.jl`), run:
+
+```
+julia --project=.
+using InfrastructureSystems
+InfrastructureSystems.generate_structs(
+    "./src/descriptors/power_system_structs.json",
+    "./src/models/generated",
+)
+```
+
+## Pre-Step c: Run the Formatter (Before Submitting a Pull Request)
+
+To automatically format the documentation to conform with the [style guide](@ref style_guide),
+run in a terminal at the root of the repository:
+
+```
+julia scripts/formatter/formatter_code.jl
+```
+
+Resolve any errors and re-run until error-free. See how to [Troubleshoot Common Errors](@ref)
+for help.
+
+This is not a necessary step to compile, but needs to be done at least once to pass pull
+request checks.
+
+## Step 1: Compile
+
+To compile, run in a terminal at the root of the repository:
+
+```
+julia --project=docs docs/make.jl 
+```
+
+Resolve any errors and re-run until error-free. See how to [Troubleshoot Common Errors](@ref)
+for help.
+
+## Step 2: View
+
+Click on the newly-created `index.html` file (e.g.,
+`SomeSiennaPackage/docs/build/index.html`) to view your locally compiled documentation in a
+browser.
+
+Visually verify formatting and that code blocks compile as expected.

docs/src/docs_best_practices/how-to/general_formatting.md

@@ -0,0 +1,84 @@
+# Clean Up General Formatting
+
+These recommendations are to make navigating our documentation effortless for users, while
+addressing common markdown formatting issues in the existing Sienna documentation:
+
+## Format in back-ticks
+
+All package names, types, functions, methods, and parameters, etc. should formatted in
+back-ticks:
+
+!!! tip "Do"
+
+    ```
+    `max_active_power`
+    ```
+
+    compiles as `max_active_power`
+
+!!! warning "Don't"
+
+    ```
+    max_active_power
+    ```
+
+    compiles as max_active_power
+
+## [Put hyperlinks everywhere](@id hyperlinks)
+
+All types, function, and methods should have hyperlinks to the correct docstring, accounting
+for multiple methods of the same name due to Julia's multiple dispatch.
+[`Documenter.jl`](https://documenter.juliadocs.org/stable/) will link to the first
+occurrance in documentation. If that's not the one you're referring to, copy the entire
+signature with types into the hyperlink reference.
+
+!!! tip "Do"
+
+    ```
+    [`get_time_series_values`](@ref)
+    ```
+
+    Or
+
+    ```
+    [`get_time_series_values` from a `ForecastCache`](@ref get_time_series_values(
+    owner::TimeSeriesOwners,
+    forecast::Forecast,
+    start_time::Dates.DateTime;
+    len::Union{Nothing, Int} = nothing,
+    ignore_scaling_factors = false,
+    ))
+    ```
+
+!!! warning "Don't"
+
+
+``get_time_series_values``
+Or
+`get_time_series_values`
+
+## Add links to other Sienna packages
+
+All other Sienna package names should have documentation (not Git repo) hyperlinks:
+
+!!! tip "Do"
+
+    ```[`PowerSystems.jl`](https://nrel-sienna.github.io/PowerSystems.jl/stable/)```
+
+!!! warning "Don't"
+
+    ```
+    `PowerSystems.jl`
+    ```
+
+    Or
+
+    ```
+    PSY
+    ```
+
+    Or
+
+    ```
+    [`PowerSystems.jl`](https://github.com/NREL-Sienna/PowerSystems.jl)
+    ```

docs/src/docs_best_practices/how-to/requirements_checklist.md

@@ -0,0 +1,38 @@
+# Complete Basic Requirements Checklist
+
+Sienna packages should follow the [Diataxis](https://diataxis.fr/)
+framework, be strictly compiled with [`Documenter.jl`](https://documenter.juliadocs.org/stable/)
+v1.0 or greater, and be automatically formatted with
+[`JuliaFormatter.jl`](https://domluna.github.io/JuliaFormatter.jl/stable/).
+
+## For New Packages
+
+The [`SiennaTemplate.jl`](https://github.com/NREL-Sienna/SiennaTemplate.jl) Git repo has the
+required environments and formatting and documentation code. Start from this template.
+
+## For Existing Packages
+
+Existing Sienna packages will need to be updated with these requirements, but these will
+only need to be addressed once:
+
+ 1. Organize the top-level documentation to follow the [Diataxis](https://diataxis.fr/)
+    framework (plus a welcome page/section). This might be a significant undertaking. See:
+
+      + How to [Write a How-to Guide](@ref)
+      + How to [Write a Tutorial](@ref)
+      + How to [Organize APIs and Write Docstrings](@ref)
+
+ 2. Replace `compat` requirements of `Documenter = "0.27"` in `docs/` environment with
+    `Documenter = "1.0"`
+ 3. Update the `docs/make.jl` file to call
+    [`Documenter.makedocs`](https://documenter.juliadocs.org/stable/lib/public/#Documenter.makedocs)
+    *without* the `warnonly` `kwarg` (i.e., all errors caught by `makedocs` must be resolved before
+    merging). [See an example here](https://github.com/NREL-Sienna/InfrastructureSystems.jl/blob/768438a40c46767560891ec493cf87ed232a2b2b/docs/make.jl#L47).
+
+      + See How-to [Troubleshoot Common Errors](@ref) if this results in a host of errors.
+ 4. Update the `scripts/formatter/formatter_code.jl` to format the markdown .md files in the
+    `docs/` folder, calling `format`() with the `kwarg` `format_markdown = true`. See
+    [these](https://github.com/NREL-Sienna/InfrastructureSystems.jl/blob/768438a40c46767560891ec493cf87ed232a2b2b/scripts/formatter/formatter_code.jl#L13)
+    [three](https://github.com/NREL-Sienna/InfrastructureSystems.jl/blob/768438a40c46767560891ec493cf87ed232a2b2b/scripts/formatter/formatter_code.jl#L8)
+    [links](https://github.com/NREL-Sienna/InfrastructureSystems.jl/blob/768438a40c46767560891ec493cf87ed232a2b2b/scripts/formatter/formatter_code.jl#L23)
+    for examples of the updated lines.