add cli docs draft

docs/source/cli.md

@@ -0,0 +1,71 @@
+# Datapipe CLI
+
+Datapipe provides `datapipe` CLI tool which can be useful for inspecting
+pipeline, tables, and running steps.
+
+`datapipe` CLI is build using `click` and provides several levels of commands
+and subcommands each of which can have parameters. `click` parameters are
+level-specific, i.e. global-level arguments should be specified at global level
+only: 
+
+`datapipe --debug run`, but NOT `datapipe run --debug`
+
+## Global arguments
+
+### `--pipeline`
+
+By default `datapipe` looks for a file `app.py` in working directory and looks
+for `app` object of type `DatapipeApp` inside this file. `--pipeline` argument
+allows user to provide location for `DatapipeApp` object.
+
+Format: `<module.import.path>:<symbol>`
+
+Format is similar to other systems, like uvicorn.
+
+Example: `datapipe --pipeline my_project.pipeline:app` will try to import module
+`my_project.pipeline` and will look for object `app`, it will expect this object
+to be of type `DatapipeApp`.
+
+### `--executor`
+
+Possible values:
+
+* `SingleThreadExecutor`
+* `RayExecutor`
+
+TODO add separate section which describes Executor
+
+### `--debug`, `--debug-sql`
+
+`--debug` turns on debug logging in most places and shows internals of
+datapipe processing.
+
+`--debug-sql` additionally turns on logging for all SQL queries which might be
+quite verbose, but provides insight on how datapipe interacts with database.
+
+### `--trace-*`
+
+* `--trace-stdout`
+* `--trace-jaeger`
+* `--trace-jaeger-host HOST`
+* `--trace-jaeger-port PORT`
+* `--trace-gcp`
+
+This set of flags turns on different exporters for OpenTelemetry
+
+## `db`
+
+### `create-all`
+
+`datapipe db create-all` is a handy shortcut for local development. It makes
+datapipe to create all known SQL tables in a configured database.
+
+## `lint`
+
+Runs checks on current state of database. Can detect and fix commong issues.
+
+## `run`
+
+## `step`
+
+## `table`

docs/source/extending-cli.md

@@ -27,5 +27,5 @@ Plugin can expect some information in `click.Context`:
 
 ## Example
 
-To see example of extending `datapipe` cli see
-[`datapipe_app.cli`](https://github.com/epoch8/datapipe-app/blob/master/datapipe_app/cli.py)
+To see example of extending `datapipe` cli see `datapipe_app.cli`:
+[https://github.com/epoch8/datapipe-app/blob/master/datapipe_app/cli.py](https://github.com/epoch8/datapipe-app/blob/master/datapipe_app/cli.py)

docs/source/index.rst

@@ -3,14 +3,18 @@
 .. toctree::
     :caption: Introduction
     :maxdepth: 2
-    :hidden:
 
     introduction
 
+.. toctree::
+    :caption: Command Line Interface
+    :maxdepth: 2
+
+    cli
+
 .. toctree::
     :caption: Advanced Topics
     :maxdepth: 2
-    :hidden:
 
     transformation-lifecycle
     sqlite
@@ -19,7 +23,6 @@
 .. toctree::
     :caption: Migration
     :maxdepth: 2
-    :hidden:
 
     migration-v013-to-v014