Dev

.github/workflows/build_publish_docs.yaml

@@ -0,0 +1,63 @@
+name: Build and Publish Docs to GitHub Pages
+
+on:
+  push:
+    branches:
+      - docs
+      - master
+
+jobs:
+  build-and-publish:
+    name: Build and Publish Docs
+    runs-on: ubuntu-latest
+    permissions:
+      contents: write
+      pages: write
+      id-token: write
+    steps:
+      - uses: actions/checkout@v3
+
+      - name: Set up Python 3.11
+        uses: actions/setup-python@v5
+        with:
+          python-version: 3.11
+
+      - name: Install docs dependencies.
+        run: pip install -r docs/requirements-docs.txt
+
+      - name: Auto generate API Reference.
+        run: cd docs && pydoc-markdown
+
+      - uses: actions/setup-node@v3
+        with:
+          node-version: 18
+          cache: yarn
+          working-directory: docs/voluseg-docs-app
+          cache-dependency-path: docs/voluseg-docs-app/yarn.lock
+
+      - name: Install dependencies
+        run: cd docs/voluseg-docs-app && yarn install --frozen-lockfile
+
+      - name: Build website
+        run: cd docs/voluseg-docs-app && yarn build
+
+      - name: Commit pydoc-markdown files
+        run: |
+          git config --local user.email "github-actions[bot]@users.noreply.github.com"
+          git config --local user.name "github-actions[bot]"
+          GIT_STATUS=$(git status -s)
+          [[ ! -z "$GIT_STATUS" ]] && git add docs/* && git commit -m "auto-commit-docs" -a || echo "No changes to commit"
+
+      - name: Push changes
+        uses: ad-m/github-push-action@master
+        with:
+          github_token: ${{ secrets.GITHUB_TOKEN }}
+          branch: ${{ github.ref }}
+
+      # Popular action to deploy to GitHub Pages:
+      # Docs: https://github.com/peaceiris/actions-gh-pages#%EF%B8%8F-docusaurus
+      - name: Deploy to GitHub Pages
+        uses: peaceiris/actions-gh-pages@v3
+        with:
+          github_token: ${{ secrets.GITHUB_TOKEN }}
+          publish_dir: ./docs/voluseg-docs-app/build

.github/workflows/run_tests.yaml

@@ -7,6 +7,7 @@ on:
   pull_request:
     branches:
       - master
+      - dev
 
 jobs:
   test:

Dockerfile

@@ -1,7 +1,5 @@
 FROM python:3.12.4-slim
 
-USER root
-
 WORKDIR /voluseg
 
 RUN apt-get update && apt-get install -y wget unzip && \

README.md

@@ -53,7 +53,7 @@ import voluseg
 voluseg.update()
 
 # Download sample data
-voluseg._tools.download_sample_data("/path/to/input/")
+voluseg.tools.download_sample_data("/path/to/input/")
 
 # set and save parameters
 parameters0 = voluseg.parameter_dictionary()

app/app.py

@@ -2,7 +2,7 @@
 import typer
 from typing_extensions import Annotated
 import voluseg
-from voluseg._tools.aws import export_to_s3
+from voluseg.tools.aws import export_to_s3
 
 
 app = typer.Typer()

docs/README.md

@@ -0,0 +1,31 @@
+# Documentation
+
+This folder contains the documentation for the Voluseg project. The docs are a docusaurus app that is built and deployed to GitHub Pages.
+
+
+### Automatically generating the API Reference documentation
+The API Reference documentation is automatically generated from the docstrings and type annotations in the codebase using [pydoc-markdown](https://github.com/NiklasRosenstein/pydoc-markdown).
+
+1. Install pydoc-markdown:
+First install `pydoc-markdown` package following their guide [here](https://niklasrosenstein.github.io/pydoc-markdown/#installation-).
+
+2. In the `docs/` directory, run the following command to generate the API Reference documentation:
+```bash
+pydoc-markdown
+```
+This command will generate the API Reference documentation from the project and save it in the `docs/voluseg-docs-app/docs/reference/voluseg` folder.
+
+
+### Running the documentation app locally
+To run the documentation app locally, follow these steps:
+
+1. Install the dependencies: go to the `docs/voluseg-docs-app` directory and run:
+```bash
+# Be sure using node > 18
+yarn
+```
+2. Start the development server:
+```bash
+yarn start
+```
+

docs/pydoc-markdown.yml

@@ -0,0 +1,11 @@
+loaders:
+  - type: python
+    search_path: ["../src/voluseg"]
+processors:
+  - type: numpy
+renderer:
+  type: docusaurus
+  docs_base_path: voluseg-docs-app/docs
+  relative_output_path: reference
+  relative_sidebar_path: sidebar.json
+  sidebar_top_level_label: null

docs/requirements-docs.txt

@@ -0,0 +1,2 @@
+# pydoc-markdown==4.8.2
+git+https://github.com/luiztauffer/pydoc-markdown.git@develop

docs/voluseg-docs-app/.gitignore

@@ -0,0 +1,20 @@
+# Dependencies
+/node_modules
+
+# Production
+/build
+
+# Generated files
+.docusaurus
+.cache-loader
+
+# Misc
+.DS_Store
+.env.local
+.env.development.local
+.env.test.local
+.env.production.local
+
+npm-debug.log*
+yarn-debug.log*
+yarn-error.log*

docs/voluseg-docs-app/README.md

@@ -0,0 +1,41 @@
+# Website
+
+This website is built using [Docusaurus](https://docusaurus.io/), a modern static website generator.
+
+### Installation
+
+```
+$ yarn
+```
+
+### Local Development
+
+```
+$ yarn start
+```
+
+This command starts a local development server and opens up a browser window. Most changes are reflected live without having to restart the server.
+
+### Build
+
+```
+$ yarn build
+```
+
+This command generates static content into the `build` directory and can be served using any static contents hosting service.
+
+### Deployment
+
+Using SSH:
+
+```
+$ USE_SSH=true yarn deploy
+```
+
+Not using SSH:
+
+```
+$ GIT_USER=<Your GitHub username> yarn deploy
+```
+
+If you are using GitHub pages for hosting, this command is a convenient way to build the website and push to the `gh-pages` branch.

docs/voluseg-docs-app/babel.config.js

@@ -0,0 +1,3 @@
+module.exports = {
+  presets: [require.resolve('@docusaurus/core/lib/babel/preset')],
+};

docs/voluseg-docs-app/blog/2021-08-26-welcome/index.md

@@ -0,0 +1,8 @@
+---
+slug: welcome
+title: Welcome
+authors: [author1]
+tags: [blog, voluseg]
+---
+
+Welcome to Voluseg Blog

docs/voluseg-docs-app/blog/authors.yml

@@ -0,0 +1,5 @@
+author1:
+  name: Mikail Rubinov
+  title: Maintainer of Voluseg
+  url: https://github.com/mikarubi/voluseg
+  image_url: https://avatars.githubusercontent.com/u/10106032?v=4

docs/voluseg-docs-app/docs/faq.mdx

@@ -0,0 +1,15 @@
+---
+id: faq
+title: FAQ
+sidebar_position: 4
+---
+
+### Frequently Asked Questions
+<details>
+  <summary>What is Voluseg?</summary>
+  <p>
+    Voluseg is a software providing a pipeline for automatic volumetric cell segmentation
+    from calcium imaging recordings.
+  </p>
+</details>
+

docs/voluseg-docs-app/docs/getting_started/_category_.json

@@ -0,0 +1,8 @@
+{
+    "label": "Getting Started",
+    "position": 2,
+    "link": {
+        "type": "generated-index",
+        "description": "Getting Started"
+    }
+}

docs/iac_aws_batch.md → docs/voluseg-docs-app/docs/getting_started/iac_aws_batch.mdx

@@ -1,4 +1,7 @@
-# Running Voluseg on AWS Batch
+---
+title: Running on AWS Batch
+sidebar_position: 5
+---
 
 Here we provide instructions for setting up your Voluseg service to run jobs in AWS Batch. First you will need to provision the base AWS Batch infrastructure using CDK. This includes IAM roles, VPC, Security Group, Batch Compute Environments and Batch Job Queues. Next you will need to configure your compute resource controller to submit jobs to AWS Batch. Finally, when you submit jobs from the web interface, you must select aws_batch as the run method.
 
@@ -15,21 +18,25 @@ Here we provide instructions for setting up your Voluseg service to run jobs in
 AWS CDK is a very convenient tool, it helps automate the provisioning of AWS infrastructure and organizes all created resources in a CloudFormation stack, which can be easily updated or deleted.
 
 Follow these steps if you're running the Voluseg CDK stack for the first time:
-- go to `cloned_voluseg_repo_path/iac/aws_batch` directory.
-- run `pip install -r requirements.txt` to install the required Python packages.
-- run `cdk bootstrap` to set up the CDK deployment infrastructure (CKDToolkit) in your AWS account. This only needs to be done once.
-- run `cdk deploy` to deploy the stack to your default AWS account/region.
-- you will be prompted to confirm the deployment. Review the changes, then type `y` and hit enter.
+1. go to `cloned_voluseg_repo_path/iac/aws_batch` directory.
+2. run `pip install -r requirements.txt` to install the required Python packages.
+3. run `cdk bootstrap` to set up the CDK deployment infrastructure (CKDToolkit) in your AWS account. This only needs to be done once.
+4. run `cdk deploy` to deploy the stack to your default AWS account/region.
+5. you will be prompted to confirm the deployment. Review the changes, then type `y` and hit enter.
 
 If deployment is successful, you will be able to see the stacks in the CloudFormation page in AWS console.
 
-Other useful CDK commands:
+:::tip[Other useful CDK commands:]
 - `cdk ls` - list all stacks in the app.
 - `cdk synth` to synthesize the CloudFormation template without deploying it.
 - `cdk diff` - compare deployed stack with current state.
 - `cdk destroy` - destroy the stack.
 - `cdk docs` - open CDK documentation.
 
+:::
+
+
+
 This deployment will create the following AWS resources in your account:
 
 - IAM roles:
@@ -50,7 +57,7 @@ This deployment will create the following AWS resources in your account:
 Voluseg provides an utility function to submit jobs to AWS Batch. You can use this function to submit jobs from your local machine. You must provide the url of the remote file stored in S3, plus any other voluseg arguments you want to pass to the job. Example:
 
 ```python
-from voluseg._tools.aws import run_job_in_aws_batch
+from voluseg.tools.aws import run_job_in_aws_batch
 
 r = run_job_in_aws_batch(
     job_name="my-voluseg-job",

docs/voluseg-docs-app/docs/getting_started/installation.md

@@ -0,0 +1,35 @@
+---
+title: Installation
+sidebar_position: 2
+---
+
+
+### Prerequisites
+In order to use Voluseg you need to have the following dependencies installed on your machine:
+
+- Python >= 3.11
+- Java SE Development Kit 17
+- [ANTs - Advanced Normalization Tools](https://github.com/ANTsX/ANTs)
+
+It is recommended to install Voluseg in a separate environment, using [Anaconda](https://www.anaconda.com/distribution/) or [Virtual Environment](https://docs.python.org/3/library/venv.html).
+
+
+### Install with pip
+```python
+pip install voluseg
+```
+
+
+### Install from Github repository
+
+1. Clone the Voluseg repository to your local machine by running
+```bash
+git clone https://github.com/mikarubi/voluseg.git
+```
+
+2. Install Voluseg from local source
+Install Voluseg with pip in your active virtual environment by running
+```bash
+cd voluseg
+pip install .
+```