Skip to content

Commit

Permalink
packaging: Use Hatch (#201)
Browse files Browse the repository at this point in the history
  • Loading branch information
edgarrmondragon authored Jan 2, 2024
1 parent 8fe97ec commit 39c9438
Show file tree
Hide file tree
Showing 10 changed files with 169 additions and 1,527 deletions.
5 changes: 1 addition & 4 deletions .github/workflows/constraints.txt
Original file line number Diff line number Diff line change
@@ -1,5 +1,2 @@
nox==2023.4.22
nox-poetry==1.0.3
hatch==1.9.1
pip==23.3.2
poetry==1.7.1
poetry-dynamic-versioning==1.1.1
41 changes: 13 additions & 28 deletions .github/workflows/dynamic-publish.yaml
Original file line number Diff line number Diff line change
@@ -1,45 +1,30 @@
name: Publish with Dynamic Versioning
name: Build

on:
release:
types: [published]
push:

permissions:
contents: write
id-token: write

jobs:
publish:
name: Publish to PyPI
build:
runs-on: ubuntu-latest
env:
PIP_CONSTRAINT: .github/workflows/constraints.txt
steps:
- name: Checkout code
uses: actions/checkout@v4.1.1
- uses: actions/checkout@v4
with:
fetch-depth: 0
- uses: hynek/build-and-inspect-python-package@v2

- name: Set up Python
uses: actions/setup-python@v4.7.1
publish:
runs-on: ubuntu-latest
needs: build
if: startsWith(github.ref, 'refs/tags/')
steps:
- uses: actions/download-artifact@v4
with:
python-version: "3.10"

- name: Upgrade pip
run: |
pip install pip
pip --version
- name: Install Poetry
run: |
pipx install poetry
pipx inject poetry poetry-dynamic-versioning[plugin]
poetry --version
poetry self show plugins
- name: Build
run: poetry build

name: Packages
path: dist
- name: Upload wheel to release
uses: svenstaro/upload-release-action@v2
with:
Expand Down
43 changes: 14 additions & 29 deletions .github/workflows/test.yml
Original file line number Diff line number Diff line change
Expand Up @@ -6,8 +6,6 @@ on:
paths:
- tap_checkly/**
- tests/**
- noxfile.py
- poetry.lock
- pyproject.toml
- .github/workflows/test.yml
- .github/workflows/constraints.txt
Expand All @@ -16,8 +14,6 @@ on:
paths:
- tap_checkly/**
- tests/**
- noxfile.py
- poetry.lock
- pyproject.toml
- .github/workflows/test.yml
- .github/workflows/constraints.txt
Expand All @@ -26,51 +22,40 @@ jobs:
tests:
runs-on: ubuntu-latest
env:
PIP_CONSTRAINT: .github/workflows/constraints.txt
NOXSESSION: ${{ matrix.session }}-${{ matrix.python-version }}
FORCE_COLOR: "1"
PIP_CONSTRAINT: .github/workflows/constraints.txt
strategy:
fail-fast: false
matrix:
script: ["test:integration"]
python-version: ["3.8", "3.9", "3.10", "3.11", "3.12"]
include:
- {python-version: "3.10", session: "mypy"}
- {python-version: "3.11", session: "tests"}
- {python-version: "3.10", session: "tests"}
- {python-version: "3.9", session: "tests"}
- {python-version: "3.8", session: "tests"}
- { script: "test:dependencies", python-version: "3.12" }
- { script: "typing:check", python-version: "3.12" }

steps:
- name: Checkout code
uses: actions/checkout@v4.1.1
- uses: actions/checkout@v4
with:
fetch-depth: 0

- name: Set up Python ${{ matrix.python-version }}
uses: actions/setup-python@v4.7.1
- uses: actions/setup-python@v5
with:
python-version: ${{ matrix.python-version }}
cache: pip

- name: Upgrade pip
run: |
pip install pip
pip --version
- name: Install Poetry
run: |
pipx install poetry
pipx inject poetry poetry-dynamic-versioning[plugin]
poetry --version
poetry self show plugins
- name: Install Nox
- name: Install Hatch
run: |
pipx install nox
pipx inject nox nox-poetry
nox --version
pipx install hatch --python "python${{ matrix.python-version }}"
hatch --version
- name: Run Nox
- name: Run
env:
TAP_CHECKLY_ACCOUNT_ID: ${{ secrets.TAP_CHECKLY_ACCOUNT_ID }}
TAP_CHECKLY_TOKEN: ${{ secrets.TAP_CHECKLY_TOKEN }}
TAP_CHECKLY_START_DATE: ${{ secrets.TAP_CHECKLY_START_DATE }}
run: |
nox
hatch run ${{ matrix.script }}
51 changes: 24 additions & 27 deletions .pre-commit-config.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -5,33 +5,30 @@ ci:
autoupdate_commit_msg: "chore(deps): pre-commit autoupdate"

repos:
- repo: https://github.com/pre-commit/pre-commit-hooks
rev: v4.5.0
hooks:
- id: check-json
exclude: "\\.vscode/.*.json"
- id: check-toml
- id: check-yaml
- id: end-of-file-fixer
exclude: "plugins/.*/.*\\.lock"
- id: trailing-whitespace
- repo: https://github.com/pre-commit/pre-commit-hooks
rev: v4.5.0
hooks:
- id: check-json
exclude: "\\.vscode/.*.json"
- id: check-toml
- id: check-yaml
- id: end-of-file-fixer
exclude: "plugins/.*/.*\\.lock"
- id: trailing-whitespace

- repo: https://github.com/tox-dev/pyproject-fmt
rev: "1.5.3"
hooks:
- id: pyproject-fmt
- repo: https://github.com/tox-dev/pyproject-fmt
rev: "1.5.3"
hooks:
- id: pyproject-fmt

- repo: https://github.com/astral-sh/ruff-pre-commit
rev: "v0.1.9"
hooks:
- id: ruff
args: [--fix, --exit-non-zero-on-fix, --show-fixes]
- id: ruff-format
- repo: https://github.com/astral-sh/ruff-pre-commit
rev: "v0.1.10"
hooks:
- id: ruff
args: [--fix, --exit-non-zero-on-fix, --show-fixes]
- id: ruff-format

- repo: https://github.com/pre-commit/mirrors-mypy
rev: v1.8.0
hooks:
- id: mypy
pass_filenames: true
additional_dependencies:
- types-requests
- repo: https://github.com/pre-commit/pre-commit
rev: v3.6.0
hooks:
- id: validate_manifest
43 changes: 21 additions & 22 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -81,48 +81,47 @@ tap-checkly --config CONFIG --discover > ./catalog.json
### Initialize your Development Environment

```bash
pipx install poetry
poetry install
pipx install hatch
```

### Create and Run Tests

Create tests within the `tests` subfolder and then run:
Run integration tests:

```bash
poetry run pytest
hatch run tests:integration
```

You can also test the `tap-checkly` CLI interface directly using `poetry run`:
You can also test the `tap-checkly` CLI interface directly:

```bash
poetry run tap-checkly --help
hatch run sync:console -- --about --format=json
```

### Testing with [Meltano](https://www.meltano.com)

_**Note:** This tap will work in any Singer environment and does not require Meltano.
Examples here are for convenience and to streamline end-to-end orchestration scenarios._

Your project comes with a custom `meltano.yml` project file already created.
Install Meltano (if you haven't already) and any needed plugins:
Your project comes with a custom `meltano.yml` project file already created. Go ahead and [install Meltano](https://docs.meltano.com/getting-started/installation/) if you haven't already.

```bash
# Install meltano
pipx install meltano
# Initialize meltano within this directory
cd tap-checkly
meltano install
```
1. Install all plugins

Now you can test and orchestrate using Meltano:
```bash
meltano install
```

```bash
# Test invocation:
meltano invoke tap-checkly --version
# OR run a test `elt` pipeline:
meltano elt tap-checkly target-jsonl
```
1. Check that the extractor is working properly

```bash
meltano invoke tap-checkly --version
```

1. Execute an ELT pipeline

```bash
meltano run tap-checkly target-jsonl
```

### SDK Dev Guide

Expand Down
58 changes: 0 additions & 58 deletions noxfile.py

This file was deleted.

Loading

0 comments on commit 39c9438

Please sign in to comment.