Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add zip_lists filter to yacfg | Improve and cleanup the github actions #316

Draft
wants to merge 3 commits into
base: main
Choose a base branch
from
Draft

Add zip_lists filter to yacfg | Improve and cleanup the github actions #316

Show file tree
Hide file tree
Changes from 2 commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
50925c1
Add 'zip_lists' filter
Apr 9, 2024
3361d94
Unify and clean up the github actions.
Apr 18, 2024
6dd84f6
Let profile_test.sh be executable.
Apr 18, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
53 changes: 13 additions & 40 deletions .github/workflows/main.pull_request.workflow.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -18,11 +18,10 @@ jobs:
fail-fast: false
matrix:
os: ["Ubuntu"]
python-version: ["3.11", "pypy-3.9"]
python-version: ["3.11", "pypy-3.10", "pypy-3.9"]
runs-on: "${{ matrix.os }}-latest"

steps:

- uses: "actions/checkout@v4"

- name: "Set up Python ${{ matrix.python-version }}"
Expand All @@ -41,41 +40,15 @@ jobs:
python -m pip install --upgrade coverage[toml] tox tox-gh-actions
poetry install

# - name: "Configure poetry"
# run: poetry config virtualenvs.in-project true
#
# - name: "Set up cache"
# uses: actions/cache@v2
# id: cache
# with:
# path: .venv
# key: venv-${{ runner.os }}-${{ steps.full-python-version.outputs.version }}-${{ hashFiles('**/poetry.lock') }}
#
# - name: "Ensure cache is healthy"
# if: steps.cache.outputs.cache-hit == 'true'
# run: timeout 10s poetry run pip --version || rm -rf .venv

- name: "Run tox targets for ${{ matrix.python-version }}"
run: tox

# - name: "Get coverage"
# if: "contains(env.USING_COVERAGE, matrix.python-version)"
# run: |
# set -xe
# tox -e coverage-report

# - name: "Upload coverage to Codecov"
# if: "contains(env.USING_COVERAGE, matrix.python-version)"
# uses: "codecov/codecov-action@v1"
# with:
# fail_ci_if_error: true
#
# - name: "Upload coverage artifacts"
# if: "contains(env.USING_DOCS, matrix.python-version)"
# uses: actions/upload-artifact@v4
# with:
# name: coverage
# path: .coverage-reports
- name: "Upload docs artifacts"
uses: actions/upload-artifact@v4
if: "contains(env.USING_DOCS, matrix.python-version)"
with:
name: docs
path: docs/_build/html

- name: "Build package"
run: |
Expand All @@ -87,9 +60,9 @@ jobs:
name: package-dist-${{ runner.os }}-${{ steps.full-python-version.outputs.python-version }}
path: dist/

- name: "Upload docs artifacts"
uses: actions/upload-artifact@v4
if: "contains(env.USING_DOCS, matrix.python-version)"
with:
name: docs
path: docs/_build/html
- name: "Run help"
run: "poetry run yacfg --help"

- name: "Run the test script"
run: ./profile_test.sh

86 changes: 21 additions & 65 deletions .github/workflows/main.push.workflow.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,6 @@ on:
jobs:
Tests:
name: "Tests | ${{ matrix.os }} | Python ${{ matrix.python-version }}"

env:
USING_COVERAGE: '3.11'
USING_DOCS: '3.11'
Expand All @@ -19,10 +18,10 @@ jobs:
fail-fast: false
matrix:
os: ["Ubuntu", "Windows"]
python-version: ["3.11", "3.10"]
python-version: ["3.11", "pypy-3.10", "pypy-3.9"]
runs-on: "${{ matrix.os }}-latest"
steps:

steps:
- uses: "actions/checkout@v4"

- name: "Set up Python ${{ matrix.python-version }}"
Expand All @@ -37,60 +36,36 @@ jobs:
run: |
python -VV
python -m site
python -m pip install --upgrade pip setuptools wheel coverage[toml] tox tox-gh-actions
python -m pip install --upgrade pip setuptools wheel
python -m pip install --upgrade coverage[toml] tox tox-gh-actions
poetry install

# - name: "Configure poetry"
# run: poetry config virtualenvs.in-project true
#
# - name: "Set up cache"
# uses: actions/cache@v2
# id: cache
# with:
# path: .venv
# key: venv-${{ runner.os }}-${{ steps.full-python-version.outputs.python-version }}-${{ hashFiles('**/poetry.lock') }}
#
# - name: "Ensure cache is healthy"
# if: steps.cache.outputs.cache-hit == 'true'
# run: timeout 10s poetry run pip --version || rm -rf .venv

- name: "Run tox targets for ${{ matrix.python-version }}"
run: tox

# - name: "Get coverage"
# if: "contains(env.USING_COVERAGE, matrix.python-version)"
# run: |
# set -xe
# tox -e coverage-report

# - name: "Upload coverage to Codecov"
# if: "contains(env.USING_COVERAGE, matrix.python-version)"
# uses: "codecov/codecov-action@v1"
# with:
# fail_ci_if_error: true

- name: "Upload docs artifacts"
if: "contains(env.USING_DOCS, matrix.python-version)"
uses: actions/upload-artifact@v4
with:
name: docs
path: docs/_build/html
#
# - name: "Upload coverage artifacts"
# if: "contains(env.USING_DOCS, matrix.python-version)"
# uses: actions/upload-artifact@v4
# with:
# name: coverage
# path: .coverage-reports
#
# - name: "Publish package to TestPyPI"
# if: "contains(env.TESTPYPI_PUBLISH, matrix.python-version)"
# uses: pypa/gh-action-pypi-publish@v1.3.1
# with:
# skip_existing: true
# user: __token__
# password: ${{ secrets.TESTPYPI_TOKEN }}
# repository_url: https://test.pypi.org/legacy/

- name: "Build package"
run: |
poetry build

- name: "Upload dist artifacts"
uses: actions/upload-artifact@v4
with:
name: package-dist-${{ runner.os }}-${{ steps.full-python-version.outputs.python-version }}
path: dist/

- name: "Run help"
run: "poetry run yacfg --help"

- name: "Run the test script"
run: ./profile_test.sh


Build:
needs: Tests
Expand All @@ -112,20 +87,6 @@ jobs:

- name: "Install poetry"
uses: "abatilo/actions-poetry@v3.0.0"
#
# - name: "Configure poetry"
# run: poetry config virtualenvs.in-project true
#
# - name: "Set up cache"
# uses: actions/cache@v2
# id: cache
# with:
# path: .venv
# key: venv-${{ runner.os }}-${{ steps.full-python-version.outputs.python-version }}-${{ hashFiles('**/poetry.lock') }}
#
# - name: "Ensure cache is healthy"
# if: steps.cache.outputs.cache-hit == 'true'
# run: timeout 10s poetry run pip --version || rm -rf .venv

- name: "Build package"
id: build
Expand All @@ -152,11 +113,6 @@ jobs:
with:
name: docs
path: pages
#
# - uses: actions/download-artifact@master
# with:
# name: coverage
# path: pages/coverage

# Push the docs' HTML to github-pages
- name: "GitHub Pages action"
Expand Down
7 changes: 7 additions & 0 deletions src/yacfg/yacfg.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -136,6 +136,11 @@ def override_value_map_keys(value: Dict[str, Any]) -> Dict[str, Any]:
"""
return {override_value(key, key): val for key, val in value.items()}

def zip_lists(value: List[Any], pairing: List[Any]) -> List[(Any, Any)]:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Isn't this the invalid syntax for tuple types we discussed last week?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think so. It currently runs for me. Will check to make sure and resolve this.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more. 

>>> from typing import List, Any
>>> List[(Any, Any)]
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib64/python3.12/typing.py", line 384, in inner
    return func(*args, **kwds)
           ^^^^^^^^^^^^^^^^^^^
  File "/usr/lib64/python3.12/typing.py", line 1436, in __getitem__
    _check_generic(self, params, self._nparams)
  File "/usr/lib64/python3.12/typing.py", line 304, in _check_generic
    raise TypeError(f"Too {'many' if alen > elen else 'few'} arguments for {cls};"
TypeError: Too many arguments for typing.List; actual 2, expected 1

if not isinstance(value, List) or not isinstance(pairing, List):
return []
return zip(value, pairing)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would be nice to have a test/usage example for this. Having the actual template this will operate on only in a separate repo sucks.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree and will try to add the test into profile_test.sh.


# Pass empty filter for performance if extra_properties_data is not defined (no more conditions)
env.filters["overridevalue"] = (
override_value if extra_properties_data else empty_filter
Expand All @@ -144,6 +149,8 @@ def override_value_map_keys(value: Dict[str, Any]) -> Dict[str, Any]:
override_value_list_map_keys if extra_properties_data else empty_filter
)

env.filters["zip_lists"] = zip_lists

template_list = get_main_template_list(env)
if output_filter:
template_list = filter_template_list(template_list, output_filter)
Expand Down
Loading