[Devops] update metrics ci with new tests #1801
Workflow file for this run
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| name: model | |
| on: | |
| push: | |
| paths: # Run only when files under neuralprophet/ are modified | |
| - neuralprophet/** | |
| branches: # Run only on these branches | |
| - main | |
| - develop | |
| pull_request: # Run on pull requests | |
| branches: | |
| - main | |
| - develop | |
| workflow_dispatch: | |
| jobs: | |
| metrics: | |
| runs-on: ubuntu-latest | |
| container: docker://ghcr.io/iterative/cml:0-dvc2-base1 | |
| steps: | |
| - name: Checkout | |
| uses: actions/checkout@v3 | |
| with: | |
| ref: ${{ github.event.pull_request.head.sha }} | |
| - name: Install Python "3.10" | |
| uses: actions/setup-python@v4 | |
| with: | |
| python-version: "3.10" | |
| - name: Install Poetry | |
| uses: snok/install-poetry@v1 | |
| - name: Install dependencies | |
| run: poetry install | |
| - name: Train model | |
| run: | | |
| pytest tests/test_model_performance.py | |
| - name: Download metrics from main | |
| uses: dawidd6/action-download-artifact@v2 | |
| with: | |
| repo: ourownstory/neural_prophet | |
| branch: main | |
| name: metrics | |
| path: tests/metrics-main/ | |
| if_no_artifact_found: warn | |
| - name: Compare performance | |
| run: | | |
| echo "## Model Benchmark" >> report.md | |
| python tests/metrics/compareMetrics.py >> report.md | |
| - name: Publish report | |
| env: | |
| REPO_TOKEN: ${{ secrets.GITHUB_TOKEN }} | |
| run: | | |
| echo "<details>\n<summary>Model training plots</summary>\n" >> report.md | |
| echo "## Model Training" >> report.md | |
| echo "### PeytonManning" >> report.md | |
| cml asset publish tests/metrics/PeytonManning.svg --md >> report.md | |
| echo "### PeytonManning_test30" >> report.md | |
| cml asset publish tests/metrics/PeytonManning_test30.svg --md >> report.md | |
| echo "### YosemiteTemps" >> report.md | |
| cml asset publish tests/metrics/YosemiteTemps.svg --md >> report.md | |
| echo "### YosemiteTemps_test20" >> report.md | |
| cml asset publish tests/metrics/YosemiteTemps_test20.svg --md >> report.md | |
| echo "### AirPassengers" >> report.md | |
| cml asset publish tests/metrics/AirPassengers.svg --md >> report.md | |
| echo "### AirPassengers_test30" >> report.md | |
| cml asset publish tests/metrics/AirPassengers_test30.svg --md >> report.md | |
| echo "\n</details>" >> report.md | |
| # Post reports as comments in GitHub PRs | |
| cml comment update --pr report.md # post to PR | |
| cml check create --title=ModelReport report.md # update status of check in PR | |
| - name: Upload metrics if on main | |
| uses: actions/upload-artifact@v3 | |
| with: | |
| name: metrics | |
| path: tests/metrics/ |