release 2.0.0alpha1 #32

mbjones · 2024-09-19T01:10:23Z

This PR is for an initial 2.0.0-alpha1 release which prepares the vegbank2 repository for our normal release cycle. It contains the following:

Updated project structure and layout (set up git repositories #31)
Contributing guidelines (set up git repositories #31)
License files (set up git repositories #31)
Initial work on database migration to newer Postgres version (Create process to migrate existing VegBank DB to new Postgres instance #29)
Initial work on Flyway database migration scripts (Create process to migrate existing VegBank DB to new Postgres instance #29)
Initial poetry build infrastructure for python package builds.

….sh' and respective files to generate the base 'vegbank.sql' to initialize postgres

…odel_vegbank.xml'

….xml' back slash typo to reference sql file in 'throwAways' folder

…O item for missing data dictionary sql

…ing migration files

…ST, add required dependencies to execute bash script

…V#.#__name)

…by adding 'IF EXISTS' to 'DROP INDEX'

…ng 'IF EXISTS' to 'DROP VIEW'

…ion sql files

…'IF EXISTS' to 'DROP VIEW'

…d update migrations versioning in file names

…and context

…ble commands were moved from 'V1.0__vegbank.sql'

…l) alter migrations 'V1.6__vegbank_populate_configtables.sql' sequence to restart key value to resolve constraint issues

…_preassignacccode_dba_requestnumber_seq'

…-only dump file with only one exception (hydrologicregime), but unable to apply last constraints migration

…y 'sql'

…to 50

…to 255

…s been commented out/disabled

…estnumber_seq' in 'V1.12' migrations, which was re-added previously into 'V1.0' migration

…t, and revise output file name from

…fully restore vegbank's postgres from a data-only dump file with the existing migrations

Feature-29: Flyway Migration with Data-Only Dump File

…y dump file

…b and add new logs file for reference/context

…th helm files

mbjones · 2024-09-30T23:30:17Z

Ping: @RWShelton @regetz I'd like to get this merged as a starting point to provide a better public face to the github repo and a starting point for incremental releases. Waiting on review from one of you.

RWShelton · 2024-09-30T23:38:30Z

Looks good to me, I've been working with the flyway stuff for a while and it's working well. My only question - I know it's a small commit, but does my last one with the baseline helm chart need to be reviewed individually before it goes in with this pull request? More of a process question for my own knowledge going forward than anything particular about this one.

mbjones · 2024-10-01T00:06:31Z

@RWShelton thanks. for your recent commits, in general and typically, each new major feature would indeed be in its own branch named after the feature it is implementing, and you would create a PR to pull that into develop. This process is described in detail in the CONTRIBUTING.md file. I think your current commit is fine though as a starting point in an alpha release.

I just added in the tests structure for pytest, so this branch now includes that as well. I will add a github action to run the tests in a sec. Then we can merge.

mbjones · 2024-10-01T00:50:11Z

Added GH Action to run poetry run pytest which is now passing. Can definitely be improved (it emits some warnings about doing a pip install, but its a start.

regetz

I left a bunch of small comments where I saw especially DataONE and/or boilerplate links/etc that probably need to change, either as part of this PR or with the next one. Still marking as Approved to unblock this merge.

regetz · 2024-10-01T06:07:36Z

README.md

@@ -9,17 +18,19 @@ VegBank provides a common storage system and web portal for accessing:
 - Plant taxonomy data
 - Community data

+VegBank is an open source, community projects.  We [welcome contributions](./CONTRIBUTING.md) in many forms, including code, graphics, documentation, bug reports, testing, etc.  Use the [DataONE discussions](https://github.com/DataONEorg/dataone/discussions) to discuss these contributions with us.


Do we want to keep the DataONE discussions link or is there something more appropriate?

That's a good question. I will leave this as is, but I opened a ticket (#34) to update this and replace it with a vegbank-appropriate solution. The README is a template and needs a lot of work still.

regetz · 2024-10-01T06:09:22Z

CONTRIBUTING.md

+
+In short:
+
+- add an [issue](https://github.com/DataONEorg/REPONAME/issues) describing your planned changes, or add a comment to an existing issue;


I assume we want to link to https://github.com/NCEAS/vegbank2/issues

yes, fixed locally and I will push shortly.

regetz · 2024-10-01T06:10:04Z

CONTRIBUTING.md

+We welcome all types of contributions, including bug fixes, feature enhancements,
+bug reports, documentation, graphics, and many others.  You might consider contributing by:
+
+- Report a bug or request a new feature in our [issue tracker](https://github.com/DataONEorg/REPONAME/issues)


I assume we want to link to https://github.com/NCEAS/vegbank2/issues

yes, will push shortly

regetz · 2024-10-01T06:11:28Z

CONTRIBUTING.md

+- Fix a bug and contribute the code with a Pull Request
+- Write or edit some documentation
+- Sharing helpful tips or FAQ-type answers to users or future contributors
+- Create screenshots or tutorials of features of MetacatUI


Delete this bullet, and probably the next one unless we create a VegBank discussion board

I generalized it rather than deleting it.

regetz · 2024-10-01T06:11:56Z

CONTRIBUTING.md

+In short:
+
+- add an [issue](https://github.com/DataONEorg/REPONAME/issues) describing your planned changes, or add a comment to an existing issue;
+- on GitHub, fork the [repository](https://github.com/DataONEorg/REPONAME)


https://github.com/NCEAS/vegbank2

regetz · 2024-10-01T06:12:09Z

CONTRIBUTING.md

+
+## 🚀 Development Workflow
+
+Development is managed through the git repository at https://github.com/DataONEorg/REPONAME.  The repository is organized into several branches, each with a specific purpose.  


https://github.com/NCEAS/vegbank2

regetz · 2024-10-01T06:16:03Z

README.md

+- **License**: [Apache 2](http://opensource.org/licenses/Apache-2.0)
+- [Package source code on GitHub](https://github.com/NCEAS/vegbank2)
+- [**Submit Bugs and feature requests**](https://github.com/NCEAS/vegbank2/issues)
+- Contact us: support@dataone.org


Can/should we use an apporpriate VegBank email list?

Changed to help@vegbank.org for now.

doulikecookiedough added 30 commits July 23, 2024 15:29

Add new 'sql-transformations' folder, with bash script 'getvegbanksql…

7f4330c

….sh' and respective files to generate the base 'vegbank.sql' to initialize postgres

Add required .sql files defined in 'updateSQLFile' parameter in 'db_m…

4b2da2d

…odel_vegbank.xml'

Add 'INSTALL.md' to provide WIP context, and modify 'db_model_vegbank…

bff0705

….xml' back slash typo to reference sql file in 'throwAways' folder

Add initial migrations WIP flow, not yet tested with flyway, and TO-D…

5e611b6

…O item for missing data dictionary sql

Rename migration file, add additional context for 'INSTALL.md' regard…

39300ef

…ing migration files

Add additional context to 'INSTALL.md' for clarity

8ad27da

Add 'populate-datadictionary.sql' doc to migrations, update INSTALL.R…

3acdc72

…ST, add required dependencies to execute bash script

Add TODO item in 'INSTALL.md' to review 'dbVerify' target

c0afa7a

Remove dbVerify TODO in 'INSTALL.md' - not necessary

9312f9c

Add 'create_admin_user.sql' to migrations

cea47f6

Add 'resources' folder with initial 'flyway.conf' file

dbcf685

Update 'flyway.conf' user and password

317fa8c

Rename migrations sql files to follow flyway naming conventions (Ex. …

be6a270

…V#.#__name)

Fix exceptions causing migration to fail in V1.3__createIndices file …

e7663fe

…by adding 'IF EXISTS' to 'DROP INDEX'

Rename some migration files for consistency

9db046c

Fix exception causing migration to fail in V1.4__drop... file by addi…

6d5ded2

…ng 'IF EXISTS' to 'DROP VIEW'

Update INSTALL.md with additional context to manual changes to migrat…

6bb395a

…ion sql files

Fix exception causing migration to vail in V1.5__create... by adding …

a88f462

…'IF EXISTS' to 'DROP VIEW'

Add new migration file step to drop views before re-creating them, an…

10d3631

…d update migrations versioning in file names

Add flyway 'README.md' in /resources to provide some flyway know-how …

ebeda73

…and context

Update flyway 'README.md' for more context and wording

305cc9a

Create new migration file 'V1.12__add_constraints.sql' where alter ta…

97e215d

…ble commands were moved from 'V1.0__vegbank.sql'

Disable migration sql commands that conflicts with the data dump process

a493778

Add logs for dump file execution with constraints moved, (unsuccessfu…

e5de662

…l) alter migrations 'V1.6__vegbank_populate_configtables.sql' sequence to restart key value to resolve constraint issues

Complete comment out V1.6 migrations

d5d11be

Add missing table 'temp_ks_commconcept_acccodelist' and sequence 'dba…

d3f7b88

…_preassignacccode_dba_requestnumber_seq'

Add log to 'sql-logs' where successfully loaded all records from data…

6955cc1

…-only dump file with only one exception (hydrologicregime), but unable to apply last constraints migration

Move sql-logs and sql-transformations folder into new parent director…

934e2cd

…y 'sql'

Revise 'V1.0' migration observation table's hydrologicRegime varchar …

d7e7c75

…to 50

Revise 'V1.0' migration observation table's hydrologicRegime varchar …

203ce2f

…to 255

doulikecookiedough and others added 14 commits August 24, 2024 09:41

Add note into 'V1.6' migrations file to clarify why this migration ha…

21ceaba

…s been commented out/disabled

Remove redundant sequence creation for 'dba_preassignacccode_dba_requ…

a10247d

…estnumber_seq' in 'V1.12' migrations, which was re-added previously into 'V1.0' migration

Rename sql-log files

a85bcd4

Rename 'INSTALL.md' in sql-transformations to 'README', revise conten…

ab7240d

…t, and revise output file name from

Add README.md

0cc1d4b

Add new 'INSTALL.md' document in repo root to describe how to success…

645ef29

…fully restore vegbank's postgres from a data-only dump file with the existing migrations

Update readme docs

ea95718

Merge pull request #30 from NCEAS/feature-29-constraint-migration

c406ec0

Feature-29: Flyway Migration with Data-Only Dump File

Add additional context to 'INSTALL.md' for when generating a data-onl…

fe0a00a

…y dump file

Update 'INSTALL.md' for docker pg16 restoration process for vegbank d…

80aa80d

…b and add new logs file for reference/context

Add license and contributing docs.

7b0dbe8

Update READEM to follow the standard repo template.

41ee62c

Moved database files to src/database directory.

fd20d31

Initial example pyproject.toml file for poetry build

133651f

mbjones added the D-2.2 Deployed and refactored, open-source VegBank label Sep 19, 2024

mbjones added this to the v2.0.0alpha01 milestone Sep 19, 2024

mbjones requested review from regetz and RWShelton September 19, 2024 01:10

Adding helm chart with baseline postgres setup, updating gitignore wi…

b0f3b6b

…th helm files

Add basic pytest framework.

bfae4a4

mbjones added 2 commits September 30, 2024 16:41

Iniital poetry-based Github Action for CI.

32072e3

Fix syntax errors in authors list.

767aaac

Add description of Github action (and act) to README

2fca575

regetz approved these changes Oct 1, 2024

View reviewed changes

Misc editorial fixes from feedback on PR #32

e35b3dc

mbjones merged commit ec99052 into main Oct 1, 2024
2 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

release 2.0.0alpha1 #32

release 2.0.0alpha1 #32

mbjones commented Sep 19, 2024

mbjones commented Sep 30, 2024

RWShelton commented Sep 30, 2024

mbjones commented Oct 1, 2024

mbjones commented Oct 1, 2024

regetz left a comment

regetz Oct 1, 2024

mbjones Oct 1, 2024

regetz Oct 1, 2024

mbjones Oct 1, 2024

regetz Oct 1, 2024

mbjones Oct 1, 2024

regetz Oct 1, 2024

mbjones Oct 1, 2024

regetz Oct 1, 2024

mbjones Oct 1, 2024

regetz Oct 1, 2024

mbjones Oct 1, 2024

regetz Oct 1, 2024

mbjones Oct 1, 2024


		In short:

		- add an [issue](https://github.com/DataONEorg/REPONAME/issues) describing your planned changes, or add a comment to an existing issue;


		## 🚀 Development Workflow

		Development is managed through the git repository at https://github.com/DataONEorg/REPONAME. The repository is organized into several branches, each with a specific purpose.

release 2.0.0alpha1 #32

release 2.0.0alpha1 #32

Conversation

mbjones commented Sep 19, 2024

mbjones commented Sep 30, 2024

RWShelton commented Sep 30, 2024

mbjones commented Oct 1, 2024

mbjones commented Oct 1, 2024

regetz left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment