Merge pull request #3 from NRCan/dev

small bug fixes and initial review updates

CONTRIBUTING.md

@@ -0,0 +1,63 @@
+# CanCurve development
+
+## Installation
+Typically, development should employ a virtual environment with pyqgis bindings. 
+We usually develop to target the most recent QGIS LTR. 
+Some additional packages for testing are specified in `./requirements.txt`
+
+### PYTHONPATH
+only the source directory should be included (`./CanCurve` not `./CanCurve/cancurve`)
+
+## Tests
+pytests are in `./tests`
+
+## Compiling
+the only compiling typically requied is when the qt `resources.qrc` file is changed. This needs to be ported to a python module (typically using  `pyrcc5 -o resources.py resources.qrc` as in `./dev_tools/plug_compile.bat`)
+
+
+
+## Deploying Plugin
+
+### Active Development
+Typically a `dev` QGIS profile is maintained with a custom `QGIS_PLUGINPATH` pointed to the project source directory. This facilitates plugin updating with the `plugin reload` (ie no real deployment) 
+
+### Pre-Release testing
+Pre-release testing (and full deployment) employ a zip of the plugin directory (see `./dev_tools/plug_zip.bat`):
+1) remove all `__pychace__`
+2) zip/archive the plugin directory
+
+This zip file can then be distributed using a git-hub release (upload the zip file to the github release... NOT the git repo tracking)
+
+
+### Full QGIS Repository release
+- [ ] create the plugin zip as above
+
+- [ ] in git-hub, create a new release tag (e.g., v1.2.0), summarize new features for developers. upload the zip file. 
+
+- [ ] login to [plugins.qgis.org](https://plugins.qgis.org/accounts/login/?next=/plugins/my) using the CanFlood credentials (ask Nicky). Navigate to **Upload a plugin** and select the zip file.
+
+- [ ] In QGIS, refresh the repository and ensure that the new version is available (may take ~10mins for the version to be available). Upgrade and check that it works.
+
+- [ ] notify project team
+
+## Developing an Update
+
+the dev branch is where new features and fixes are collected and tested before release. The following should be executed on the dev branch in preparation for pushing to the main branch:
+
+- [ ] add/update documentation where applicable
+
+- [ ] backwards merge master into dev to capture any upstream changes (these should be minor and limited to documentation tweaks as all development is done on the dev branch)
+
+- [ ] ensure the version tag is updated on `.\cancurve\__init__.py`
+
+- [ ] update the README.md to summarize any new features for users
+
+- [ ] similarly update cancurve\metadata.txt
+
+- [ ] execute all tests. investigate warnings. fix errors. 
+
+- [ ] perform a 'person test' by having a non-developer follow relevant tutorials. investigate warnings and fix errors.
+
+- [ ] Once these tests are complete **and passing**, a pull request should be completed and the dev branch merged into the main.
+
+- [ ] Follow the above **Deploying Plugin/Full QGIS Repository Release** on the main branch 

README.md

@@ -5,13 +5,19 @@ CanCurve is an open-source tool which can develop depth-damage (stage-damage) fu
   <img src="./cancurve/img/icon.png" alt="CanCurve Icon"> 
 </p>
 
+
+## Updates
+- 2024-05-09: small fixes and updates based on initial comments
+- 2024-05-01: initial working release (un tested) [v0.0.1](https://github.com/NRCan/CanCurve/releases/tag/v0.0.1)
+
 
 ## Installation
 - Install [QGIS 3.34.5](https://download.qgis.org/downloads/) (with Qt 5.15.13)
 - download the `cancurve.zip` file from the latest release to your local machine
 - in QGIS, `Manage and Install Plugins...` > `Install from ZIP` > select the downloaded file
 - it is recommended to also install the **First Aid** plugin for more detailed error messages. 
 - it is recommended to set up the QGIS Debug Log file as shown [here](https://stackoverflow.com/a/61669864/9871683)
+- CanCurve backend and project data is implemented in SQLite relational databases. For enhanced customization and debugging, it is recommended to install a SQLite viewer + editor like [DB Browser for SQLite](https://sqlitebrowser.org/) for working with these files.  
 
 
 

cancurve/__init__.py

@@ -1,6 +1,11 @@
+#===============================================================================
+# plugin metadata
+#===============================================================================
+__version__='0.0.2'
 
-__version__='0.0'
-
+#===============================================================================
+# plugin entry point
+#===============================================================================
 # noinspection PyPep8Naming
 def classFactory(iface):  # pylint: disable=invalid-name
     """Load CanCurve class from file CanCurve.
@@ -10,4 +15,23 @@ def classFactory(iface):  # pylint: disable=invalid-name
     """
     #
     from .plugin import CanCurve
-    return CanCurve(iface)
+    return CanCurve(iface)
+
+#===============================================================================
+# dependency check
+#===============================================================================
+
+
+
+import importlib, warnings
+
+def check_package(package_name):
+    spec = importlib.util.find_spec(package_name)
+    if spec is not None:
+        print(f'module {package_name} is installed')
+    else:
+        warnings.warn(f'module \'{package_name}\' not installed')
+
+
+check_package('openpyxl')
+

cancurve/bldgs/assertions.py

@@ -9,6 +9,7 @@
 import pandas as pd
 
 from .parameters import colns_index, colns_dtypes, bldg_meta_rqmt_df
+from ..hp.basic import view_web_df as view
 
 
 expected_tables_base = ['project_meta','project_settings','c00_bldg_meta', 'c00_cost_items','c00_drf']
@@ -39,9 +40,10 @@ def assert_ci_df(df):
     # Check data types
     for coln, dstr in df.dtypes.items():
         if coln not in colns_dtypes:
-            raise IOError(f'Unrecognized column name in estimate data: \'{coln}\'')
-        if dstr != colns_dtypes[coln]:  # More specific check
-            raise AssertionError(f"Incorrect data type for column '{coln}'. Expected: {colns_dtypes[coln]}, Found: {dstr}")
+            print(f'Unrecognized column name in estimate data: \'{coln}\'')
+        else:
+            if dstr != colns_dtypes[coln]:  # More specific check
+                raise AssertionError(f"Incorrect data type for column '{coln}'. Expected: {colns_dtypes[coln]}, Found: {dstr}")
 
 
 #===============================================================================
@@ -118,6 +120,8 @@ def assert_drf_df(df):
     Raises:
         TypeError: If the input is not a DataFrame or columns have non-float types.
         KeyError: If the DataFrame's index names are incorrect.
+        
+    view(df)
     """
 
     # Check if it's a DataFrame
@@ -128,14 +132,15 @@ def assert_drf_df(df):
     if not set(df.index.names).difference(['cat', 'sel', 'bldg_layout']) == set():
         raise KeyError("Incorrect index names in DataFrame")
 
-    # Check data types
+    # Check the columns all conform to float depths
     if not 'float' in df.columns.dtype.name:
-        raise TypeError('bad type on columns')
+        raise TypeError(f'DRF column headers expected as dtype float. instead got \'{df.columns.dtype.name}\'')
 
     # Check data types (more accurate)
-    for col in df.columns:
-        if df[col].dtype != 'float64':  # Assuming you want specifically float64
-            raise TypeError(f"Column '{col}' is not a float type")
+
+    for i, col in enumerate(df.columns):
+        if not 'float' in df[col].dtype.name:  # Assuming you want specifically float64
+            raise TypeError(f'DRF column \'{col}\' ({i}) expected as dtype float. instead got \'{df[col].dtype}\'')
 
 def assert_bldg_meta_d(bldg_meta):
     """check the bldg_meta_d meets expectations"""

cancurve/bldgs/bldg_meta_rqmts.csv

@@ -1,39 +1,38 @@
-varName_core,varName_ui,varName_canflood,type,required_core,required_canflood,default_canflood,widgetName,case1,case2
-,,tag,str,FALSE,TRUE,?,,test_case1,test_case2
-,,location,str,FALSE,FALSE,?,,,
-,,date,str,FALSE,FALSE,?,,,
-,,source,str,FALSE,FALSE,?,,,
-,currency,impact_units,str,FALSE,TRUE,$CAD,currency_ComboBox,,
-,costBasis,impact_var,str,FALSE,FALSE,damage,costBasis_ComboBox,,
-,,exposure_units,str,FALSE,FALSE,m,comboBox_tab3dataInput_expoUnits,,
-,,exposure_var,str,FALSE,FALSE,flood depth above floor,,,
-,,scale_var,str,FALSE,FALSE,building footprint,,,
-bldg_layout,buildingLayout,,str,TRUE,FALSE,,buildingLayout_ComboBox,default,default
-,basementHeight,,float,FALSE,FALSE,,basementHeight_DoubleSpinBox,1.8,2.1
-,basementHeightUnits,,str,FALSE,FALSE,,basementHeightUnits_ComboBox,m,m
-basement_height_m,,,float,TRUE,FALSE,,,1.8,2.1
-,sizeOrAreaValue,,float,FALSE,FALSE,,sizeOrAreaValue_DoubleSpinBox,232.1,232.2
-,sizeOrAreaUnits,scale_units,str,FALSE,FALSE,m2,sizeOrAreaUnits_ComboBox,m2,m2
-scale_value_m2,,,float,TRUE,FALSE,,,232.1,232.2
-,,exposure,str,FALSE,TRUE,impact,,,
-,occupancyClassification,,str,FALSE,FALSE,,occupancyClassification_ComboBox,,
-,subClassification,,str,FALSE,FALSE,,subClassification_ComboBox,,
-,storeys,,str,FALSE,FALSE,,storeys_ComboBox,,
-,heatingType,,str,FALSE,FALSE,,heatingType_ComboBox,,
-,coolingType,,str,FALSE,FALSE,,coolingType_ComboBox,,
-,garageType,,str,FALSE,FALSE,,garageType_ComboBox,,
-,garageSize,,str,FALSE,FALSE,,garageSize_ComboBox,,
-,buildingLayout,,str,FALSE,FALSE,,buildingLayout_ComboBox,,
-,foundationType,,str,FALSE,FALSE,,foundationType_ComboBox,,
-,qualityOfBuildingMaterials,,str,FALSE,FALSE,,qualityOfBuildingMaterials_ComboBox,,
-,provinceTerritory,,str,FALSE,FALSE,,provinceTerritory_ComboBox,,
-,taxesIncluded,,str,FALSE,FALSE,,taxesIncluded_ComboBox,,
-,priceListSource,,str,,,,,,
-,country,,str,,,,,,
-,BathroomCount,,int,,,,,,
-,pricingDate,,str,,,,,,
-,BedroomCount,,int,,,,,,
-,basementFinish,,str,,,,,,
-,createdBy,,str,,,,,,
-,locationCityTownRegion,,str,,,,,,
-,yearOfBuildingConstruction,,int,,,,,,
+varName_core,varName_ui,varName_canflood,type,required_core,required_canflood,default_canflood,widgetName,case1,case2,case3
+,,tag,str,FALSE,TRUE,?,,test_case1,test_case2,heather_0509
+,,location,str,FALSE,FALSE,?,,,,
+,,date,str,FALSE,FALSE,?,,,,
+,,source,str,FALSE,FALSE,?,,,,
+,currency,impact_units,str,FALSE,TRUE,$CAD,currency_ComboBox,,,
+,costBasis,impact_var,str,FALSE,FALSE,damage,costBasis_ComboBox,,,Depreciated Costs
+,,exposure_units,str,FALSE,FALSE,m,comboBox_tab3dataInput_expoUnits,,,
+,,exposure_var,str,FALSE,FALSE,flood depth above floor,,,,
+,,scale_var,str,FALSE,FALSE,building footprint,,,,
+bldg_layout,buildingLayout,,str,TRUE,FALSE,,buildingLayout_ComboBox,default,default,default
+,basementHeight,,float,FALSE,FALSE,,basementHeight_DoubleSpinBox,1.8,2.1,0
+,basementHeightUnits,,str,FALSE,FALSE,,basementHeightUnits_ComboBox,m,m,m
+basement_height_m,,,float,TRUE,FALSE,,,1.8,2.1,0
+,sizeOrAreaValue,,float,FALSE,FALSE,,sizeOrAreaValue_DoubleSpinBox,232.1,232.2,344
+,sizeOrAreaUnits,scale_units,str,FALSE,FALSE,m2,sizeOrAreaUnits_ComboBox,m2,m2,m2
+scale_value_m2,,,float,TRUE,FALSE,,,232.1,232.2,344
+,,exposure,str,FALSE,TRUE,impact,,,,
+,occupancyClassification,,str,FALSE,FALSE,,occupancyClassification_ComboBox,,,Commercial
+,subClassification,,str,FALSE,FALSE,,subClassification_ComboBox,,,Construction
+,storeys,,str,FALSE,FALSE,,storeys_ComboBox,,,1
+,BedroomCount,,int,FALSE,FALSE,,BedroomCount_QSpinBox,,,0
+,BathroomCount,,float,FALSE,FALSE,,BathroomCount_DoubleSpinBox,,,1
+,heatingType,,str,FALSE,FALSE,,heatingType_ComboBox,,,Forced Air - Gas
+,coolingType,,str,FALSE,FALSE,,coolingType_ComboBox,,,Central air
+,garageType,,str,FALSE,FALSE,,garageType_ComboBox,,,None
+,garageSize,,str,FALSE,FALSE,,garageSize_ComboBox,,,
+,foundationType,,str,FALSE,FALSE,,foundationType_ComboBox,,,other
+,qualityOfBuildingMaterials,,str,FALSE,FALSE,,qualityOfBuildingMaterials_ComboBox,,,Average
+,taxesIncluded,,str,FALSE,FALSE,,taxesIncluded_ComboBox,,,
+,priceListSource,,str,FALSE,FALSE,,priceListSource_LineEdit,,,exactimate
+,country,,str,FALSE,FALSE,Canada,country_ComboBox,,,Canada
+,provinceTerritory,,str,FALSE,FALSE,,provinceTerritory_ComboBox,,,ON
+,pricingDate,,str,FALSE,FALSE,,,,,
+,basementFinish,,float,FALSE,FALSE,,basementFinish_DoubleSpinBox,,,100
+,createdBy,,str,FALSE,FALSE,,createdBy_LineEdit,,,hmcgrath
+,locationCityTownRegion,,str,FALSE,FALSE,,,,,
+,yearOfBuildingConstruction,,int,FALSE,FALSE,,yearOfBuildingConstruction_SpinBox,,,2500