Upgrade validation methods (#1911)

* reconcile io path validator differences

* make io and path output the same

* update tests for io validation

* add test that io and path output is same

* update tests for io output with status

* add validate helper function and fix status tracking

* update arguments and status checks

* fix test comparing io and path outputs

* update error message for io inputs

* add tests to compare all io and path outputs

* remove script for testing

* add initial json export option

* update validator inputs and outputs, remove hdf5io references

* update tests for non status output

* add tests for json validator output

* move pynwb.validate into validation module

* update json report

* add validation entry point

* separate cli and validation function files

* update validation tutorial

* move get_backend to pynwb init

* update example validation for new io behavior

* update ruff ignores

* fix test comments

* update CHANGELOG

* add tests for _get_backend

* update backend imports for optional zarr

* fix test name

* update test filename

* close io after validation

* fix test assertion

* add condition for ros3 validation

* Update CHANGELOG.md

* Apply suggestions from code review

Co-authored-by: Ryan Ly <rly@lbl.gov>

* update cli args and docs

* fix formatting

* update json-file-path argname

* update extension tutorial

* Apply suggestions from code review

Co-authored-by: Ryan Ly <rly@lbl.gov>

* warn for positional args in validation

---------

Co-authored-by: Ryan Ly <rly@lbl.gov>

CHANGELOG.md

@@ -2,6 +2,13 @@
 
 ## PyNWB 3.0.0 (Upcoming)
 
+### Breaking changes
+- The validation methods have been updated with multiple breaking changes. @stephprince [#1911](https://github.com/NeurodataWithoutBorders/pynwb/pull/1911)
+   - The behavior of `pynwb.validate(io=...)` now matches the behavior of `pynwb.validate(path=...)`. In previous pynwb versions, `pynwb.validate(io=...)` did not use the cached namespaces during validation. To obtain the same behavior as in previous versions, you can update the function call to `pynwb.validate(io=..., use_cached_namespaces=False)`
+   - `pynwb.validate` will return only a list of validation errors instead of a tuple: (list of validation_errors, status code)
+   - The validate module has been renamed to `validation.py`. The validate method can be 
+   imported using `import pynwb; pynwb.validate` or `from pynwb import validate`
+
 ### Deprecations
 - The following deprecated classes will now raise errors when creating new instances of these classes: ``ClusteringWaveforms``, ``Clustering``, ``SweepTable``. Reading files using these data types will continue to be supported.
 - The following methods and arguments have been deprecated:
@@ -30,6 +37,9 @@
   - Added support for `model_number`, `model_name`, and `serial_number` fields to `Device`. @stephprince [#1997](https://github.com/NeurodataWithoutBorders/pynwb/pull/1997)
   - Deprecated `EventWaveform` neurodata type. @rly [#1940](https://github.com/NeurodataWithoutBorders/pynwb/pull/1940)
   - Deprecated `ImageMaskSeries` neurodata type. @rly [#1941](https://github.com/NeurodataWithoutBorders/pynwb/pull/1941)
+- Added enhancements to the validation CLI. @stephprince [#1911](https://github.com/NeurodataWithoutBorders/pynwb/pull/1911)
+  - Added an entry point for the validation module. You can now use `pynwb-validate "file.nwb"`.
+  - Added the `--json-outpath-path` CLI argument to output validation results in a machine readable format.
 - Removed python 3.8 support, added python 3.13 support. @stephprince [#2007](https://github.com/NeurodataWithoutBorders/pynwb/pull/2007)
 - Added warnings when using positional arguments in `Container` constructor methods. Positional arguments will raise errors in the next major release. @stephprince [#1972](https://github.com/NeurodataWithoutBorders/pynwb/pull/1972)
 

docs/gallery/general/extensions.py

@@ -37,8 +37,7 @@
 ns_builder = NWBNamespaceBuilder(
     "Extension for use in my Lab", "mylab", version="0.1.0"
 )
-
-ns_builder.include_type("ElectricalSeries", namespace="core")
+ns_builder.include_namespace("core")
 
 ext = NWBGroupSpec(
     "A custom ElectricalSeries for my lab",
@@ -264,7 +263,7 @@ def __init__(self, **kwargs):
 ext_source = name + ".extensions.yaml"
 
 ns_builder = NWBNamespaceBuilder(name + " extensions", name, version="0.1.0")
-ns_builder.include_type("NWBDataInterface", namespace="core")
+ns_builder.include_namespace("core")
 
 potato = NWBGroupSpec(
     neurodata_type_def="Potato",

docs/source/validation.rst

@@ -3,12 +3,21 @@
 Validating NWB files
 ====================
 
+.. note:: 
+
+  The pynwb validation CLI checks for structural compliance of NWB files with the NWB schema. 
+  It is recommended to use the `NWBInspector CLI <https://nwbinspector.readthedocs.io/en/dev/>`_ 
+  for more comprehensive validation of both structural compliance with the NWB schema and 
+  compliance of data with NWB best practices. The NWBInspector runs both PyNWB validation as 
+  described here and additional data checks.
+
+
 Validating NWB files is handled by a command-line tool available in :py:mod:`~pynwb`.
 The validator can be invoked like so:
 
 .. code-block:: bash
 
-  python -m pynwb.validate test.nwb
+  pynwb-validate test.nwb
 
 If the file contains no NWB extensions, then this command will validate the file ``test.nwb`` against the
 *core* NWB specification. On success, the output will be:
@@ -29,35 +38,36 @@ within the ``test.nwb`` file.
 
 .. code-block:: bash
 
-  python -m pynwb.validate -n ndx-my-extension test.nwb
+  pynwb-validate -n ndx-my-extension test.nwb
 
 To validate against the version of the **core** NWB specification that is included with the installed version of
 PyNWB, use the ``--no-cached-namespace`` flag. This can be useful in validating files against newer or older versions
 of the **core** NWB specification that are installed with newer or older versions of PyNWB.
 
 .. code-block:: bash
 
-  python -m pynwb.validate --no-cached-namespace test.nwb
+  pynwb-validate --no-cached-namespace test.nwb
 
 .. Last updated 8/13/2021
 .. code-block:: text
 
-  $python -m pynwb.validate --help
-  usage: validate.py [-h] [-n NS] [-lns] [--cached-namespace | --no-cached-namespace] paths [paths ...]
+  $pynwb-validate --help
+  usage: pynwb-validate [-h] [-lns] [-n NS] [--json-output-path JSON_OUTPUT_PATH] [--no-cached-namespace] paths [paths ...]
 
   Validate an NWB file
 
   positional arguments:
     paths                 NWB file paths
 
-  optional arguments:
+  options:
     -h, --help            show this help message and exit
-    -n NS, --ns NS        the namespace to validate against
     -lns, --list-namespaces
                           List the available namespaces and exit.
-    --cached-namespace    Use the cached namespace (default).
+    -n NS, --ns NS        the namespace to validate against
+    --json-output-path JSON_OUTPUT_PATH
+                          Write json output to this location.
     --no-cached-namespace
-                          Don't use the cached namespace.
+                          Use the namespaces installed by PyNWB (true) or use the cached namespaces (false; default).
 
   If --ns is not specified, validate against all namespaces in the NWB file.
 

pyproject.toml

@@ -48,6 +48,9 @@ dynamic = ["version"] # the build backend will compute the version dynamically f
 "Homepage" = "https://github.com/NeurodataWithoutBorders/pynwb"
 "Bug Tracker" = "https://github.com/NeurodataWithoutBorders/pynwb/issues"
 
+[project.scripts]
+pynwb-validate = "pynwb.validation_cli:validation_cli"
+
 [tool.hatch.version]
 source = "vcs"
 
@@ -111,7 +114,7 @@ line-length = 120
 "docs/gallery/*" = ["E402", "T201"]
 "src/*/__init__.py" = ["F401"]
 "src/pynwb/_version.py" = ["T201"]
-"src/pynwb/validate.py" = ["T201"]
+"src/pynwb/validation_cli.py" = ["T201"]
 "scripts/*" = ["T201"]
 
 # "test_gallery.py" = ["T201"] # Uncomment when test_gallery.py is created

src/pynwb/__init__.py

@@ -23,7 +23,7 @@
 CORE_NAMESPACE = 'core'
 
 from .spec import NWBDatasetSpec, NWBGroupSpec, NWBNamespace  # noqa E402
-from .validate import validate  # noqa: F401, E402
+from .validation import validate  # noqa: F401, E402
 
 try:
     # see https://effigies.gitlab.io/posts/python-packaging-2023/
@@ -344,6 +344,27 @@ def get_sum(self, a, b):
     return __TYPE_MAP.get_dt_container_cls(neurodata_type, namespace)
 
 
+@docval({'name': 'path', 'type': str, 'doc': 'Path to the NWB file which can be an HDF5 file or a Zarr store.'},
+        {"name": "method", "type": str, "doc": "the method to use when opening the file", 'default': None},
+        is_method=False)
+def _get_backend(path: str, method: str = None):
+    if method == "ros3":
+        return NWBHDF5IO  # TODO - add additional conditions for other streaming methods
+
+    try:
+        from hdmf_zarr import NWBZarrIO
+        backend_io_classes = [NWBHDF5IO, NWBZarrIO]
+    except ImportError:
+        backend_io_classes = [NWBHDF5IO]
+
+    backend_options = [b for b in backend_io_classes if b.can_read(path=path)]
+    if len(backend_options) == 0:
+        raise ValueError(f"Could not find an IO to read the file '{path}'. If you are trying to read "
+                         f"a Zarr file, make sure you have hdmf-zarr installed.")
+    else:
+        return backend_options[0]
+
+
 class NWBHDF5IO(_HDF5IO):
 
     @staticmethod

src/pynwb/testing/testh5io.py

@@ -163,12 +163,12 @@ def getContainer(self, nwbfile):
     def validate(self):
         """ Validate the created files """
         if os.path.exists(self.filename):
-            errors, _ = pynwb_validate(paths=[self.filename])
+            errors = pynwb_validate(path=self.filename)
             if errors:
                 raise Exception("\n".join(errors))
 
         if os.path.exists(self.export_filename):
-            errors, _ = pynwb_validate(paths=[self.export_filename])
+            errors = pynwb_validate(path=self.export_filename)
             if errors:
                 raise Exception("\n".join(errors))
 
@@ -366,11 +366,11 @@ def roundtripExportContainer(self, cache_spec=True):
     def validate(self):
         """Validate the created files."""
         if os.path.exists(self.filename):
-            errors, _ = pynwb_validate(paths=[self.filename])
+            errors = pynwb_validate(path=self.filename)
             if errors:
                 raise Exception("\n".join(errors))
 
         if os.path.exists(self.export_filename):
-            errors, _ = pynwb_validate(paths=[self.export_filename])
+            errors = pynwb_validate(path=self.export_filename)
             if errors:
                 raise Exception("\n".join(errors))