Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Pydantic model for some napari_workflows_wrapper arguments #404

Closed
tcompa opened this issue Jun 8, 2023 · 7 comments · Fixed by #422
Closed

Add Pydantic model for some napari_workflows_wrapper arguments #404

tcompa opened this issue Jun 8, 2023 · 7 comments · Fixed by #422
Labels
High Priority Current Priorities & Blocking Issues JSON Schemas

Comments

@tcompa
Copy link
Collaborator

tcompa commented Jun 8, 2023

As in #386, we should introduce Pydantic models for what is currently only described in the docstring

   """
        # Examples of allowed entries for input_specs and output_specs
        input_specs = {
            "in_1": {"type": "image", "wavelength_id": "A01_C02"},
            "in_2": {"type": "image", "channel_label": "DAPI"},
            "in_3": {"type": "label", "label_name": "label_DAPI"},
        }
        output_specs = {
            "out_1": {"type": "label", "label_name": "label_DAPI_new"},
            "out_2": {"type": "dataframe", "table_name": "measurements"},
        }
   """
@jluethi
Copy link
Collaborator

jluethi commented Jun 8, 2023

That's a great idea!
Would also allow us to then validate the valid types (image, label & dataframe) and what name/id parameter needs to be present with them!

@tcompa tcompa changed the title Introduce Pydantic model for some napari_workflows_wrapper arguments Add Pydantic model for some napari_workflows_wrapper arguments Jun 12, 2023
@tcompa
Copy link
Collaborator Author

tcompa commented Jun 15, 2023

We have a bunch of different "channel" models, to be used in different tasks:

  1. The current Channel, which is the Omero one (used in create_ome_zarr`)
  2. The current BaseChannel (preliminary name), that simply requires either wavelength_id or label and is used in cellpose/napari tasks

I propose to rename 1 to OmeroChannel and 2 to Channel. Any better idea @jluethi?


As per the input/output specs of the napari-workflows task, I'm currently using

@validate_arguments
def napari_workflows_wrapper(
    ...
    input_specs: Dict[str, NapariWorkflowsInput],
    output_specs: Dict[str, NapariWorkflowsOutput],
    ...
):

Once again: better ideas are welcome.

(note that it'd be best to keep names of custom Pydantic models simple/short, since they will likely be displayed somewhere in fractal-web)

@jluethi
Copy link
Collaborator

jluethi commented Jun 15, 2023

I propose to rename 1 to OmeroChannel and 2 to Channel. Any better idea @jluethi?

Agreed.

Regarding NapariWorkflowsInput & NapariWorkflowsOutput: Makes sense. The way we currently use them, we do support different things for Input & Output (i.e. input supports images, output supports tables, both support labels). Long-term, they may become the same.

@tcompa
Copy link
Collaborator Author

tcompa commented Jun 15, 2023

Current models (to be merged with #422 after some clean up):

"""
Pydantic models for some task parameters
"""
from typing import Literal
from typing import Optional

from pydantic import BaseModel
from pydantic import validator


class Channel(BaseModel):
    """
    A channel which is specified by either ``wavelength_id`` or ``label``.
    """

    wavelength_id: Optional[str] = None
    label: Optional[str] = None

    @validator("label", always=True)
    def mutually_exclusive_channel_attributes(cls, v, values):
        wavelength_id = values.get("wavelength_id")
        label = v
        if wavelength_id and v:
            raise ValueError(
                "`wavelength_id` and `label` cannot be both set "
                f"(given {wavelength_id=} and {label=})."
            )
        if wavelength_id is None and v is None:
            raise ValueError(
                "`wavelength_id` and `label` cannot be both `None`"
            )

        return v


class NapariWorkflowsInput(BaseModel):
    """
    A value of the ``input_specs`` argument in ``napari_workflows_wrapper``.
    """

    type: Literal["image", "label"]
    label_name: Optional[str]
    channel: Optional[Channel]

    @validator("label_name", always=True)
    def label_name_is_present(cls, v, values):
        _type = values.get("type")
        if _type == "label" and not v:
            raise ValueError(
                f"Input item has type={_type} but label_name={v}."
            )
        return v

    @validator("channel", always=True)
    def channel_is_present(cls, v, values):
        _type = values.get("type")
        if _type == "image" and not v:
            raise ValueError(f"Input item has type={_type} but channel={v}.")
        return v


class NapariWorkflowsOutput(BaseModel):
    """
    A value of the ``output_specs`` argument in ``napari_workflows_wrapper``.
    """

    type: Literal["label", "dataframe"]
    label_name: Optional[str] = None
    table_name: Optional[str] = None

    @validator("label_name", always=True)
    def label_name_only_for_label_type(cls, v, values):
        _type = values.get("type")
        if (_type == "label" and (not v)) or (_type != "label" and v):
            raise ValueError(
                f"Output item has type={_type} but label_name={v}."
            )
        return v

    @validator("table_name", always=True)
    def table_name_only_for_dataframe_type(cls, v, values):
        _type = values.get("type")
        if (_type == "dataframe" and (not v)) or (_type != "dataframe" and v):
            raise ValueError(
                f"Output item has type={_type} but table_name={v}."
            )
        return v

@jluethi
Copy link
Collaborator

jluethi commented Jun 15, 2023

I don't fully understand what the inputs in label_name_only_for_label_type(cls, v, values) are. But other than that, looks great and very useful that this validation moves outside of the task functions now!

@tcompa
Copy link
Collaborator Author

tcompa commented Jun 15, 2023

v is the value which is the first argument of @validator. For instance for @validator("label_name", always=True) the value of v is the value of the attribute label_name.

values is the set of all the previous attribute values (note: order matter), as you see from the fact that we extract _type via values.get("type").

cls is similar to self, but for classmethods (we're not really using it, but validator is a classmethod and then we have to include it)

@tcompa
Copy link
Collaborator Author

tcompa commented Jun 15, 2023

very useful that this validation moves outside of the task functions now!

Agreed.

In principle we could go a step further and create a more complex set of models such that part of the validation is also exposed in the JSON Schema. I'd refrain from this kind of optimization until we gain a bit more experience of how usable the multi-nested schemas are in fractal-web, but eventually we should revisit the models.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
High Priority Current Priorities & Blocking Issues JSON Schemas
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants