Integrations

[Nintorac]

No software can function without a myriad of other components that enhance and support one another. Currently nvflare is a little siloed in this regard, implementing your own; config manager, rbac, authorization, experiment management, job management etc.

A great example of a project that does integrations well is Dagster, https://docs.dagster.io/integrations

It might be beneficial for NVFlare to take some lessons from this approach, build out rather horizontally (in terms of integrating with the existing ecosystem) rather than vertically (building the full ecosystem internally).

Some interesting integrations that I am thinking of at the moment

• Weights and Biases
  In the secure environments where we can expect to be using federated data it is not always a given that the public internet will be routable. This makes troubles for applications such as weights and biases where a connection to their server is required. It would be cool to have an integration that could route experiment tracking from these routeless environments to the weights and biases server.
  
  Obviously a lot of thought and considerations need to be made here to avoid the possibility that a bad actor could use this integration as a vector to exfil data

• Dagster
  Dagster is coming up strong and looks to be the pioneer in the data orchestration space (at least IMO), it would be very cool if a Dagster Ops could be annotated somehow to execute via NVFlare in a remote environment. Users could then build Dagster Graphs that could run Ops that snake through any number of secure environments in order to produce the final result.

• Kubernetes
  In the long term I imagine a flare system that is running thousands of Apps, continuous training, inference and alerting, backtesting and validation etc. As such it may not be feasible to have a client running for all these services all the time. It may be useful for nvflare to be able to execute workflows as Kubernetes jobs, this way the lifetime of the client is the time it take to do the workflow. This could help optimizing resource allocation.

• Hydra and/or OmegaConf
  Not sure if this would be an integration or just a drop in replacement for the current configuration engine but Hydra is very useful as it allows you to write component based configuration and then decide which components to pull in at runtime. You can also use it to enforce type checking and some other nifty features.
  
  Hydra- https://github.com/facebookresearch/hydra
  OmegaConf - https://github.com/omry/omegaconf

• OSO and/or Casbin
  In some instances there may be institutional requirements around assurances for RBAC. In these cases it might be preferable to use a well known engine that has had public scrutiny and/or professional audits. These libraries may also offer extra features that would be distracting to implement for nvflare eg. SSO
  
  Casbin - https://casbin.org/
  OSO - https://docs.osohq.com

One big advantage of having a simple interface for defining integrations is that eventually once the project has enough traction, other projects will write their own integrations which will help to bolster the standing of NVFlare in the ecosystem.

[yanchengnv]

This is one of the design principles we try to follow: defining API specs for the things that might have different ways of doing (e.g. things in the app_common/abstract and in the apis). There will be more of these (e.g. messaging layer).

However we also need to make a judgement call of what we want to do concretely. The task-based interaction of work coordination is such an example; and there are many others.

Nothing is set in stone. It is very possible that what we do today may be abstracted tomorrow (become API spec based) to allow different implementations from others. Config management could be such an example. But this kind of decisions will have to be practical: customer asks, product timeline, other priorities, etc.

About running jobs in Kubernetes, you can do that now. If you look closely, all NVFLARE cares is that there is an executor to run a task assigned by the workflow. It says nothing about how to execute the task and where to run it. You can choose to run it on the client machine directly, or simply submits to a cloud/cluster, and then get the result back to the Controller.

[Nintorac]

Yup, agreed that this is all possible because of the nice modular design of NVFlare.

Just ideating and trying to spark some discussion. There is a value proposition between time and benefit. How much time does it take, how much benefit does it give. Integrating with existing tools will greatly reduce time to get out new features and the benefit of existing tools is that you get to reap in the network effect of their popularity (and the functionality they off of course).

What is the core offering of nvflare and how does that tie into the ecosystem? What is being done to make that core benefit accessible within the existing eco-system? How do the features that already exist tie into the existing eco system?

Please post as well any other integrations in this discussion that the team is planning or thinking of

[YuanTingHsieh]

Our future integration will be inside the "integration" folder: https://github.com/NVIDIA/NVFlare/tree/dev/integration

We have only one integration with MONAI (https://github.com/Project-MONAI/MONAI) right now.

[chesterxgchen]

@Nintorac a lot of great ideas

• Along with W&B, there could be MLFlow. How to perserve the data privacy in this can be a challenge
• Dagster and Airflow are along the same category. This belong to the data and ML pipeline schedules. Not sure how this fit into the FL
• K8s deployment, resource management, job schedule requires cloud native plugins, which needs a lot of work.