How can fpm better handle the external link dependencies of fpm projects?

[zoziha]

Introduction

From my personal experience using fpm, I have found that currently, fpm's usage experience in handling external link dependencies and operating system environments is relatively troublesome. The former involves two situations:

1. External link libraries without mod files compiled by Fortran 77, including OpenBLAS.
2. External link libraries with mod files compiled by Fortran 90, including hdf5, netcdf, fgsl, and fortran-stdlib.

These libraries may be libraries of the open source community or inhouse link libraries of small teams. Currently, fpm cannot handle external link dependencies with mod files well, mainly involving the following three points:

1. -l/-L to link the link library;
2. -I to introduce the header file path;
3. Use build.external-modules in fpm.toml to declare external mod file references.

Hope brought by the hdf5 meta-package and pkg-config

The implementation of the hdf5 meta-package, through pkg-config, has achieved references to external packages such as hdf5. This enables us to use hdf5 as a meta-package and thus correctly link the hdf5 link library. Meson and CMake also have the ability to query external dependencies through pkg-config. Using pkg-config is a good way forward.

However, compared to Meson, fpm needs to explicitly declare the correct build.external-modules. Otherwise, fpm will report an error, which is equivalent to fpm intercepting such errors instead of the compiler. This leads to when packages with Fortran 90 interfaces such as hdf5 and netcdf are used as meta-packages of fpm, we need to hardcode all the *.mod names of packages like hdf5 into fpm. This leads to a problem. Fpm cannot infinitely increase potential meta-packages and constantly track and record build.external-modules. Moreover, the fpm team also needs to test them in CI.

Above all, this raises two questions:

1. Should fpm, like Meson and CMake, open the built-in pkg-config usage method of fpm to users? For example:

   fortran_stdlib = "*"  # Check if fortran_stdlib is a meta-package. If not, look for the external dependency `fortran_stdlib.pc`

2. Should fpm consider unfound mod references as belonging to some external link dependency, (perhaps with a warning) but not treat this problem as an error and instead leave it to the compiler to report an error?

Statement: I'm not quite sure if my description of CMake is correct. I only have a basic understanding of it.

[perazz]

we need to hardcode all the *.mod names of packages like hdf5 into fpm.

Regarding the HDF5 metapackage, this is not the case: the metapackage automatically defines all HDF5 Fortran modules as external:

fpm/src/fpm_meta.f90

Lines 1879 to 1900 in 448d47c

	!> Add HDF5 modules as external
	this%has_external_modules = .true.
	this%external_modules = [string_t('h5a'), &
	string_t('h5d'), &
	string_t('h5es'), &
	string_t('h5e'), &
	string_t('h5f'), &
	string_t('h5g'), &
	string_t('h5i'), &
	string_t('h5l'), &
	string_t('h5o'), &
	string_t('h5p'), &
	string_t('h5r'), &
	string_t('h5s'), &
	string_t('h5t'), &
	string_t('h5vl'), &
	string_t('h5z'), &
	string_t('h5lib'), &
	string_t('h5global'), &
	string_t('h5_gen'), &
	string_t('h5fortkit'), &
	string_t('hdf5')]

If there are any missing module names, please open an Issue page so we can implement fixes as needed. The examples in the CI test for the base hdf5 module and hdf5 library initialization, so they should work on all platforms.

[zoziha]

Yes, I saw this code before. For well-known packages like hdf5 as meta-packages, the fpm team hardcodes the *.mod names of these packages into the fpm source code, so users no longer need to declare them in build.external-modules in fpm.toml. This requires the fpm team to continuously maintain the *.mod list of hdf5, reducing the pressure on users. However, for other less-known packages, the fpm team may not be able to take care of them. So it seems best to allow users to access any possible pkg-config package through fpm and eliminate the trouble for users to declare build.external-modules themselves. That is, if a *.mod cannot be found, fpm does not report an error?

[perazz]

That is, if a *.mod cannot be found, fpm does not report an error?

Correct: external modules are those for which a source file is not available.