Prepare Java release to Maven Central

[javagl]

Addresses #624

There is already some discussion about the related tasks and possible approaches in that issue, mainly starting at this comment. I'll try to summarize the current state of this PR and the open questions.

---

Organizational

One question that is not (yet) addressed with the PR is that of the actual deployment of the JARs to Maven Central. I'm not deeply familiar with (or may have forgotten) some of the details here. Maven is incredibly powerful, and it is possible to handle much of that release with Maven itself. In a perfect world (i.e. one that only involves Java 🙂 ), it is possible to set up Maven in a way so that a single Maven run can 0. define the version number for the release, 1. create a Git tag for this release, 2. check out this tag, 3. perform the build, 4. run the tests, 5. sign the JARs, 6. upload the JARs to the Maven Central 'staging' repository, 7. update the version numbers, ... (and more... heck, it can do anything, for what it's worth...)

However, for libraries that involve native libraries (as it is the case here), there are apparent caveats. For example, the run to build Linux- and Windows binaries can hardly happen on the same machine. (In theory, maybe, yes, but ... now throw in MacOS, and you're screwed...).

One reasonably painless path that I've been taking (for JCuda and JOCL) is:

• Build the native libraries for Windows, Linux, and MacOS, somewhere and somehow
• Throw these natives into a common directory
• During the Maven build: Copy these natives into the target directory, to include them in the JAR files
• Semi-manually sign these JARs
• Upload these JAR files to Maven, manually, via Sonatype, "Bundle upload"

Maybe someone wants to chime in with better approaches. (Preferably, going beyond the comment "You should use a better approach" ... ). In any case - regardless of whether it's fully automatic or semi-manual - there are some questions about authorization and authentification (for signing and the actual upload) where some Khronos Webmaster/Maven-Admin will have to contribute.

The current main state of the pom.xml contains a few fragments of the infrastructure that are currently <!-- commented out --> . Maybe some of that can be re-used?

---

Technical

The current state of this PR contains a few building blocks for the technical part of the deployment.

I added a copy of the LibUtils that I've also been using for JOCL. It is a class that can load native libraries, depending on the target OS and architecture, either from local files, or from a JAR file. In order to do that, it unpacks the files from the JAR into a "temp" directory, and loads them from there. This class can be used to conveniently ensure that the required native libraries are loaded when any of the classes is loaded that uses native functions.

I updated the Maven pom.xml to prepare the handling of the native libraries:

(One detail: I lowered the Java version requirement to 1.8. There is no reason to require Java 11 here...)

I added an execution of the maven-resources-plugin that takes all native libraries that are found in a /nativeLibraries subdirectory of the project, and puts them into the target/lib directory. From there, they will be packed into the JAR. (The libraries should be put into the nativeLibraries directory by the native build processes)

Right now, this is tailored to creating a single JAR that contains all native libraries. (This is the approach that I took for JOCL). Having "one JAR to rule them all" is convenient (e.g. for deployment). But it has drawbacks:

• The resulting JAR may be huge (depending on the size of the native libraries)
• It's not easy to create that JAR on one system (because it needs all the native libraries at once)

A better approach usually is to create dedicated JARs for each of the native libraries. This basically means that there would be

• ktx-0.0.0.jar for the main Java part
• ktx-natives-0.0.0-windows-x86_64.jar containing only the Windows DLL
• ktx-natives-0.0.0-linux-x86_64.jar containing only the Linux natives...
• ...

This does involve some overhead for the POM: The right dependencies have to be added using profiles, but I already did that for JCuda (e.g. in the JCuda parent POM), and the same approach is used for other native libraries (like the LWJGL family of libraries, or java-cpp).

---

The next steps here will be to prepare the build infrastructure for creating the dedicated ktx-natives... libraries where each contains only the natives that are required for the respective OS/architecture.

[javagl]

One point about the naming of the native libraries that may look like a detail, but is pretty important:

The native libraries that are supposed to be packed into the JAR will need a version number suffix, like ktx-4.3.2.dll.

The reason:

When loading a native library in Java with System.loadLibrary("example"), then it will try to load the library (like example.dll) from a variety of possible sources, including the system PATH (environment variable). When someone has installed KTX-Software, then the directory that contains the ktx.dll and ktx-jni.dll will be added to the PATH. So when someone has installed KTX-Software 4.2.0, but uses the Java JAR for version 4.3.0, then the Java bindings would load that "old" ktx-jni.dll from the PATH, and may try to call a function that simply didn't exist in 4.2.0 - causing a crash...

[MarkCallow]

A few things to note:

1. CI is set up so when a release tag is added (e.g. v4.3.2) is added to the repo, it builds and deploys a release with that name. At present the Java part of that deployment consists of copying packages to GitHub releases.
2. The Java build is taking the version number from the CMake variable that is set by the tag. It is important that we do not have to set tags or version numbers anywhere else.
3. The native libraries already are named with full version numbers and said name is available in a CMake variable, LIBKTX_FULL_VERSION, I think. This is standard practice for shared/dynamic libraries. The common name used by developers is a link to the file with the full name.
4. We handle the multiple native builds problem for Python. We have a GitHub Action that is manually run after all builds for a release have completed successfully. It downloads the native libs from GitHub Releases, assembles the packages to upload to PyPI and then uploads them. See .github/workflows/publish-pyktx.yml. We can do something similar for Java. This cannot be automated at present because there is no way to programmatically determine that all builds have completed then rigger this action. If all builds are moved to GitHub Actions it may become possible.

[javagl]

@MarkCallow All that sounds OK to me. It sounds like there could be a path to properly handle the versioning, naming of the native libs, and (if the Travis->GitHub transition progresses) the packaging of the natives into JARs. Ideally, this could include the upload of the JARs to Sonatype/MavenCentral (maybe not strictly required, but certainly desirable).

However, there already are places that carry a version number that would have to be adjusted manually. Specifically, the main pom.xml currently declares the Java version of the KTX libraries to be 4.0.0-SNAPSHOT at

KTX-Software/interface/java_binding/pom.xml

Line 11 in 3038f7c

<revision>4.0.0-SNAPSHOT</revision>

. I preliminarily and manually changed this to 4.3.3-SNAPSHOT in this branch, but we'd have to check where this version will come from when all this is to be automated.

[MarkCallow]

However, there already are places that carry a version number that would have to be adjusted manually. Specifically, the main pom.xml currently declares the Java version of the KTX libraries to be 4.0.0-SNAPSHOT at

KTX-Software/interface/java_binding/pom.xml

Line 11 in 3038f7c

<revision>4.0.0-SNAPSHOT</revision>

. I preliminarily and manually changed this to 4.3.3-SNAPSHOT in this branch, but we'd have to check where this version will come from when all this is to be automated.

revision is overridden when Maven is run because the command run by the CMakeLists.txt is

 ${MAVEN_EXECUTABLE} --quiet -Drevision=${PROJECT_VERSION} -Dmaven.test.skip=true package

PROJECT_VERSION is set to KTX_VERSION in the top-level CMakeLists.txt and KTX_VERSION is the version with major.minor.patch.

So version naming is already handled.

and (if the Travis->GitHub transition progresses) the packaging of the natives into JARs. Ideally, this could include the upload of the JARs to Sonatype/MavenCentral (maybe not strictly required, but certainly desirable).

The Travis->GitHub transition is not essential. As I wrote, Java can use exactly the same techniques as we are using for python to package the native libraries into JARS and upload them to Sonatype/MavenCentral. Take a look at the workflow file I pointed you at.