Add Java binding using the existing SWIG infrastructure

[kkofler]

CMakeLists.txt: Add NLOPT_JAVA option. If set, check for C++, JNI, and Java.

src/swig/nlopt-java.i: New SWIG input file (not standalone, but to be included into nlopt.i). Contains the Java-specific declarations.

src/swig/nlopt.i: Add syntax highlighting declaration for KatePart-based editors. Instantiate vector as doublevector instead of nlopt_doublevector for SWIGJAVA, because everything is in an nlopt package in Java, so the nlopt_ part is redundant. Ignore nlopt_get_initial_step because at least for Java, SWIG wants to generate a binding for it, using SWIGTYPE_p_* types that cannot be practically used. (It is in-place just like nlopt::opt::get_initial_step, which is already ignored.) For SWIGJAVA, %include nlopt-java.i.

src/swig/CMakeLists.txt: Set UseSWIG_MODULE_VERSION to 2 so that UseSWIG cleans up old generated source files before running SWIG, useful for Java. (In particular, it prevents obsolete .java files from old .i file versions, which might not even compile anymore, from being included in the Java compilation and the JAR.) If JNI, Java, and SWIG were found, generate the Java binding with SWIG, and compile the JNI library with the C++ compiler and the Java JAR with the Java compiler.

src/swig/glob_java.cmake: New helper script whose only purpose is to invoke the CMake file(GLOB ...) command at the correct stage of the build process, after the SWIG run, not when CMake is run on the main CMakeLists.txt, because the latter happens before anything at all is built. The script is invoked through cmake -P by an add_custom_command in CMakeLists.txt, whose dependencies order it to the correct spot of the build. This is the only portable way to automatically determine which *.java files SWIG has generated from the *.i files. The result is written to java_sources.txt, which is dynamically read by the add_jar command thanks to the @ indirection.

test/t_java.java: New test. Java port of t_tutorial.cxx/t_python.py.

test/CMakeLists.txt: If JNI, Java >= 1.8, and SWIG were found, run the t_java test program with the algorithms 23 (MMA), 24 (COBYLA), 30 (augmented Lagrangian), and 39 (SLSQP). All 4 tests pass. (Java < 1.8 should be supported by the binding itself, but not by the test.)

This code is mostly unrelated to the old unmaintained nlopt4j binding (https://github.com/dibyendumajumdar/nlopt4j), which used handwritten JNI code. Instead, it reuses the existing SWIG binding infrastructure, adding Java as a supported language to that. Only the code in the func_java and mfunc_java wrappers is loosely inspired by the nlopt4j code. The rest of the code in this binding relies mostly on SWIG features and uses very little handwritten JNI code, so nlopt4j was not useful as a reference for it. This binding is also backwards-incompatible with nlopt4j due to differing naming conventions.

Note that this binding maps the C++ class and method names identically to Java. As a result, it does not use idiomatic Java case conventions, but uses snake_case for both class and method names instead of the usual UpperCamelCase for class names and lowerCamelCase for method names in Java. The C++ namespace nlopt is mapped to the Java package nlopt.

[kkofler]

Does the binding make sense to you as is? Or do you think I should use the SWIG %rename feature aggressively to convert all the class and method names to the Java camel-case conventions?

[stevengj]

It would be nice to make the naming somewhat more idiomatic for Java. On the other hand, it adds maintenance overhead if we expand the API in the future.

[kkofler]

In the new revision (commit 32a4a64), I have fixed all the actual bugs you have found.

What is still unresolved is:

• the missing documentation – I will look into that some time next week.
• deciding how to handle the minimum Java version – I am not a huge fan of requiring a newer version than actually required, though I can see why you do not like allowing an older version than what the test requires, since it cannot be automatically tested. (Technically, one could build the binding with, e.g., JDK 1.6, then build and run the test with JDK 1.8, but not in one CMake run.) JDK 1.8 is the oldest we care about at my employer, so it should not be a big issue if it were required.

[jschueller]

* deciding how to handle the minimum Java version

this was not a strong requirement, just a remark to make the code simpler, if you feel java < 1.6 will be useful you can leave the code as is

[kkofler]

It would be nice to make the naming somewhat more idiomatic for Java. On the other hand, it adds maintenance overhead if we expand the API in the future.

It turns out that SWIG supports global renaming directives that can do most of the work, as in:

// use Java naming conventions
%rename("%(camelcase)s", match="enum") "";
%rename("%(camelcase)s", match="class") "";
%rename("%(lowercamelcase)s", %$isfunction) "";

so I have implemented just that in the latest revision, commit bb97515.

I needed only a handful other changes (with 2 added #ifdefs in nlopt.i, the rest is centralized in nlopt_java.i) on top of those.

[kkofler]

The latest revision (commit 7f09d01) should hopefully also fix the CI failure on Windows. (The one on macOS was due to the Guile issue that I fixed in the previous revision.) The problem was that the classpath was using the wrong separator between paths on Windows. (It was hardcoded to :, most software on Windows uses ; instead because : is used for the drive letter.)

[kkofler]

Hmmmph, Windows is still unhappy, probably because CMake is munging the ; somewhere.

Also, I noticed strict-aliasing warnings in the GNU/Linux CI run, nloptJAVA_wrap.cxx needs to be built with -fno-strict-aliasing (I have not seen those warnings locally), I need to look into that, too. (And of course the documentation that is still missing.)

[kkofler]

Hmmmph, Windows is still unhappy, probably because CMake is munging the ; somewhere.

This and …

Also, I noticed strict-aliasing warnings in the GNU/Linux CI run, nloptJAVA_wrap.cxx needs to be built with -fno-strict-aliasing (I have not seen those warnings locally)

… this should both be fixed in the latest revision (commit 4cab95d).

[kkofler]

The latest revision (commit 8c481a9) now simplifies the syntax of the DoubleVector constructor so that you can write, e.g.:

DoubleVector x0 = new DoubleVector(1.234, 5.678);

instead of:

DoubleVector x0 = new DoubleVector(new double[]{1.234, 5.678});

This is done by using the varargs syntax double... initialElements introduced in Java 1.5 instead of the array syntax double[] initialElements. Note that, with Java varargs, you can still pass an explicitly constructed array (so both snippets above are now valid), useful if you already have an array.

This raises the minimum Java version for the Java binding to 1.5. (The test program still requires 1.8.) For some perspective, Java 1.5 was released in 2004 and reached EOL in 2009, Java 1.4 (which is no longer supported) was released in 2002 and reached EOL in 2008.

SWIG is unfortunately still unconditionally using the legacy Java 1.4 syntax in 2024 and requiring a manual replacement in the output file to fix this. (The [] is hardcoded in a %proxycode block in std_vector.i, so it cannot even be customized via %typemap.) Backwards compatibility of the generated output is not even an argument, because as I mentioned above, varargs also accept an explicit array, so it really seems to be about not wanting to require Java 1.5 in SWIG. So I added the replacement to the glob_java.cmake snippet that runs at the right spot of the build and is needed anyway.

EDIT: And actually, the binding already required 1.5 anyway because it uses Java enums.

[kkofler]

The Java tests are still failing on Windows, and I do not know why it (still) does not find the class in the classpath there. I do not have a Windows computer to test with locally.

[kkofler]

There are actually several issues with the Windows CI build, e.g., it explicitly installs NumPy with pip install numpy, but then CMake does not find it. There is also some peculiar behavior by MSBuild, or the CMake support for it: There are a bunch of errors occurring during the build and it just proceeds, triggering more errors. And it seems to also not honor ordering dependencies.

[jschueller]

I fixed the numpy detection on windows, but dlls are still manually copied to the swig binary dir, so if the target dir changes you will have to modify the destination

[kkofler]

The latest revision (commit fe30d98) adds the missing documentation that was requested in the review.

It also fixes the exception handling to work as in Python (noticed while adapting the documentation): Runtime exceptions thrown by the Java callbacks now get rethrown as is (with the original class, message, and stack trace) rather than as a ForcedStopException that lost all the original information. (Another way in Java would be to set the original exception as a cause in the ForcedStopException, but that is mostly useful when wrapping checked exceptions into a runtime exception or into another checked exception class. It is unnecessary for runtime exceptions, where we can avoid wrapping the exception to begin with.)

[jschueller]

seems there is a conflict with master (I fixed the numpy detection on the CI and improved it on the cmake side)

[kkofler]

What I still do not fully understand is the reason for the Windows CI failure. As far as I can tell, the first error is this:

  Generating java_sources.txt
  CMake Error at glob_java.cmake:11 (file):
    file failed to open for reading (No such file or directory):
  
      D:/a/nlopt/nlopt/src/swig/java/nlopt/DoubleVector.java

which means that the add_custom_command rule for java_sources.txt executes too early, before the SWIG run that generates those .java files (also the other ones that should end up listed in java_sources.txt). I believe that the remaining errors are probably followup errors.

The rule has a DEPENDS on nlopt_java_swig_compilation. On the "UNIX Makefiles" target, this appears to be sufficient to get the correct ordering, but not on the MSBuild target.

[kkofler]

The latest revision (commit f4a1e4e) attempts to fix this, but I cannot test it on Windows, so I can only hope for the best.

What this does is add a dependency on "${swig_generated_file_fullname}", which is a variable set by UseSWIG.cmake to the main SWIG output file (e.g., in this case, src/swig/nloptJAVA_wrap.cxx), since, looking at UseSWIG.cmake, it looks like nlopt_java_swig_compilation is only used for Makefile targets. In addition, it also makes this the MAIN_DEPENDENCY, because the documentation for add_custom_command says that doing that "also suggests to Visual Studio generators where to hang the custom command." So I really hope the ordering will be right now.

To be clear, this is all I changed, ignoring the 2 slightly botched intermediate revisions:
https://github.com/stevengj/nlopt/compare/427ca219a7ebd670513b8f0a4229facddb7dd229..f4a1e4efcc3aa3bb056ba53514911795ccfe2889

[kkofler]

Hmmmph, the added MAIN_DEPENDENCY is definitely not good, it now generates an empty libnloptjni.so for me! Looks like I forgot to run the tests.

[kkofler]

So I have now made that dependency a regular DEPENDS. No idea whether it will be good enough for Windows, but at least it does not break GNU/Linux.

Using it as a MAIN_DEPENDENCY violates this constraint: "Each source file may have at most one command specifying it as its main dependency." that I had overlooked.

[kkofler]

The latest revision (commit 819ee46) fixes the build with CMake < 3.24, e.g, I tried it in a CentOS 7 chroot with the cmake3 from EPEL, which is 3.17.5. (The regular cmake on the ancient CentOS 7 is 2.8.12.2 and too old to build NLopt altogether.)

[jschueller]

great!
just a rebase to master and it should pass CI

[kkofler]

Rebased now.