From 3c23aeeda5dbe24384ff48f7b426a3bed92c0624 Mon Sep 17 00:00:00 2001 From: AMZN-Gene Date: Tue, 19 Dec 2023 16:20:37 -0800 Subject: [PATCH 1/7] Add documentation for setting up packages using prebuilt libs Signed-off-by: AMZN-Gene --- README.md | 42 +++++++++++++++++++++++++++++++++++++++++- 1 file changed, 41 insertions(+), 1 deletion(-) diff --git a/README.md b/README.md index bffe86b6..b7539069 100644 --- a/README.md +++ b/README.md @@ -16,4 +16,44 @@ Some notable examples * Lua - this one uses a script called pull_and_build_from_git.py (in Scripts/extras) to build the package image. See the documentation (README.md in 3p-package-scripts repo for a full description of how to author packages.) - \ No newline at end of file + +## Setup Packages Using Prebuilt Libraries +3rdParty packages depend on prebuilt libraries. This allows 3rdParty authors to ship their libraries (.dll, .lib, .so, .a, etc) so customers do not need source code. This section covers how to author 3rdParty packages to avoid linker errors. + +### The Problem + +There was problems early on with O3DE 3rdParty libraries. Many 3rdParties used [CMake interface libraries](https://cmake.org/cmake/help/latest/command/add_library.html#interface-libraries). The problem is that interfaces can only control their dependencies, not the hierarchy. For example, the O3DE LibTIFF 3rdParty depends on a prebuilt libtiff.a, and libtiff.a depends on ZLib. As an interface, the old LibTIFF used `target_link_libraries` to link in libtiff.a and ZLib. + +``` + +add_library(TIFF::TIFF INTERFACE IMPORTED GLOBAL) + +target_link_libraries(TIFF::TIFF INTERFACE ZLIB::ZLIB "${TIFF_LIBRARY}") # No actual dependency between ZLib and TIFF and thus has undefined link order. + +``` + +This is a flat dependency list, and so there was no way to tell that libtiff.a depends on ZLib. When CMake generates a Makefile it is free to list those libraries in any order. Depending on the order, libtiff could fail find ZLib definitions. As a result, a program using the LibTIFF 3rdParty would fail to link. + +### The Proper Way to Declare a Prebuilt Library Dependency + +Instead of INTERFACE, use whatever library target type has been prebuilt. + +- `add_library( STATIC IMPORTED)` for a static library + +- `add_library( SHARED IMPORTED)` for a shared library + +- `add_library( MODULE IMPORTED)` for a runtime loadable-ONLY library + +Today's LibTIFF is a proper example of how to declare static library dependencies. The 3rdParty [specifies the path to the library file on disk](https://cmake.org/cmake/help/latest/prop_tgt/IMPORTED_LOCATION.html). In this case, 3rdParty::TIFF points to the prebuilt libtiff.a. The 3rdParty, now acting as a wrapper, can tack on dependencies required by the static library. + +``` + +# Add the CMake standard TIFF::TIFF library. It is a static library. + +add_library(3rdParty::TIFF STATIC IMPORTED GLOBAL) + +set_target_properties(3rdParty::TIFF PROPERTIES IMPORTED_LOCATION "${TIFF_LIBRARY}") + +target_link_libraries(3rdParty::TIFF INTERFACE ZLIB::ZLIB) + +``` From 071878d8594a97953138cb1130683e3a9ddd585d Mon Sep 17 00:00:00 2001 From: AMZN-Gene Date: Tue, 19 Dec 2023 16:22:59 -0800 Subject: [PATCH 2/7] Minor, whitespace edit Signed-off-by: AMZN-Gene --- README.md | 2 -- 1 file changed, 2 deletions(-) diff --git a/README.md b/README.md index b7539069..ab0f7359 100644 --- a/README.md +++ b/README.md @@ -21,7 +21,6 @@ Some notable examples 3rdParty packages depend on prebuilt libraries. This allows 3rdParty authors to ship their libraries (.dll, .lib, .so, .a, etc) so customers do not need source code. This section covers how to author 3rdParty packages to avoid linker errors. ### The Problem - There was problems early on with O3DE 3rdParty libraries. Many 3rdParties used [CMake interface libraries](https://cmake.org/cmake/help/latest/command/add_library.html#interface-libraries). The problem is that interfaces can only control their dependencies, not the hierarchy. For example, the O3DE LibTIFF 3rdParty depends on a prebuilt libtiff.a, and libtiff.a depends on ZLib. As an interface, the old LibTIFF used `target_link_libraries` to link in libtiff.a and ZLib. ``` @@ -35,7 +34,6 @@ target_link_libraries(TIFF::TIFF INTERFACE ZLIB::ZLIB "${TIFF_LIBRARY}") # No a This is a flat dependency list, and so there was no way to tell that libtiff.a depends on ZLib. When CMake generates a Makefile it is free to list those libraries in any order. Depending on the order, libtiff could fail find ZLib definitions. As a result, a program using the LibTIFF 3rdParty would fail to link. ### The Proper Way to Declare a Prebuilt Library Dependency - Instead of INTERFACE, use whatever library target type has been prebuilt. - `add_library( STATIC IMPORTED)` for a static library From 61326f8ed9fa54d8d964d7384b19906a9b256b01 Mon Sep 17 00:00:00 2001 From: AMZN-Gene Date: Tue, 19 Dec 2023 16:38:15 -0800 Subject: [PATCH 3/7] Minor, whitespace edit Signed-off-by: AMZN-Gene --- README.md | 5 ----- 1 file changed, 5 deletions(-) diff --git a/README.md b/README.md index ab0f7359..0694cd2d 100644 --- a/README.md +++ b/README.md @@ -24,11 +24,9 @@ Some notable examples There was problems early on with O3DE 3rdParty libraries. Many 3rdParties used [CMake interface libraries](https://cmake.org/cmake/help/latest/command/add_library.html#interface-libraries). The problem is that interfaces can only control their dependencies, not the hierarchy. For example, the O3DE LibTIFF 3rdParty depends on a prebuilt libtiff.a, and libtiff.a depends on ZLib. As an interface, the old LibTIFF used `target_link_libraries` to link in libtiff.a and ZLib. ``` - add_library(TIFF::TIFF INTERFACE IMPORTED GLOBAL) target_link_libraries(TIFF::TIFF INTERFACE ZLIB::ZLIB "${TIFF_LIBRARY}") # No actual dependency between ZLib and TIFF and thus has undefined link order. - ``` This is a flat dependency list, and so there was no way to tell that libtiff.a depends on ZLib. When CMake generates a Makefile it is free to list those libraries in any order. Depending on the order, libtiff could fail find ZLib definitions. As a result, a program using the LibTIFF 3rdParty would fail to link. @@ -45,13 +43,10 @@ Instead of INTERFACE, use whatever library target type has been prebuilt. Today's LibTIFF is a proper example of how to declare static library dependencies. The 3rdParty [specifies the path to the library file on disk](https://cmake.org/cmake/help/latest/prop_tgt/IMPORTED_LOCATION.html). In this case, 3rdParty::TIFF points to the prebuilt libtiff.a. The 3rdParty, now acting as a wrapper, can tack on dependencies required by the static library. ``` - # Add the CMake standard TIFF::TIFF library. It is a static library. - add_library(3rdParty::TIFF STATIC IMPORTED GLOBAL) set_target_properties(3rdParty::TIFF PROPERTIES IMPORTED_LOCATION "${TIFF_LIBRARY}") target_link_libraries(3rdParty::TIFF INTERFACE ZLIB::ZLIB) - ``` From 9994547a8a7237746312a57872e1fd36a394e006 Mon Sep 17 00:00:00 2001 From: AMZN-Gene Date: Tue, 19 Dec 2023 16:46:50 -0800 Subject: [PATCH 4/7] Updated the first paragraph to better describe who would want to read this section Signed-off-by: AMZN-Gene --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index 0694cd2d..c536cf82 100644 --- a/README.md +++ b/README.md @@ -18,7 +18,7 @@ Some notable examples See the documentation (README.md in 3p-package-scripts repo for a full description of how to author packages.) ## Setup Packages Using Prebuilt Libraries -3rdParty packages depend on prebuilt libraries. This allows 3rdParty authors to ship their libraries (.dll, .lib, .so, .a, etc) so customers do not need source code. This section covers how to author 3rdParty packages to avoid linker errors. +3rdParty packages depend on prebuilt libraries. This allows 3rdParty authors to ship their libraries (.dll, .lib, .so, .a, etc) so customers do not need source code. This section covers how to author packages that depend on prebuilt libraries who also have dependencies of their own. ### The Problem There was problems early on with O3DE 3rdParty libraries. Many 3rdParties used [CMake interface libraries](https://cmake.org/cmake/help/latest/command/add_library.html#interface-libraries). The problem is that interfaces can only control their dependencies, not the hierarchy. For example, the O3DE LibTIFF 3rdParty depends on a prebuilt libtiff.a, and libtiff.a depends on ZLib. As an interface, the old LibTIFF used `target_link_libraries` to link in libtiff.a and ZLib. From f2f793efb9c55536510f41a762b4ecbbc37f16f5 Mon Sep 17 00:00:00 2001 From: AMZN-Gene Date: Tue, 19 Dec 2023 16:49:36 -0800 Subject: [PATCH 5/7] LibTIFF proper noun Signed-off-by: AMZN-Gene --- README.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/README.md b/README.md index c536cf82..62300a92 100644 --- a/README.md +++ b/README.md @@ -29,10 +29,10 @@ add_library(TIFF::TIFF INTERFACE IMPORTED GLOBAL) target_link_libraries(TIFF::TIFF INTERFACE ZLIB::ZLIB "${TIFF_LIBRARY}") # No actual dependency between ZLib and TIFF and thus has undefined link order. ``` -This is a flat dependency list, and so there was no way to tell that libtiff.a depends on ZLib. When CMake generates a Makefile it is free to list those libraries in any order. Depending on the order, libtiff could fail find ZLib definitions. As a result, a program using the LibTIFF 3rdParty would fail to link. +This is a flat dependency list, and so there was no way to tell that libtiff.a depends on ZLib. When CMake generates a Makefile it is free to list those libraries in any order. Depending on the order, LibTIFF could fail find ZLib definitions. As a result, a program using the LibTIFF 3rdParty would fail to link. ### The Proper Way to Declare a Prebuilt Library Dependency -Instead of INTERFACE, use whatever library target type has been prebuilt. +Instead of using INTERFACE, use whatever library target type has been prebuilt. - `add_library( STATIC IMPORTED)` for a static library From ee114645e51d41712c27dd982216b269ba1202f3 Mon Sep 17 00:00:00 2001 From: AMZN-Gene Date: Tue, 19 Dec 2023 16:50:57 -0800 Subject: [PATCH 6/7] TIFF::TIFF aliases to 3rdParty::TIFF. I think the 3rdParty version is easier for readers to understand Signed-off-by: AMZN-Gene --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index 62300a92..c2c6fbbf 100644 --- a/README.md +++ b/README.md @@ -43,7 +43,7 @@ Instead of using INTERFACE, use whatever library target type has been prebuilt. Today's LibTIFF is a proper example of how to declare static library dependencies. The 3rdParty [specifies the path to the library file on disk](https://cmake.org/cmake/help/latest/prop_tgt/IMPORTED_LOCATION.html). In this case, 3rdParty::TIFF points to the prebuilt libtiff.a. The 3rdParty, now acting as a wrapper, can tack on dependencies required by the static library. ``` -# Add the CMake standard TIFF::TIFF library. It is a static library. +# Add the CMake standard 3rdParty::TIFF library. It is a static library. add_library(3rdParty::TIFF STATIC IMPORTED GLOBAL) set_target_properties(3rdParty::TIFF PROPERTIES IMPORTED_LOCATION "${TIFF_LIBRARY}") From e504d7c5afe3c74644b3c1de484f949ec8f73eda Mon Sep 17 00:00:00 2001 From: AMZN-Gene Date: Tue, 19 Dec 2023 16:53:04 -0800 Subject: [PATCH 7/7] TIFF::TIFF aliases to 3rdParty::TIFF. I think the 3rdParty version is easier for readers to understand Signed-off-by: AMZN-Gene --- README.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/README.md b/README.md index c2c6fbbf..be9da6f3 100644 --- a/README.md +++ b/README.md @@ -24,9 +24,9 @@ Some notable examples There was problems early on with O3DE 3rdParty libraries. Many 3rdParties used [CMake interface libraries](https://cmake.org/cmake/help/latest/command/add_library.html#interface-libraries). The problem is that interfaces can only control their dependencies, not the hierarchy. For example, the O3DE LibTIFF 3rdParty depends on a prebuilt libtiff.a, and libtiff.a depends on ZLib. As an interface, the old LibTIFF used `target_link_libraries` to link in libtiff.a and ZLib. ``` -add_library(TIFF::TIFF INTERFACE IMPORTED GLOBAL) +add_library(3rdParty::TIFF INTERFACE IMPORTED GLOBAL) -target_link_libraries(TIFF::TIFF INTERFACE ZLIB::ZLIB "${TIFF_LIBRARY}") # No actual dependency between ZLib and TIFF and thus has undefined link order. +target_link_libraries(3rdParty::TIFF INTERFACE ZLIB::ZLIB "${TIFF_LIBRARY}") # No actual dependency between ZLib and TIFF and thus has undefined link order. ``` This is a flat dependency list, and so there was no way to tell that libtiff.a depends on ZLib. When CMake generates a Makefile it is free to list those libraries in any order. Depending on the order, LibTIFF could fail find ZLib definitions. As a result, a program using the LibTIFF 3rdParty would fail to link.