Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update scripts/Build*.sh to build the libraries listed in the current README #5

Open
wants to merge 3 commits into
base: main
Choose a base branch
from
Open

Update scripts/Build*.sh to build the libraries listed in the current README #5

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
b3fbeb4
Update scripts/BuildLibsRelease.sh to build the libraries listed in t…
nkaminski Dec 3, 2024
a46fe2b
Refactor build scripts for wxWidgets to operate per the README as well
nkaminski Dec 3, 2024
7d61a28
Release build need not test for libdbg
nkaminski Dec 3, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions README.macOS
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -277,9 +277,9 @@ make -j 12 ; make
find . -name "*.a" -exec cp {} /opt/local/libdbg/ \;


libxslwriter:
libxslxwriter:
git clone https://github.com/jmcnamara/libxlsxwriter.git
cd libxlswriter
cd libxlsxwriter
git checkout v1.1.9
cd cmake
export CXXFLAGS="-g -O3 -flto=thin ${XL_TARGETS} ${OSX_VERSION_MIN} "
Expand Down
251 changes: 178 additions & 73 deletions scripts/BuildLibsRelease.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,113 +1,157 @@
sudo mkdir -p /opt/local/lib
sudo mkdir -p /opt/local/bin
#!/bin/bash -e
nkaminski marked this conversation as resolved.
Show resolved Hide resolved

#setup permissions on /opt/local
sudo chgrp -R staff /opt/local*
sudo chmod -R g+w /opt/local*
# System dependency checks...
if [ ! -d /opt/local/lib ] || [ ! -d /opt/local/libdbg ] || [ ! -d /opt/local/bin ] || [ ! -d /opt/local/bin ] ; then
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if you're restricting to bash, more reliable to use [[ instead of test

Copy link
Contributor Author

@nkaminski nkaminski Dec 3, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fair, can refactor.

echo "/opt/local/bin, /opt/local/lib, /opt/local/libdbg and /opt/local/include must exist and be writable!"
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

mild recommend for using printf over echo

here are some sample helpers that will do what you might expect
https://gist.github.com/DanInProgress/e444b7815745da4c1117f78dc9ebb2d3

exit 1
fi
if ! command -v brew > /dev/null ; then
echo 'Some libraries require Homebrew Tools, to install Homebrew:'
echo '/bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)"'
exit 1
fi

export MACOSX_DEPLOYMENT_TARGET=10.14
export MACOSX_DEPLOYMENT_TARGET=${MACOSX_DEPLOYMENT_TARGET:-11.0}
export OSX_VERSION_MIN="-mmacosx-version-min=${MACOSX_DEPLOYMENT_TARGET}"
export ARM64_TARGETS="-target arm64-apple-macos10.14 -arch arm64"
export X86_64_TARGETS="-target x86_64-apple-macos10.14 -arch x86_64"
export ARM64_TARGETS="-target arm64-apple-macos11.0 -arch arm64"
export X86_64_TARGETS="-target x86_64-apple-macos11.0 -arch x86_64"

# Absolute path to directory containing the xLights source tree
# Default assumes that xlights is cloned in your cwd
export XL_DIR="${XL_DIR:-$(realpath ./xlights)}"

# Core count of the build host
export NUM_CPUS="${NUM_CPUS:-$(sysctl -n hw.ncpu)}"
echo "Building with $NUM_CPUS cores"

#need ONE of these lines
export XL_TARGETS="${X86_64_TARGETS} ${ARM64_TARGETS}"
# export XL_TARGETS="${X86_64_TARGETS}"
# export XL_TARGETS="${ARM64_TARGETS}"

# need ONE of these
# export BUILD_HOST=x86_64
export BUILD_HOST=arm
export BUILD_HOST="$(uname -p)"
echo "Build host architecture is $BUILD_HOST"

#Note: Some libraries require Homebrew Tools, to install Homebrew:
/bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)"

#Autotools and LibTool are required from Homebrew:
set -x
# install dependencies from homebrew
brew install automake libtool cmake nasm

echo "libzstd"
git clone https://github.com/facebook/zstd
cd zstd
git checkout v1.5.2
if [ ! -d "zstd" ]; then git clone https://github.com/facebook/zstd ; fi
pushd zstd
git checkout v1.5.6
export CFLAGS="-g -flto=thin ${OSX_VERSION_MIN} ${XL_TARGETS}"
export LDFLAGS="-flto=thin ${OSX_VERSION_MIN} ${XL_TARGETS} "
make clean
make -j 8 HAVE_LZMA=0 HAVE_LZ4=0 lib-mt
make -j ${NUM_CPUS} HAVE_LZMA=0 HAVE_LZ4=0 lib-mt
cp lib/libzstd.a /opt/local/lib
export CFLAGS="-g ${OSX_VERSION_MIN} ${XL_TARGETS}"
export LDFLAGS=" ${OSX_VERSION_MIN} ${XL_TARGETS}"
make clean
make -j ${NUM_CPUS} HAVE_LZMA=0 HAVE_LZ4=0 lib-mt
cp lib/libzstd.a /opt/local/libdbg
unset CFLAGS
nkaminski marked this conversation as resolved.
Show resolved Hide resolved
unset LDFLAGS
popd

echo "log4cpp"
#Download latest src release (current 1.1.3)
wget https://nchc.dl.sourceforge.net/project/log4cpp/log4cpp-1.1.x%20%28new%29/log4cpp-1.1/log4cpp-1.1.3.tar.gz
# Download latest src release (current 1.1.3)
if [ -d "log4cpp" ] ; then rm -r log4cpp ; fi
curl -LO https://nchc.dl.sourceforge.net/project/log4cpp/log4cpp-1.1.x%20%28new%29/log4cpp-1.1/log4cpp-1.1.3.tar.gz
tar -xzf log4cpp-1.1.3.tar.gz
cd log4cpp
patch -p1 < ~/working/xLights/macOS/patches/log4cpp.patch
export CXXFLAGS="-g -O2 -flto=thin ${OSX_VERSION_MIN} ${XL_TARGETS} -std=c++11 -stdlib=libc++ -fvisibility-inlines-hidden"
pushd log4cpp
patch -p1 < "${XL_DIR}/macOS/patches/log4cpp.patch"
export CXXFLAGS="-g -O2 -flto=thin ${OSX_VERSION_MIN} ${XL_TARGETS} -std=c++11 -stdlib=libc++ -fvisibility-inlines-hidden "
export LDFLAGS="-flto=thin ${XL_TARGETS} "
./configure --prefix=/opt/local -host ${BUILD_HOST}
make clean
make -j 8
make -j ${NUM_CPUS}
cp src/.libs/liblog4cpp.a /opt/local/lib
export CXXFLAGS="-g ${OSX_VERSION_MIN} ${XL_TARGETS} -std=c++11 -stdlib=libc++ -fvisibility-inlines-hidden "
export LDFLAGS="${XL_TARGETS} "
./configure --prefix=/opt/local -host ${BUILD_HOST}
make clean
make -j ${NUM_CPUS}
cp src/.libs/liblog4cpp.a /opt/local/libdbg
unset CXXFLAGS
unset LDFLAGS

popd

echo "liquidfun"
# requires cmake to be installed, most likely need to have
# homebrew installed and then "brew install cmake"
# aternatively, install CMAKE for OSX from https://cmake.org/download/
# and add the full path to cmake to PATH
# PATH=$PATH:/Applications/CMake.app/Contents/bin/
git clone https://github.com/google/liquidfun
cd liquidfun/liquidfun/Box2D
if [ ! -d "liquidfun" ] ; then git clone https://github.com/google/liquidfun ; fi
pushd liquidfun/liquidfun/Box2D
git status --ignored -s | colrm 1 2 | xargs rm -rf
export CXX=clang++
export CXXFLAGS="-g -O3 -flto=thin ${XL_TARGETS} ${OSX_VERSION_MIN}"
export CXXFLAGS="-g -O3 -flto=thin ${XL_TARGETS} ${OSX_VERSION_MIN} "
cmake -G "Unix Makefiles" -DCMAKE_BUILD_TYPE=Release -DBOX2D_BUILD_EXAMPLES=OFF
echo "CXX_FLAGS += -Wno-unused-but-set-variable -Wno-error " >> ./Box2D/CMakeFiles/Box2D.dir/flags.make
make clean
make -j 8
make -j $NUM_CPUS
cp ./Box2D/Release/libliquidfun.a /opt/local/lib
git status --ignored -s | colrm 1 2 | xargs rm -rf
export CXXFLAGS="-g ${XL_TARGETS} ${OSX_VERSION_MIN} "
cmake -G "Unix Makefiles" -DCMAKE_BUILD_TYPE=Release -DBOX2D_BUILD_EXAMPLES=OFF
echo "CXX_FLAGS += -Wno-unused-but-set-variable -Wno-error " >> ./Box2D/CMakeFiles/Box2D.dir/flags.make
make clean
make -j $NUM_CPUS
cp ./Box2D/Release/libliquidfun.a /opt/local/libdbg
unset CXXFLAGS
unset CXX
popd


echo "SDL2" #currently using 2.0.22
git clone https://github.com/libsdl-org/SDL
cd SDL
echo "SDL2" currently using 2.30.9
if [ ! -d "SDL" ] ; then git clone https://github.com/libsdl-org/SDL ; fi
pushd SDL
git fetch -v
git reset --hard
git checkout release-2.0.22
export CXXFLAGS="-g -O3 -flto=thin ${XL_TARGETS} ${OSX_VERSION_MIN}"
export CFLAGS="-g -O3 -flto=thin ${XL_TARGETS} ${OSX_VERSION_MIN}"
export LDFLAGS="-g -O3 -flto=thin ${XL_TARGETS} ${OSX_VERSION_MIN}"
./configure --disable-shared --enable-static --disable-render-metal --disable-video-metal --disable-video-dummy --disable-video-opengl --disable-video-opengles --disable-video-opengles2 --disable-video-vulkan --disable-haptic --disable-joystick
git checkout release-2.30.9
export CXXFLAGS="-g -O3 -flto=thin ${XL_TARGETS} ${OSX_VERSION_MIN} "
export CFLAGS="-g -O3 -flto=thin ${XL_TARGETS} ${OSX_VERSION_MIN} "
export LDFLAGS="-g -O3 -flto=thin ${XL_TARGETS} ${OSX_VERSION_MIN} "
./configure --disable-shared --enable-static --disable-render-metal --disable-video-metal --disable-video-dummy --disable-video-x11 --disable-video-opengles --disable-video-opengles2 --disable-video-vulkan --disable-haptic --disable-joystick --prefix=/opt/local
make clean
make -j 8
make -j $NUM_CPUS
cp ./build/.libs/libSDL2.a /opt/local/lib
export CXXFLAGS="-g ${XL_TARGETS} ${OSX_VERSION_MIN} "
export CFLAGS="-g ${XL_TARGETS} ${OSX_VERSION_MIN} "
export LDFLAGS="-g ${XL_TARGETS} ${OSX_VERSION_MIN} "
./configure --disable-shared --enable-static --disable-render-metal --disable-video-metal --disable-video-dummy --disable-video-x11 --disable-video-opengles --disable-video-opengles2 --disable-video-vulkan --disable-haptic --disable-joystick --prefix=/opt/local
make clean
make -j $NUM_CPUS
cp ./build/.libs/libSDL2.a /opt/local/libdbg
unset CXXFLAGS
unset CFLAGS
unset LDFLAGS
popd


echo "ffmpeg" #currently using 6.0
echo "ffmpeg" # currently using 6.1.2
# Note: requires nasm to be install. Easiest option is via "brew install nasm"
git clone https://git.ffmpeg.org/ffmpeg.git
cd ffmpeg
git checkout n6.0
make clean
if [ ! -d "ffmpeg" ] ; then git clone https://git.ffmpeg.org/ffmpeg.git ; fi
pushd ffmpeg
git fetch -v
git checkout n6.1.2
make clean || true # can exit nonzero
rm -rf x86_64
git status --ignored -s | colrm 1 2 | xargs rm -rf
./configure --disable-inline-asm --enable-static --disable-shared --disable-securetransport --extra-cflags="${OSX_VERSION_MIN}" --disable-indev=lavfi --disable-libx264 --disable-lzma --enable-gpl --enable-opengl --disable-programs --arch=x86_64
sed -i -e "s/^CFLAGS=/CFLAGS=-g ${X86_64_TARGETS} ${OSX_VERSION_MIN} /" ffbuild/config.mak
sed -i -e "s/^CXXFLAGS=/CXXFLAGS=-g ${X86_64_TARGETS} ${OSX_VERSION_MIN} /" ffbuild/config.mak
sed -i -e "s/^CFLAGS=/CFLAGS=-g ${X86_64_TARGETS} ${OSX_VERSION_MIN} -DGL_SILENCE_DEPRECATION=1 -Wno-incompatible-function-pointer-types /" ffbuild/config.mak
sed -i -e "s/^CXXFLAGS=/CXXFLAGS=-g ${X86_64_TARGETS} ${OSX_VERSION_MIN} -DGL_SILENCE_DEPRECATION=1 /" ffbuild/config.mak
sed -i -e "s/^LDFLAGS=/LDFLAGS=-g ${X86_64_TARGETS} ${OSX_VERSION_MIN} /" ffbuild/config.mak
make -j 16
make -j $NUM_CPUS ; make
mkdir ./x86_64
find . -name "*.a" -exec cp {} ./x86_64 \;
make clean
./configure --disable-inline-asm --enable-static --disable-shared --disable-securetransport --extra-cflags="${OSX_VERSION_MIN}" --disable-indev=lavfi --disable-libx264 --disable-lzma --enable-gpl --enable-opengl --disable-programs --arch=arm64
sed -i -e "s/^CFLAGS=/CFLAGS=-g ${ARM64_TARGETS} ${OSX_VERSION_MIN} /" ffbuild/config.mak
sed -i -e "s/^CXXFLAGS=/CXXFLAGS=-g ${ARM64_TARGETS} ${OSX_VERSION_MIN} /" ffbuild/config.mak
find . -name "*.a" -exec cp -f {} ./x86_64 \;
make clean || true # can exit nonzero
git status --ignored -s | colrm 1 2 | grep -v x86_64 | xargs rm -rf
./configure --enable-static --disable-shared --disable-securetransport --extra-cflags="${OSX_VERSION_MIN}" --disable-indev=lavfi --disable-libx264 --disable-lzma --enable-gpl --enable-opengl --disable-programs --arch=arm64
sed -i -e "s/^CFLAGS=/CFLAGS=-g ${ARM64_TARGETS} ${OSX_VERSION_MIN} -DGL_SILENCE_DEPRECATION=1 -Wno-incompatible-function-pointer-types /" ffbuild/config.mak
sed -i -e "s/^CXXFLAGS=/CXXFLAGS=-g ${ARM64_TARGETS} ${OSX_VERSION_MIN} -DGL_SILENCE_DEPRECATION=1 /" ffbuild/config.mak
sed -i -e "s/^LDFLAGS=/LDFLAGS=-g ${ARM64_TARGETS} ${OSX_VERSION_MIN} /" ffbuild/config.mak
make -j 16
make -j $NUM_CPUS ; make
lipo -create -output /opt/local/lib/libavutil.a ./libavutil/libavutil.a ./x86_64/libavutil.a
lipo -create -output /opt/local/lib/libavfilter.a ./libavfilter/libavfilter.a ./x86_64/libavfilter.a
lipo -create -output /opt/local/lib/libavcodec.a ./libavcodec/libavcodec.a ./x86_64/libavcodec.a
Expand All @@ -116,41 +160,102 @@ lipo -create -output /opt/local/lib/libavformat.a ./libavformat/libavformat.a ./
lipo -create -output /opt/local/lib/libavdevice.a ./libavdevice/libavdevice.a ./x86_64/libavdevice.a
lipo -create -output /opt/local/lib/libswresample.a ./libswresample/libswresample.a ./x86_64/libswresample.a
lipo -create -output /opt/local/lib/libswscale.a ./libswscale/libswscale.a ./x86_64/libswscale.a
make clean || true # can exit nonzero
git status --ignored -s | colrm 1 2 | xargs rm -rf
./configure --disable-asm --disable-x86asm --enable-static --disable-shared --disable-securetransport --extra-cflags="${OSX_VERSION_MIN}" --disable-indev=lavfi --disable-libx264 --disable-lzma --enable-gpl --enable-opengl --disable-programs --disable-optimizations
sed -i -e "s/^CFLAGS=/CFLAGS=-g ${XL_TARGETS} ${OSX_VERSION_MIN} -DGL_SILENCE_DEPRECATION=1 -Wno-incompatible-function-pointer-types /" ffbuild/config.mak
sed -i -e "s/^CXXFLAGS=/CXXFLAGS=-g ${XL_TARGETS} ${OSX_VERSION_MIN} -DGL_SILENCE_DEPRECATION=1 /" ffbuild/config.mak
sed -i -e "s/^LDFLAGS=/LDFLAGS=-g ${XL_TARGETS} ${OSX_VERSION_MIN} /" ffbuild/config.mak
nkaminski marked this conversation as resolved.
Show resolved Hide resolved
make -j $NUM_CPUS ; make
find . -name "*.a" -exec cp {} /opt/local/libdbg/ \;
popd

echo "libxslwriter"
git clone https://github.com/jmcnamara/libxlsxwriter.git
cd libxlsxwriter/cmake
export CXXFLAGS="-g -O3 -flto=thin ${XL_TARGETS} ${OSX_VERSION_MIN}"
export CFLAGS="-g -O3 -flto=thin ${XL_TARGETS} ${OSX_VERSION_MIN}"
export LDFLAGS="-g -O3 -flto=thin ${XL_TARGETS} ${OSX_VERSION_MIN}"
echo "libxslxwriter"
if [ ! -d "libxlsxwriter" ] ; then git clone https://github.com/jmcnamara/libxlsxwriter.git ; fi
pushd libxlsxwriter
git checkout v1.1.9
cd cmake
export CXXFLAGS="-g -O3 -flto=thin ${XL_TARGETS} ${OSX_VERSION_MIN} "
export CFLAGS="-g -O3 -flto=thin ${XL_TARGETS} ${OSX_VERSION_MIN} "
export LDFLAGS="-g -O3 -flto=thin ${XL_TARGETS} ${OSX_VERSION_MIN} "
cmake -DCMAKE_OSX_ARCHITECTURES="x86_64;arm64" ..
make
cp libxlsxwriter.a /opt/local/lib/
git status --ignored -s | colrm 1 3 | xargs rm -rf
export CXXFLAGS="-g ${XL_TARGETS} ${OSX_VERSION_MIN} "
export CFLAGS="-g ${XL_TARGETS} ${OSX_VERSION_MIN} "
export LDFLAGS="-g ${XL_TARGETS} ${OSX_VERSION_MIN} "
cmake -DCMAKE_OSX_ARCHITECTURES="x86_64;arm64" ..
make
cp libxlsxwriter.a /opt/local/libdbg/
cp -a ../include/* /opt/local/include
make clean
git status --ignored -s | colrm 1 3 | xargs rm -rf
unset CXXFLAGS
unset CFLAGS
unset LDFLAGS
popd

echo "liblua" #(currently using v5.4.4)
git clone https://github.com/lua/lua
cd lua
git checkout v5.4.4

echo "liblua" # currently using v5.4.7
if [ ! -d "liblua" ] ; then git clone https://github.com/lua/lua ; fi
pushd lua
git fetch -v
git checkout v5.4.7
make clean
export CFLAGS="-g -O3 -flto=thin -Wall -fno-stack-protector -fno-common ${XL_TARGETS} ${OSX_VERSION_MIN}"
make -j4 all CFLAGS="$CFLAGS" MYLDFLAGS="$CFLAGS"
export CFLAGS="-g -O3 -flto=thin -Wall -fno-stack-protector -fno-common ${XL_TARGETS} ${OSX_VERSION_MIN} "
make -j $NUM_CPUS all CFLAGS="$CFLAGS" MYLDFLAGS="$CFLAGS"
Copy link

@DanInProgress DanInProgress Dec 3, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

with the large number of builds, it might be more legible debuggable to any of:

  • split this into multiple shells scripts that can succeed or fail individually
  • use make with per-target exported flags to update the dependent builds and track their success
  • shelling out will also help with env/workdir management and isolation

edit to add context from other comment

you can subshell out without a separate script using ( ... ), but beware changes in set -e behavior due to the compound command with either {...} or (...)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would be in favor of this however this would be a functional change from the steps in the readme. Would be interested as to other's thoughts here.

cp liblua.a /opt/local/lib
cp lua.h luaconf.h lualib.h lauxlib.h /opt/local/include
make clean
export CFLAGS="-g ${XL_TARGETS} ${OSX_VERSION_MIN} "
make -j $NUM_CPUS all CFLAGS="$CFLAGS" MYLDFLAGS="$CFLAGS"
cp liblua.a /opt/local/libdbg
unset CFLAGS
popd

echo "libhidapi" #(currently using 0.11.2)
git clone https://github.com/libusb/hidapi
cd hidapi
git checkout hidapi-0.11.2
export CFLAGS="-g -O3 -flto=thin -Wall -fno-stack-protector -fno-common ${XL_TARGETS} ${OSX_VERSION_MIN}"
echo "libhidapi" # currently using 0.14.0
if [ ! -d "hidapi" ] ; then git clone https://github.com/libusb/hidapi ; fi
pushd hidapi
git fetch -v
git checkout hidapi-0.14.0
export CFLAGS="-g -O3 -flto=thin -Wall -fno-stack-protector -fno-common ${XL_TARGETS} ${OSX_VERSION_MIN} "
./bootstrap
./configure --prefix=/opt/local
make clean
make
cp ./mac/.libs/libhidapi.a /opt/local/lib
make clean
export CFLAGS="-g -Wall -fno-stack-protector -fno-common ${XL_TARGETS} ${OSX_VERSION_MIN} "
./configure --prefix=/opt/local
make
rm -f /opt/local/libdbg/libhidapi*
cp ./mac/.libs/libhidapi.a /opt/local/libdbg
make clean
unset CFLAGS
popd

echo "done"
echo "libwebp" # currently using v1.4.0
if [ ! -d "libwebp" ] ; then git clone https://chromium.googlesource.com/webm/libwebp ; fi
pushd libwebp
git fetch -v
git checkout v1.4.0
./autogen.sh
export CFLAGS="-g -O3 -flto=thin -Wall -fno-stack-protector -fno-common ${XL_TARGETS} ${OSX_VERSION_MIN} "
./configure --prefix=/opt/local --disable-shared
make clean
make -j $NUM_CPUS
cp ./src/.libs/libwebp.a /opt/local/lib
cp ./src/demux/.libs/libwebpdemux.a /opt/local/lib
cp ./sharpyuv/.libs/libsharpyuv.a /opt/local/lib
make clean
export CFLAGS="-g -Wall -fno-stack-protector -fno-common ${XL_TARGETS} ${OSX_VERSION_MIN} "
./configure --prefix=/opt/local --disable-shared
make -j $NUM_CPUS
cp ./src/.libs/libwebp.a /opt/local/libdbg
cp ./src/demux/.libs/libwebpdemux.a /opt/local/libdbg
cp ./sharpyuv/.libs/libsharpyuv.a /opt/local/libdbg
unset CFLAGS
popd
echo "Libraries built"
ls -ltR /opt/local