Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Support fully-static linking #8

Open
allevato opened this issue Jun 6, 2018 · 6 comments
Open

Support fully-static linking #8

allevato opened this issue Jun 6, 2018 · 6 comments
Labels
P3 Would be nice, but probably next quarter at the earliest type: feature request New feature or request

Comments

@allevato
Copy link
Member

allevato commented Jun 6, 2018

Right now, we only support kind-of-static linking: the swift_library targets that the binary depends on are statically linked into the final binary, but the Swift runtime libraries are dynamically linked. This is fine (and in fact, desired) on Apple's mobile platforms (iOS, tvOS, watchOS), but on Linux (and probably macOS), we should offer static linking against the runtime as well.

One interesting catch: Linux toolchains do not come with a static version of libdispatch, which would seem to mean you can't do a whole lot of interesting stuff in a statically linked environment. So this might reduce the usefulness and urgency of this feature.
@allevato allevato added the type: feature request New feature or request label Jun 6, 2018
@allevato
Copy link
Member Author

allevato commented Jun 6, 2018

There don't appear to be any major fundamental issues preventing libdispatch from being statically linked, it's just not there yet. So having this functionality will be good for when that happens. See SR-7039, SR-7085.

@allevato allevato added the P2 We should fix this this quarter label Jun 7, 2018
@liuliu
Copy link

liuliu commented Nov 5, 2020 •
edited
Loading

It seems Swift 5.3 packaged all static libs (Dispatch, Foundation etc.) for their Linux dist. Let me try to hack it a bit see if it roughly works.

@keith keith added P3 Would be nice, but probably next quarter at the earliest and removed P2 We should fix this this quarter labels Oct 26, 2021
@liuliu
Copy link

liuliu commented Oct 27, 2021 •
edited
Loading

Got some time to do some investigations in what's needed:

  1. Pass -static-stdlib to swiftc, in this way, the autolink will embed the information of the right static libraries to link to;
  2. During link phase, pass static-stdlib-args.lnk to include other necessary libraries to link to.

With these, a minimal viable patch I made will generate binary that doesn't link to any Swift runtime (still have glibc / pthread etc): https://gist.github.com/liuliu/9450547b0c12488a95ad7f2485395407. Note that at the moment, this has to be compiled with bazel build xxx:yyy --swiftcopt=-static-stdlib

By doing this, I successfully compiled https://github.com/liuliu/s4nnc/tree/main/examples/ddpg with the following readelf output:

liu@sz77:~/workspace/s4nnc$ readelf -d bazel-bin/examples/ddpg

Dynamic section at offset 0x2e929b0 contains 52 entries:
  Tag        Type                         Name/Value
 0x0000000000000001 (NEEDED)             Shared library: [libcublas.so.11]
 0x0000000000000001 (NEEDED)             Shared library: [libcudart.so.11.0]
 0x0000000000000001 (NEEDED)             Shared library: [libcudnn.so.7]
 0x0000000000000001 (NEEDED)             Shared library: [libcufft.so.10]
 0x0000000000000001 (NEEDED)             Shared library: [libcurand.so.10]
 0x0000000000000001 (NEEDED)             Shared library: [libnccl.so.2]
 0x0000000000000001 (NEEDED)             Shared library: [libm.so.6]
 0x0000000000000001 (NEEDED)             Shared library: [libcblas.so.3]
 0x0000000000000001 (NEEDED)             Shared library: [libatlas.so.3]
 0x0000000000000001 (NEEDED)             Shared library: [libpng16.so.16]
 0x0000000000000001 (NEEDED)             Shared library: [libjpeg.so.8]
 0x0000000000000001 (NEEDED)             Shared library: [libfftw3.so.3]
 0x0000000000000001 (NEEDED)             Shared library: [libfftw3f.so.3]
 0x0000000000000001 (NEEDED)             Shared library: [libpthread.so.0]
 0x0000000000000001 (NEEDED)             Shared library: [liblinear.so.4]
 0x0000000000000001 (NEEDED)             Shared library: [libtesseract.so.4]
 0x0000000000000001 (NEEDED)             Shared library: [libgsl.so.23]
 0x0000000000000001 (NEEDED)             Shared library: [libgslcblas.so.0]
 0x0000000000000001 (NEEDED)             Shared library: [libstdc++.so.6]
 0x0000000000000001 (NEEDED)             Shared library: [librt.so.1]
 0x0000000000000001 (NEEDED)             Shared library: [libdl.so.2]
 0x0000000000000001 (NEEDED)             Shared library: [libutil.so.1]
 0x0000000000000001 (NEEDED)             Shared library: [libomp.so]
 0x0000000000000001 (NEEDED)             Shared library: [libc.so.6]
 0x0000000000000001 (NEEDED)             Shared library: [ld-linux-x86-64.so.2]
 0x000000000000001d (RUNPATH)            Library runpath: [$ORIGIN/../_solib_local/_U@local_Uconfig_Ucuda_S_Scuda_Ccublas___Uexternal_Slocal_Uconfig_Ucuda_Scuda_Scuda_Slib:$ORIGIN/../_solib_local/_U@local_Uconfig_Ucuda_S_Scuda_Ccudart___Uexternal_Slocal_Uconfig_Ucuda_Scuda_Scuda_Slib:$ORIGIN/../_solib_local/_U@local_Uconfig_Ucuda_S_Scuda_Ccudnn___Uexternal_Slocal_Uconfig_Ucuda_Scuda_Scuda_Slib:$ORIGIN/../_solib_local/_U@local_Uconfig_Ucuda_S_Scuda_Ccufft___Uexternal_Slocal_Uconfig_Ucuda_Scuda_Scuda_Slib:$ORIGIN/../_solib_local/_U@local_Uconfig_Ucuda_S_Scuda_Ccurand___Uexternal_Slocal_Uconfig_Ucuda_Scuda_Scuda_Slib:$ORIGIN/../_solib_local/_U@local_Uconfig_Unccl_S_S_Cnccl___Uexternal_Slocal_Uconfig_Unccl:/usr/local/lib:/opt/swift/usr/lib/swift_static/linux]
 0x000000000000000c (INIT)               0x143000
 0x000000000000000d (FINI)               0xdaee34
 0x0000000000000019 (INIT_ARRAY)         0x2e38b48
 0x000000000000001b (INIT_ARRAYSZ)       368 (bytes)
 0x000000000000001a (FINI_ARRAY)         0x2e38cb8
 0x000000000000001c (FINI_ARRAYSZ)       8 (bytes)
 0x000000006ffffef5 (GNU_HASH)           0x3a8
 0x0000000000000005 (STRTAB)             0x3be8
 0x0000000000000006 (SYMTAB)             0x528
 0x000000000000000a (STRSZ)              9247 (bytes)
 0x000000000000000b (SYMENT)             24 (bytes)
 0x0000000000000015 (DEBUG)              0x0
 0x0000000000000003 (PLTGOT)             0x2e93d30
 0x0000000000000002 (PLTRELSZ)           12912 (bytes)
 0x0000000000000014 (PLTREL)             RELA
 0x0000000000000017 (JMPREL)             0x13fc48
 0x0000000000000007 (RELA)               0x6748
 0x0000000000000008 (RELASZ)             1283328 (bytes)
 0x0000000000000009 (RELAENT)            24 (bytes)
 0x000000000000001e (FLAGS)              BIND_NOW
 0x000000006ffffffb (FLAGS_1)            Flags: NOW PIE
 0x000000006ffffffe (VERNEED)            0x6498
 0x000000006fffffff (VERNEEDNUM)         10
 0x000000006ffffff0 (VERSYM)             0x6008
 0x000000006ffffff9 (RELACOUNT)          52680
 0x0000000000000000 (NULL)               0x0

Before this patch, the output looks like this:

liu@sz77:~/workspace/s4nnc$ readelf -d bazel-bin/examples/ddpg

Dynamic section at offset 0x6b58a8 contains 61 entries:
  Tag        Type                         Name/Value
 0x0000000000000001 (NEEDED)             Shared library: [libcublas.so.11]
 0x0000000000000001 (NEEDED)             Shared library: [libcudart.so.11.0]
 0x0000000000000001 (NEEDED)             Shared library: [libcudnn.so.7]
 0x0000000000000001 (NEEDED)             Shared library: [libcufft.so.10]
 0x0000000000000001 (NEEDED)             Shared library: [libcurand.so.10]
 0x0000000000000001 (NEEDED)             Shared library: [libnccl.so.2]
 0x0000000000000001 (NEEDED)             Shared library: [libm.so.6]
 0x0000000000000001 (NEEDED)             Shared library: [libcblas.so.3]
 0x0000000000000001 (NEEDED)             Shared library: [libatlas.so.3]
 0x0000000000000001 (NEEDED)             Shared library: [libpng16.so.16]
 0x0000000000000001 (NEEDED)             Shared library: [libjpeg.so.8]
 0x0000000000000001 (NEEDED)             Shared library: [libfftw3.so.3]
 0x0000000000000001 (NEEDED)             Shared library: [libfftw3f.so.3]
 0x0000000000000001 (NEEDED)             Shared library: [libpthread.so.0]
 0x0000000000000001 (NEEDED)             Shared library: [liblinear.so.4]
 0x0000000000000001 (NEEDED)             Shared library: [libtesseract.so.4]
 0x0000000000000001 (NEEDED)             Shared library: [libgsl.so.23]
 0x0000000000000001 (NEEDED)             Shared library: [libgslcblas.so.0]
 0x0000000000000001 (NEEDED)             Shared library: [libdispatch.so]
 0x0000000000000001 (NEEDED)             Shared library: [libBlocksRuntime.so]
 0x0000000000000001 (NEEDED)             Shared library: [libstdc++.so.6]
 0x0000000000000001 (NEEDED)             Shared library: [librt.so.1]
 0x0000000000000001 (NEEDED)             Shared library: [libdl.so.2]
 0x0000000000000001 (NEEDED)             Shared library: [libswift_Concurrency.so]
 0x0000000000000001 (NEEDED)             Shared library: [libswiftCore.so]
 0x0000000000000001 (NEEDED)             Shared library: [libswiftSwiftOnoneSupport.so]
 0x0000000000000001 (NEEDED)             Shared library: [libswiftGlibc.so]
 0x0000000000000001 (NEEDED)             Shared library: [libutil.so.1]
 0x0000000000000001 (NEEDED)             Shared library: [libFoundation.so]
 0x0000000000000001 (NEEDED)             Shared library: [libswiftDispatch.so]
 0x0000000000000001 (NEEDED)             Shared library: [libswift_Differentiation.so]
 0x0000000000000001 (NEEDED)             Shared library: [libomp.so]
 0x0000000000000001 (NEEDED)             Shared library: [libc.so.6]
 0x0000000000000001 (NEEDED)             Shared library: [ld-linux-x86-64.so.2]
 0x000000000000001d (RUNPATH)            Library runpath: [$ORIGIN/../_solib_local/_U@local_Uconfig_Ucuda_S_Scuda_Ccublas___Uexternal_Slocal_Uconfig_Ucuda_Scuda_Scuda_Slib:$ORIGIN/../_solib_local/_U@local_Uconfig_Ucuda_S_Scuda_Ccudart___Uexternal_Slocal_Uconfig_Ucuda_Scuda_Scuda_Slib:$ORIGIN/../_solib_local/_U@local_Uconfig_Ucuda_S_Scuda_Ccudnn___Uexternal_Slocal_Uconfig_Ucuda_Scuda_Scuda_Slib:$ORIGIN/../_solib_local/_U@local_Uconfig_Ucuda_S_Scuda_Ccufft___Uexternal_Slocal_Uconfig_Ucuda_Scuda_Scuda_Slib:$ORIGIN/../_solib_local/_U@local_Uconfig_Ucuda_S_Scuda_Ccurand___Uexternal_Slocal_Uconfig_Ucuda_Scuda_Scuda_Slib:$ORIGIN/../_solib_local/_U@local_Uconfig_Unccl_S_S_Cnccl___Uexternal_Slocal_Uconfig_Unccl:/usr/local/lib:/opt/swift/usr/lib/swift/linux]
 0x000000000000000c (INIT)               0x26000
 0x000000000000000d (FINI)               0x41bc64
 0x0000000000000019 (INIT_ARRAY)         0x6b0f08
 0x000000000000001b (INIT_ARRAYSZ)       336 (bytes)
 0x000000000000001a (FINI_ARRAY)         0x6b1058
 0x000000000000001c (FINI_ARRAYSZ)       8 (bytes)
 0x000000006ffffef5 (GNU_HASH)           0x3a8
 0x0000000000000005 (STRTAB)             0x5288
 0x0000000000000006 (SYMTAB)             0x3f8
 0x000000000000000a (STRSZ)              24253 (bytes)
 0x000000000000000b (SYMENT)             24 (bytes)
 0x0000000000000015 (DEBUG)              0x0
 0x0000000000000003 (PLTGOT)             0x6b6cb8
 0x0000000000000002 (PLTRELSZ)           11592 (bytes)
 0x0000000000000014 (PLTREL)             RELA
 0x0000000000000017 (JMPREL)             0x223f0
 0x0000000000000007 (RELA)               0xb998
 0x0000000000000008 (RELASZ)             92760 (bytes)
 0x0000000000000009 (RELAENT)            24 (bytes)
 0x000000000000001e (FLAGS)              BIND_NOW
 0x000000006ffffffb (FLAGS_1)            Flags: NOW PIE
 0x000000006ffffffe (VERNEED)            0xb7d8
 0x000000006fffffff (VERNEEDNUM)         10
 0x000000006ffffff0 (VERSYM)             0xb146
 0x000000006ffffff9 (RELACOUNT)          3308
 0x0000000000000000 (NULL)               0x0

@keith , I think this makes a good case to move forward with this task, given other discussions such as https://forums.swift.org/t/pre-pitch-statically-link-by-default-on-linux/52837

What I need for guidance, mostly due to that I am not familiar with how rules_swift organized:

  1. How to make this an option in swift_binary and propagate it to the place I modified?
  2. How to do above and propagate swiftcopt based on that?
  3. Should we differentiate static_stdlib and static_executable like SwiftPM currently does?

Above investigations only done with Swift 5.5.1 on Ubuntu 20.04.

@keith
Copy link
Member

keith commented Oct 27, 2021

  1. I think we probably want to enable this only when using some feature saying you want this when building
  2. this would be possible using the feature i'm talking about above
  3. this shouldn't have to be changed with this PR, right now technically every swift_binary from rules_swift is going to be statically linked (maybe ignoring some C++ stuff), but that's fine, so this feature should be specific to the stdlib, but it shouldn't make things harder for you.

I don't see from your patch what static-stdlib-args.lnk is?

@liuliu
Copy link

liuliu commented Oct 27, 2021

I don't see from your patch what static-stdlib-args.lnk is?

It is a file under /usr/lib/swift_static/ directory shipped with every Swift toolchain since 5.3.1: https://forums.swift.org/t/static-linking-on-linux-in-swift-5-3-1/41989

@liuliu
Copy link

liuliu commented Oct 28, 2021

@keith open a PR with your suggestion: #706

I still have issue with Bazel's cache not properly dirtied for autolink extract result (it seems).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
P3 Would be nice, but probably next quarter at the earliest type: feature request New feature or request
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@liuliu @keith @allevato