Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Introduce a Swift C++ runfiles library wrapper #1470

Open
wants to merge 6 commits into
base: master
Choose a base branch
from

Conversation

cerisier
Copy link

@cerisier cerisier commented Dec 29, 2024

Second shot at implementing #890

This PR introduces a Swift runfiles lookup library.

After discussion, it was decided to give a go to wrapping the C++ runfiles library instead of writing one in pure Swift.

Implementation

This implementation is done in 2 parts:

  1. A C++ wrapper to the original C++ library, exposed as a C API.
  2. A Swift wrapper to the C API.

C++ wrapper as C API

I took the decision to expose the C++ runfiles library to Swift via a C API for the following reasons:

The original C++ runfiles library was not compatible with Swift C++ interop out of the box:

  1. References to types such as std::unique_ptr in the public API (https://www.swift.org/documentation/cxx-interop/#unique-reference-types)
  2. Swift considers C++ classes as value types and requires them to have a copy constructor unless the type itself is annotated accordingly which is not possible when wrapping (https://www.swift.org/documentation/cxx-interop/#mapping-c-types-to-swift-reference-types)
    It would fail with an error like this: note: record 'Runfiles' is not automatically available: does not have a copy constructor or destructor; does this type have reference semantics?

Using C++ interop currently forces users to provide -cxx-interoperability-mode=default down the line, otherwise the header from the generted module map is treated as C and not C++.

For those reasons, I chose the C API approach which makes everything invisible to users of the library.

Swift Wrapper

The Swift wrapper is just a wrapper around around the C API with 3 exceptions:

  1. We use CommandLine.arguments[0] automatically instead of requiring users to pass argv0 like it is the case in the C++ library.
  2. We automatically compute the caller's canonical repository name if it is not provided (See explanation below).
  3. rlocation returns nil instead of an empty string if repository mappings fail.

Source Repository

The runfiles library needs to know the canonical name of the repository that is trying to resolve a particular runfile path.
This is required to map apparent repository names used in the runfile path, to its canonical repo name on the filesystem.

Every ruleset has its own way of making this transparent to the users of the runfiles library (Except C++ and Java), most of them rely on runtime accessors or language specific features that do not exist in Swift.

Here, we rely on the built-in #filePath magic identifier which has the property to be expanded at callsite when used as a default argument, the same way assert and fatalError work.

I couldn't find the specification for this behavior except the below mention in an accepted Swift proposal:
https://github.com/swiftlang/swift-evolution/blob/80c3616bb23e7bf751f886aaf178acb3df09abe5/proposals/0274-magic-file.md#provide-separate-modulename-and-filename-magic-identifiers:

Magic identifiers only give you the caller's location if they are the only thing in the default argument

From that #filePath, we deduce the canonical repository name the same way rules_go does (thanks @fmeum).
See https://github.com/bazel-contrib/rules_go/blob/6505cf2e4f0a768497b123a74363f47b711e1d02/go/runfiles/global.go#L53-L80

Additional notes

  1. I didn't include tests at this stage on purpose before we settle on the API. Also, because this is a wrapper reusing a battle tested C++ library, I would like to discuss which tests make sense to write :)

  2. Because we always compute the source canonical repository name, I didn't expose variants of the C++ API that do not require it.

  3. In the original pure Swift implementation, I had rlocation return a URL instead of a String, let me know if I should use that instead.

  4. Finally, all rulesets providing a runfiles library provide a env and args attributes to *_binary that are subject to Make Variable substitution (for $(rlocationpath //target) most notably) , let me know this is something we want and if I should submit a PR for that.

TODO

  • Discuss test strategy
  • Implement tests

Links

Comment on lines 42 to 44
If you want to start subprocesses, and the subprocess can't automatically
find the correct runfiles directory, you can explicitly set the right
environment variables for them:
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The contract for runfiles libraries requires each process to pass these variables to subprocesses. Not setting them will result in non-hermetic runfiles access, so I would suggest to strengthen the language.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure

///
/// This method looks at the RUNFILES_MANIFEST_FILE and TEST_SRCDIR
/// environment variables.
public static func createForTest(
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This overload isn't necessary, tests can use create just fine. The separate function in the C++ lib exists for historical purposes only.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great, i'll remove it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants