Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve Windows support and make the top-level buildozer target public #17

Closed
wants to merge 2 commits into from
Closed
Show file tree
Hide file tree
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion .bcr/presubmit.yml
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,8 @@ bcr_test_module:
# Workaround for https://github.com/bazelbuild/bazel/issues/3268
- echo bazel-bcr/>> .bazelignore
run_targets:
- //:run_buildozer
- //:buildozer_alias # Run the alias with no args.
- //:run_buildozer # Will modify a test target.
build_targets:
- //...
test_targets:
Expand Down
15 changes: 4 additions & 11 deletions BUILD.bazel
Original file line number Diff line number Diff line change
@@ -1,15 +1,8 @@
sh_binary(
# A buildozer binary that will run in the root repository's workspace directory.
alias(
name = "buildozer",
srcs = ["buildozer.sh"],
data = [
"@buildozer_binary//:buildozer.exe",
],
env = {
"BUILDOZER_RLOCATIONPATH": "$(rlocationpath @buildozer_binary//:buildozer.exe)",
},
deps = [
"@bazel_tools//tools/bash/runfiles",
],
actual = "@buildozer_binary//:source_buildozer",
visibility = ["//visibility:public"],
)

exports_files(["buildozer.bzl"])
Expand Down
1 change: 1 addition & 0 deletions private/BUILD.bazel
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
exports_files(["buildozer.template.sh", "buildozer.template.bat"])
23 changes: 23 additions & 0 deletions private/buildozer.template.bat
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
@echo off
SETLOCAL ENABLEEXTENSIONS
SETLOCAL ENABLEDELAYEDEXPANSION

set MF=MANIFEST
set PATH=%SYSTEMROOT%\system32

REM Lookup the buildozer binary in the runfiles manifest.
for /F "tokens=2* usebackq" %%i in (`findstr.exe /l /c:"%%BUILDOZER_RLOCATIONPATH%% " "%MF%"`) do (
set BUILDOZER=%%i
set BUILDOZER=!BUILDOZER:/=\!
)
if "!BUILDOZER!" equ "" (
echo>&2 ERROR: %%BUILDOZER_RLOCATIONPATH%% not found in runfiles.
exit /b 1
)

REM Run the newly found full path to the executable within the build's workspace.
cd %BUILD_WORKSPACE_DIRECTORY%
!BUILDOZER! %*
if %ERRORLEVEL% neq 0 (
exit /b %ERRORLEVEL%
)
7 changes: 2 additions & 5 deletions buildozer.sh → private/buildozer.template.sh
Original file line number Diff line number Diff line change
Expand Up @@ -5,10 +5,6 @@ if [[ -z "${BUILD_WORKING_DIRECTORY+x}" ]]; then
exit 1
fi

# Disable MSYS path conversion, which can otherwise mess up Bazel labels.
export MSYS2_ARG_CONV_EXCL=*
export MSYS_NO_PATHCONV=1

# --- begin runfiles.bash initialization v3 ---
# Copy-pasted from the Bazel Bash runfiles library v3.
set -uo pipefail; set +e; f=bazel_tools/tools/bash/runfiles/runfiles.bash
Expand All @@ -20,7 +16,8 @@ source "${RUNFILES_DIR:-/dev/null}/$f" 2>/dev/null || \
{ echo>&2 "ERROR: cannot find $f"; exit 1; }; f=; set -e
# --- end runfiles.bash initialization v3 ---

buildozer_path=$(rlocation $BUILDOZER_RLOCATIONPATH)
BUILDOZER_RLOCATIONPATH=%%BUILDOZER_RLOCATIONPATH%%
buildozer_path=$(rlocation "$BUILDOZER_RLOCATIONPATH")
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Instead of injecting this variable, we should be able to just use buildozer_binary/buildozer (plus the OS-dependent extension). That avoids having to stamp in the path, which in turn allows us to avoid the .bat script. It looks pretty good (thanks for giving me an idea of what that would look like!), but I'm not sure that it fully follows the runfiles discovery process - for example, what if someone builds with runfiles enabled on Windows?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Having an inner repo for the binary requires a rendered script. As far as I know, there is no other way to locate the binary when runfiles are disabled - which is the default state of a Windows box.

When running in command prompt, batch is also required. Here is a stock windows box with bazelisk only:

C:\Users\clzdg\Projects\buildozer>bazel run @buildozer -- -h
WARNING: Build option --enable_runfiles has changed, discarding analysis cache (this can be expensive, see https://bazel.build/advanced/performance/iteration-speed).
INFO: Analyzed target //:buildozer (1 packages loaded, 7 targets configured).
INFO: Found 1 target...
Target @@+buildozer_binary+buildozer_binary//:source_buildozer up-to-date:
  bazel-bin/external/+buildozer_binary+buildozer_binary/source_buildozer.sh
INFO: Elapsed time: 0.618s, Critical Path: 0.02s
INFO: 5 processes: 5 internal.
INFO: Build completed successfully, 5 total actions
INFO: Running command line: bazel-bin/external/+buildozer_binary+buildozer_binary/source_buildozer.sh <args omitted>
FATAL: ExecuteProgram(C:\users\clzdg\_bazel_clzdg\2qmadfcc\execroot\_main\bazel-out\x64_windows-fastbuild\bin\external\+buildozer_binary+buildozer_binary\source_buildozer.sh) failed: ERROR: src/main/native/windows/process.cc(202): CreateProcessW("C:\users\clzdg\_bazel_clzdg\2qmadfcc\execroot\_main\bazel-out\x64_windows-fastbuild\bin\external\+buildozer_binary+buildozer_binary\source_buildozer.sh"  -h): %1 is not a valid Win32 application.
 (error: 193)

I updated the batch file to be a bit smarter. This is generally modeled after https://github.com/bazelbuild/bazel-skylib/blob/main/rules/diff_test.bzl, but unfortunately the test wrapper provides the runfiles envvars that make their runfiles discovery simpler, so we have a sillier MANIFEST direct lookup.

I think we need a batch runfiles library, only 10k lines of batch and you've got a repo mapping baby.

Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The batch script looks pretty accurate now, but I think that we can still avoid it. Please take a look at #23, which is based on your PR but hardcodes the rlocationpath with an apparent repository name. It doesn't work on 6.2.0 as the runfiles library had a bug, but it works on 6.x.

if [[ ! -f "$buildozer_path" ]]; then
echo "buildozer.exe not found at runfiles path $BUILDOZER_RLOCATIONPATH and resolved path $buildozer_path."
exit 1
Expand Down
67 changes: 65 additions & 2 deletions private/buildozer_binary.bzl
Original file line number Diff line number Diff line change
@@ -1,6 +1,58 @@
load("@bazel_features//:features.bzl", "bazel_features")

visibility("//")
def _source_buildozer_impl(ctx):
executable = ctx.actions.declare_file(ctx.label.name + (".bat" if ctx.attr.is_windows else ".sh"))

ctx.actions.expand_template(
template = ctx.file._windows_template if ctx.attr.is_windows else ctx.file._linux_template,
output = executable,
is_executable = True,
substitutions = {
"%%BUILDOZER_RLOCATIONPATH%%": ctx.file.buildozer_binary.short_path.lstrip("../"),
}
)
files = [ctx.file.buildozer_binary]
if not ctx.attr.is_windows:
files.append(ctx.file._bash_runfiles)

return [
DefaultInfo(
files = depset([executable]),
executable = executable,
runfiles = ctx.runfiles(files),
)
]

source_buildozer = rule(
doc = "An executable buildozer binary that changes directory into a workspace before running.",
implementation = _source_buildozer_impl,
attrs = {
"buildozer_binary": attr.label(
doc = "Buildozer binary to run. Must come from an external repository.",
allow_single_file = True,
executable = True,
cfg = "target",
mandatory = True,
),
"is_windows": attr.bool(
doc = "Whether the buildozer binary will be running on Windows.",
mandatory = True,
),
"_bash_runfiles": attr.label(
allow_single_file = True,
default = Label("@bazel_tools//tools/bash/runfiles"),
),
"_linux_template": attr.label(
allow_single_file = [".sh"],
default = Label("//private:buildozer.template.sh"),
),
"_windows_template": attr.label(
allow_single_file = [".bat"],
default = Label("//private:buildozer.template.bat"),
)
},
executable = True,
)

def _get_buildozer_os(rctx_os):
name = rctx_os.name
Expand All @@ -26,7 +78,6 @@ def _get_buildozer_arch(rctx_os):

def _buildozer_binary_repo_impl(repository_ctx):
repository_ctx.file("WORKSPACE")
repository_ctx.file("BUILD.bazel", """exports_files(["buildozer.exe"])""")

os = _get_buildozer_os(repository_ctx.os)
arch = _get_buildozer_arch(repository_ctx.os)
Expand All @@ -35,6 +86,18 @@ def _buildozer_binary_repo_impl(repository_ctx):
if not sha256:
fail("No match for '{os_arch}' in sha256".format(os_arch = os_arch))

repository_ctx.file("BUILD.bazel", """\
load("@buildozer//private:buildozer_binary.bzl", "source_buildozer")
exports_files(["buildozer.exe"])

source_buildozer(
name = "source_buildozer",
buildozer_binary = "buildozer.exe",
is_windows = {is_windows},
visibility = ["//visibility:public"],
)
""".format(is_windows = os == "windows"))

repository_ctx.download(
url = [
"https://github.com/bazelbuild/buildtools/releases/download/v{version}/buildozer-{os_arch}{extension}".format(
Expand Down
5 changes: 5 additions & 0 deletions tests/bcr/BUILD.bazel
Original file line number Diff line number Diff line change
Expand Up @@ -5,3 +5,8 @@ sh_binary(
},
srcs = ["run_buildozer.sh"],
)

alias(
name = "buildozer_alias",
actual = "@buildozer",
)
Loading