Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

app: support alias commands #716

Merged
merged 3 commits into from
Sep 6, 2024
Merged

Conversation

pdgendt
Copy link
Collaborator

@pdgendt pdgendt commented Jul 17, 2024

This PR allows to add alias west commands to a configuration, similar to how git aliases work.

It can override existing commands to have default arguments, and iteratively replace alias commands.

Some examples:

# Add a global "west run" shortcut (will override pristine build)
$ west config --global alias.run "build --pristine=never -t run"

# Add a global "west menuconfig" shortcut
$ west config --global alias.menuconfig "build --pristine=never -t menuconfig"

# A shortcut to build my current project (prefix the path with $PWD/ to build anywhere from the workspace)
$ west config alias.my_proj "build -b native_sim samples/hello_world -- -DCONFIG_ASSERT=y"

# WARNING: overriding built-in or other commands is at your own risk
# build pristine by default locally
$ west config alias.build "build --pristine=always"

After applying you can run:

$ west run         #> west build --pristine=always --pristine=never -t run
$ west menuconfig  #> west build --pristine=always --pristine=never -t menuconfig
$ west my_proj     #> west build --pristine=always -b native_sim samples/hello_world -- -DCONFIG_ASSERT=y

Copy link
Collaborator

@marc-hb marc-hb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice! Amazing how little code this requires.

Can you please submit a documentation PR for https://docs.zephyrproject.org/latest/develop/west/config.html#built-in-configuration-options ?

It sucks that it is a in different repo but that's just the way it is now.

I would like the interface design to be reviewed before the implementation. Please do mention that it's recursive. Also mention briefly that overriding/shadowing built-in commands is possible but please change the (commit messages) examples that do that: that looks like a sure support nightmare! I think it's fine if "advanced" users who know what they're doing override built-in commands but we really don't want beginners to do that and then have their friends, colleagues and support spend hours scratching their heads.

src/west/app/main.py Outdated Show resolved Hide resolved
src/west/app/main.py Outdated Show resolved Hide resolved
src/west/app/main.py Outdated Show resolved Hide resolved
src/west/app/main.py Outdated Show resolved Hide resolved
tests/test_alias.py Show resolved Hide resolved
@pdgendt pdgendt force-pushed the alias branch 2 times, most recently from 5ebbddd to 6aec00d Compare July 18, 2024 08:33
@pdgendt
Copy link
Collaborator Author

pdgendt commented Jul 18, 2024

Updated the PR to have west help <alias> print out info about the alias.

@pdgendt pdgendt force-pushed the alias branch 3 times, most recently from f2195e6 to c7234bf Compare July 19, 2024 11:32
@pdgendt
Copy link
Collaborator Author

pdgendt commented Jul 19, 2024

Calling west help now also lists alias commands.

@pdgendt pdgendt force-pushed the alias branch 3 times, most recently from 17df658 to 3fcad1f Compare July 22, 2024 18:16
@carlescufi carlescufi requested a review from marc-hb August 27, 2024 11:32
src/west/app/main.py Show resolved Hide resolved
src/west/app/main.py Outdated Show resolved Hide resolved
tests/test_alias.py Outdated Show resolved Hide resolved
tests/test_alias.py Outdated Show resolved Hide resolved
src/west/app/main.py Outdated Show resolved Hide resolved
@pdgendt pdgendt force-pushed the alias branch 2 times, most recently from 5fa1280 to fe632c5 Compare August 27, 2024 21:03
@pdgendt
Copy link
Collaborator Author

pdgendt commented Aug 27, 2024

@marc-hb Thanks for the review!
I will look into the difference between west help some_alias and west -h some_alias and the alias issue on windows.

@marc-hb
Copy link
Collaborator

marc-hb commented Aug 28, 2024

I will look into the difference between west help some_alias and west -h some_alias and the alias issue on windows.

I just did some interactive west config testing with both Powershell and CMD.EXE. You can nest single quotes inside double quotes in both but not the other way round. Don't ask me why. That's hopefully the only reason why https://github.com/zephyrproject-rtos/west/actions/runs/10585812818/job/29333273125?pr=716 failed.

If we really wanted to test quoting more then we should check some pre-canned config file in git and avoid command line peculiarities entirely. But for now I think restricting tests to single quotes nested in double quotes is portable and plenty good enough. We just want to make sure no one breaks the most basic whitespace usage and then trust shlex for the rest.

@pdgendt pdgendt force-pushed the alias branch 5 times, most recently from 2524057 to afa24da Compare August 28, 2024 09:03
@pdgendt
Copy link
Collaborator Author

pdgendt commented Aug 28, 2024

@marc-hb I've handled all comments, I think.

Copy link
Collaborator

@marc-hb marc-hb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let me try getting rid of shlex in conftest.py first.

tests/conftest.py Outdated Show resolved Hide resolved
marc-hb added a commit to marc-hb/west that referenced this pull request Aug 28, 2024
Stop using `shlex.split()` in conftest.py as a way to prepare test
commands for subprocess.run() because it does not always work. When we
have quoting issues, just avoid them entirely by passing the test
command as a list.

Replace shlex.split() with a simple str.split() for convenience in
simple cases. This simple split() will immediately fail when trying to
use quote which is working exactly as intended.

This also makes the cmd() interface more similar to subprocess.run() and
its many friends, which is good thing.

This is similar to commit 4100764 ("Project.git(list/str): reduce
reliance on shlex.split()") but applied to tests/

Before commit 624880e, shlex.split() was used unconditionally in
conftest.py. As expected, this eventually broke on Windows: shlex is
not compatible across all operating systems and shells.

https://docs.python.org/3/library/subprocess.html#security-considerations

> If the shell is invoked explicitly, via shell=True, it is the
> application’s responsibility to ensure that all whitespace and
> metacharacters are quoted appropriately to avoid shell injection
> vulnerabilities. On _some_ platforms, it is possible to use shlex.quote()
> for this escaping.

(Emphasis mine)

Then commit 624880e made the "interesting" decision to stop using
shlex.split()... only on Windows. This was discussed in
zephyrproject-rtos#266 (comment)

So after this commit, parsing of test commands was delegated to the
shell but... only on Windows! This worked for a long time but eventually
broke testing white spaces for the new `west alias` zephyrproject-rtos#716. Rather than
making that Windows-specific hack even more complex with a special case,
finally do the right thing and ask more complex test commands to use a
list. Now we can drop shlex.

Signed-off-by: Marc Herbert <marc.herbert@intel.com>
@marc-hb marc-hb dismissed their stale review August 29, 2024 23:17

outdated

src/west/app/main.py Outdated Show resolved Hide resolved
tests/test_alias.py Show resolved Hide resolved
tests/test_alias.py Outdated Show resolved Hide resolved
tests/test_alias.py Show resolved Hide resolved
src/west/app/main.py Outdated Show resolved Hide resolved
@marc-hb
Copy link
Collaborator

marc-hb commented Sep 4, 2024

Just tried the latest 93f9715 and infinite loops do not seem handled well. From reading the code I would expect this to print west: unknown command alpha but instead I get:

west help

aliases:
  alpha:                beta
  beta:                 alpha

west alpha

Traceback (most recent call last):
  File "~/.local/bin/west", line 8, in <module>
    sys.exit(main())
             ^^^^^^
  File "src/west/app/main.py", line 1172, in main
    app.run(argv or sys.argv[1:])
  File "src/west/app/main.py", line 255, in run
    self.run_command(argv, early_args)
  File "src/west/app/main.py", line 556, in run_command
    self.run_extension(args.command, argv)
  File "src/west/app/main.py", line 682, in run_extension
    self.cmd = self.extensions[name].factory()
               ~~~~~~~~~~~~~~~^^^^^^
KeyError: 'alpha'

More tests needed maybe?

I warned you recursion (and overriding built ins) was fraught with peril! :-)

@pdgendt pdgendt force-pushed the alias branch 2 times, most recently from 7883b31 to b389b3c Compare September 4, 2024 08:16
@pdgendt
Copy link
Collaborator Author

pdgendt commented Sep 4, 2024

I warned you recursion (and overriding built ins) was fraught with peril! :-)

Yes, the early argument handling does make it a bit more complicated, but I think I've managed to make it more robust now.

More tests needed maybe?

I added a test for the infinite recursion to contain "unknown command".

@pdgendt
Copy link
Collaborator Author

pdgendt commented Sep 4, 2024

Update: handle empty aliases and add a test

@marc-hb
Copy link
Collaborator

marc-hb commented Sep 4, 2024

Update: handle empty aliases

west help still broken with empty aliases and latest commit f8c61d1

west config alias.bar ""

west bar
usage: west [-h] [-z ZEPHYR_BASE] [-v] [-q] [-V] <command> ...
west: empty alias "bar"

west help

Traceback (most recent call last):
  File "/var/home/.local/bin/west", line 8, in <module>
    sys.exit(main())
             ^^^^^^
  File "src/west/app/main.py", line 1175, in main
    app.run(argv or sys.argv[1:])
  File "src/west/app/main.py", line 255, in run
    self.run_command(argv, early_args)
  File "src/west/app/main.py", line 557, in run_command
    self.run_builtin(args, unknown)
  File "src/west/app/main.py", line 673, in run_builtin
    self.cmd.run(args, unknown, self.topdir,
  File "src/west/commands.py", line 194, in run
    self.do_run(args, unknown)
  File "src/west/app/main.py", line 862, in do_run
    app.west_parser.print_help(top_level=True)
  File "src/west/app/main.py", line 927, in print_help
    print(self.format_help(top_level=top_level), end='',
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "src/west/app/main.py", line 1014, in format_help
    self.format_command(append, alias, width)
  File "src/west/app/main.py", line 1039, in format_command
    self.format_thing_and_help(append, thing, command.help, width)
  File "src/west/app/main.py", line 1075, in format_thing_and_help
    help_lines[0] = thing + help_lines[0][thinglen:]
                            ~~~~~~~~~~^^^
IndexError: list index out of range

@pdgendt
Copy link
Collaborator Author

pdgendt commented Sep 4, 2024

west help still broken with empty aliases and latest commit f8c61d1

Ugh, sorry about that, and thanks for all the testing efforts! Empty help strings seem to get stripped. I now set an <empty> placeholder. Added it to the test.

@pdgendt
Copy link
Collaborator Author

pdgendt commented Sep 4, 2024

Fixed typo 🤦

Copy link
Collaborator

@marc-hb marc-hb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A few questions and minor suggestions left.

src/west/app/main.py Show resolved Hide resolved
src/west/app/main.py Outdated Show resolved Hide resolved
src/west/app/main.py Outdated Show resolved Hide resolved
src/west/app/main.py Outdated Show resolved Hide resolved
src/west/app/main.py Show resolved Hide resolved
tests/test_alias.py Outdated Show resolved Hide resolved
pdgendt and others added 3 commits September 5, 2024 20:15
Let the builtin help command handle the passed arguments.

Signed-off-by: Pieter De Gendt <pieter.degendt@gmail.com>
Replace west commands with alias set in configuration files

Signed-off-by: Pieter De Gendt <pieter.degendt@basalte.be>
Add simple test cases to verify alias' functionality.

Signed-off-by: Pieter De Gendt <pieter.degendt@basalte.be>
@marc-hb marc-hb merged commit d4ad256 into zephyrproject-rtos:main Sep 6, 2024
9 checks passed
@pdgendt pdgendt deleted the alias branch September 6, 2024 14:13
@pdgendt pdgendt added this to the v1.3.0 milestone Sep 13, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants