Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Makefile.in: fix occasional parallel build failures around bison rule #1167

Merged
merged 1 commit into from
Nov 4, 2024

Conversation

trofi
Copy link
Contributor

@trofi trofi commented Oct 28, 2024

Without the change make -j16 --shuffle occasinally fails to build as:

$ make -j16 --shuffle
...
bison -y -d -o util/configparser.c ./util/configparser.y
...
/libtool --tag=CC --mode=compile gcc -I.  -I...-openssl-3.3.2-dev/include -I...-libevent-2.1.12-dev/include -I...-expat-2.6.3-dev/include -DSRCDIR=. -g -O2 -flto -fPIE -pthread  -o configparser.lo -c util/configparser.c
...
util/configparser.c:755:3: error: expected ',' or '}' at end of input
755 |   YYSYMBOL_server_low_rtt = 626,           /* server_low_rtt  */
    |   ^

The build failure happens due to this Makefile.in rule:

util/configparser.c util/configparser.h:  $(srcdir)/util/configparser.y
    @-if test ! -d util; then $(INSTALL) -d util; fi
    $(YACC) -d -o util/configparser.c $(srcdir)/util/configparser.y

For GNU make that means that each of the targets will attempt the rule execution when the file is missing: one for .c file and another for .h file:

https://www.gnu.org/software/make/manual/html_node/Multiple-Targets.html

The workaround is to only run $(YACC) for .c target and use .c as a pre-requisite for an .h file.

Before the change the build fails about every 10-th run. After the change no build failures after 100 successful builds.

Without the change `make -j16 --shuffle` occasinally fails to build as:

    $ make -j16 --shuffle
    ...
    bison -y -d -o util/configparser.c ./util/configparser.y
    ...
    /libtool --tag=CC --mode=compile gcc -I.  -I...-openssl-3.3.2-dev/include -I...-libevent-2.1.12-dev/include -I...-expat-2.6.3-dev/include -DSRCDIR=. -g -O2 -flto -fPIE -pthread  -o configparser.lo -c util/configparser.c
    ...
    util/configparser.c:755:3: error: expected ',' or '}' at end of input
  755 |   YYSYMBOL_server_low_rtt = 626,           /* server_low_rtt  */
      |   ^

The build failure happens due to this `Makefile.in` rule:

    util/configparser.c util/configparser.h:  $(srcdir)/util/configparser.y
        @-if test ! -d util; then $(INSTALL) -d util; fi
        $(YACC) -d -o util/configparser.c $(srcdir)/util/configparser.y

For GNU make that means that each of the targets will attempt the rule
execution when the file is missing: one for .c file and another for .h
file:

    https://www.gnu.org/software/make/manual/html_node/Multiple-Targets.html

The workaround is to only run $(YACC) for .c target and use .c as a
pre-requisite for an .h file.

Before the change the build fails about every 10-th run.
After the change no build failures after 100 successful builds.
Copy link
Member

@wcawijngaards wcawijngaards left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The fix looks very short and nice to be able to build parallel without failure. Thank you for the contribution!

@wcawijngaards wcawijngaards merged commit 46cfbf3 into NLnetLabs:master Nov 4, 2024
1 check passed
wcawijngaards added a commit that referenced this pull request Nov 4, 2024
- Merge #1167: Makefile.in: fix occasional parallel build failures
  around bison rule.
@wcawijngaards
Copy link
Member

The same issue is fixed with a similar fix, for NSD, in NLnetLabs/nsd@c2f82f4 . It had a similar problem, so it could be fixed there too.

@trofi trofi deleted the parallel-build-fix branch November 4, 2024 13:12
jedisct1 added a commit to jedisct1/unbound that referenced this pull request Dec 26, 2024
* nlnet/master: (26 commits)
  - For NLnetLabs#1175, update serve-expired tests.
  - Fix NLnetLabs#1175: serve-expired does not adhere to secure-by-default   principle. The default value of serve-expired-client-timeout   is set to 1800 as suggested by RFC8767.
  - Fix comparison to help static analyzer.
  Changelog entry for NLnetLabs#1169: - Merge NLnetLabs#1169 from Sergey Kacheev, fix: lock-free counters for   auth_zone up/down queries.
  fix: lock-free counters for auth_zone up/down queries
  - Fix for NLnetLabs#1183: release nsec3 hashes per test file.
  - Fix NLnetLabs#1183: the data being used is released in method   nsec3_hash_test_entry.
  - Complete fix for max-global-quota to 200.
  - More descriptive text for 'harden-algo-downgrade'.
  - Increase the default of max-global-quota to 200 from 128 after   operational feedback. Still keeping the possible amplification   factor (CAMP related issues) in the hundreds.
  Changelog entry for: - Fix SETEX check during Redis (re)initialization.
  - Fix SETEX check during Redis (re)initialization.
  - Fix to log redis timeout error string on failure.
  - Fix for the serve expired DNSSEC information fix, it would not allow   current delegation information be updated in cache. The fix allows   current delegation and validation recursion information to be   updated, but as a consequence no longer has certain expired   information around for later dnssec valid expired responses.
  Changelog note for NLnetLabs#1167 - Merge NLnetLabs#1167: Makefile.in: fix occasional parallel build failures   around bison rule.
  Makefile.in: fix occasional parallel build failures around bison rule (NLnetLabs#1167)
  - Fix redis that during a reload it does not fail if the redis   server does not connect or does not respond. It still logs the   errors and if the server is up checks expiration features.
  - Fix redis that during a reload it does not fail if the redis   server does not connect or does not respond. It still logs the   errors and if the server is up checks expiration features.
  Changelog entry for NLnetLabs#1157: - Merge NLnetLabs#1157 from Liang Zhu, Fix heap corruption when calling   ub_ctx_delete in Windows.
  Fix heap corruption when calling ub_ctx_delete in Windows (NLnetLabs#1157)
  ...
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants