Node.js 20 Upgrade: Segmentation Fault Core Dump During Pipeline Lage Build Step

[Umesh-daiict]

Version

20.15.0

Platform

linux-gnu

Subsystem

No response

What steps will reproduce the bug?

1. Use a Linux-based CI agent.
2. Execute the following pipeline commands:
   yarn lage build bundle stage-deployment --concurrency 32
3. Observe the segmentation fault during execution.

How often does it reproduce? Is there a required condition?

The issue with the segmentation fault during the yarn lage build bundle step occurs intermittently. Sometimes the build and bundle process passes successfully, while other times it results in a core dump. There is no specific required condition mentioned for the issue to reproduce.

What is the expected behavior? Why is that the expected behavior?

The expected behavior is for the yarn lage build bundle command to successfully build and bundle the project without encountering segmentation faults. This is expected because the same process worked correctly with Node.js 18, and the upgrade to Node.js 20 should not introduce such critical issues.

What do you see instead?

We are encountering segmentation fault (core dump) when running our pipeline after upgrading to Node.js 20. This issue specifically arises during the execution of yarn lage build bundle steps on a Linux-based CI agent. The core dump logs do not provide sufficient insights into the root cause. This behaviour was not observed prior when running with Node.js 18, it's only been seen when we are trying to upgrade from Node 18.15.0 to Node 20.15.0

Additional information

We have tried multiple of above troubleshooting approaches but are still unable to resolve this issue. If anyone has encountered similar segmentation faults with Node.js 20 or has suggestions for further debugging, please share your thoughts.

1. System Vitals Monitoring:
   ○ Monitored memory usage, agent configurations, and system vitals to identify any anomalies.
   ○ Found no indications of memory pressure or system resource limitations.
   2. Core Dump Analysis:
   ○ Installed Valgrind and a segfault handler to capture and analyze core dump logs.
   ○ Unfortunately, no meaningful insights were captured from the VM or agent machine logs.
   3. Node Version Update:
   ○ Verified and updated all Node.js native modules and dependencies for compatibility with Node.js 20.
   4. Heap Space Configuration:
   ○ Ensured no misconfigurations targeting the "new" space in the heap.
   ○ Adjusted heap settings with the following commands:
   node --max-old-space-size=8192 dist/server.js
   ○ Introduced the --max-semi-space-size parameter to configure the "new" space in the heap.
   5. Node Environment Cleanup:
   ○ Removed older versions of Node.js from the environment to prevent conflicts during pipeline execution.
   Any assistance or guidance would be appreciated. Please tag anyone relevant who can help us out.