Selective independent package management in Intergrated mode, and Regarding colocation.

[p-iknow]

Hello. I am reaching out because I need information about our team's technical decision-making on using Nx.

question

• Is it possible to selectively manage certain packages separately in nx integrated mode? We have packages that need to adhere to the single version policy and those that don't, at the same time. Therefore, we would like some packages to optionally not follow the single version policy. e.g.(Migrate From nx to turbo woocommerce/woocommerce#32656)

• Is there a way to efficiently utilize the caching strategy without creating a separate feature library when using the next.js app folder in nx? I aim to achieve colocation through this. Just like the structure suggested by Next.js below.
  

Backgorund

NX monorepo offers two options: package-based and integrated. After reading the article at https://nx.dev/concepts/integrated-vs-package-based, we chose 'integrated' for a more unified environment setup. However, after making this choice, we faced two issues.

Problem

One root Package is allowed, and Managing selectively certain packages is not allowed.

Integrated mode enforces a single package.json dependency definition in the root file. However, sometimes specific packages need to be used differently for each service. Otherwise, a regression test of all versions that depend on that package is necessary to update one package.

Given our team's resources and circumstances, performing all regression tests automatically or manually is difficult. As a result, package updates become a source of fear, and nobody performs package updates. Or, a package bump is carried out without verification, which can lead to unexpected runtime bugs.

Is there a way to have separate dependencies for each service only for a few packages that are not shared in integrated mode?

Colocation for next.js app folder

Another issue is with the folder structure NX suggests in this article https://nx.dev/more-concepts/monorepo-nx-enterprise. In this structure, there's a folder called "feature".

This folder is a library that is dependent on a specific app (service). This means it should not be relied upon by other apps. Ultimately, code dependent on a specific app ends up in a library separate from the app, reducing the cohesion of the code. We assume this folder strategy was chosen to enable individual caching and build strategies for each feature.

However, recently, the importance of colocation is often mentioned in the frontend scene. The claim is that "code that should be together is more efficient when it is together." Recently, next.js is promoting a transition to the app folder and presenting a folder structure that can express colocation.

the structure suggested by Next.js below.

Having features for a specific app under the lib folder due to caching undermines colocation. Our team is contemplating how much loss there is in the caching strategy when such features are included in the apps folder or whether there is a way for the caching strategy to work effectively still, even if feature libraries exist in the apps folder.

Do you have any advice for this issue?

Another Background

we're a web frontend team developing dedicated web pages for browsers and web view pages. Our stack mainly consists of React and Next.js, and we aim to integrate non-conforming codes into Next.js.

We operate in units called squads, each handling a specific area of the application. The frontend codes are divided into poly repos according to the squad units. Common library functions, including the design system, are uploaded as packages to npm, and each service repo depends on these common libraries.

The existing poly repo structure had the following problems

• Each service was not consistently updated with the common library, causing a significant version bump whenever we resumed development on a particular service. This led to the daunting task of catching up with all missed updates.

• Any changes to the common library required testing across all dependent repositories. Given the npm-based dependency management, this involved releasing the package and then installing it in the service repository for testing. This process was quite cumbersome and often led to code changes without fully verifying the coverage of all dependent codes.

• Given the risks of modifying the common library, teams started writing codes within their assured scope. Consequently, code-solving the same problems was scattered across different services, leading to duplication of problem-solving efforts across different teams.

• There were instances when, due to project requirements, we needed to use an interface identical to another team. We would copy the code from the other team's repository and paste it into ours. However, whenever the API specifications in the original code changed, the copied code in our repository was not updated, leading to occasional failures.

We chose NX to resolve these issues, and some services have been migrated to an NX monorepo.

[ZackDeRose]

Is it possible to selectively manage certain packages separately in nx integrated mode? We have packages that need to adhere to the single version policy and those that don't, at the same time. Therefore, we would like some packages to optionally not follow the single version policy. e.g.(woocommerce/woocommerce#32656)

Yes, even with an integrated approach, you can enable project-level dependency overrides.

You'd do this by:

• turning on the workspaces functionality of your package manager of choice
• adding the projects that need to override dependencies to that workspaces collection
• specifying dependencies in the package.json of those projects
• now your package manager will provide the correct deps to the project in question while maintaining an integrated style
  (@juristr or @jaysoo to confirm this is correct - I hadn't done this specifically myself yet)

While you can do this, I'd advise against it if possible. You mention:

Given our team's resources and circumstances, performing all regression tests automatically or manually is difficult. As a result, package updates become a source of fear, and nobody performs package updates. Or, a package bump is carried out without verification, which can lead to unexpected runtime bugs.

It seems like the issue of updating packages wreaking havoc would happen whether you have a single version policy or not - a single version policy will likely surface the thing going wrong faster, but splitting the dependencies so you have different versions of the same package throughout the repo will have the same problems, but more drawn out - while also adding a second problem of remembering which project uses which version.

Is there a way to efficiently utilize the caching strategy without creating a separate feature library when using the next.js app folder in nx? I aim to achieve colocation through this. Just like the structure suggested by Next.js below.

This one gets tricky. The straight-forward answer here is that Nx caches tasks around project lines. Next seems to be pursuing a strategy that enables caching at a much more granular level with their investment in Turbopack. If I understand their intentions correctly, they want to cache your build down to almost the AST level. If they can make it work this would be very exciting. But from Nx's standpoint, Turbopack is simply another builder. Nx will cache the artifacts listed as a Turbopack target's output just fine, and our @nx/next:server executor supports the --turbo flag to turn on the experimental Turbopack builder.

So, leveraging turbopack while still using Nx may give you the situation you are looking for. (note turbopack is a bundler that Nx will support, vs. Turborepo, which is a tool that operates at the same level as Nx. As Turborepo and Nx are redundant, we don't intent to support integration there).

Depending on your motivations though, the ability to nest Nx projects may be intriguing. In an integrated monorepo, a "project" defined as any directory that contains a valid project.json file. This could potentially give you the file structure you are looking for. Rather than:

apps/
  my-app/
    src/
    project.json
libs/
  my-button/
    lib/
    project.json

As you might see in a typical Nx integrated approach, you could do:

projects/
  my-app/
    my-button/
        <ts files>
        project.json
    <ts files>
    project.json

Your Nx project graph in either case would be the same, but your folder structure would be different.

The latter does make it more confusing if you want to share a single projects between multiple apps. The former setup is often chosen so that my-button can be re-used across multiple applications as your repo grows. Your reasoning here:

This folder is a library that is dependent on a specific app (service). This means it should not be relied upon by other apps.

Seems a bit confused on this point specifically. My general advice is actually to not break off code from an application UNTIL you want to reuse it across other apps or libs.

I hope this advice is helpful. Please feel free to follow up if any other questions come up.

[melutovich]

@ZackDeRose you wanted confirmation from @juristr and @jaysoo but I do not see an explicit confirmation here from either yet, has this been confirmed yet?

[melutovich]

@p-iknow Do you have any update or advice to share with others?