Replies: 4 comments 6 replies
-
|
This will lead to a community fork that everyone will adopt or mass exodus to something else. |
Beta Was this translation helpful? Give feedback.
-
|
Looks like there are not so much people care... Maybe because NX docs are so complex, that majority of people even dont understand how all this features work and dont use them. |
Beta Was this translation helpful? Give feedback.
-
|
This change is indeed critical for businesses of all sizes. Custom task runners provide immense flexibility and adaptability, especially for teams with unique workflows or security compliance requirements. For example, using custom runners allows us to:
Given our constraints and compliance needs, we’re already exploring Nx alternatives. Removing custom task runners creates a significant gap that many teams, including ours, may find challenging to bridge without substantial effort or compromise. We truly hope the Nx team reconsiders this change or provides a viable alternative for self-managed setups. |
Beta Was this translation helpful? Give feedback.
-
|
https://turbo.build/blog/free-vercel-remote-cache Looks like Vercel are caching in on this. |
Beta Was this translation helpful? Give feedback.
-
Well played. |
Beta Was this translation helpful? Give feedback.
-
|
Thank you for raising these use cases to us. We understand that custom task runners have been used for a myriad of reasons. We're not trying to block users from doing those things, but instead trying to get better control over how things are done. Some of the use cases discussed are being or have already been addressed:
This is something we are working on. Stay tuned for more information.
Root tasks are indeed a thing. The root project of a workspace can have targets and are a better place for many of these things. Many of the use cases discussed do not and should not require a custom task runner.
Each executor and generator should read the feature flags. I don't think that belongs in the task runner. In addition, this is an example of how some of the assumptions of Nx can be broken by these custom task runners. Nx will cache outputs of executors based on the inputs. It will not invalidate cache for feature flags from an external source unless there is a runtime input which prints the feature flag information.
This is the behavior of
Neither of these checks should be done as part of running tasks. This can either be a conformance rule or a root task which is run before others in the pipeline.
This is available with
Sorry, I'm not familiar with Projen, if you create a separate feature request we can discuss it there.
Both Nx and Nx Cloud can be run on prem without talking to any external cloud providers and are used in organizations that have some of the most stringent security policies. Let us know what you are currently doing and we can advise you on an alternative method which does not involve custom task runners.
What kind of performance improvements have you made? Would you mind sharing them with us so we can provide those improvements to more Nx users?
What are your security policies? Reach out to us and we can review your policies and see if Nx Cloud can be used. Again, thank you for providing these use cases to us. We've learned many interesting use cases recently some of which we are looking to provide to more users in an official manner. Please continue to discuss the use cases of custom task runners. Let us know your use cases and we can advise on an alternative method to do the same things. |
Beta Was this translation helpful? Give feedback.
-
|
Thanks for the responses.
To be clear, I'm saying we have our task runner wired into our own analytics tooling. I don't think Nx itself would be able to help in this area.
After writing this, I did end up exploring root targets. It was a bit awkward to make it act like a project that wasn't a project, but it did work. It doesn't really solve most of the problems above though as just being another task in the chain is very different than being the orchestrator of the tasks. The only use case that ended up working as a root target was for a "run once at the end for the whole repo" report task.
This is one of the "run once" use cases I referred to. We use file-based feature flags in all projects and in the root. (Think like how most JS tools like ESLint work with merging configs.) We have to crawl the repo and assemble a list of those flags and other metadata around package types. That's an awaited operation aka takes time. With a task runner, we can cache the result of that in module state and then when ever executor asks for feature flags, it gets that cached response. You could store it on disk, but that'd be much slower. This should sound pretty familiar to how parts of Nx work. ;)
It's not that simple. We have several other cache options beyond local cache, and with other systems like package and tool managers doing their own caches. With a task runner, we can take env variable data from the CI runner that are used by other tools and then set the flag that Nx wants to turn off its cache, the
I disagree. There's nothing wrong with using the entrypoint to the task graph to run some precursor logic. I'm pretty sure we did this long before root tasks were a thing anyways. I'm talking about states that we would consider invalid for a repo to be in and we need to immediately throw on them. There's no point in muddying up the task graph on the inference side to try to get special tasks to run as dependencies of many other ones. It's a separation of concerns. It's possible the config errors could lead to inference errors too and then we're worse off.
As many people have said, this was always encouraged by the Nx team and is now being paywalled after the fact. There's nothing particularly magical about a remote cache implementation that warrants the cost. It's around 100-200 lines of code for us.
You can think of it as a more advanced generator. It lets you express project files and configs as classes and assemble them into a hierarchy. We glue its capabilities to the virtual file system and in a couple other ways. |
Beta Was this translation helpful? Give feedback.
-
I just wanted to say +1 for the custom analytics/observability We use datadog and it's really useful to be able to see what tasks are being used, and how long they take, and what errors are happening, etc If Nx is working on a feature to support this, then I'd think it would need to be pretty extensible/customizable, in order to be adaptable to the different observability modeling idiosyncrasies of a given org |
Beta Was this translation helpful? Give feedback.
-
|
Thank you so much for Zooming with us yesterday. Jason and I have a design that should address your use cases. We will publish an RFC early next week and provide an implementation for you to try. Other folks on the thread: I will link the RFC here once it is published. Please check it out to see if it addresses your use case. |
Beta Was this translation helpful? Give feedback.
-
|
This is the RFC: #29637 Please take a look and leave feedback. If you have a lot of nuanced feedback that requires a lot of back and forth, we can jump on a call to discuss it. |
Beta Was this translation helpful? Give feedback.
-
I wanted to take a minute and inform the team of the myriad of other ways we use custom task runners, beyond what the blog mentioned. I don't see how any of this will work after Nx 21.
In no particular order
As you can see, a lot of it is run once/at the beginning use cases that only task runners provide. It gives us guardrails and integrations with other tools. If root tasks were a thing, it might work, but task runners are at the perfect spot to do all of these things today.
Beta Was this translation helpful? Give feedback.
All reactions