Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Bug fix: unable to run migration from compiled files #376

Merged
merged 16 commits into from
Jan 12, 2025

Conversation

lelinhtinh
Copy link
Contributor

I wrote migration using ts but when building and running on production environment, compiled js files have module not found error. The reason is because the extension is fixed as .ts.
Realizing that nodejs and typescript do not require extension so I removed them in this PR.

src/migrator.ts Outdated Show resolved Hide resolved
tests/migrator.test.ts Outdated Show resolved Hide resolved
@ilovepixelart
Copy link
Owner

ilovepixelart commented Jan 11, 2025

Also just a general question you are the second person who compiles migrations, what is the use-case for that?
The whole idea of the package that you put your migrations outside of source then transpile and run them on demand
Check full example here: https://github.com/ilovepixelart/ts-express-swc

Basically you expanding on this idea that was recently added, now I'm thinking this was a mistake to add in the first place)
#365

@lelinhtinh
Copy link
Contributor Author

Also just a general question you are the second person who compiles migrations, what is the use-case for that? The whole idea of the package that you put your migrations outside of source then transpile and run them on demand Check full example here: ilovepixelart/ts-express-swc

Basically you expanding on this idea that was recently added, now I'm thinking this was a mistake to add in the first place) #365

I build migrations with source code, and run it using nodejs, no tsc.

@ilovepixelart
Copy link
Owner

ilovepixelart commented Jan 11, 2025

Also just a general question you are the second person who compiles migrations, what is the use-case for that? The whole idea of the package that you put your migrations outside of source then transpile and run them on demand Check full example here: ilovepixelart/ts-express-swc
Basically you expanding on this idea that was recently added, now I'm thinking this was a mistake to add in the first place) #365

I build migrations with source code, and run it using nodejs, no tsc.

Yeah but why you do it this way? Is there a reason for it? Wait you don't use typescript, in your project?

@ilovepixelart
Copy link
Owner

ilovepixelart commented Jan 11, 2025

We also have programmatic approach of running migrations
https://github.com/ilovepixelart/ts-migrate-mongoose/tree/main/examples
Both programmatic mode and CLI will transpile your migrations on the fly without need to compile, as long as they written in Typescript and your project should use CommonJS compile target

@lelinhtinh
Copy link
Contributor Author

lelinhtinh commented Jan 11, 2025

Also just a general question you are the second person who compiles migrations, what is the use-case for that? The whole idea of the package that you put your migrations outside of source then transpile and run them on demand Check full example here: ilovepixelart/ts-express-swc
Basically you expanding on this idea that was recently added, now I'm thinking this was a mistake to add in the first place) #365

I build migrations with source code, and run it using nodejs, no tsc.

Yeah but why you do it this way? Is there a reason for it? Wait you don't use typescript?

Because if the migration contains other dependencies, for example lodash. Then node_modules must be kept in the build, which is not very reasonable.

@ilovepixelart
Copy link
Owner

ilovepixelart commented Jan 11, 2025

Also just a general question you are the second person who compiles migrations, what is the use-case for that? The whole idea of the package that you put your migrations outside of source then transpile and run them on demand Check full example here: ilovepixelart/ts-express-swc
Basically you expanding on this idea that was recently added, now I'm thinking this was a mistake to add in the first place) #365

I build migrations with source code, and run it using nodejs, no tsc.

Yeah but why you do it this way? Is there a reason for it? Wait you don't use typescript?

Because if the migration contains other dependencies, for example lodash. Then node_modules must be kept in the build, which is not very reasonable.

Whether or not you need node_modules, depends on how you build your stuff, in your case I suspect you bundle your code if you say you don't need node_modules after your build process. Look I'm genuinely interested in your build process now, so can you explain to me what are the benefits for you of bundling code on backend?
The only thing I can can think off, is I/O lookups for files but in that case you should bundle into one file instead of multiple. But if you run your migration files they are stand alone...
Also can you tell me what tool do you use to build?
Just in case you use esbuild as your bundler: https://github.com/ilovepixelart/ts-express-esbuild

@lelinhtinh
Copy link
Contributor Author

For small applications, bundling is useful using tree-shaking, some tools like ncc, webpack supports this. For applications built with nestjs I still keep node_modules, because its dependencies often cause errors.

@ilovepixelart
Copy link
Owner

Yeah exactly what I thought, I guess ncc is a good example surprisingly built on top of web-pack (thought it was dead), but at this point you do basically serverless.

Ok now lets look at this from standpoint of the lib, do we really need all these after compilation use cases
When the whole point of lib is on demand transpile and run from TS files
Lets say you bundle your stuff to js and then run it what is the purpose of the lib I think at this stage we loosing the plot)

What if we improve documentation instead, and describe the intended use

@lelinhtinh
Copy link
Contributor Author

If the library can accommodate both usages, with just a few minor changes, that's even better.

@ilovepixelart
Copy link
Owner

Your tests are hanging indefinitely, probably you are not closing migrator or some dangling connections

@lelinhtinh
Copy link
Contributor Author

lelinhtinh commented Jan 12, 2025

Your tests are hanging indefinitely, probably you are not closing migrator or some dangling connections

The connection in getModels is causing this issue

await mongoose.connect(process.env.MIGRATE_MONGO_URI ?? 'mongodb://localhost/my-db', mongooseOptions)

@ilovepixelart ilovepixelart added the bug Something isn't working label Jan 12, 2025
src/template.ts Outdated Show resolved Hide resolved
.vscode/settings.json Outdated Show resolved Hide resolved
@ilovepixelart
Copy link
Owner

Lets update docs about connection passed as first prop in migration functions

@ilovepixelart ilovepixelart self-requested a review January 12, 2025 11:06
@ilovepixelart ilovepixelart merged commit d02add8 into ilovepixelart:main Jan 12, 2025
4 checks passed
@ilovepixelart
Copy link
Owner

Released in: https://github.com/ilovepixelart/ts-migrate-mongoose/releases/tag/v3.8.8

@bensquire
Copy link

Thanks for this. Just an FYI we also run migrations once they've been compiled into JS.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants