Avoid loading the schema from db #242
Closed
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
DBConnection loads the whole schema in its __init__, which takes a lot of time and is not actually needed apart from getting meta data. Though that means the meta checks (schema_type) can't be done that readily, so I've overriden that part for the purpose of my script.
Merged
|
Better version: #295 |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
DBConnection loads the whole schema in its init, which takes a lot of time and is not actually needed apart from getting meta data. In this PR I override the init without the schema loading (that does make the pylint fail obviously).
NB: this is mostly to accelerate the db_factory (which needs to get metadata for all the databases it lists). Without this fix, the db_factory takes minutes to run (instead of seconds) when it tries to read lots of dbs.
That means the meta checks (schema_type) can't be done that readily, so I've overriden that part for the purpose of my script (that said it should probably not even use the SQLalchemy Meta model here: a raw SQL query might actually be better in the initial code).
This PR is not meant to be merged as is as it is probably not the best solution (though it does work).