Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Made changes to
eval.py
to accommodate creation of temporary postgres databases not found in the standard 7 dbs of defog-data.There is now an option to create (and drop) temporary DBs if the
db_name
contains the suffix "_temp". Thetable_metadata_string
col containing the DDL statements for this temp DB must also be provided in the samedata/your_custom_qns.csv
file. Note that it's important that the DDL statements contain the insertion of VALUES so that we can evaluate on non-empty dataframes.Additional note: to prevent race conditions during creation and dropping your temporary databases, it's best to give each question a unique db_name like
temp_db_name = db_name + question + "_temp"
ortemp_db_name = db_name + row_index + "_temp"