Improve the project API and documentation. #668
Conversation
nsthorat
changed the title
| From CLI: | ||
|
|
||
| ```sh | ||
| lilac init ~/my_project |
There was a problem hiding this comment.
curious what the use-case is for having 'lilac init', w/o starting the web server versus 'lilac start'. Asking since rilldata doesn't have init
There was a problem hiding this comment.
Good question -- rill is basically UI-only (and configs). When using the python API, it's really weird that you have to start a server to start a project.
docs/projects/projects.md
Outdated
| lilac.yml | ||
| datasets/ | ||
| open-orca/ | ||
| concepts/ |
There was a problem hiding this comment.
rename to concept/ or change CONCEPTS_DIR to concepts (though we will have to deal with backcompat)
There was a problem hiding this comment.
Renamed, we can deal with that later
lilac/sources/gmail_source.py
Outdated
| @@ -46,8 +49,8 @@ class GmailSource(Source): | |||
| name = 'gmail' | |||
|
|
|||
| credentials_file: str = PydanticField( | |||
| description='Path to the OAuth credentials file.', | |||
| default=os.path.join(_GMAIL_CONFIG_DIR, _CREDS_FILENAME)) | |||
| description=f'Path to the OAuth credentials file. Defaults to {_CREDS_FILENAME} in your Lilac ' | |||
There was a problem hiding this comment.
Defaults to os.path.join(gmail_config_dir, _CREDS_FILENAME)
lilac/sources/gmail_source.py
Outdated
| description='Path to the OAuth credentials file.', | ||
| default=os.path.join(_GMAIL_CONFIG_DIR, _CREDS_FILENAME)) | ||
| description=f'Path to the OAuth credentials file. Defaults to {_CREDS_FILENAME} in your Lilac ' | ||
| 'project directory.') |
There was a problem hiding this comment.
Since credentials_file is marked as required in line 59 below, we should set a default
There was a problem hiding this comment.
Done, I made this a default_factory so that it is fresh when environment variables change.
| config: A Lilac config or the path to a json or yml file describing the configuration. The | ||
| contents should be an instance of `lilac.Config` or `lilac.DatasetConfig`. When not defined, | ||
| uses `LILAC_PROJECT_DIR`/lilac.yml. | ||
| overwrite: When True, runs all data from scratch, overwriting existing data. When false, only |
There was a problem hiding this comment.
curious why we no longer pass overwrite to _compute_embedding?
There was a problem hiding this comment.
We were doing the check for existence of signals/embeddings in 2 places, outside the worker, and inside. Now I just check overwrite outside the worker before scheduling a task. I have a unit test to make sure overwrite actually overwrites.
Docs:
API:
lilac initandlilac loadset_project_dirto set the environment variable globally.Details:
LILAC_DATA_PATHin favor ofLILAC_PROJECT_DIR.