You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
source-googledrive supports grabbing unstructured documents, puts them through Unstructured, and then emits the outputs as rows.
For some usecases, we want google-drive to work in file-transfer mode, similar to what we implemented for source-file and source-s3.
Solution
Config option to use file-transfer mode
In file transfer mode, file filtering glob is still applied
Fetch all files maching the filters, extract them as the corresponding office format (docx for docs, ditto spreadsheets and presentations, pdf for all else)
Transfer these raw files with file transfers mode, do NOT put them thorugh unstructured flow.
Using unstructured.io is still available in the flow that exists today
The text was updated successfully, but these errors were encountered:
Problem
source-googledrive
supports grabbing unstructured documents, puts them through Unstructured, and then emits the outputs as rows.For some usecases, we want google-drive to work in file-transfer mode, similar to what we implemented for source-file and source-s3.
Solution
The text was updated successfully, but these errors were encountered: