This repository has been archived by the owner on Nov 1, 2023. It is now read-only.
The agent should delete output files after they are uploaded to Azure #3447
Labels
enhancement
New feature or request
As stated, for any output files uploaded from a task, the 1F agent should delete the source files from disk after the
azcopy sync
operation has finished.We need this because we produce some pretty large files from our
generic_analysis
task, and they have the potential to fill up the OS disk if they aren't cleaned up after they have been uploaded.The most likely place that this could be implemented is in
SyncedDir::sync
, adding an additionaldelete_src: bool
parameter.Also note that it does not appear that the
azcopy
team will implement this functionality:Azure/azure-storage-azcopy#990
AB#164737
The text was updated successfully, but these errors were encountered: