From 87be5b1d43274b8ff498eab5def5a50c8a26b250 Mon Sep 17 00:00:00 2001 From: AnthonyDShaw Date: Mon, 7 Oct 2024 11:00:07 +1300 Subject: [PATCH] Update 02-unix-shell.md --- _episodes/02-unix-shell.md | 60 ++++++++++++++++++++++++++++++++++++++ 1 file changed, 60 insertions(+) diff --git a/_episodes/02-unix-shell.md b/_episodes/02-unix-shell.md index 00194109..a51c90f1 100644 --- a/_episodes/02-unix-shell.md +++ b/_episodes/02-unix-shell.md @@ -28,6 +28,66 @@ keypoints: - "Directory names in a path are separated with `/` on Unix, but `\\` on Windows." - "`..` means 'the directory above the current one'; `.` on its own means 'the current directory'." --- +The NeSI filesystem looks something like this: + +![The file system is made up of a root directory that contains sub-directories +titled home, nesi, and system files](../fig/NesiFiletree.svg) + +The directories that are relevant to us are. + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
LocationDefault StorageDefault FilesBackupAccess Speed
Home is for user-specific files such as configuration files, environment setup, source code, etc./home/<username>20GB1,000,000DailyNormal
Project is for persistent project-related data, project-related software, etc./nesi/project/<projectcode>100GB100,000DailyNormal
Nobackup is a 'scratch space', for data you don't need to keep long term. Old data is periodically deleted from nobackup/nesi/nobackup/<projectcode>10TB1,000,000NoneFast
+ +### Managing your data and storage (backups and quotas) + +NeSI performs backups of the `/home` and `/nesi/project` (persistent) filesystems. However, backups are only captured once per day. So, if you edit or change code or data and then immediately delete it, it likely cannot be recovered. Note, as the name suggests, NeSI does **not** backup the `/nesi/nobackup` filesystem. + +Protecting critical data from corruption or deletion is primarily your +responsibility. Ensure you have a data management plan and stick to the plan to reduce the chance of data loss. Also important is managing your storage quota. To check your quotas, use the `nn_storage_quota` command, eg + +{% include {{ site.snippets }}/filedir/sinfo.snip %} + +As well as disk space, 'inodes' are also tracked, this is the *number* of files. + +Notice that the project space for this user is over quota and has been locked, meaning no more data can be added. When your space is locked you will need to move or remove data. Also note that none of the nobackup space is being used. Likely data from project can be moved to nobackup. `nn_storage_quota` uses cached data, and so will no immediately show changes to storage use. + +For more details on our persistent and nobackup storage systems, including data retention and the nobackup autodelete schedule, +please see our [Filesystem and Quota](https://docs.nesi.org.nz/Storage/File_Systems_and_Quotas/NeSI_File_Systems_and_Quotas/) documentation. + > ## The Unix Shell > > This episode will be a quick introduction to the Unix shell, only the bare minimum required to use the cluster.