Overview of your Storage on DeepSense

Overview

DeepSense is a platform for AI/ML for oceans research data. All users will have a home directory, and access to various filesystems. Each filesystem has a default quota, with more space available upon request. While some of the filesystems are backed up, it is the responsibility of the user to maintain their original data on their own site.

DeepSense is not meant for long term data storage. Data will only be stored so long as your project is ongoing. Once a project is completed, it is expected that the users will remove their data in a timely fashion.

DeepSense is not intended to be used for data sharing. While each user in your project/group will have access to shared space, it won't be accessible by any other users. We do not host databases for sharing data, or for web access.

The filesystems provided are a resource shared by all users. It is expected that users make sensible use of the space, and follow the guidelines outlined here. It is possible the quotas and policies will change in the future, but we will strive to provide plenty of notice.

Filesystems

There are several different filespaces available to users. They are shared filesystems that can be accessed from any of the nodes. They each have separate purposes. See also Transferring Data.

Home directory

Each user has a home directory in /dshome/subdirectory/. The subdirectory (visitor, research, faculty, grad, etc.) will depend on the type of LDAP account you have. This is primarily designed for your personal use, and only you have permission to access it. It is ideal for managing scripts, source code and test data sets. It is not meant for large data storage.

Data

Each user/project will have access to a directory in the data filesystem.

• If you are a member of a project, the directory will be /data/projectname (or groupname). Everyone in your project group will have access to this directory.
• If you are an individual student, your directory is /data/username. Only you will have access to this directory.

The data filesystem will house the bulk of your data, and it has a larger default quota. This is also the primary location for transferring large amounts of data. It is accessible via samba.

Scratch

Each user/project will have access to a directory in the scratch filesystem.

• If you are a member of a project, the directory will be /scratch/projectname (or groupname). Everyone in your project group will have access to this directory.
• If you are an individual student, your directory is /scratch/username. Only you will have access to this directory.

The scratch filesystem is intended to support data used during job execution. It has a larger default quota, and can support a larger number of files. Note: this is temporary space, and is not backed up. Data which has not been accessed in 60 days may be purged, though we will contact you prior to doing so. Data needed for longer storage should be stored elsewhere.

Delete files that you no longer need as soon as you are done with them, rather than leaving large amounts of data sitting untouched.