Overview of your Storage on DeepSense

Overview

DeepSense is a platform for AI/ML for oceans research data. You have store the project data for your DeepSense projects. DeepSense is not meant for long term data storage. Data will only be stored so long as your project is ongoing. Once a project is completed, it is expected that the users will remove their data in a timely fashion.

DeepSense is not intended to be used for data sharing. While each user in your project/group will have access to shared space, it won't be accessible by any other users.

Your data is stored on cloud storage. The cloud storage types are similar to the storage types in the on-premises data centers. You can store your data as objects in cloud buckets, cloud block devices like a hard drive on your computer, or even NFS shared file systems which provides very high performance.

Your data is very safe in the cloud storage because DeepSense applies security measures at every layer of the cloud architecture. DeepSense also helps you migrate your data from your local disk to the secured cloud storage. DeepSense provides you detailed instructions on how to access your and operate your data.

You may be confused which storage is the right choice for your workload. DeepSense will help you analyze the nature of your data and the requirements of your workload to provide you the best storage solutions.

Below are the explanations of the cloud storage solutions mentioned above.

Amazon EBS - Elastic Block Store (hard drives in cloud)

Amazon Elastic Block Store (Amazon EBS) provides persistent block storage volumes in the AWS Cloud for use with Amazon EC2 instances. To protect you from component failure and to provide high availability and durability, each Amazon EBS volume is automatically replicated within its Availability Zone. Amazon EBS volumes provide the reliable and low-latency performance required to run your workloads. You can scale your usage up or down in minutes with Amazon EBS, all while paying a low price for only what you provision.

Amazon S3 - Simple Storage Service (object storage in cloud)

Amazon S3 is an object storage service that provides industry-leading scalability, data availability, security, and performance. Customers of all sizes and industries can use it to store and protect any amount of data for a variety of use cases, including websites, mobile apps, backup and restore, archiving, enterprise applications, IoT devices, and big data analytics. Amazon S3 offers simple management features that allow you to organize your data and configure fine-grained access controls to meet your specific business, organizational, and compliance needs. Amazon S3 is built to have super high durability of 99.999999999% (11 9s) and stores data for millions of applications for businesses all over the world. Amazon S3 supports versioning and deletion protections. You never worry the accidental deletion of your files. Also, you will never have to stop your workload after running for a long time due to storage quota issues that happen frequently on on-premises systems. Amazon S3 provides unlimited storage.

Amazon S3 Storage Tiers

Amazon S3 provides several tiers of storage for different use cases. This can be very cost-effective if you choose a storage tier that matches your access pattern. For example, Amazon S3 Glacier (S3 Glacier) is a safe and long-lasting service for low-cost data archiving and backup. With S3 Glacier, you can store your data for months, years, or even decades at a low cost. You can consult DeepSense to learn the best storage solution for your use cases.