Selecting storage solutions for ML datasets
AWS Cloud provides a wide range of storage solutions that can be used to store inference and training data. When choosing an optimal storage solution, you may consider the following factors:
- Data volume and velocity
- Data types and associated metadata
- Consumption patterns
- Backup and retention requirements
- Security and audit requirements
- Integration capabilities
- Price to store, write, and read data
Carefully analyzing your specific requirements may suggest the right solution for your use case. It’s also typical to combine several storage solutions for different stages of your data life cycle. For instance, you could store data used for inference consumption with lower latency requirements in faster but more expensive storage; then, you could move the data to cheaper and slower storage solutions for training purposes and long-term retention.
There are several types of common storage types with...