How do you take advantage of data access and usage patterns to support your sustainability goals?
8 best practices
"Implement a data classification policy" creates control points that keep operations aligned with business policy, risk, and compliance obligations. Treat ownership, exception han…
"Use technologies that support data access and storage patterns" should be delivered as a standard operating capability with explicit scope, controls, and validation checkpoints.…
"Use policies to manage the lifecycle of your datasets" helps remove wasted effort, unused capacity, and inefficient patterns that degrade cost and performance outcomes. Focus on…
"Use elasticity and automation to expand block storage or file system" should be delivered as a standard operating capability with explicit scope, controls, and validation checkpo…
"Remove unneeded or redundant data" helps remove wasted effort, unused capacity, and inefficient patterns that degrade cost and performance outcomes. Focus on continuous tuning ba…
"Use shared file systems or storage to access common data" aligns people, process, and communication so operational execution remains predictable under pressure. Define responsibi…
"Minimize data movement across networks" should be delivered as a standard operating capability with explicit scope, controls, and validation checkpoints. Embed it into day-to-day…
"Back up data only when difficult to recreate" should be delivered as a standard operating capability with explicit scope, controls, and validation checkpoints. Embed it into day-…
Key Concepts
Sustainability Design Foundations
Data access optimization: Use this concept to guide architecture and operating decisions for this question area. Define measurable targets, assign clear ownership, and review results regularly against expected business outcomes.
Storage lifecycle design: Use this concept to guide architecture and operating decisions for this question area. Define measurable targets, assign clear ownership, and review results regularly against expected business outcomes.
Data minimization: Use this concept to guide architecture and operating decisions for this question area. Define measurable targets, assign clear ownership, and review results regularly against expected business outcomes.
Operational Sustainability Controls
Query efficiency: Use this concept to guide architecture and operating decisions for this question area. Define measurable targets, assign clear ownership, and review results regularly against expected business outcomes.
Data movement reduction: Use this concept to guide architecture and operating decisions for this question area. Define measurable targets, assign clear ownership, and review results regularly against expected business outcomes.
Retention governance: Use this concept to guide architecture and operating decisions for this question area. Define measurable targets, assign clear ownership, and review results regularly against expected business outcomes.
Implementation Approach
1. Understand data usage
- Classify data by access frequency and criticality
- Identify hot, warm, and cold datasets
- Map expensive data movement paths
- Define retention and deletion requirements
2. Optimize storage and access
- Place hot data on performant tiers and archive cold data
- Use lifecycle policies for automated transitions
- Cache repeated reads and precompute frequent aggregations
- Reduce redundant data copies across environments
3. Improve processing efficiency
- Tune queries and partition strategies
- Process data incrementally rather than full scans
- Run batch processing during efficient windows
- Use compression and efficient formats for analytics
4. Govern and refine continuously
- Audit retention policy adherence
- Monitor cost and performance of data workflows
- Retire stale datasets and unused pipelines
- Update access patterns as application behavior changes
AWS Services to Consider
Amazon S3
Delivers highly durable object storage with storage classes and lifecycle controls for performance and cost optimization.
AWS Glue
Builds and automates data cataloging and ETL pipelines to improve data processing efficiency.
Amazon Athena
Runs serverless SQL queries on data in S3 for analytics and operational reporting.
Amazon EMR
Runs scalable big data frameworks for batch and streaming data workloads.
Amazon CloudWatch
Collects metrics, logs, alarms, and dashboards so teams can detect issues early and track operational outcomes.
Common Challenges and Solutions
Challenge: Cold data kept on high-performance tiers
Solution: Automate tiering and lifecycle policies based on access telemetry.
Challenge: Large repeated full-table scans
Solution: Adopt partitioning, pruning, and incremental processing techniques.
Challenge: Data sprawl across environments
Solution: Use governance controls and retention enforcement to remove unnecessary copies.