AWS Cost and Usage Reports is a service provided by Amazon Web Services (AWS) designed to simplify cloud cost management for their customers and provide greater visibility into Reserved Instances and more recently, Savings Plan usage.
To help organizations best utilize the services available to them for cloud cost management and optimization, this article will help explain the benefits of using AWS Cost and Usage Reports, how they work, and a few points to keep in mind as organizations mature in their cloud journey and begin to face more challenging cloud financial management scenarios. Let’s dive in!
What are AWS Cost and Usage Reports?
AWS Cost and Usage Reports is a tool provided by AWS to help customers visualize and track AWS cloud usage and spend. The reports include millions of line items tracking costs, utilization, Savings Plans, Reserved Instances, and other relevant information regarding how much an organization is using and spending on AWS services.
How do AWS Cost and Usage Reports work?
AWS Cost and Usage Reports are published daily in .csv format to an S3 bucket specified by the customer during setup. From there, the report can either be downloaded from the Amazon S3 console and queried using Amazon Athena, or uploaded into other AWS services, such as Amazon Redshift or Amazon QuickSight. These additional AWS services enable customers to query and analyze large datasets.
How do you set up AWS Cost and Usage Reports?
To start creating reports, go to the Cost and Usage Reports page of the Billing and Cost Management console and follow these steps:
- Select “Create Report” and enter a name for the report
- Select “Resource ID” if you want the report to produce an individual line for each resource
- The “Data Refresh Option” gives you the choice of whether to receive a finalized report with or without refunds, credits, support fees and AWS Marketplace costs include
- Either create a new S3 bucket for the reports to be published to or select an existing S3 bucket from the drop-down list and give the service permission to access the bucket
- If you wish to download reports to Athena, or upload reports to Redshift or QuickSight, enter the report path prefix
- For the “Time Granularity” option, select either:
- Hourly, if you want the line items in the report to be aggregated by the hour
- Daily, if you want the line items in the report to be aggregated by the day
- Monthly, if you want the line items in the report to be aggregated by month
- For the “Report Versioning” option, select either:
- Overwrite existing report to save space on S3 storage
- Publish new report to improve auditability of billing data
- If you will be using Athena, Redshift, or QuickSight to query the reports, click “Enable Report Data Integration”
- Review your settings for the report before clicking “Review and Complete” (you can always edit the settings later if necessary)
Note that it can take up to 24 hours for AWS to start delivering reports to your S3 bucket, after which reports are updated up to three times a day. Thereafter, details of how you can query AWS Cost and Usage Reports using Athena, Amazon Redshift, or Amazon QuickStart can be found on the following links:
- Querying Cost and Usage Reports using Amazon Athena
- Loading report data to Amazon Redshift
- Loading report data to Amazon QuickSight
What to consider about AWS Cost and Usage Reports for cloud financial management
AWS Cost and Usage Reports is a valuable tool to help organizations gain visibility and answer questions about cloud costs and usage. However, as organizations mature in their cloud journey, more difficult cloud financial management challenges can arise.
Here are a few points to keep in mind:
Piecing together data among disparate tools
While AWS offers various tools to provide visibility into cloud spend, choosing the right combinations of these tools and understanding how to best use them can be extremely complex.
The complexity is exacerbated when different teams or departments across the organization are using these tools in different ways.
AWS recommends customers use AWS CloudFormation to “streamline and automate the integration of Cost and Usage Reports with Athena.” However, AWS CloudFormation doesn’t support cross-Region resources. The Athena integration setup process also removes any Amazon S3 events that your bucket might already have. AWS recommends that you create both a new Amazon S3 bucket and a new AWS Cost and Usage Report to use with Athena.
AWS tools can come with additional costs that have the potential to get out of hand, especially when distributed teams are operating autonomously without cost management guardrails or policies in place.
For example, if you’re using Amazon Athena to query your Cost and Usage Report, keep in mind that Athena is priced per query, and charges are based on the amount of data scanned by the query (currently, this is $5 per TB).
Additionally, when using AWS Cost and Usage Reports, you’ll continue to be charged standard S3 rates for the storage, requests, and data transfers associated with the specified S3 bucket.
To ensure you know what you’re being billed and how you can optimize your cloud storage costs, see our article explaining S3 storage costs here, and our complete guide to Amazon Cloud Storage pricing here.
Visibility across multiple clouds
When it comes to budgeting and accurately allocating costs, public cloud cost data cannot be limited to just the AWS environment. Even if your organization operates in just the AWS cloud today, it may not be long before your teams begin to embrace a multi-cloud approach, leveraging services offered by other cloud providers.
In 2020, our global assessment of cloud management maturity found that 46.7% of respondents reported operating in multiple clouds (public and/or private), while just 24.3% reported operating in a single public cloud environment.
Many organizations quickly find the need to aggregate visibility and reporting of cloud costs and usage across all environments, which is not currently possible using only AWS services.
Manual work configuring and maintaining data for reporting
As a customer’s cloud usage grows, they often find their teams inundated with manual tasks related to maintaining the efficacy of the reports. As an example, according to the AWS Cost and Usage Reports user guide, report names cannot be edited. If you delete a report set to Overwrite and create a new report with the same name, S3 bucket, and path prefix, the data could corrupt and become inaccurate.
Tagging and creating cloud cost management policies
Cost and Usage Reports—and other AWS services used in combination with the reports—rely on consistent and correct tagging in order to be effective. To help with consistent resource tagging, AWS offers the Tag Policies tool, which enables customers to define tagging policies that apply to accounts and groups established in AWS Organizations.
However, it’s important to keep in mind that customers using AWS Tag Policies will need to anticipate all potential tag and key/value pairs in advance, and will not receive notifications for any usage that does not align with those expectations, or for any resources that are launched without tags. This means that businesses could be alerted to multiple cost anomalies every day with no indication of where or how they are occurring.
Learn more about why the key to a meaningful cloud cost analysis is a global tagging policy.
Simplify AWS cost management
The granular information customers receive from AWS Cost and Usage Reports can be beneficial, but for the reasons we’ve outlined above, it can be overwhelming trying to turn that information into actionable insights for different users across the organization.
Many AWS customers rely on CloudHealth to create distinct Perspectives that focus on the relevant KPIs for different users, so the IT and finance teams are presented with the information that’s most relevant to them in a complete, comprehensive, and easy-to-understand way.
CloudHealth customers can also configure policies directly from the platform’s interface based on custom, defined guidelines—start, stop, terminate, or reboot instances, run lights on/lights off policies, detect and eliminate unattached volumes, and more. Automation can be set up without coding so business users can set governance policies without IT support.
The platform’s policy-driven automation capabilities enable businesses to enforce tag policies that notify you when resources are being launched without tags or with misspelled and misconfigured tags.
If you’d like to learn more about how you can simplify AWS cost management, here are a few of our most popular and helpful resources:
- Whitepaper: Comparing AWS Tools for Cloud Cost Management and Optimization
- Webinar on-demand: AWS Native Tools Fireside Chat and Q&A
- Whitepaper: AWS Cost Optimization: Best Practices to Lower Your Cloud Bill
If you have any further questions or you’re interested in learning more about CloudHealth’s cloud cost optimization capabilities, please don’t hesitate to reach out to our team of experts. We’d be happy to provide you with an inside look into the platform and answer any questions you may have.