With Amazon Web Services (AWS) planning to retire its Detailed Billing Reports, AWS users are being encouraged to enable Cost and Usage Reporting. Our Guide to AWS Cost and Usage Reports explains the differences between the two reports and suggests an easier route to better understanding the new format.
It’s hard to believe that it wasn’t until 2013 that AWS users were able to gain programmatic access to billing statements that included activity by resources and by tags. That was the year AWS released detailed billing reports (DBR)—a statement of all billing charges on an hourly basis with both tag and resource information, all delivered to an S3 bucket.
However, in December 2015, AWS announced its new Cost and Usage Reports. The new customizable reports had the advantages of making cost data much easier to understand and analyze, and giving greater visibility into Reserved Instance usage.
To rationalize and simplify their cost reports, AWS announced that Detailed Billing Reports and the data-intensive Detailed Billing Reports with Resources and Tags—along with AWS Monthly Reports and AWS Cost Allocation Reports—would be phased out in due course. Although this has not yet happened, businesses that have not yet enabled AWS Cost and Usage Reports are advised to do so as soon as possible.
What are AWS Cost and Usage Reports?
AWS Cost and Usage Reports track the usage of AWS resources and estimate the charges associated with each account. By default, data is presented for each service category used by an account and its IAM users in hourly or daily line items as indicated in the simplified image below:
The reason for the image being simplified is due to the vast number of reporting options available. Only seven of the possible twenty “Line Item Details” have been included in the image, plus the option exists to include the following in each AWS Cost and Usage Report:
- Identity Details
- Billing Details
- Reservation Details
- Pricing Details
- Product Details
- Resource Tags
Whereas most of the above “details” are self-explanatory, the Resource Tags option is possibly a little more extensive than many users will assume. Not only can you use the “user:NameTag” option to track which resources are associated with a specific user, you can also use the tag option to identify who created a resource, who owns the resource, and the purpose of the resource being created.
As with the Detailed Billing Reports, AWS Cost and Usage Reports are automatically updated periodically and uploaded to a user-specified assigned S3 bucket as a .csv file. The reports are cumulative per accounting period and so the estimated charges for each accounting period can vary according to the increased or decreased usage of resources. AWS finalizes the report at the end of the month to include any refunds, credits, support fees and AWS Marketplace costs.
The difference to Detailed Billing Reports
The difference between AWS Cost and Usage Reports and Detailed Billing Reports mostly revolves around the number of reporting options available. A few of the former options have disappeared (i.e. RecordType, RecordId, RateId, and PricingPlanId) and several have been renamed (i.e. the former “ReservedInstance” field has been renamed “pricing/PurchaseOption”), but overall there is a vast increase in the number of reporting options available.
A further difference is how Reserved Instance usage is reported. Reserved Instances are given a unique reservation ID number in order that users can track each Reserved Instance and identify when it is applied to an instance and for how long. The Reservation Details section of the new AWS Cost and Usage Reports also include amortization data for Reserved Instances so users can see how much of the charge for All Upfront or Partial Upfront Reserved Instances has been used during the month.
The increased number of reporting options and more detailed information about Reserved Instances will make it easier for AWS' customers to analyze and control their cloud spend. As with Detailed Billing Reports, the AWS Cost and Usage Report .csv file in the S3 bucket can be uploaded to Amazon Redshift or Amazon QuickSight for deeper analysis—although it is important to note that columns in Amazon Redshift have stricter character limitations than user-defined tags, and problems can occur when additional columns are added to Cost and Usage Reports mid-month.
How to set up cost and Usage Reporting
Even if you are not yet ready to switch from Detailed Billing Reports to Cost and Usage Reporting, it is recommended you set up the new reporting facility at the earliest possible opportunity. This will enable you to run the two reports side-by-side to familiarize yourself with the new format and options, and it will also allow a history of data to build up in your S3 bucket for analysis. The process for setting up Cost and Usage Reporting is straightforward:
- Open the Billing and Cost Management Console within the AWS Management Console
- Select “Reports” from the navigation pane
- Select “Create Report” and enter a name for the report
- Select either an “Hourly” or “Daily” Time Unit.
- Select “Resource IDs” to include reservation ID numbers for Reserved Instances
- If you wish to upload reports to Redshift or QuickSight, select “Enable Support for [your choice]”
- The “Data Refresh Option” gives you the choice of whether to receive a finalized report with or without refunds, credits, support fees and AWS Marketplace costs included.
- Finally enter the name of the S3 bucket reports should be delivered to and the report path prefix you want prepended to the name of your reports.
It can take up to 24 hours for AWS to start delivering reports to your S3 bucket, after which reports are updated up to three times a day. Ultimately, businesses will have no option but to use AWS Cost and Usage Reporting, so although the prospect of change can appear daunting if you have been using Detailed Billing Reports for a long time, it is well worth migrating to the new reports sooner rather than later. The likelihood is you will benefit from making the switch due to new cost-saving opportunities being identified.
How to further simplify AWS Cost Analyses
Receiving more data with greater amounts of detail and having greater visibility into Reserved Instance usage, is undoubtedly beneficial; but for some businesses this may be overwhelming to analyze and interpret. Additionally, in many cases, more than one department is interested in how AWS resources are being utilized and how costs and performance can be optimized, and compiling different data sets for different departments can be both complicated and time-consuming.
CloudHealth´s cloud management platform overcomes this potential problem via “Perspectives” - a capability of the platform that enables users to group assets into dynamic business groups by environment, function, finance, owner and more. The groups can be arranged to resolve individual requirements, or combined to create specific data intersections such as “Environment by Customer”, or “Product Line by Environment”—simplifying AWS cost analyses, and saving time and money.
The savings that can be achieved are further enhanced by CloudHealth's rightsizing recommendations, policy-driven automation tools, and recently-announced Reserved Instance Exchanger - an engine that analyzes Convertible Reserved Instance inventories, evaluates how well Reserved Instances are being used, analyzes on-demand usage, and runs the permutations to identify the mathematically optimal exchange businesses can make.
To find out more about the cost-saving capabilities of CloudHealth, the differences between AWS' Detailed Billing Reports and Cost and Usage Reporting, or the “Perspectives” capability of our cloud management platform, our team will be happy to answer your questions.