A recent report from Gartner - “How to Identify Solutions for Managing Costs in Public Cloud IaaS” - identifies the three key challenges to managing cloud costs, and makes recommendations about how best to evaluate cloud service expense management (CSEM) solutions to overcome these challenges.
At the beginning of the year, Gartner forecasted: “Through 2020, 80% of organizations will overshoot their cloud IaaS budgets due to a lack of cost optimization approaches”. The company attributed the failure to manage cloud costs to three key challenges organizations were struggling to overcome:
- Complex multi-cloud environments are becoming commonplace, and billing details vary dramatically by the provider.<
- Many I&O teams are operationalized for traditional data center principles rather than cloud IaaS, and they lack the organizational processes to manage costs in the cloud.
- There are many options to address cloud expense management. As a result, I&O leaders may struggle to align options with the organizational cloud strategy.
In the report - “How to Identify Solutions for Managing Costs in Public Cloud IaaS” - Gartner highlights the complexity of managing cloud costs by listing the number of components that contribute towards the price point of a single instance (Regions, Operating Systems, Instance Types, etc.) and discusses how much more complicated it becomes to manage cloud costs in a multi-cloud environment.
According to previous research conducted by Gartner, 86% of organizations with a public cloud adoption strategy expect to use multiple public cloud providers by 2020 - more than half of which will be using three or more providers. The company feels that, as organizations grow in their use of multi-cloud environments, their ability to manage cloud costs often fails to keep pace.
The solution to managing cloud costs
Gartner makes several recommendations to overcome the challenges of managing cloud costs - the first being to deploy a cloud service expense management (CSEM) solution. However, the recommendation comes with the caveat of deciding which solution to deploy only after taking a holistic view of the overall cloud management strategy, and not selecting a solution on an exclusively cost management basis.
It is pointed out in the report that many CSEM solutions have been built out from their core functionalities to eliminate the need for overlapping solutions that cover provider native services, cloud management, and IT financial management. Indeed, Gartner notes that “cost in the cloud is tied directly to ongoing consumption, so managing utilization is inextricable from managing expenses”.
The report also notes “as an organization matures in the cloud, the oversight of a CSEM tool mitigates risk in addition to providing cost optimization. For example, when development teams are able to self-service infrastructure, the value of CSEM solutions to the infrastructure team is in operational integrity as much as explicit cost savings”. Consequently, any CSEM solution deployed within an organization should factor in performance optimization and security in addition to managing cloud costs.
Another solution is to establish a Cloud Financial Management practice to help manage cloud costs. Cloud Financial Management (CFM), also known as or Cloud Cost Management, is a function that helps align and develop financial goals, drive a cost-conscious culture, establish guardrails to meet financial targets, and gain greater business efficiencies. Learn more about establishing a Cloud Financial Management practice here.
Evaluating cloud cost management solutions
With this in mind, Gartner recommends evaluating cloud cost management solutions based on eleven core functions. The list of functions is reproduced below - each one being followed with a description of how the CloudHealth platform fulfills the evaluation criteria. However, please note this is not the full range of CloudHealth services - further details of which can be found here.
- Allocation and Tagging - Manage the provider's native tagging and augment it with an overlay of allocation tags. Many also allow administrators to detect resources without tags.
We strongly recommend implementing a global tagging policy that takes into account inconsistencies between Cloud Service Providers with regard to the characters allowed and the maximum value lengths. The CloudHealth platform monitors compliance with the global tagging policy to identify untagged resources, misspelled Tags and Labels, and non-conforming values.
- Role-Based Access Control — Support for roles on users and groups, controlling access and privileges.
Cloud security starts with users and access controls. Without proper access controls and identity management, users can intentionally or unintentionally create security flaws. CloudHealth allows administrators to set policies that will validate securely configured access to resources, and help organizations stay ahead of breaches by monitoring for non-compliant activity.
- Discount Management — Ability to display the savings generated by discounts or enterprise agreements (for example, AWS's Enterprise Discount Program [EDP] or Microsoft's Enterprise Agreement).
The CloudHealth platform utilizes a “greedy knapsack” algorithm that compares all possible discounts and enterprise agreements to achieve the optimal outcome. The algorithm allows you to see what your infrastructure will look like after taking advantage of the savings, and export different scenarios for comparison.
- Budget Management — Define budgets over various time periods and by cost allocation segment, user or group. Visualize current spend and project budget overruns.
CloudHealth enables organizations to define budgets by customizable business units in order to compare actual spend against budget. Policies can be created to notify budget owners when spend exceeds month-to-date budget - or when projected spend is likely to exceed budget - in order to help with managing cloud costs and keeping them under control.
- Dashboard and Reporting — Visualize data on multiple interactive dashboards, create and customize dashboards or reports, and associate them with specific roles.
Using the Perspectives capability, CloudHealth lets organizations decide how they want to visualize and assemble cloud and hybrid infrastructure assets and services for analysis, management, evaluation, monitoring, and measurement. By being able to create reallocation rules, organizations can identify which individuals, teams, departments, or applications are accountable for driving costs.
- Spending Anomaly Detection — Identify changes in consumption or trends, and report or trigger actions upon detection.
It is not always advisable to trigger stop or terminate actions when spending anomalies are detected because there may be a good reason for the anomalies. Although CloudHealth provides this capability, a more appropriate solution is to use policy-driven automation to alert administrators when spending increases by more than a certain percentage within a specified timeframe.
- Cost Forecasting — Configurable forecasting to compute and display future costs based on previous trends. Estimate costs for migrating workloads into a given provider.
The CloudHealth platform provides the opportunity for organizations to evaluate month-over-month spending trends and anomalies in multiple formats, and leverage historical data to forecast future spend over a custom period of time. It is also possible to construct “what if” scenarios to assist with the decision making process.
- Events and Notifications — Generate emails, UI alerts or provider-specific API calls to modify resource states, both out-of-the-box and customizable.
This is one of CloudHealth´s strengths. Administrators can apply many different types of policies with a choice of automated actions depending on the nature of the policy breach. One of the most useful actions is executing FaaS functions to prevent security breaches by revoking user access when suspicious activity is identified.
- Cost Optimization — Detect orphaned or underutilized/overprovisioned resources. Report or automate the cost optimization recommendations.
CloudHealth not only detects orphaned or underutilized/overprovisioned resources - and reports or automates cost optimization recommendations - but also resources that are under-provisioned and would perform better if their capacities were increased. As was noted earlier, any CSEM solution deployed within an organization should factor in performance optimization.
- Reserved Instance Management — Identify opportunities to save by purchasing reserved instances. Manage existing reserved instances with alert policies, and buy or sell from the solution.
As mentioned above, the “greedy knapsack” algorithm compares all possible discounts to identify opportunities to reduce costs. With regard to Reserved Instance management, CloudHealth ensures discount opportunities are fully utilized - including convertible discount opportunities - based on historical usage, reservation types, and expiring reservations.
- Service Provider Capabilities — Manage cloud MSP client access to specified portions of data, support invoicing of clients and add markups to the bill.
In 2014, CloudHealth Technologies launched a version of the CloudHealth platform specifically designed for Managed Service Providers (MSPs), Systems Integrators (SIs) and Value-Added Resellers (VARs) that helps third-party service providers better manage multiple customer accounts through a single console. Since then, we have listened to client feedback and made several improvements to the platform.
To read the full Gartner report - How to Identify Solutions for Managing Costs in Public Cloud IaaS, Brandon Medford, Craig Lowery, 22 September 2018 - please click here. Alternatively, if you have any questions about managing cloud costs with the CloudHealth platform, do not hesitate to contact us.