Reviewing and maintaining your cloud environment on a regular basis is an excellent habit to cultivate. Just like flossing and brushing your teeth, it prevents infections, decay, and saves a lot of money in the long run.
I’m all for automating cloud operations and security. At a minimum, you should have robust identity management, log management, and conditional access control solutions in place. But, this article is all about low-tech, practical steps you can take now to keep your cloud clean, no special tooling needed.
A great reason to utilize cloud computing is how easy it is to spin up new virtual networks, compute instances and other services at the click of a button. The downside of all this goodness is that your cloud resources can quickly spiral out of control and lead to poor manageability, gaping security holes, and budget overruns.
The good news is that by developing a few essential hygiene habits, it’s easy to keep your cloud orderly, secure, and cost-effective. At New Edge Labs, we primarily use Amazon Web Services (AWS), so I’ll borrow their terminology in a few places, but these recommendations apply pretty well to any cloud provider. On with the list.
Cadence: 30 minutes, once a month
An AWS invoice, for example, provides a consolidated itemization of spend across various services, like EC2, S3, etc. It also provides a breakdown of spending on services within each linked account. Usually, your finance department reviews and pays invoices. But, they’re probably only looking out for spending irregularities, perhaps a big spike in overall costs during a given period.
I recommend having your finance person review your cloud provider’s invoice together with your DevOps team every month.
Why do I think reviewing the invoice is so useful? First, unexpected spend on any particular service should be a red flag to your team, especially managers and security leads. Second, the invoice provides a clear listing of what services you have consumed and what accounts were active during the period. Any significant anomalies will probably jump right off the page – “Why are we spending so much money on network traffic egress?”, “EC2 spend is crazy. Is somebody mining bitcoin?”
This idea may seem counterintuitive, depending on your role on the team, but I almost guarantee that you’ll be surprised at what you discover.
Cadence: 30 minutes, once a week
Using the invoice to guide you, review each service associated with each account using your cloud provider’s management console. The console is where you can drill down into each account, virtual network, and service to see what’s going on right now.
Again, I recommend performing this review as a team. The various members of your team probably have different areas of responsibility. Each person will be able to quickly answer questions regarding their area and tell the group which resources are needed and which can be stopped or terminated. It’s common for people to spin up resources that are needed only temporarily, and then get busy with another task and just forget about them.
It may seem daunting or tedious to click on the console to review each service, but this process can usually be completed pretty quickly when your whole DevOps team is involved. You don’t need to take any specific actions during the review, like shutting down services immediately. Just have someone, like the manager, take notes on what the team has agreed to do and assign those actions to the appropriate person to handle later.
The team console review is a decidedly low-tech approach to system maintenance, but it is a highly effective way of keeping costs under control and keeping security in check.
Cadence: once now, as needed
People in the DevOps community tend to gravitate towards one or the other of two extremes when it comes to coordinating their cloud environments. They either organize all of their resources under a single “master” account. Or, they deploy each application, or even microservice, into a dedicated VPC (virtual network) within a distinct account.
Unless you’re very early in your development project and are not ready to go to production, the single master account approach is, generally speaking, a bad idea. This type of setup eventually creates all sorts of management and security problems and is very difficult to unwind once your service is live (in production). My advice: don’t do it.
At the other extreme, is the highly isolated “separate accounts for everything” model. This approach may be appropriate in some particular cases, but it requires a lot of overhead to manage user access and infrastructure monitoring and makes it difficult to get a clear picture of your overall architecture.
In most cases, I’d recommend you consider taking a more middle-of-the-road approach. Start by separating the resources needed to support your primary functions into different accounts. Many well-designed cloud environments encompass three core areas: production, dev/test/staging, and internal IT.
Production is where your customer and partner facing services run. Production services usually have high uptime and security requirements, and may also need to meet service level agreements (SLAs). Dev/test/staging is typically accessed only by your DevOps team (developers, QA, security, etc.). Internal IT is where you run applications and services used by employees and contractors.
There are variations on this theme. For example, you may have a good reason to separate your staging environment out to a separate account. Or, you may have staging and production versions of your internal IT systems. The real point here is, don’t add more complexity than is needed to ensure a smooth and secure cloud operation.
Following is a simple visual outline of what I described.
Master Account |--- Production (linked account) |--- VPC A (virtual network) |--- Dev/test/staging (linked account) |--- VPC B (virtual network) |--- VPC C (virtual network) |--- VPC D (virtual network) |--- Internal IT (linked account) |--- VPC E (virtual network)
Develop essential cloud hygiene habits to improve manageability and security, and keep costs under control. Implement simple, low-tech processes, like the “team invoice review,” and “team console review.” Organize cloud resources into separate accounts where it makes sense.
How do you organize cloud resources? What’s your favorite approach? Feel free to share your ideas in the comments section.
New Edge is the best way to provide secure access to both cloud and datacenter services. Contact us, to learn more about how we can help secure your critical application infrastructure.