The idea of having multiple AWS accounts in your organization broken out not just by team but also by cost center isn’t the terrible idea that some people (including me until a year or two ago!) tend to assume it is. Permit me to explain why.
Imagine a mythical organization that has a single AWS account. (For purposes of this exercise, ignore the technical problems of account-wide rate limits, the operational hazards of “oops that was Production,” the governance implications of a lack of a discrete audit trail, etc.) Everything’s in one place, it gets charged to one bill, one person in Finance pays the bill (presumably, anyway; you’d be amazed how many Disaster Recovery plans overlook “the credit card stops working” as a potential risk factor). All is well.
Your finance group is absolutely fine with this. They ask you what percentage of your infrastructure spend is for development work (“oh, about forty percent”) and then take it as gospel for years. This is never revisited or questioned again.
Years pass. The company scales. Your development spend is now roughly five percent of your infrastructure bill. Somehow this never gets reported through the four organizational layers between the engineer who builds things and the finance person who gets the bill. It’s probably not that big of a deal, but one day an engineer reads this blog post and thinks to ask a question of the finance team. “Hey, what are you using the Prod vs Dev numbers for, anyway?”
Finance will be thrilled that someone cares enough to ask them about their area. They’ll happily give an answer that touches on a bunch of terms of art from the world of finance that bears no relevance to anything engineering ever touches. Internal P&Ls, pitch deck stuff numbers, calculating out the unit economic model… and then they drop in a statement that resembles “oh, and we’re claiming a federal R&D tax credit.”
record scratch
freeze frame
“You’re probably wondering why the CFO just pooped a literal abacus.”
Suddenly it’s no longer just about how Finance thinks about things– now there’s a very real risk that the IRS would like a few words with you, and when they’re done the SEC is just down the hall if you’re publicly traded. Suddenly the API rate limits aren’t really your driving concern anymore. Suddenly the CFO isn’t the only one pooping abaci. Instantly “one AWS account” doesn’t sound like the best idea anymore.
So how do you break up your environment? An initial approach that makes an awful lot of sense is to split your environment a minimum of two accounts– production, or as finance calls it “COGS (Cost of Goods Sold)”, and pre-production, or “R&D (Research and Development)”.
Those readers following along at home may suddenly realize why a lot of large companies were thrilled to pieces by AWS recently allowing you to prevent RIs from being distributed to or from certain accounts in Organizations. This feels really dumb in terms of dollars and cents, until you factor in the cost of failing an audit– at which point paying 30% more for a few instances isn’t the worst thing in the world.
It’s tempting to stop here. It feels like you can get by just fine from two accounts. Except by this point, your organization has grown not just in size, but in complexity. Let’s say you’re on the ball and automatically apply cost allocation tags. “Instance X” gets a tag for whatever service it provides, whichever team owns it, what color tie the CEO wore the day it was spun up, etc. Now finance can happily allocate that instance’s cost to various cost centers, and all is well– except for the EBS volumes, the snapshots of those volumes, the data transfer the instance incurs, the load balancer tied to it, the CloudFront cost, the percentage of the AWS support fee, the Lambda functions that fire off incessantly because for some godforsaken reason we’ve collectively decided that cron jobs are way too understandable, etc.
So you’re now in an unpleasant reality where you’re tagging ~20% of the cost, with the remainder being allocated to HEY LOOK OVER THERE IT’S A SQUIRREL IMPROPERLY DEPRECIATING AN ASSET! and hoping finance takes the bait so you can escape the conversation.
It’s not an easy migration process for most shops– but if you have an AWS account that’s just for the Sunrise Service or the Data Team or the Cornflower Blue tie, you can roll everything inside of it up to the cost center without crossing the streams. Mostly.
Ultimately, an account per cost center makes an awful lot of sense for finance, except for the part where managing things cross-account still requires non-trivial amounts of work despite the presence of things like aws-vault. There is no API that exists between finance and engineering– translating requirements back and forth can be exhausting. Finance has no idea what an ELB is, or why EBS is required even if you’re storing no data on it– and engineering doesn’t know the best way to amortize an RI purchase, or whether GAAP permits the RI to be depreciated over that span.
If you’re having trouble determining how to best attribute or reduce your AWS spend, win hearts and minds across the finance / engineering divide, or ultimately just want to chat about the Cloud, we should talk. This is exactly what I do for a living.
