Mending Leaky Buckets: Overcoming the Unsecured Cloud Server Crisis

To its credit, AWS has rolled out a number of extra layers of safeguards in the wake of high-profile security breaches. For example, public buckets are clearly signposted on the dashboard and appear at the top of the page by default.

Yet treasure troves of data continue to surface unguarded – and not just from companies with small IT budgets. Netflix, the Dow Jones, Accenture and even the Pentagon have all spewed sensitive data thanks to a misconfigured S3 bucket.

So, how does it keep on happening, despite AWS’ best efforts? VpnMentor security researcher Noam Rotem believes it is all down to human error.

“AWS is taking this seriously and is making it harder for people to make these mistakes,” says Rotem who, along with Ran Locar, has made a name for himself uncovering data repositories with poor security.

“However, you cannot stop idiots from being idiots. AWS cannot prevent the ability to open the database to external connections. They can advise against it and they are doing a good job at it. But if someone actively goes against best practices and common sense, there is little they can do about it.”

David Locke, EMEA chief technology advisor at World Wide Technology, agrees that AWS shouldn’t be held accountable for the poor security practices of its users.

“Their remit is to provide a platform. How you choose to use that platform is really then down to the person consuming it and buying that product and service,” he says.

“In the same way that people can buy a car and drive it into a building to rob it, it's not the car manufacturers that have done something illegal with their product.”

But Ian Massingham, director of developer technology & evangelism at AWS, doesn’t think the blame should land squarely at the feet of the user.

“We've got to do a better job of ensuring that customers are aware of the potential risk posed by having lax security policies on all cloud based resources – not just on S3 buckets,” he says.

“And of course, we can put in place steps of the type that we have done to make it more difficult, and to warn customers that are attempting to create these insecure configurations as they are attempting to create them.”

AWS has done this with ServiceNow, which provides an extra layer of oversight into customers’ resources.

The cloud giant has also introduced Service Control Policy, which can be used to create a hierarchy of AWS accounts with different access privileges.

“So I can apply a policy at the top of the hierarchy and say, for example, don't permit the creation of S3 buckets with a lax public access policy in this hierarchy at all,” explains Massingham.

Additionally, AWS Config Rules can be set to respond automatically to any changes in a customer’s AWS environment.

“So you could have a automation step so that whenever somebody applies an AWS policy to an Amazon S3 bucket to another S3 bucket, you can verify that policy and potentially strip it away from the bucket programmatically,” explains Massingham.

In addition, AWS provides a variety of best practices documents, encryption tools, and other guidance to customers, as well as a machine learning tool that detects open buckets.

“I think the tools are all there for organisations to secure their web buckets and check their security,” says independent security expert Graham Cluley.

“It’s not really fair to blame the likes of Amazon. It’s the customers’ responsibility.”

It is worth noting that only a small percentage of S3 buckets are misconfigured, with even fewer resulting in data breaches. Figures from 2017 put the number of unrestricted public access S3 buckets at 7%, which against AWS’ million or so customers means potentially a hundred thousand unsecured buckets waiting to be uncovered - although it is impossible to tell how many of these legitamately need to be made public. 

Nor is the misconfiguration problem limited to AWS. Rivals Microsoft Azure and Google Cloud Platform are and have been susceptible to user error. According to a recent analysis by cybersecurity firm McAfee, 99% of all misconfigured cloud servers go undetected, suggesting the problem is far more widespread than even the breathless headlines suggest.

Given the financial and reputational risk of a data breach, why bother to make a bucket public at all? As Massingham points out, there are legitimate reasons for making a bucket public. For example, hosting a website, running a game engine, and so on.

Because of these genuine use cases, applying a unilateral, retrospective private access policy would “break” customer applications.

“There isn’t a switch that can be flicked inside our service to remove this issue,” says Massingham. “It can’t work like that. If there was, we would have flipped it.”

In some cases, users who make a bucket public for a short period can forget to make it private again. Others might make something public to get rid of an error message out of laziness or a lack of technical understanding.

“At the moment they're having to balance ease of use and kind of a consumer type model – having come from consumer – against enterprise,” says Locke.

For Massingham, education is key to preventing such security slip ups. AWS has been proactive in this area, sending large email campaigns to raise awareness among its customers, as well as sending reminders to those with public buckets that they are accessible by anyone.

However, there is the danger of too much communication. Sending too many emails to those who haven’t left buckets unsecured could generate “false positive communication” says Massingham.

“You've also got to be cognisant of communications overload.”

Massingham also points out that many breaches “predate the increased awareness that customers have about the issue today”. Sister publication Verdict, for example, recently discovered that British holiday firm Teletext Holidays had left an S3 bucket containing 212,000 customer call recordings unsecured since 2016.

It is certain that there will be more incidents like this, but AWS’ main option is to keep raising awareness.

“The job of educating customers about how to use the platform in a secure way will never be complete,” says Massingham.

“There are always new customers, there are always beginners that just come out of university or just come out of school or in their first software engineering job, that don't know about best practices.

“So it was always going to be an opportunity to educate customers more effectively, about how to use the services in a secure way.”

In a way, AWS is a victim of its own success – at least when it comes to unsecured S3 buckets.

S3 was the first AWS service that made it out of beta, back in 2006. “At that point, Amazon had no idea how popular AWS would be,” says Massingham. “It was an experiment. We were lucky that nobody else entered the market for quite a while.”

As cloud adoption grew – buoyed by the extra freedom granted to businesses by offloading the heavy lifting to the likes of AWS – the security risks became more apparent.

“Maybe if we’d known the service would be so heavily adopted, we might have made a few different design decisions right at the beginning,” says Massingham.

One possibility, suggested by cybersecurity firm Upguard, is creating two separate products – private buckets and public buckets – to leave users in no doubt as to the access levels of their data.

With the benefit of hindsight, it’s an idea that would have made sense from the outset, says Massingham.

“If we went back in time and created the service again, we probably would create S3 public and S3 private and we wouldn't have the issue,” says Massingham.

“You'd have to create a resource in the public service in order to do that. But we don't have a time machine, unfortunately.”

Massingham’s main takeaway message is for customers to verify the configuration of their existing buckets.

“If they’ve got any usage of S3 in their organisation, no matter how old it is, or how they use it, it would be great for them to check the bucket policy they have matches the use case they have in mind.”