AWS Middle East Central Zone (UAE) down, apparently struck in war
93 points by carlana
93 points by carlana
Has an AWS data center ever been destroyed in war before?
I've always been surprised data centers are not bigger targets during war. Maybe I was too early in my thinking. Maybe this is the first.
I think this mostly reflects which countries have had wars since 2010. The first time a country had a war on its soil with nuclear power plants was Ukraine. History is long and technology is still relatively new.
There is nothing that I read that said this was explicitly targeted. They say "At around 4:30 AM PST, one of our Availability Zones (mec1-az2) was impacted by objects that struck the data center, creating sparks and fire." My guess is if it was actively targeted, the impact (no pun intended) would be greater. This could be debris from an interception, for example.
Edit: If this page is correct, the data center is located right next to Dubai airport, which has been a target, so this is likely an accidental circumstance from data centers colocating near to airports in many places.
The wording is also ominous. It can be interpreted as corp speak for "rocket", but it could also just to debris for missiles shot down by missile defense, in which case the corp speak is quite accurately describing what it is.
It could also be "we don't have any clue yet what hit us, but there's sparks and fire". Normal fog of war really.
Agree. (I wish lobste.rs had an "agree" reaction)
They probably have to get the fire out first before they know what it was exactly. Claiming "it was a rocket" (implying a deliberate attack) without knowing also sets off a very different chain of events.
At least in the US, AWS data centers have been the (as-of-yet unsuccessful) targets of domestic terrorism.
ProTip: avoid hosting based in military dictatorships.
But us-east-1 is mandatory for some services/functionality.
The list is here, which includes core things like IAM:
https://docs.aws.amazon.com/whitepapers/latest/aws-fault-isolation-boundaries/global-services.html
well, the entire mec1 is down for us. cannot even access the EC2 landing page.
apparently, the complete region goes out if two AZs in the same region goes out.
TBH, I never thought of missiles as a DoS factor.
I mean, we’ve talked about cutting undersea fiber cables as a means of disrupting nations for the last 5 years a la Taiwan, Japan.
The Russians have also been accused of this in the Baltic.
Yes, although I believe the gas pipeline cut was revealed to have been done by Ukraine.
Strictly speaking: It is true that several Ukrainian individuals were under investigation by Germany, especially involving the theory around the yacht Andromeda. There's still multiple competing theories on what actually happened and no final conclusion.
I'm still (unironically) a little skeptic of the "someone took a yacht and sabotaged a 110m deep pipeline with military explosives", that is not an easy dive operation off a rented yacht vessel with no dive equipment as it seems. Not at all impossible though.
However, accepting the Ukraine theory - bombing Nord Stream is strategically valuable to the Ukraine. It curbs Russian exports massively and attacks one of the biggest symbols of Russia <> Europe integration.
Cutting a baltic sea cable advances exactly zero goals of Ukraine, angering Europe in the process.
Agreed. Just following the rule of “who benefits” and assuming false flags are rare is a good policy in general.
The initial goal of the internet was to route communications in case of nuclear war destroying telephone switching stations https://en.wikipedia.org/wiki/ARPANET
Does the affected AZ consist of only a single DC? That isn't normal I think, but that's what this sentence implies:
We want to provide some additional information on the power issue in a single Availability Zone in the ME-CENTRAL-1 Region. At around 4:30 AM PST, one of our Availability Zones (mec1-az2) was impacted by objects that struck the data center, creating sparks and fire.
Why are they recommending migrating away from the whole region if only a single AZ was struck?
For customers that can, we recommend considering using alternate AWS Regions.
I suspect many of AWS’ small regions have AZs that are a single DC, if not a fraction of a DC.
I’m sure if losing one AZ is a significant fraction of that region, it would benefit the overall availability of the region for some customers to leave
According to this upstream page UAE has 3 AZs (as do all Middle East regions).
Edit: AFAICT every AWS region on the globe has at least 3 Availability Zones.
Right, so losing an AZ could mean 1/3 of their capacity in the region is gone, and so customers may not be able to easily move workloads to another AZ in the same region
Oh, I misread your comment and thought you were suggesting that there were single-DC/AZ regions. (I thought that an AZ basically meant a DC, and I'm surprised that your comment seems to be implying that an AZ may have multiple DCs in larger regions. Unless I'm misunderstanding, again?)