| By Nickolay Bunev, Senior Cloud Infrastructure Consultant at HeleCloud |
Houston, we have lift-off.
Yesterday kicked off AWS re:Invent 2020, which due to the global pandemic is all virtual this year, however, as usual, the event commenced with AWS CEO Andy Jassy’s keynote. Outlined here is the major announcements in each of the main AWS components to catch the eys of our consulting team. Let’s start.
The day within Compute started with a pre-re:Invent announcement of Amazon EC2 Mac instances powered by Mac mini and the AWS Nitro system, helping customers build and test MacOS or IOS applications. This was followed by the new Graviton2 based instance types like the 100GBps C6gn.
2021 will also see EC2 instances build specifically for Machine Learning (ML) training powered by Haban Gaudi processors from Intel as well as the new AWS Trainium chip specifically optimised for deep learning training and cost-effectiveness. Predominantly, the major announcements in Compute were in the Containers and Serverless.
Containers and Serverless
Andy Jassy broke the news of AWS ECS and EKS Anywhere, which will allow you to use the same AWS APIs on-premise. Plus, the EKS Kubernetes distribution has been released as open source under the name EKS Distro. This will help many organisations starting their journey in the Cloud to use the same stack in the Cloud and on-prem whilst they move their workloads to AWS.
Chiefly all AWS customers are using Lambda, it’s practically the serverless stepping-stone. Lambda now supports up to 10 GB of memory and 6 vCPU cores and the billing duration granularity has been scaled down from 100ms to 1ms, leading to direct cost savings for clients with Lambda functions that usually finish faster than 100ms.
The major game-changer for Lambda is the ability to package your code as a container image from which you’ll start your function. One of the major pain points with Lambda was the fact that the deployment package was 50MB zipped and 250MB unzipped. We’ve experienced clients with custom Step Functions workloads hitting those limits before, and having the option to build code in a docker image will solve that. Also, Container Image Factory offering can be used to build images not only for your ECS and EKS clusters, but for your Lambda functions too.
Last but not least, following up on the Docker Hub pull rate-limiting AWS are now introducing Elastic Container Registry Public (ECR). ECR was always there, but anonymous access to it was impossible, and if you wanted to make an image public you had to rely on other means and offerings. This is no longer valid, you can now publish your fancy docker images to the world. And whilst there is a monthly 500GB bandwidth limit, if the image is used for an AWS workload you get unlimited bandwidth.
Two major announcements in the storage worth mentioning.
gp3 EBS Volumes
gp3 allows enhacned IOPS performance for a lower price than gp2. It feels a bit like a hybrid between gp2 and io1 as you can now scale the IOPS independently. Previously the IOPS performance of the gp2 volumes was tied to the storage capacity.
io2 and EBS Block Express
The io2 volumes were announcement in August. In essence, they provide increased durability and higher IOPS performance. AWS announced EBS Block Express, delivering up to 260,000 IOPS (with AWS Nitro instances), effectively a SAN for the Cloud.
Intentionally Databases is the last topic here, as this is where we were most impressed (so far).
Together with Aurora Serverless v2, during the keynote AWS introduced BabelFish for Aurora PostgreSQL.
Why are we so blown away by this announcement? Let us share a story. During a recent customer project we built a truly robust AWS infrastructure. By successfully combining EKS, Step Functions, Transit Gateway and much more we created a self-healing and cost-effective, multi-region application setup. Our customer gained enormous performance for a fraction of their previous cost. However, we had one issue that we were unable to resolve – Microsoft SQL Server. As AWS mentioned in their BabelFish blog post: “The greatest force in legacy databases is inertia”. Similarly to their description, our customer had spent years building data models in SQL Server, creating a challenge to migrate to anything that would allow them to span between regions. BabelFish will enable this whilst relying on the best open-source sequential databases out there – Postgres. To note, BabelFIsh will be open source too.
Advanced Query Accelerator (AQUA) is very interesting and will provide up to 10x faster query performance than other cloud data warehouses, definitely something we should keep an eye on.
AWS Infrastructure & Hybrid Cloud
Apart from the ECS and EKS Anywhere, the AWS team increased the options for a truly consistent hybrid experience with the announcement of AWS Outposts 1U and 2U form factors.
The AWS global infrastructure is expanding with 12 new Local Zones in the States, plus the new AWS Wavelength Zone in Las Vegas.
In recent years the lack of tools providing consistent deployment of the microservice applications was obvious. As usual, the biggest Cloud provider is addressing the challenges their customers are facing in order to help them focus on building the business value components. Hence the reason AWS Proton landed as the first fully managed deployment service for container and serverless applications.
The announcement of the DevOps Guru is really cool because currently it’s a huge effort to manage and proactively detect anomalies in complex cloud-native applications. DevOps Guru is an ML-powered service that will help to measure and improve an application’s operational performance and availability, and reduce expensive downtime without having ML expertise.
Security is priority number one for AWS. Not surprisingly the new Security Detectors feature introduced in CodeGuru will review the code using ML and automated reasoning to help identify security risks from the top ten Open Web Application Security Project (OWASP).
SageMaker Pipelines is the last but not least impressive announcement, having a dedicated service for building continuous integration and continuous delivery (CI/CD) for ML will simplify and speed up the process of automating and managing the end to end workflow at scale.
Today saw many announcements, and abundantly more is expected throuhgout the next three weeks. In short, the key takeaways of Day 1:
- For developers – Lambda Container Image Support & AWS Proton
- For enterprises – BabelFish for Aurora PostgreSQL
If you want to discuss this and other announcements join us for our live event this Friday.
AWS Nitro System
AWS ECS and EKS Anywhere
Amazon Elastic Container Registry Public
Babelfish for Amazon Aurora PostgreSQL
AWS Outposts 1U and 2U form factors