cloudonaut

Get insights into the day-to-day challenges of builders. In this issue, Monika Oblonczek from our partner tecRacer talks about how cloud migrations succeed.

I was excited when AWS announced Aurora Serverless at re:Invent 2017. Disappointment followed shortly after. Even after Aurora Serverless became a generally available service in August 2018, it was missing important features like multi-AZ deployments and read replication. Unfortunately, the innovative service never achieved a breakthrough. Therefore, I used Aurora Serverless in exceptional cases only. Four years later, AWS is making a fresh start with Aurora Serverless v2. Reason enough to take a closer look at the new service.

Andreas invited John Culkin and Mike Zazon to talk about their latest book: the AWS Cookbook which includes 70 self-contained recipes to help you creatively solve common AWS challenges you'll encounter on your cloud journey. This show includes the following recipes: * Testing IAM Policies with the IAM Policy Simulator * Automatically Scanning Images in ECR for Security * Redacting PII from text using Amazon Comprehend And don't forget to get the whole book!

Stephen Kuenzli and I lead several cloud migration projects. In this conversation, we shared our learnings focusing on AWS security and IAM (Identity and Access Management). The result is advice and inspiration that will help you in your daily work. Our conversation is available as a video or podcast episode. In the following, you will also find a summary of our discussion.

Architecting applications on AWS is challenging. On the one hand, you need a broad understanding of AWS services. On the other hand, you have to know the details as well. In this episode, Michael outlines the mindset you need to build on AWS successfully.

Launching an EC2 instance takes minutes. Keeping your virtual machines secure and maintaining your VMs is more work. In this episode, I share seven things to do after launching a Linux, Windows, or macOS instance.

Writing CloudFormation templates from scratch is a lot of work. You will run into many issues along the way: the documentation is incomplete, magic values are required, unsupported combinations of attributes, etc. The feedback cycles are long. In the end, we have to provision real infrastructure to test the template. If you ever created an Elastisearch cluster, you feel the pain. We also observe that AWS architectures follow similar patterns (aka best practices). So why not make a collection of templates and share them with the world? That's what we did in late 2015. We launched Free Templates for AWS CloudFormation. In this episode, Michael provides you an overview of the project and show you typical use cases.

AWS allows us to run applications distributed across EC2 instances and availability zones. By adding load balancers or message queues to the architecture, we can achieve fault tolerance or high availability. But how can we test that our system can survive faults in reality? Assuming an application has five consumers and seven downstream dependencies. What happens if one of them fails? Are all timeouts configured accurately? Are all applications retrying? What happens if the network is slow? So many things can go wrong. It is not possible to understand all consequences upfront. Therefore, a new approach emerged: Chaos Engineering. With chaos engineering, we simulate faults in our systems and observe the consequences. The trick is that we can simulate faults as often as we wish. We don't have to wait for the one day a year where things go horribly wrong. AWS released Fault Injection Simulator (FIS) as a tool to run controlled fault experiments within our AWS accounts.

We coached developers building their first Serverless applications for a large company recently and want to share our learnings and observations with you.

I completed my first AWS certification in 2014: AWS Solutions Architect Associate. During the following years, I accomplished all five associate and professional certificates. However, Michael and I decided not to renew our AWS certifications about a year ago. In the following, I will share five good reasons not to get AWS certified!

Elastic Block Storage (EBS) provides solid state drives (SSD) and hard disk drives (HDD) for EC2 instances. The virtual machine accesses the persistent storage via the network. In December 2020, AWS announced another volume type called General Purpose SSD (gp3). So now there are three volume types based on SSDs. In this episode, Andreas compares gp2, gp3, and io2 volumes and guides how to choose the volume type that fits best a specific scenario.

Werner Vogels's keynote was a blast and definitely the highlight of re:Invent 2020. Michael and I are going through the announced features and services. As usual, we also take a look at the technical details.

Are you following the Infrastructure as Code approach using CloudFormation? If so, I bet you encountered a situation where CloudFormation misses support for a service's latest features. I run into those issues weekly! So what can we do about it?

Andreas Wittig and Michael Wittig from cloudonaut are discussing Andy Jassy's keynote from re:Invent 2020. The focus is on the newly announced services and features: ECS Anywhere, EBS volumes (gp3), Aurora Serverless v3, Lambda Container Support, and many more.

The Elastic Container Service (ECS) is Amazon's container orchestration service. Besides that there is Elastic Kubernetes Service (EKS) the managed Kubernetes offering by AWS. Both container clusters support EC2 and Fargate as the underlying compute engine.

AWS offers five different types of API Gateways. Which option fits your needs?

Michael shares his learnings about IPv6 on AWS. Enabling IPv6 is highly recommended for public endpoints like CloudFront and ALB. On top of that, Michael explains how to enable IPv6 for your VPCs.

In this episode, Michael unboxes Amazon Timestream for us. A recently launched time-series database. Andreas asks questions like: What is Amazon Timestream? How does it work? What are typical use cases? And Michael tells us why his first job was all about time-series data.

Are you using a container registry already? Andreas Wittig and Michael Hausenblas discuss different scenarios and options. The episode focuses on ECR including recent announcements and upcoming features. On top of that, the episode includes a comparison of different container registry options: Amazon ECR, Docker Hub, and GitHub Container Registry.

Have you ever looked at an IAM policy and wondered: Is it really necessary to grant access to this specific action? Or do you need to know which API calls a legacy or 3rd party application is actually sending to come up with a secure IAM policy? CloudTrail can help here, but there is something better: Record API calls with the AWS SDKs and CLI (including the stuff that is not visible in CloudTrail).

It seems to me like everyone is talking about service meshes these days - definetly a hot topic in the world of containers and microservices. A service mesh promises reducing latency, increasing observability, and simplifying security within microservice architectures. AWS announced a preview for App Mesh in November 2018 and the general availability in March 2019. Therefore, it is about time to take a closer look at App Mesh. As always, my review focuses on the technical details and educates about pitfalls. There is a lot more to know about the service than written on the official marketing page or demonstrated by technical evangelists.

I was recently invited to a CloudFormation workshop with a group of early CloudFormation users. I soon realized that the group had a good understanding of the basics, so I started to introduce more advanced features. Today, I would like to share with you six CloudFormation features that have inspired the workshop participants most.

Choosing storage service is critical when designing a cloud architecture. Read on to learn about the characteristics, limitations, typical use cases, and a decision tree for the following options to store data on AWS: Instance Store, EBS, EFS, FSx, and S3.

In this episode, Michael invited a guest: [Philipp Garbe](https://twitter.com/pgarbe). Philipp is an AWS Container Hero, Working in the Cloud, and we hope you enjoy his Bavarian accent as much as the knowledge he shares with us.

In this episode, Michael compares the available messaging options on AWS. The goal of messaging is to decouple the producers of messages from consumers. The messaging pattern allows us to process the messages asynchronously. This has several advantages. You can roll out a new version of consumers of messages while the producers can continue to send new messages at full speed. You can also scale the consumers independently from the producers. You get some kind of buffer in your system that can absorb spikes without overloading it.

AWS offers different types of API gateways as a managed service. This review takes a closer look at the new service API Gateway HTTP APIs announced in December 2019 and generally since available in March 2020. The cloud provider promises that HTTP APIs are faster and cheaper than it's predecessor. We will look at hard technical facts instead of flowery marketing promises.

There are countless reasons why your website is not working as your users expect. From a technical point of view, you can monitor your load balancers, your web servers, and your database. But what if that external script that you embed is breaking your site? Expired TLS certificate? Something wrong with DNS? How can you test that your website works for real users? In this episode, we introduce CloudWatch Synthetics as a solution to monitor your website from a user perspective.

Containers are a powerful tool to streamline your development and deployment process. However, a container cluster - no matter if you are using ECS (Elastic Container Service), EKS (Elastic Kubernetes Service), or self-managed Kubernetes - increases complexity. You are not only managing virtual machines anymore, but you are also operating containers on top of those virtual machines. Luckily, AWS offers a few approaches to minimize the effort of providing the computing capacity for your container cluster. - ECS with Cluster Auto Scaling - ECS with DIY Auto Scaling based on CloudWatch Events and Metrics - ECS on Fargate - EKS with Cluster Autoscaler and Managed Node Group - EKS on Fargate

The most reliable way to automate creating, updating, and deleting your cloud resources is to describe the target state of your infrastructure and use a tool to apply it to the current state of your infrastructure. AWS CloudFormation and Terraform are the most valuable tools to implement Infrastructure as Code on AWS. But what are the differences between both tools?

Using multiple AWS accounts to isolate workloads has been a best practice, not only since AWS introduced consolidated billing in 2010. AWS made a huge step by introducing AWS Organizations in 2017 and has added more and more features on top of the formerly boundary of an AWS account. In my opinion, we have passed the sweet spot between centralism and isolated accounts. The possibilities powered by AWS Organizations ruin the concept of isolated accounts with limited blast radius. I recommend, to manage no more than 50 AWS accounts per AWS organization. Use multiple AWS organizations instead. Also, think twice before using SCP or Trusted Organization Access, both features make centralism permanent. I haven't seen a thriving, innovative, and centralized IT organization so far. Correct me if I'm wrong.

Amazon CloudWatch improved significantly over the years. It's time to look at its monitoring capabilities again. CloudWatch is an excellent starting point to implement enhanced monitoring on AWS. In this episode, Michael demonstrates what you can do with CloudWatch metrics and alarms. Metrics provide a time-series database for telemetry (e.g., CPU utilization of an EC2 instance). Alarms watch a metric and trigger actions if a threshold is reached.

AWS offers shiny and powerful networking services. However, you should know about the pitfalls when designing advanced networking architectures for AWS. I will share some pitfalls that came to my attention when consulting clients to get the most out of AWS. You will learn how to answer the following questions: VPC Peering or Transit Gateway NAT Gateway or Public Subnet? VPC Endpoints or NAT Gateway? CloudFront or Akamai, Cloudflare, Fastly ...? Route 53 Resolver or Public Hosted Zone?

You can run your application on virtual machines using EC2. If you prefer containers, ECS Fargate is your choice. But you can also use the latest Serverless capabilities to run your application on Lambda. But what's the best option for your project?

Do you provide services to consumer or business clients? Which channels do you provide for clients to get support, leave feedback, or let off frustration? Amazon Connect provides a contact center solution in the cloud. Your clients contact you via phone or chat. A group of agents answers their phone calls and chat conversations. The workflows are fully customizable to your specific needs. This review puts Amazon Connect to the test. I set up Amazon Connect for our consulting agency, recently. While doing so I had a look into the technical details as well.

Managing a mutable EC2 instance comes with many responsibilities. In this episode, Michael shows you how to solve everyday challenges by leveraging the latest and greatest capabilities of the AWS platform.

Are you planning to start the cloud journey for your organization soon? Learn from others to turn your initiative into a huge success. Michael and I have accompanied medium-sized businesses and enterprises in their transformation projects and would like to share our learnings with you.

re:Invent was a blast: five days packed with announcements of new services and features. We have created a top 10 list for our re:Invent recap. Here is all you need to know about re:Invent 2019.

AWS made a prominent announcement on November 6th, 2019: AWS Savings Plans. It was never easier to get a discount on compute capacity by committing to a monthly consumption and paying upfront. This blog post introduced AWS Savings Plans and compares them to other options to reduce your AWS bill as well.

Andreas is reviewing the AWS Global Accelerator. AWS introduced Global Accelerator at re:Invent in 2018. A year after that, it is about time to review the service. AWS Global Accelerator makes use of Amazon’s worldwide infrastructure and is designed to improve the performance and reliability of your applications.

We love simplicity! Our blog runs on CloudFront and S3 which is maintenance free and does handle traffic spikes easily. We use the static website generator hexo to publish our content. Lambda@Edge handles redirects and generates optimized images on the fly. Instead of Google Analytics we are using Athena and QuickSight to get statistics about our blog and posts.

Not a week goes by without a frightening announcement that an organization has leaked confidential data from Amazon S3 accidentally. Most often, the root cause of a security breach is a misconfiguration of S3 access control. Andreas presents four rules to avoid S3 data leaks to Michael in this episode.

There are many options available when you are looking for ways to implement a deployment pipeline. You might have heard about Jenkins, CircleCi, BitBucket Pipelines, GitLab Pipelines, and many others. AWS, on the other hand, offers services for CI/CD itself: CodeBuild and CodePipeline. AWS CodePipeline orchestrates deployment pipelines. Unfortunately, the learning curve is steep and the implementation is often complicated. Therefore, I recommend a more simple approach: use CodeBuild. In general, CodeBuild feels like CircleCI or GitLab Pipelines. However, CodePipeline offers tighter security controls and excellent integration into your AWS infrastructure.

It was never easier to scale your compute layer. EC2 Auto Scaling, Fargate, and Lambda enable horizontal scaling. But how do you scale your database? Use a NoSQL database like DynamoDB, one could say. But what if you don't want to miss all the advantages of an SQL database? You should check out Amazon Aurora Serverless, a cloud-native SQL database.

AWS Marketplace allows you to sell software to AWS customers. The customer can either run the software on its own (using AMIs and optional CloudFormation), or you can offer the software as a service (SaaS). You can also offer containers and machine learning algorithms in the AWS Marketplace. In this episode, you will learn how to sell pay per use SaaS in the AWS Marketplace. I will show you the overall process and finish with code snippets to implement the process.

AWS released a new feature called EC2 Instance Connect. Unfortunately, the defaults are insecure. You likely can open an SSH connection to every EC2 instance in your AWS account now.

AWS releases a new service with a lot of marketing noise. You can’t resist, you want to use that new thing now. But soon you discover that the service is missing essential features. As a result, you stumble upon a show stopper and get frustrated. Why is that? AWS ships new services with a lot of limitations and rough edges. That’s a good strategy for AWS to get early feedback. But it’s painful for us, the customer. Therefore, we start a little series where we review new AWS services to give you a more balanced view of the capabilities. We start with AWS Backup. AWS Backup aims to become a centralized place for managing backups. If possible, AWS Backup uses existing features to create backups (e.g., RDS snapshots). Sometimes, AWS Backup is the only way to create a backup (e.g., EFS file systems).