AWS re:Invent 2022 – Werner Vogels and Event-Driven Architecture

If given the chance, would you want to live in a synchronous world? “Nope,” said Dr. Werner Vogels, VP and CTO of Amazon.com, in a The Matrix-inspired opening video to his Thursday keynote address at AWS re:Invent 2022. What follows is an overview of Dr. Vogels’ reasoning behind his “nope” and a look at how AWS products can offer a better way.

Synchrony is an illusion—the world is asynchronous

In the video, Vogels takes the red pill offered him by a would-be Morpheus, transporting him to a perfectly synchronous world. The video goes on to show the flaws with such a world—every task must be performed individually by a single person, and one task must be completed before another can be started.

At lunch, Vogels has to order his burger and fries separately, one at a time. His meal is then prepared by the server, who has to prepare every aspect of the meal one at a time, even frying every individual French fry separately and plating it before starting on the next one. It may sound heavy-handed, but the point is clear: if the world were synchronous, nothing would ever get done.

So why would we want to build software this way? Monolithic, synchronous architectures were the status quo in the 90s and early aughts, but software developers gradually started incorporating elements of parallelism and asynchrony, leading to staged event-driven architectures and eventually the event-driven architectures of today.

Loosely coupled systems built on event-driven architectures are more true to the way nature works, Vogels said, but they create an illusion of synchrony when seen from afar. In reality, these systems are highly complex, with processes responding to multiple different simultaneous events to increase throughput and reduce latency.

Evolvable architectures lead to loosely coupled systems

But even the software behind Amazon.com started out as a tightly-coupled system built on a monolithic architecture. Over time, the company’s software engineers transitioned from a monolith to a service-oriented architecture and finally to a microservices architecture.

This paved the way for a shared services architecture, which is the current infrastructure of Amazon retail and the foundation of Amazon S3. By building an asynchronous architecture, Amazon engineers could build a complex, loosely coupled system one component at a time, building gradually without having to worry about breaking the entire system if one component failed.

> Which AWS Certification Should You Get?

The interesting part is that Amazon engineers were onto this as early as 1998. Vogels shared a report published internally to the company called “The Distributed Computing Manifesto” in which Amazon engineers outlined the need to transition to what we would now call a services-oriented architecture. Vogels admits it reads a bit outdated now, but at the time, this was cutting-edge programming that almost no one else was doing.

Amazon started with a relatively simple system that could evolve into a complex system. Vogels flashed a quote of Gall’s law on the screen: “All complex systems that work evolved from a simple system that worked.” Building an evolvable architecture is the best way to build a complex system, and event-driven architecture is the best way to accomplish that.

New AWS products and features for easier architecting

From here, Vogels pivoted from sharing the why of event-based architecture to sharing the how. AWS already offers tools for building infrastructure, but Vogels announced some new products and features on Thursday to make that easier:

• New product – AWS Step Function Distributed Map. Lets you orchestrate large-scale parallel workloads in serverless applications using very simple Lambda functions.
• New product – AWS Application Composer. Uses a visual canvas interface with drag and drop functionality to link components together in a serverless configuration for building applications faster.
• New feature – Amazon EventBridge Pipes. One of the fastest growing products, according to Vogels, EventBridge now lets you manipulate events flowing through your pipes before they reach the consumer. You can also build advanced integrations without all the coding.
• New product – Amazon CodeCatalyst. A unified software development and delivery service that eases a lot of the heavy lifting involved with software development. You can create a new project using a blueprint, which comes in pre-built, custom, and organizational templates. Supports collaboration with team members as well as code coverage, unit testing, automation, and report generation.

3D will one day be as common as video

After making new product and feature announcements, Vogels shifted gears to discuss the power of 3D modeling and simulation building. He talked about the importance of neural radiance fields (NeRFs) to “fill in the gaps” of images using machine learning and how that translates to what he sees as the coming prevalence of 3D models in the near future.

“3D will soon be as pervasive as video,” he said.

> How to Attract the Best AWS Talent

No new product announcements here, but Vogels did highlight some pretty exciting functionality of existing and recently announced products. For example, AWS IoT TwinMaker lets you embed live data in a digital “twin” of a room or object, bringing together models, sensors, and data. And Unreal Engine—the incredible computer graphics engine developed by Epic Games—uses AWS Ambit Scenario Designer to create photorealistic 3D worlds at scale for some groundbreaking video games.

Simulations are the future

Vogels saved what might have been the most exciting use case of an AWS product for the end of his keynote. Announced on Tuesday, November 29th during CEO Adam Selipsky’s keynote address, AWS SimSpace Weaver is an incredible new product that lets you run massive spatial simulations without managing infrastructure.

You can use this tool to build all kinds of spatial simulations, but Vogels highlighted a recent use case of the product from climate tech startup Terraformation. Terraformation builds tools to help with reforestation and biodiversity efforts, and they use SimSpace Weaver to run various simulations to help clients identify the best ways to reforest or increase biodiversity in an area.

In the example Vogels shared, Terraformation could run multiple different simulations to see which tree planting locations gave foresters the best results in reforesting a tract of barren land. The simulation includes an incredible amount of detail, even showing how factors like rivers and streams and even the presence of deer in an area can help or harm reforestation efforts.

Vogels listed a number of benefits simulations can offer, from innovation to safety improvements to experimentation to designing entirely new systems. Simulations, though highly sophisticated, are still approximations right now, he said. But as quantum computing technology improves, we’ll be able to run simulations on a scale that is currently impossible with modern computing technologies.

To help advance quantum technology, Vogels directed the audience to Amazon Braket, which lets you build, test, and run quantum algorithms on your choice of quantum computers to help advance research and development of quantum computing.

Look to nature for designing computer systems

Biomimicry can be found in all kinds of unexpected areas, but computing may not be the first one that comes to mind. However, Vogels said that, just as the asynchronous nature of the world informs event-driven design, programmers should look for other ways to implement lessons from nature when designing computer systems.

If technologists can take inspiration from natural processes like the flight patterns of Starlings and the way our brains generate peripheral vision, who knows what kinds of exciting new technologies humans can create when taking a closer look at the world around us?

Related articles