Asynchronous Computing at Scale : Overview and Learnings
Sayak Kundu

Abstract

A talk about the architectural changes in Meta’s event driven asynchronous computing platform that enabled easy integration with multiple event-sources. We share our learnings from handling various types of workload and how to tackle trade offs made with certain design choices in building the platform.

Event Driven Architectures
Date:
May 18, 2023
 
This talk was hosted at an aleios event. The speaker / company does not work for aleios.

Amirali Sobhgol

Amirali is a Senior Production Engineer at Meta. He works closely with an 11-person team to design and build Function as a Service (FaaS) products akin to AWS Event Bridge and Lambda for Meta's internal cloud. Their primary objective was to manage millions of function executions per second running across hundreds of thousands of machines worldwide. The technology stack they used was heavily reliant on big data platforms like Hive, Scribe (a Kafka-like system), and sharded MySQL clusters for storing events and various execution runtimes such as HHVM.

Omar Abdou

Omar is a Software Engineer on the Serverless Computing Team at Meta. After building Media Matching Infrastructure for Integrity, he decided to help build the Infrastructure that powers it all. Since then,he has focused on growing reliable and scalable data processing systems, making them available to every engineer at Meta.

Sayak Kundu

Sayak is a software engineering Lead at Meta. He is part of Meta's Serverless computing platform team which is responsible for enabling transport of event driven workloads for Serverless execution in the scale of 4+ Trillions per day. Earlier in his career at AWS, he led the refactoring of monitoring plane of a critical consumer facing AWS service to enable faster deployments to production, as well as faster configuration push to toggle knobs to mitigate operational issues.

But wait... there's more...