System architects have recently faced myriad of challenges that demand them to build a scalable system that can be deployed everywhere with millisecond response time and 100% uptime. On top of that, data is measured in PB rather than GB or TB. Due to similar demands across different industries, they are now attempted to solve it in collaborative fashion and an new system architecture patten is drafted out from this.
We believe that a coherent approach to systems architecture is needed, and we believe that all necessary aspects are already recognized individually: we want systems that are Responsive, Resilient, Elastic and Message Driven. We call these Reactive Systems.
Translated to the terms I am more familiar with, the initiative is to build a scalable system that delivers fast user experience consistently in load over the good and bad days. And such a system is believed to be achieved thru orchestrating microservices on top of an asynchronous messaging infrastructure. I start seeing companies taking this initiative in practice and achieve great result like Gilt.
Follow up reference
- Gilt’s Microservice Architecture
- Scaling Microservices at Gilt with Scala, Docker and AWS
- What is reactive programming
- Gilt Tech Blog
- Learning reactive programming with code from egghead.io
The core of the Reactive System is how it resiliently deals with requests or data that could dramatically change in seconds. If consumers are way faster than producers, nothing is really special here. However, if producers are way faster than consumers (either out of traffic peak or some parts of the system is down), it will post a challenge like avoiding buffer overflow or out of memory issues. Reactive streaming is drafted to focus on this aspect.
…more people started using Hadoop and other batch-based frameworks. They needed real-time online streaming. Once you need that, then you don’t know up front how big your input is because it’s continuous…you need a means to control the rate at which you consume that data. You need to have this back pressure in your system to make sure the producer of data doesn’t overwhelm the consumer of data. It’s a problem that becomes visible once you start going to real-time streaming from batch-based.