Lambda Architecture
Jump to navigation
Jump to search
A Lambda Architecture is a data processing architecture designed to handle massive data streams by using batch processing of batch data to optimize for space, while simultaneously using real-time stream processing of online data to optimize for time.
- Context:
- It can (typically) be composed of a Lambda Serving Layer, Lambda Batch Layer, and a Lambda Speed Layer.
- …
- Counter-Example(s):
- See: Map-Reduce, Data Processing, Batch Processing, Stream Processing, Latency, Throughput, Fault-Tolerance, Big Data, AWS Lambda.
References
2019
- (Wikipedia, 2019) ⇒ https://en.wikipedia.org/wiki/Lambda_architecture Retrieved:2019-12-4.
- Lambda architecture is a data-processing architecture designed to handle massive quantities of data by taking advantage of both batch and stream-processing methods. This approach to architecture attempts to balance latency, throughput, and fault-tolerance by using batch processing to provide comprehensive and accurate views of batch data, while simultaneously using real-time stream processing to provide views of online data. The two view outputs may be joined before presentation. The rise of lambda architecture is correlated with the growth of big data, real-time analytics, and the drive to mitigate the latencies of map-reduce. [1] Lambda architecture depends on a data model with an append-only, immutable data source that serves as a system of record.[2] It is intended for ingesting and processing timestamped events that are appended to existing events rather than overwriting them. State is determined from the natural time-based ordering of the data.
2019
- https://databricks.com/glossary/lambda-architecture
- QUOTE: Lambda architecture is a way of processing massive quantities of data (i.e. “Big Data”) that provides access to batch-processing and stream-processing methods with a hybrid approach.
Lambda architecture is used to solve the problem of computing arbitrary functions. The lambda architecture itself is composed of 3 layers:
- QUOTE: Lambda architecture is a way of processing massive quantities of data (i.e. “Big Data”) that provides access to batch-processing and stream-processing methods with a hybrid approach.
- ↑ Interview with Nathan Marz, 6 April 2014
- ↑ Bijnens, Nathan. "A real-time architecture using Hadoop and Storm". 11 December 2013.