CS-E4640 - Big Data Platforms D, Lecture, 11.1.2023-13.4.2023
This course space end date is set to 13.04.2023 Search Courses: CS-E4640
Översikt
-
-
We study and discuss key architectural principles for designing big data platforms.
- your scenario/story of big data
- data movement in big data platforms
- basic big data pipelines
- Lambda architecture
- Kappa architecture
- big data at large-scale
- key building blocks and technologies
- reactive systems for big data platforms
- partitioning
- data concerns
- component API, interaction, orchestration and coordination
- components distribution
- scalability and elasticity
- your scenario/story of big data
-
We examine service models and integration for big data platforms.
- Bring data into platforms
- data transfer/uploading models
- examples of technology stacks (Google, AWS, Azure)
- Messaging protocols for big data
- MQTT
- AMQP
- Optimizing service requests and functionalities
- Contention, back-pressure, elasticity
- Sharding
- Discovery and consensus in big data platforms
- Key techniques
- Examples of Zookeeper, consul, etcd.
-
Big data storages, databases and services in big data platforms.
- Consistency, Availability and Partition Tolerance
- Basic models, CAP/BASE
- Data models and data management
- Data models (File, relational data, Key-value model, document-oriented model, column family, graph)
- Examples with Cloud storage, Cassandra, Mongodb, etc.