Ultimate AWS Data Engineering Bootcamp

Ultimate AWS Data Engineering Bootcamp - 15 Real-World Labs

• Lesson 01: Course Introduction

  11:01Preview

  • 1.01 Course objectives and Tech stack

    03:31

  • 1.02 Prerequisites & tools

    03:13

  • 1.03 Install Docker and AWS CLI

    04:17

• Lesson 02: Lab - Batch data processing of music streams using Airflow & Redshift

  43:10Preview

  • 2.01 Project Introduction

    01:25

  • 2.02 Introduction to Redshift serverless

    02:13

  • 2.03 Introduction to Airflow for data orchestration

    04:11

  • 2.04 Lab - Setup Redshift Serverless

    06:48

  • 2.05 Lab - Setup Airflow Environment on AWS

    05:00

  • 2.06 Lab - Local Python development of the code

    03:56

  • 2.07 Lab - Visual walkthrough of the Airflow DAG

    02:05

  • 2.08 Airflow DAG Code explanation

    08:39

  • 2.09 Lab - End to end execution

    08:53

• Lesson 03: Lab - Distributed music streams processing using Airflow, Spark & DynamoDB

  32:26Preview

  • 3.01 Project introduction

    03:49

  • 3.02 Introduction to DynamoDB

    01:58

  • 3.03 Lab - Execute Pyspark locally using Glue Docker image

    09:37

  • 3.04 Lab - Pyspark code walkthrough

    06:40

  • 3.05 Lab - End to end execution

    10:22

• Lesson 04: Lab - ETL for Rental apartments using Step Functions, AWS Glue, and Redshift

  01:02:21Preview

  • 4.01 Project Introduction

    04:17

  • 4.02 Introduction to Step Functions for workflow automation

    02:30

  • 4.03 Introduction to AWS Aurora for MySQL

    01:10

  • 4.04 Lab - Setup MySQL db on AWS Aurora

    08:29

  • 4.05 Lab - Deploy and execute your MySQL Extraction Glue Job

    13:12

  • 4.06 Lab - Deploy Redshift ingestion pipeline on AWS Glue

    13:17

  • 4.07 Lab - Deploy pipeline to process curated data in Redshift on AWS Glue

    06:19

  • 4.08 Lab - Deploy Step Functions to orchestrate workflow execution

    08:46

  • 4.09 Lab - Setup AWS Event Bridge for workflow scheduling and automation

    04:21

• Lesson 05: Lab - Build a datalake for rental vehicles store using EMR, S3 and Athena

  43:49Preview

  • 5.01 Project Introduction

    02:49

  • 5.02 Introduction to AWS Elastic Map-Reduce (EMR)

    01:06

  • 5.03 Introduction to AWS Athena

    01:05

  • 5.04 Lab - Execute Pyspark using Docker locally

    06:28

  • 5.05 Lab - Deploy and execute spark jobs on AWS EMR

    07:15

  • 5.06 Lab - Setup Glue crawlers for data catalog and Athena

    05:25

  • 5.07 Lab - Step functions code walkthrough and deployment

    12:43

  • 5.08 Lab - End to end project execution

    06:58

• Lesson 06: Lab - Build Event driven pipelines for E-Commerce using ECS and Step Functions

  01:01:15Preview

  • 6.01 Project Introduction

    03:48

  • 6.02 Introduction to AWS Elastic Container Service (ECS)

    01:38

  • 6.03 Lab - Containerize your First Python application and execute locally

    08:13

  • 6.04 Lab - Deploy containerized Python application to ECS

    12:17

  • 6.05 Lab - Deploy and execute the second Docker application in ECS

    14:19

  • 6.06 Lab - Deploy and execute Step Functions for ECS workflow automation

    15:48

  • 6.07 Lab - Setup AWS EventBridge for workflow scheduling and automation

    05:12

• Lesson 07: Lab - Build a lakehouse for an E-Commerce store using Pyspark delta tables and S3

  32:12Preview

  • 7.01 Project Introduction

    04:44

  • 7.02 Lab - Execute Pyspark code to create deltalake using Docker locally

    10:56

  • 7.03 Lab - Deploy Pyspark on AWS Glue

    04:52

  • 7.04 Lab - Setup Glue data catalog tables and query using AWS Athena

    07:23

  • 7.05 Lab - Access delta tables from Glue data catalog using Redshift

    04:17

• Lesson 08: Lab - Event driven data processing for Taxi trips using Lambda and Kinesis

  28:13Preview

  • 8.01 Project Introduction

    03:13

  • 8.02 Introduction to AWS Kinesis for event-driven & real-time data processing

    02:14

  • 8.03 Lab - Ingest streaming data into Kinesis data streams from local env

    03:49

  • 8.04 Lab - Deploy Lambda Functions with Kinesis triggers

    11:38

  • 8.05 Lab - End to end project execution

    07:19

• Lesson 09: Lab - Process mobile network logs in real time using Pyspark & Streamlit on ECS

  32:33Preview

  • 9.01 Project introduction

    02:04

  • 9.02 Lab - Pyspark code walkthrough

    05:14

  • 9.03 Lab - Setup Kinesis data stream & deploy spark streaming job on AWS Glue

    06:55

  • 9.04 Lab - Setup Glue crawlers for data catalog and Athena

    05:37

  • 9.05 Lab - Setup a real time Streamlit dashboard using Docker locally

    04:45

  • 9.06 Lab - Deploy Streamlit dashboard to AWS ECS as a containerized task

    07:58

• Lesson 10: Lab - CI/CD for AWS Services using GITHUB ACTIONS

  51:29

  • 10.01 Introduction to CI/CD using Github Actions

    03:01

  • 10.02 Lab - Learn AWS CLI for deployments & visual walkthrough of Github actions

    08:58

  • 10.03 Lab - Deploy AWS Glue using Github Actions

    10:48

  • 10.04 Lab - Deploy AWS ECS tasks using Github Actions

    16:36

  • 10.05 Lab - Deploy Lambda functions using Github Actions

    12:06

• Lesson 11: Lab - Real time data ingestion of clickstreams using Kinesis Firehose and Redshift

  20:08Preview

  • 11.01 Project Introduction

    03:07

  • 11.02 Lab - Create Redshift Tables for Firehose delivery and Lambda for data enrichment

    07:42

  • 11.03 Lab - Create Kinesis Data Streams and Firehose Streams

    05:27

  • 11.04 Lab - Real time data ingestion pipeline execution

    03:52

• Lesson 12: Assignment 1 - Setup MySQL Database in AWS Aurora RDS

  10:01

  • 12.01 Assignment Problem Statement

    02:23

  • 12.02 Assignment Solution

    07:38

• Lesson 13: Assignment 2 - Build a lakehouse on S3 for Commercial flights dataset

  18:33

  • 13.01 Assignment Problem Statement

    03:25

  • 13.02 Solution Lab - Deploy Data extraction pipeline

    06:36

  • 13.03 Solution Lab - Deploy Pyspark job for Delta Lake

    05:10

  • 13.04 Solution Lab - Setup Glue Crawlers for data catalog

    03:22

• Lesson 14: Assignment 3 - Offer dynamic discounts to E-Commerce users using Real Time Events

  10:29Preview

  • 14.01 Assignment Problem Statement

    03:07

  • 14.02 Assignment Solution

    07:22

• Lesson 15: Assignment 4 - Setup real time Pyspark streaming job for Spotify songs metrics

  10:05

  • 15.01 Assignment Problem Statement

    03:26

  • 15.02 Assignment Solution

    06:39

• Lesson 16: Assignment 5 - Automate deployment of Lambda functions using Github actions

  06:44Preview

  • 16.01 Assignment Problem Statement

    01:13

  • 16.02 Assignment Solution

    05:31