Complete Big Data Engineering Course, Part 1, 5 p.m.

Channel	Publish Date	Thumbnail & View Count	Download Video
	Publish Date not found	0 Views	View on YouTube Download Video

This course covers comprehensive topics on Big Data Engineering

Part 1 – https://youtu.be/Tyg1FVNq40g
Part 2 – https://youtu.be/k1LaWFNOa68

Resources

Hadoop Installation Steps – https://github.com/atozknowledge/bigdata/wiki/Hadoop-Single-Node-Installation

Hadoop Multi-Node Cluster Setup Installation Steps – https://bit.ly/3LRwgRi

Big Data Integration Book – https://bit.ly/3ipIlBx

Hive
Hive-site.xml – https://github.com/Gowthamsb12/hive/blob/main/hive-site.xml
Hive ACID Orders – https://bit.ly/2V9W1qT
Apache Hive ORC vs TextFile format – https://bit.ly/3cbIbNl
Hive UDF Code – https://codewithgowtham.blogspot.com/2021/09/hive-udf.html

Spark
Spark submission cluster mode code link [YARN] – https://github.com/atozknowledge/bigdata
Spark Kafka Cassandra End-to-end Streaming Project Code and Steps – https://bit.ly/3LqXXRC

Kafka installation video – https://youtu.be/XCOIp-CqGkg

Sqoop Commands – https://codewithgowtham.blogspot.com/2021/03/sqoop-commands.html

Lesson Plan

00:00 About the instructor
00:48 All about [What is Big Data]
37:52 Roadmap for Big Data Engineering
54:45 Hadoop Distributed File System (HDFS)
02:16:39 Unboxing [Hadoop Framework]
02:38:06 Setting up a Hadoop single node
03:14:24 Use cases for HDFS quotas
03:26:57 MapReduce Full Video
05:30:33 Introduction and architecture of Apache Hive
05:47:23 Installation [Apache Hive 2 with MySQL]
06:00:50 Hive SQL [Create load insert show]
06:19:05 Hive internal and external table
06:25:53 Hive partition [Static vs Dynamic]
06:43:47 Hive Bucket Explained from start to finish
07:03:09 How to decide [number of buckets] in Hive
07:14:09 Hive partition with bucket explained
07:20:42 Hive ORC file format with demo
07:28:18 ACID painting of the hive
07:35:09 UDF Hive
07:47:39 Apache Spark overview
08:35:18 Installing Apache Spark
08:52:36 Apache Spark Scala word count program (REPL)
09:05:51 Spark Autonomous Architecture