Spark SQL and Spark 3 using Scala Hands-On with Labs

A comprehensive course on Spark SQL as well as Data Frame APIs using Scala with complementary lab access

4.49 (2941 reviews)

Udemy

platform

English

language

IT Certification

What you will learn

All the HDFS Commands that are relevant to validate files and folders in HDFS.

Enough Scala to work Data Engineering Projects using Scala as Programming Language

Spark Dataframe APIs to solve the problems using Dataframe style APIs.

Basic Transformations such as Projection, Filtering, Total as well as Aggregations by Keys using Spark Dataframe APIs

Inner as well as outer joins using Spark Data Frame APIs

Ability to use Spark SQL to solve the problems using SQL style syntax.

Basic Transformations such as Projection, Filtering, Total as well as Aggregations by Keys using Spark SQL

Inner as well as outer joins using Spark SQL

Basic DDL to create and manage tables using Spark SQL

Basic DML or CRUD Operations using Spark SQL

Create and Manage Partitioned Tables using Spark SQL

Manipulating Data using Spark SQL Functions

Advanced Analytical or Windowing Functions to perform aggregations and ranking using Spark SQL

Course Gallery

Spark SQL and Spark 3 using Scala Hands-On with Labs – Screenshot 1 — Screenshot 1 – Spark SQL and Spark 3 using Scala Hands-On with Labs

Spark SQL and Spark 3 using Scala Hands-On with Labs – Screenshot 2 — Screenshot 2 – Spark SQL and Spark 3 using Scala Hands-On with Labs

Spark SQL and Spark 3 using Scala Hands-On with Labs – Screenshot 3 — Screenshot 3 – Spark SQL and Spark 3 using Scala Hands-On with Labs

Spark SQL and Spark 3 using Scala Hands-On with Labs – Screenshot 4 — Screenshot 4 – Spark SQL and Spark 3 using Scala Hands-On with Labs

Loading charts...

Comidoc Review

Our Verdict

A worthy pick for Scala and Spark SQL enthusiasts, but expect to invest time in navigating through occasional outdated content and cluster tuning concepts. Boost your Data Engineering Projects readiness while grasping HDFS commands, transformations, joins, and analytical functions through this Scala-infused learning excursion. However, Linux system administration knowledge is assumed for using Cloudera VM or personal clusters. Be prepared to troubleshoot any setup issues independently.

What We Liked

Comprehensive coverage of Spark SQL and Data Frame APIs using Scala
Excellent for gaining enough Scala to work on Data Engineering Projects
In-depth explanations of Transformations, Joins, DDL and DML Operations
Insightful: Analytical and Windowing Functions with Spark SQL

Potential Drawbacks

Last section might be showing its age; sparkContext usage may not be current
Cluster tuning extensively used in exercises but not fully explained in course
Setup issues for those who prefer working on Cloudera VM or personal clusters
First part of IntelliJ with sbt setup is not directly relevant to the rest of the course