Course Information

Big Data platforms is a 5 ECTS Master's level advanced course. This course focuses on big data platforms and on key algorithmic ideas and methods used to implement them. After completing this course you are able to list many of the key technologies used in big data processing and to select suitable methods for solving challenging big data processing tasks using cloud computing technologies. You will also be able to compare the scalability and fault tolerance implications of using the selected methodologies.

Main topics are:

• distributed computing,
• Warehouse-Scale Computers,
• fault tolerance in distributed systems,
• distributed file systems,
• distributed batch processing with the MapReduce and the Apache Spark (PySpark) computing frameworks, and
• distributed cloud based databases.

The course material will consist of lecture materials and exercises provided by the lecturer.

Course Target Audience

The course is suitable to those who are interested in big data platforms employed in cloud computing and have previous knowledge in programming, database systems and command line tools. Optional course in Data Science Master's Program. Also suitable for Computer Science Master's Program students. The course is suitable to University of Helsinki exchange students.

Course Prerequisites

To attend this course, you must have:

• basic programming skills (Python),
• skills to work with command line tools in Linux, and
• basic knowledge in database systems (SQL).

Lecture Schedule

The Lectures of the course will be will Zoom based lectures. The Zoom links to the lecture sessions and the slides will be made available below. Video recording of each of the lectures will be made available a couple of hours after the live lecture session. The link to the Zoom lectures is:

https://helsinki.zoom.us/j/65782930627?pwd=b2ovVlJvaWR1dTFORzJFUzVHWWczZz09

Lecture Slides and Videos

The Lecture slides contain all the material needed to pass the course, the videos go through this material and contain no additional information needed for the exam.

Home Exercise Schedule

The course will contain programming exercises where you will be using the Spark framework to solve Big Data processing tasks. We will be using the Python programming language based PySpark interface and will be doing several database query type analytics queries. Therefore basic programming skills using Python and knowledge about database programming, especially using the SQL query language will be very helpful for completing the home exercises. In addition to passing the Exam, you will need to get 50% (17 points) of the home exercise points to pass the course. You can also get +1 to the Exam grade if you get more than 80% (28 points or above) of the points (this is of course not applicable to grades 0 and 5). There are also six extra points beyond 100% (34 points) that students can obtain doing the tasks in an optional Extra assignment. These points are useful for students who would like to get a better overall assignment grade.

The schedule for the home exercises is as follows:

To complete the exercises you will have to use the JupyterHub notebook platform found at https://bigdata.cs.helsinki.fi. A short introduction video on how to use the platform to complete and submit the exercises can be found at https://youtu.be/F0mjmycxWUg.

The assignment grades take some time (a day or two) to get published/updated and are not instant at the moment. The current plan is to run the assignment grading scripts after late nights where there is less load on our servers. Therefore, please be aware of that and give some time before you check back on your assignment grades.Make sure to use the assignment submission validation feature before your submissions if you want to gain more confidence about your submission's correctness and final grade.

For any of your generic course-related questions and group discussions, use our Telegram group. Try to use the group to help each other out and discuss exercise and course-related issues. We will also periodically check the group to address the more important unanswered issues. For specific personal course-related questions, please use the course email address (bigdata-platforms-f20@helsinki.fi).

Course Telegram Group

The course has a Telegram group for helping fellow students. Lecturer and Course Assistants will periodically also join in the conversation. You can join to the groups through the link:

https://t.me/joinchat/TyLakk51VMItQPEcvV1Wwg

Exam Schedule

In addition to the programming exercises you need to pass the exam, which will be a Web based exam scheduled at the end of the course. The next exam is scheduled on Thursday, 21st of January 2021 at 16:00-19:30 Finnish time. The exam will be available from the Exam Website:

https://exams.mooc.fi/en

Passing the Course

Passing both home exercises with at least 50% of the points gained and passing the exam are required to pass the course.