Education: Bachelor’s degree in computer science, Information Systems or related field.

Job Experience & Skills Required:

2+ years’ experience working in Big data environment, optimising and building big data

pipelines, architectures and data sets with e.g. Java, Scala, Python, Hadoop, Apache

Spark and Kafka

Familiarity with Hadoop ecosystem and its components

Good knowledge of the concepts of Hadoop

Solid experience in a working environment in Big Data development utilising SQL or

Python

Experience in Big Data development using Spark

Experience in Hadoop, HDFS and MapReduce

Experience in database design, development and data modelling

The following additional knowledge, skills and attributes are preferred:

Good knowledge in back-end programming, specifically java

Experience with development in a Linux environment and its basic commands

Ability to write reliable, manageable, and high-performance code

Should have basic knowledge of SQL, database structures, principles, and theories

Knowledge of workflow/schedulers

Strong collaboration and communication skills

Strong analytical and problem-solving skills

Support pipelines end to end

Build enhancements and new developments

Build and deploy new data pipelines

Identify optimisation opportunities

Improvement on recovery time in case of prod failures

Test prototypes and oversee handover to the Data Operations teams

Attend and contribute to regular team and User meetings

Responsible for the actual coding or programming of Hadoop applications

High-speed querying