Senior Data Engineer

Senior Data Engineer

Remuneration: R900000 – R1150000 per year To be discussed 
Location: Cape Town
Education level: Degree
Job level: Senior
Type: Permanent
Reference: #0023 SDE
Company: Core Change


Job description

Our client is looking for a senior data engineer.

Key responsibilities:

  • Create and maintain optimal data pipeline architecture
  • Designs data integrations and data quality framework
  • Assemble large, complex data sets that meet functional/ non-functional business requirements
  • Identify, design, and implement internal process improvements: automating manual processes, optimising data delivery, re-designing infrastructure for greater scalability, etc.
  • Keep our data separated and secure across national boundaries through AWS security services
  • Writes unit/integration tests, contributes to engineering wiki, and documents work
Technologies used:
  • AWS: S3, EC2, RDS, Redshift, Lambdas, CloudWatch, etc.. 
  • Python
  • Jenkins
  • Github
  • Terraform


Requirements

As a team member:
  • Continuously keeping the team up to date with possible changes
  • Report progress in daily stand-ups
  • Weekly sprint meeting
  • Weekly meeting with the head of search and data to align, share learnings and suggest improvements
  • Participate in the scrum process and daily stand-ups
  • Seasoned communicator as regards to technical and business/ functional aspects of assigned work
  • Be comfortable working in an agile and fast-paced environment
Qualifications and experience:
  • Two years’ experience working with server-less services such as GCP or AWS
  • Four years’ experience building and optimizing ‘big data’ data pipelines, architectures and data sets
  • A successful history of manipulating, processing and extracting value from large disconnected datasets
  • Build processes supporting data transformation, data structures, metadata, dependency and workload management
  • Strong analytic skills related to working with unstructured datasets
  • Working knowledge of message queuing, stream processing and highly scalable ‘big data’ data stores


Posted on 09 Jul 08:10

Apply
Lyn

+27 83 230 6462


Partagez et envoyez cette annonce à vos amis !
Senior IoT Developer – C# Remuneration: R600 – R700 per hour  Location: Johannesburg, Midrand Education level: Degree Reference: #TM47580. Company: E-Merge IT Recruitment Job description Be in the driving seat of a…
Senior Java Developer Remuneration: negotiable Cost to company  Location: Pretoria, Centurion Education level: Degree Job level: Mid/Senior Type: Permanent Reference: #GZ38682 Company: E-Merge IT Recruitment Job description We have a new…
Solution Architect Remuneration: negotiable Basic salary  Location: Johannesburg Education level: Degree Job level: Senior Type: Permanent Reference: #DM40134 Company: E-Merge IT Recruitment Job description Our client is looking for a solutions architect…
General Details Location: , OtherJohannesburg Advertised By: Agency Job Type: Full-Time Description Exciting challenge for ICT Manager at senior level with Mining or Manufacturing company experience (unionised environment) To be…
Java Developer Remuneration: R400000 – R480000 per year Negotiable depending on experience  Location: Johannesburg, Edenvale Education level: Degree Job level: Senior Type: Permanent Reference: #GZ41001 Company: E-Merge IT Recruitment Job description South Africa’s…
.Net Core Web Application Architect Remuneration: R900000 – R1100000 per year  Location: Johannesburg, Bryanston Education level: Degree Job level: Senior Type: Contract Reference: #SP49162 Company: E-Merge IT Recruitment Are you a specialized…
Senior iOS Developer Remuneration: R600 – R700 per hour Basic salary  Location: Johannesburg Type: Contract Reference: #GZ46334 Company: E-Merge IT Recruitment Job description Introduction An industry-leading motor car manufacturer is looking for a…
C# Engineer Remuneration: R55000 – R60000 per month  Location: Johannesburg, Sandton Education level: Degree Type: Permanent Reference: #SP42916 Company: E-Merge IT Recruitment Job description An industry innovating Life Insurance group is looking…
Python Developer Remuneration: R800000 – R804000 per year  Location: Johannesburg, Melrose Education level: Degree Type: Permanent Reference: #DM48508 Company: E-Merge IT Recruitment Job description A flexible and fun environment has an exciting…