Position Title: Data Engineer
In this role you will be based within the Product Design and Engineering team's Data Platform and Insights function where our focus is on delivering a rock-solid data platform, including analytics and insights-based solutions to support our customers to make data driven decisions.
You will have an in-depth understanding of data integration and processing pipelines, cloud data platforms, development methodologies and standards. Ideally, you will have a successful track record in designing and building well-architected, highly available, scalable data pipelines based on cloud platforms. You will also have a good understanding and experience of continuous integration and delivery.
Role Description:
- Build the infrastructure and pipelines required for optimal extraction, transformation, and loading of data from a wide variety of data sources.
- Assemble large, complex data sets to meet functional / non-functional business requirements.
- Ability to design solutions based on a high-level architecture and aligned with frameworks such as the AWS and Azure Well Architected Framework and maintain effective technical documentation.
- Manage the technical communication between internal and external stakeholders.
- Monitor and maintain production systems.
- Collaborate with other development and research teams.
- Build highly scalable cloud-based platforms that allow easy development of new applications.
Essential Skills:
- 5+ years of data engineering experience.
- Strong proficiency in TSQL and advanced SQL skills (relational databases, query authoring, algorithm design, optimization, and implementation).
- Experience building and deploying cloud native ETL/ELT pipelines on platforms such as Azure and AWS.
- Proficiency in designing and developing within microservices architecture.
- Knowledge of AWS services such as EMR, S3, RDS, DynamoDB, Kinesis, SNS, and SQS.
- Experience with RESTful API design and OpenAPI specifications.
- Knowledge of common design patterns and when/how to use them.
- Experience in designing to address Non-functional requirements (e.g. security, scalability, observability, cost).
- Automated testing on different levels (unit, component, and integration test automation) using tools like dbt, Junit, TestNG, and Mockito.
- Experience with CI/CD pipelines and version control systems like Git.
- Experience with containerisation (e.g., Docker) and deployment of containerised applications using platforms like Amazon Elastic Kubernetes Service (EKS) and Azure Kubernetes Services (AKS)
- Experience with Python, Java, or Scala.
Working Hours:
Morning Shift, 4AM - 1PM (Monday - Friday)
Package and Benefits:
Highly competitive salary + benefits including HMO with 2 free dependents. Role is available for a remote setup with all hardware and tools provided.
Interested applicants may directly apply here or send their CVs and portfolios at [Confidential Information].
