As a Data/QA Engineer in our Enterprise Data department, you will be responsible for creating and managing automated tests to identify issues with the ETL process to ensure the quality of data in the Enterprise Data Warehouse.
The role involves working together with the onshore Enterprise Data team to decide on the required tests on the process, automate them through creating stored procedures, custom dbt tests and scheduling these as part of the ETL process which loads data from source into the EDW.
Job Tasks/Responsibilities:
- Use dbt to build tests that ensures the ETL process is working as intended
- Automate the common manual testing done by the QA through creating macros in dbt.
- Build and monitor automated system health checks
- Collaborate with Enterprise Data Engineers to investigate root cause of issues and suggestresolutions.
- Orchestrate data testing solutions using airflow
- Be able to support the team in doing releases
Required Skills/Experience
- You enjoy collaborating with your team members to build solutions which improves the data quality within the data warehouse
- You have a drive to automate processes that mainly focuses on ensuring data quality and process rigidity
- You have experience within data engineering.
- You have experience working with relational databases and investigating root cause of issues.
- Your SQL and data transformation skills are a key skill of yours
- You have an understanding of database management systems and scripting
Desired Skills/ Experience
- Strong SQL and data transformation skills
- Understanding of ETL/ELT process fundamentals
- Experience in dbt, snowflake, airflow (Nice to have)
Education
