Job description:- Design, build, and maintain scalable data pipelines and infrastructure using big data tools such as Hadoop, Spark, Kafka, etc.
Develop and optimize relational SQL and NoSQL databases, including Postgres and Cassandra, to support data storage and retrieval requirements.- Implement data pipeline and workflow management solutions using tools such as Azkaban, Luigi, Airflow, etc.
Utilize AWS cloud services such as EC2, EMR, RDS, and Redshift to deploy and manage data infrastructure in the cloud.
- Architect and implement stream-processing systems, including Storm, Spark-Streaming, etc., to process real-time data streams.
Write efficient, maintainable code in object-oriented/object function scripting languages such as Python, Java, C++, Scala, etc., to automate data processes and workflows.- Collaborate with cross-functional teams including data scientists, software engineers, and product managers to understand data requirements and deliver innovative solutions.
Stay up-to-date with emerging technologies and industry trends in big data and data engineering to continuously improve our data infrastructure and processes.
At least 5 years of experience as Data Engineer
- Must be a Graduate of Bachelor's Degree in Information Technology, Computer Science, or related fields.
Experience with big data tools: Hadoop, Spark, Kafka, etc.- Experience with relational SQL and NoSQL databases, including Postgres and Cassandra.
Experience with data pipeline and workflow management tools: Azkaban, Luigi, Airflow, etc.
- Experience with AWS cloud services: EC2, EMR, RDS, Redshift
Experience with stream-processing systems: Storm, Spark-Streaming, etc.- Experience with object-oriented/object function scripting languages: Python, Java, C++, Scala, etc.
Job Type: Full-time
Benefits: Health insurance
- Opportunities for promotion
Pay raise- Promotion to permanent employee
Schedule: Day shift
Supplemental pay types:
Overtime pay
Education:
Bachelor's (Required)
