Data Engineer

Skills

• Technical Strong development skills in Pyspark & Python is a must
• Must have hands on experience on PySpark Dataframes, RDD,DAG, Partitioning, SparkSQL, Optimization, Clustering
• Knowledge of Azure SQL DB, ADF is required
• Experience in working with Delta file/Tables, Parquet & CSV file formats
• Knowledge in ETL tools like Knime or Alteryx would be a plus.
• Knowledgeable in cloud technologies primarily in MS Azure, which includes Databricks, ADF, SQL DB, Storage Accounts, KeyVault, Application Gateways, Vnets, Azure Portal Management, PowerBI Integration
• Discipline in writing technical& nontechnical documentation
• 
  

Responsibilities:

• Problem solving skills Should attend to tickets raised by client, identify root cause and resolve tickets as per SLAs
• Monitor critical jobs/pipelines and have the mindset to automate manual tasks and prevent user raised incidents
• Develops and maintains ingestion pipelines
• Critical tickets may needed off hours summary as per SLA. Personnel should be ready to take owners ship of critical tickets until resolution
• Develops and maintains data wrangling code, wrangling pipelines and database integrations
• Focus on end user and customer centricity & Strong oral and written communication skills
• Passion for learning new tools languages and frameworks & Strong problem solving skills able to work with minimal direct guidance