Azure Data Engineer

 Responsibilities:

• Designing and implementing highly performant data pipelines from multiple sources using Apache Spark and/or Azure Databricks.
• Integrating the end-to-end data pipeline to take data from source systems to target data repositories ensuring the quality and consistency of data is always maintained.
• Working with other members of the project team to support delivery of additional project components.
• Evaluating the performance and applicability of multiple tools against customer requirements.
• Working within an Agile delivery / DevOps methodology to deliver proof of concept and production implementation in iterative sprints.
• Direct experience of building data pipelines using Azure Data Factory and Apache Spark.
• Expert level understanding on Azure Data Factory, Azure Synapse, Azure SQL, Azure Data Lake, and Azure App Service is required.

Key Skills:

• ETL.
• Data modeling.
• Data warehouse.
• Data pipelining.
• Data security and compliance.
• Database.
• Machine learning.
• SQL.
• Big data.