AWS Senior Data Engineer

Job description Skills Required :

- Proficient in languages : Python.

- Experience in AWS Stack - Glue, Athena, Quick sight, RDS, Redshift, Kafka, pySpark.

- Experience setting up data pipelines, archiving data, data lakes.

- Bachelors or Master's degree in computer science Maths statistics or related field.

- Expertise with designing complex Data Models and Data Engineering solutions.

- Write complex SQL for processing raw data, data transformations as well as data validations.

- Experience with infrastructure as code using shell scripting.

- Hands on experience in working with big data on AWS environments including cleaning transformingcatalogingmapping etc.

- Experience in setup of data warehouse using Amazon Redshift, creating Redshift clusters and perform data analysis queries.

- Experience in ETL and data modeling on AWS ecosystem components - AWS Glue, Redshift, DynamoDB.

- Familiarity with AWS data migration tools such as AWS DMS and AWS Data Pipeline.

- Certifications on AWS will be an added plus.

Roles and Responsibilities :

- Tests, monitors, manages and validates data warehouse activity including data extraction. transformation, movement, loading, cleansing, and updating processes.

- Building and maintaining optimised and highly available data pipelines that facilitate deeper analysis and reporting.

- Collaborate with stakeholders to understand, define and document business applications, data sourcesrelationships, and needs (metrics, dimensions, charts and dashboards).

- Analyse and Identify new data sources as needed per stakeholder requirements.

- Working closely with the analytics team and other business stakeholders to create data processing frameworks that help the business make sense of growing data .

- Manage and monitor existing data pipelines and work with other team members to ensure best practices for ETL and data warehousing.

- Work on optimisation of data storage and processing infrastructure to reduce costs.

- Assist in the design and documentation of data infrastructure and technologies.

Education UG :B.Sc in Maths, Computers, Statistics, B.A in Statistics, Maths, B.Tech/B.E. in Computers, BCA in Computers PG :M.A in Maths, Statistics, M.Tech in Computers, MCA in Computers, MCM in Computers and Management, MS/M.Sc(Science) in Maths, Computers, Statistics Doctorate :Doctorate Not Required

Key Skills AWS Data Lake Data Pipeline Py Spark Kafka Data Warehousing Data Modeling ETL Python SQL

Company Information hidden please Login to view details