BIG DATA

Big Data Lead / Engineer

Location : Houston, TX

Must have Azure and data factor

5+ years of hands on experience in Big Data technologies like Cloudera Spark, Hive, Scala or Java, Oozie, Impala, SQL queries is mandatory
3+ years of hands on experience in

o Azure Data Factory, Data Pipeline

o Databricks, Spark, PySpark and Python

o Azure Data Lake Storage

o Azure SQL Database / Azure SQL Data Warehouse

Experience with AGILE & implementing DevOps using Jenkins, Git and Maven
Good hands on experience in data extraction, ingestion, transformation and processing of large and complex data sets
Excellent communication skills and customer facing skills to drive user stories and tasks in Agile scrum meetings independently.
Strong experience with estimating tasks for user stories
Very strong experience in resolving technical challenges and managing dependencies with scrum team
Ability to work independently and self-driven
Deciphering the business requirements for data and reporting and design a solution.
Deep diving into data to understand the potential attributes/metrics, their mapping against tables and their source of origin.
Root cause analysis of potential issues
Producing useful insight and valuable information from various raw data sources; ability to identify and analyze metric variance within tables and work with appropriate business & technology groups to align on solutions.