BIG DATA

Job Description :

Big Data Lead / Engineer

Location : Houston, TX

 

Must have Azure and data factor

• 5+ years of hands on experience in Big Data technologies like Cloudera Spark, Hive, Scala or Java, Oozie, Impala, SQL queries is mandatory
• 3+ years of hands on experience in

o Azure Data Factory, Data Pipeline

o Databricks, Spark, PySpark and Python

o Azure Data Lake Storage

o Azure SQL Database / Azure SQL Data Warehouse

• Experience with AGILE & implementing DevOps using Jenkins, Git and Maven
• Good hands on experience in data extraction, ingestion, transformation and processing of large and complex data sets
• Excellent communication skills and customer facing skills to drive user stories and tasks in Agile scrum meetings independently.
• Strong experience with estimating tasks for user stories
• Very strong experience in resolving technical challenges and managing dependencies with scrum team
• Ability to work independently and self-driven
• Deciphering the business requirements for data and reporting and design a solution.
• Deep diving into data to understand the potential attributes/metrics, their mapping against tables and their source of origin.
• Root cause analysis of potential issues
• Producing useful insight and valuable information from various raw data sources; ability to identify and analyze metric variance within tables and work with appropriate business & technology groups to align on solutions.