Big Data Engineer

Responsibilities:

• Design, develop, and implement scalable data pipelines and ETL processes using Java, Python, and Spark.
• Collaborate with data scientists, analysts, and other stakeholders to understand data requirements and design efficient solutions.
• Perform data exploration, data cleaning, and data transformation tasks to prepare data for analysis and modeling.
• Develop and maintain data models and schemas to support data integration and analysis.
• Implement data quality and validation checks to ensure accuracy and consistency of data.
• Utilize REST API development skills to create and integrate data services and endpoints for seamless data access and consumption.
• Monitor and troubleshoot data pipeline performance, identifying and resolving bottlenecks and issues.
• Mentor and guide junior team members, providing technical leadership and sharing best practices.

Qualifications:

• Master's degree in Computer Science, Data Science, or a related field.
• Minimum of 3 years of professional experience in data engineering, working with Java, Python, Spark, and big data technologies.
• Strong programming skills in Java and Python, with expertise in building scalable and maintainable code.
• Proven experience in Spark cluster management, optimization, and performance tuning.
• Solid understanding of data science concepts and experience working with data scientists and analysts.

Key Skills:

• Multi-Cloud computing. 
• Visualization. 
• Machine Learning and AI. 
• NoSQL. 
• Data Pipelines. 
• Hyper Automation.
• Programming. 
• DevOps.