Data Engineer

Responsiblties:

• Create data pipelines for transforming new datasets into consistent, reliable data systems that are usable across multiple platforms
• Identify optimizations and improvements in existing data pipelines.
• Create solutions for data validation to ensure data integrity, accuracy, and consistency.
• Come up with data validation solutions for both initial data imports as well as results.
• Audit the automation process to make sure there are no gaps, redundancies, or faults in the data pipeline that would cause damage to data integrity.
• Planning query efficient data store schemas with respect to the data requirements of our internal data warehouse.
• Communicate with data scientists and other software engineers to build proper data pipelines for efficient and cost-effective querying.

Key Skills:

• Strong programming skills, particularly in languages.
•  Python, Java, Scala, and SQL.
• They should also understand database systems.
• Computing systems, and big data technologies.