Data Engineer

Responsibilities:

• Work with product managers and data analysts to build data products that power our business.
• Build and maintain data analytics reports, ETL pipelines, data lakes, and streaming solutions.
• Analyze new data sources and work with stakeholders to understand the impact of integrating new data into existing pipelines and models
• Develop & maintain critical data pipelines, using tools such as Apache Airflow, to ensure highly accurate and reliable business reporting
• Perform ad-hoc analysis and provide insights to stakeholders within the organization such as Marketing, Growth & Sales teams to drive prioritization.
• Create data visualizations and analytic reports using various tools such as Tableau, Databox, etc
• Optimize database queries to improve and maintain performance on very large data sets.
• Work closely with the business and data science team for the deliverables.

• Masters or bachelor's degree in CS/ML/AI or relevant computational/engineering discipline.
• Experience building stream-processing systems, using solutions such as Kinesis, Kafka, Storm or Spark-Streaming.
• Expertise with coding with Pyspark, Pandas, Scipy, and visualizations with Tableau & Databox. Scheduling/orchestrating pipelines using Databricks and Airflow.
• Expertise with data querying (SQL, Redshift, Spark) and analyzing billions of rows of data points.
• Familiarity with SQL, especially within cloud-based data warehouses like Snowflake, Google BigQuery, and Amazon Redshift
• Strong understanding of relational DB such as MySQL/PostgreSQL, document-based storage systems such as MongoDB or CouchDB, key-value stores Column-oriented stores graph-oriented stores; their distinct advantages, disadvantages, and trade-offs.
• Experience building machine learning, statistical and analytical models, and tuning parameters.
• Experience designing experiments and extracting insights.

Key Skills:

• Database tools. 
• Data transformation tools. 
• Data ingestion tools. 
• Data mining tools. 
• Data warehousing and ETL tools. 
• Real-time processing frameworks. 
• Data buffering tools.