Big Data Engineer

Responsibilities:

• Establishing the system environment for large data sets, considering aspects such as data capture, storage, extraction, process and analysis.
• Using data visualization tools to create roadmaps, placemats and process flows; and creatively displaying data to facilitate customer decision-making processes.
• Applying data wrangling and exploration techniques.
• Manipulating data to gather insights from large structured and unstructured data sets and developing trend analyses.
• Developing algorithms to facilitate data analysis, support identification of data errors, and developing pattern analysis from multiple disparate data sources.
• Providing input on software implementation for analytics.
• Validating customer requirements and determining feasibility of solutions and alternative options.
• Contributing to and collaborating on executive-level briefings.
• Solving complex problems through creative and effective solutions; identifying areas of improvement; and making recommendations to realize the customer’s vision.

Key Skills:

• Databases and SQL.
• ETL and data warehousing.
• Talend, IBM DataStage, Pentaho, and Informatica.
• Operating system knowledge for Unix, Linux, Windows, and Solaris.
• Hadoop.
• Apache Spark.
• Data mining and modeling.