Job Id : 4211
Jobtitle : Senior Data Engineer
Location : Houston, TX
Company Name : Fircroft
Industry : Information Technology
Salary : $80,000 - $100,000 YEAR
Job type : Fulltime
Posted on: 2019-08-20
Required Skills : Data Management ,Hadoop,Spark,Cassandra, HBASE, MongoDB,KAFKA, StreamSets, Talend, ADFv2
Benefits : No benefits are available
Purpose and Accountabilities:
Introduction: Client’s global Trading and Supply business(STASCO) is one of the largest energy trading operations in the world with key hubs in Dubai, Houston, London, Rotterdam and Singapore, trading in crude oil, natural gas, LNG, electrical power, refined products, chemical feedstocks and environmental products. This combined with an integrated network of supply and distribution activities and an industry-leading shipping and maritime capability add value for Client across its Upstream and Downstream businesses. To enable value to the Business through IT, STASCO continues to expand in the digitalization arena and, in particular data and analytics, by focusing on business outcomes, delivering fit-for-business technology solutions which enable business agility and profitable growth through:
- A single face to the business who is accountable for IT delivery and influence strategic business plans.
- Stewardship that drives data integrity and quality.
- Proactive support in driving continuous business improvement to meet business performance objectives.
- Benefits realisation of the investments throughout their entire lifecycle.
- Capture of value opportunities through innovative, differentiated and competitive IT.
- Project delivery on-time, on-budget and meeting specifications.
- IT Functional Excellence through the adoption of best practices, continuous improvements and capabilities that apply across the enterprise.
- Client staff at strategic control points to retain in-house competitive advantage while using the external market for leverage of capability and scale.
- The Senior Data engineer will work alongside the Data Science Lead and will be responsible to support the development of cutting edge algorithms using the latest data and machine learning technologies. As the Senior Data engineer, you will ingest structured, semi-structured and unstructured batch- and real-time data from numerous data sources into various data platforms. The Senior Data engineer will use the latest auto deployment, testing and monitoring techniques. The Senior data engineer will work closely with the Solution Architect and Data Architect to design the big data platform.
Purpose and Accountabilities:
- Work together with the Solution and Data Architect to design the next generation of data platforms.
- Design and implement data solutions which supports high amounts- and velocity of data, supports future growth, use latest data technologies/techniques, is cloud supported and in-line with Client’s strategies.
- Design and implement modern data pipelines to extract, clean and process data in batch and real-time from different data sources.
- Use the latest development, testing and deployment techniques to quickly deploy new releases to e.g. deploy new data pipelines and add data sources.
- Help establish best practices around data governance, data security- and privacy (GDPR).
- Drives and applies Agile Framework within his/her operating environment. Will take direct reports, when applicable, through Agile Framework and its respective tools.
- The Data Engineer will report into the Application Delivery Lead
- Commercial Awareness - Influence on programme scope of $USD 2M+
- Stakeholder management – typically works within cross-functional teams.
- Strong collaboration skills to ensure teams are all working towards the same goals.
- Highly customer focused and fast paced environment.
- Passion to lead others directly and indirectly into the new Data Engineering career development
- Able to deal with state of the art, differentiated and often unproven technologies which require some learning on the job
- Comfortable with sometimes vague and changing scope and priorities in a rapidly changing environment
- Ability to interact and build relationships both within Client (IT & Business teams) and external software suppliers.
- Fast adaptability to the every fast and changing environment within Client-Digital and in the external market.
Skills & Requirements Education
- Bachelor or master degree in information technology, computer science, data management business administration or a related discipline.
- 10+ years’ experience with data platforms; working with large data sets, large amounts of data in motion and numerous big data technologies.
- At least 3 to 5 years’ experience in leading teams/Resources.
- Knowledge and understanding of Trading and Supply businesses
- Auto software deployment technologies (e.g. Jenkins, Docker, Git)
- Experience with Cloud (AWS, Microsoft Azure).
- Hadoop/Spark-based technologies (e.g. Spark, MapReduce).
- NoSQL-based technologies (e.g. Cassandra, HBASE, MongoDB).
- Awareness of Agile Framework and its tools such as Visual Studio Team Services (VSTS)
- Data pipelining-based technologies (e.g. KAFKA, StreamSets, Talend, ADFv2).