Get job as soon as 2 days (HURRY UP) ENROLL NOW or Call   +1 (804)-454-3215
+

Spark Hadoop Data Engineer


Job ID : 15328

Job Title : Spark Hadoop Data Engineer

Location : Irving, TX

Company Name : NTT Data

Industry : Information Technology

Salary : $76,690 - $128,917  per year

Job Type : Fulltime

Posted on: 2020-09-01


Required Skills : SQL, Hive, Hadoop, PySpark, Scala, Python

Benefits : No benefits are available


Duties and Responsibilities:

  • Analyze and understand  data sources & APIs
  • Design and Develop methods to connect & collect data from different data sources
  • Design and Develop methods to filter/cleanse the data
  • Design and Develop SQL, Hive queries, APIs to extract data from the store
  • Work closely with data scientists to ensure the source data is aggregated and cleansed
  • Work with product managers to understand the business objectives
  • Work with cloud and data architects to define robust architecture in cloud setup pipelines and workflows
  • Work with DevOps to build automated data pipelines
  • Hands-on experience with the Hadoop eco-system (HDFS, MapReduce, Yarn, Hive, Pig, Impala, Spark, Kafka,)
  • Good understanding of ETL tools like Ab-initio, Talend
  • Familiarity with  HTTP and invoking web-APIs
  • Exposure to machine learning engineering
  • Exposure to NLP and text processing
  • Experienced in managing work with distributed teams
  • Experience working in SCRUM methodology
  • Proven sense of high accountability and self-drive to take on and see through big challenges
  • Confident, takes ownership, willingness to get the job done
  • Excellent verbal communications and cross-group collaboration skills

Basic Qualification:

  • 3+ years of Advanced knowledge of Hadoop ecosystem and Big Data technologies
  • 3+ years of experience and expert-level knowledge building pipelines using Spark /Pyspark
  • 3+ years of experience in programming in Scala and Python          

Education:

  • Bachelor’s Degree in Computer Science or related discipline required.

 

Latest Jobs For You

  • Software Developer Engineer in Test SDET

    Louisiana Economic Development Bossier City, La.

    Responsibilities: Design, develop and implement test plans, manual test scripts, and automated...

    View Job
  • Senior SDET Engineer

    Cynet Systems Hartford, CT

    Job Description: Middle and Large Commercial IT is looking for an experienced, energetic, and ...

    View Job
  • Sr Software Test Engineer

    COX Enterprises Irvine, CA

    Job Description: A highly qualified Senior Quality Assurance Test Automation Engineer will ...

    View Job
  • Network Engineer

    Tech USA Downingtown, PA

    Responsibilities: Review and provide input to new technical requirements Ability to ...

    View Job
  • Sr. Big Data Developer

    Synechron Weehawken, NJ

    Skill Requirements: Hands-on expertise with Big data Technologies (Apache Spark, Hadoop, HDFS,...

    View Job

Thanks For Your Feedback

We’re an equal opportunity provider.
All applicants will be considered for employment without attention to race, color, religion, sex, sexual orientation, gender identity, national origin, veteran or disability status.
OPTnation.com is not a Consulting Company/Training Company/H1B Sponsor.

Attach A Resume First