Senior Bigdata Developer with Scala/Spark
Job Id : 7482
Jobtitle : Senior Bigdata Developer With Scala/Spark
Location : Atlanta, GA
Company Name : Spar Information Systems
Industry : Information Technology
Salary : $90,000 - $148,000 YEAR
Job type : Fulltime
Posted on: 2019-11-19
Required Skills : BigData, Scala, Spark, HealthCare
Benefits : No benefits are available
- As a Senior Scala/Spark Engineer, you will work with the team to build data pipelines using a new cutting edge Data Ingestion Platform as well as several new and exciting products in the Big Data/Advanced Analytics space. You will operate in a fast-paced environment where multiple project deliverables are coordinated within specified deadlines. This is a great opportunity to work with cutting edge technologies as well as propel your career.
Principal Responsibilities and Essential Duties:
- Data Mapping and Programming
- Work as a team member in the creation and maintenance of data engineering processes using a variety of tools including T-SQL, Spark and Scala, and shell scripting. Generally focused on data ingestion for healthcare data management, data validation, statistical report generation, and program validation.
- Work as a team member to develop tools and techniques for improving process efficiencies and data performance.
- Program per data transformation specifications to convert source data to be loaded into target data warehouse tables using T-SQL and other Data Integration/ETL tools.
- Review & test the data to ensure accuracy & validity of the data prior to uploading the data to the warehouse.
- Data Troubleshooting and Analysis
- Perform data analysis, data mining and investigations and identify root cause of issues with SQL queries.
- Work with Technical Operations to troubleshoot complex database issues related to the entire environment including OS, storage and servers. Provide off hours support to resolve production issues when necessary.
- Bachelor s degree in relevant field such as Computer Science, Engineering, a related field, or equivalent experience.
- 2+ years experience with data aggregation, standardization, linking, quality check mechanisms, and reporting.
- 2+ years experience with RDBMS (Oracle, MS SQL Server) and using SQL or other data integration/ETL tools.
- Experience with big data technologies like Hadoop and Spark a plus.
- Experience within the Healthcare or Insurance space in a Data Warehouse operations organization a plus
- Experience with ticketing and documentation tools such as Jira.
- Strong knowledge of coding and ability to follow best practices while developing and deploying codes.
- Excellent written and verbal communication skills, with the ability to multitask and prioritize projects to meet scheduled deadlines.
- Ability to analyze data with a high level of detailed accuracy and identify root causes of issues using queries.
- Ability to work well independently or in a team environment.
- Proficient in Microsoft Office Suite, including PowerPoint, Word, Excel, Outlook and Project.
- BigData, Scala, Spark, HealthCare,
Associated topics: .net, backend, c+, develop, devops, java, perl, php, programming, software programmer