Hadoop Developer

Description

Responsibilitieshadoop development and implementationdesigning, building, installing, configuring and supporting Hadoopdevelop detailed system design documentationwrite highperformance, reliable and maintainable codecreate extract, transform and load processes and programs in appropriate environmentsloading from disparate data setscreate source to target maps and transformation rules for Data Mart creation, aggregates and summary tables collect appropriate metadata and document procedurescreate reconciliation procedures and data checks to ensure data qualitydesign, develop and apply best practices and document ETL processes including audit, balancing, and controlsreview schedules, objectives and priorities with service managerassist with routine work of the department as requiredprovide production support, including oncall rotation and technical assistance to enduser and IT staffsuggest improvements for systems and proceduresmentor less experienced associates Qualifications Requirements at least seven years of software development experienceat least five years of experience with JAVA, including Core Java programming, JS, shell scripting experience, and good knowledge of Java Design Patternsat least two years of experience with Hadoop preferably with HortonWorks Data Platform (HDFS, YARN, Oozie, and Hive)handson experience in extracting data from DB2, MS SQL Server using Sqoophandson experience in writing Hive Based ETL jobsknowledgeable with HDFS file system and commandsUNIX OS working skills and proficiency in Shell Scriptingexperience designing and developing exception handling, data standardization procedures and quality assurance controlsexperience implementing and refining required business transformation rules and logicexperience analyzing source systems for data profiling and creation of source to target mapping specsprogressive experience in conceptual, logical, physical data modeling; design, development and delivery of datacentric IT solutions, including relational databases, data warehouses, ETL/ELT processesPreferred Skills Altova XML Spyversion control using TFSscheduling Linux/Unix Shell scripts in Control MApache HueZeppelinSQL tools such as Squirrel, DBeaverdata modelinginsurance domain