Big Data Engineer with Spark

Description

Responsible for development of highly scaling Spark applications for Log Analysis. Involves use of Impala, Sentry, Kafka etc. Academic background (any one from the list given below) BE / B Tech/ M Tech in Computer Science or IT MCA with strong focus on mathematics / statistics Industry Experience 4 to 8 years as a Big Data Developer. Experience as a lead in developing Spark applications and dealing with multiple stakeholders. 2 to 3 years as a Hadoop Developer for application development in Hive/Impala, Sqoop, Spark or any realtime applications. Understanding of Hadoop stack and Map Reduce concepts. Any other experience of Data warehouse / ETL Technologies or tools will be an added advantage. 2+ years of experience in working as a team lead which involves team work. Tools / Skillset Expertise Expertise in Spark Expertise in Hadoop Architecture, design patterns, platform management tasks etc. Expertise in MySQL, Hive, Impala, Sentry, Spark ecosystems. Expertise in Database utilities, Unix shell /Perl scripting, Python/Scala programming. Expertise in optimization, performance tuning for batch jobs. Job Description Work with Teach Teams for developing scalable Spark applications Lead the delivery and communication for the project Contribute or lead data / process architecture and design patterns of an application. Collaborate with various groups with the organization to leverage best practices for the project. Attributes Willingness to learn multiple technologies, domain concepts and apply them for project work Flexible to work across any technology stack and industry domain. Excellent communication, documentation, coaching and interpersonal skills. Ability to work as an integral part of a high performance team. Demonstrate leadership qualities to own programs project and motivate teams. Willing to work on weekends for support/maintenance activities during upgrades or system restarts.