Hadoop Administrator

Description

* Hadoop Administrator II will install, configure, and manage Hadoop clusters, including HDFS, YARN, MapReduce, HBase, Hive, Pig, Sqoop, and Flume
* Monitor cluster performance, troubleshoot issues, and optimize cluster performance to ensure high availability and reliability
* Implement and manage Hadoop security policies and practices, including Kerberos authentication, encryption, and access controls
* Perform regular Hadoop cluster maintenance activities, such as upgrades, patches, backups, and disaster recovery planning
* Collaborate with data engineers, developers, and other stakeholders to support data ingestion, transformation, and analytics workflows
* Develop and maintain scripts for automation of administrative tasks and system monitoring
* As a Hadoop Administrator II, you will monitor and manage Hadoop cluster resource usage and capacity planning to ensure efficient utilization
* Provide support and troubleshooting for Hadoop-related issues, ensuring timely resolution and minimal disruption to business operations
* Document Hadoop cluster configurations, procedures, and best practices for maintenance and troubleshooting
* Stay up to date with the latest Hadoop ecosystem developments, tools, and best practices, and recommend improvements to the existing infrastructure

Responsibilities

• * Bachelor's degree in Computer Science, Information Technology, or a related field
• * Minimum of 3 years of experience in Hadoop administration
• * Proficiency in managing Hadoop clusters using tools like Ambari, Cloudera Manager, or Hortonworks Data Platform
• * Strong understanding of Hadoop ecosystem components (HDFS, YARN, MapReduce, HBase, Hive, Pig, Sqoop, and Flume)
• * Experience with Linux/Unix operating systems and scripting languages (e.g., Bash, Python)
• * Knowledge of Hadoop security practices, including Kerberos authentication and data encryption
• * Strong problem-solving and troubleshooting skills with the ability to perform root cause analysis
• * Excellent communication and collaboration skills, with the ability to work effectively in a team environment
• * Experience with cloud-based Hadoop services (e.g., AWS EMR, Google Cloud Dataproc, Azure HDInsight) is a plus