Hadoop Administrator

Skills

1. Expert in setting up Hadoop cluster from scratch and server maintenance. 2. Commissioning and decommissioning of the nodes to/from Hadoop cluster. 3. Implementing managing and administering the overall Hadoop infrastructure. 4. Capacity planning and estimating the requirements for lowering or increasing the capacity of the Hadoop cluster. 5. Monitoring the Hadoop cluster connectivity and performance of the cluster for application teams. 6. Replicating huge amount data from one Hadoop cluster to another. 7. Applying encryption to secure the data in the Hadoop cluster. 8. Applying the configuration changes and testing the cluster in collaboration with the application teams and business users. 9. Sound knowledge of Hadoop services - HDFS Hive HBase Impala Hue Ranger etc. 10. Experience with Cloudera CDP Hadoop upgrade. 11. Experience with access management and setting up user roles in Hadoop. 12. Able to create and debug complex Unix shell scripts.

Description

Morgan Stanley is a leading global financial services firm providing a wide range of investment banking, securities, investment management and wealth management services. We advise, originate, trade, manage and distribute capital for governments, institutions and individuals. As a market leader, the talent and passion of our people is critical to our success. Together, we share a common set of values rooted in integrity, excellence and strong team ethic. We provide you a superior foundation for building a professional career where you can learn, achieve and grow.



Technology is the key differentiator that ensures that we manage our global businesses and serve clients on a market-leading platform that is resilient, safe, efficient, smart, fast and flexible. The Technology division partners with our business units and leading technology companies to redefine how we do business in ever more global and dynamic financial markets.

Responsibilities

• 1. Analyzing and troubleshooting the issues with the clusters.
• 2. Set up and maintain appropriate infrastructure to maintain performance and data integrity.
• 3. Experience with ELK Stack: Elasticsearch, Logstash, Kibana, Apache Hadoop
• 4. Ensuring appropriate monitoring & alerting of Elasticsearch cluster.
• 5. Expert in networking and security settings to support network related pipelines.
• 6. Good to have Ruby and/or Python, Unix and GIT Knowledge.