Big Data Administrator

Description

Responsibilities
Expertise in Elastic Cloud Enterprise
Working with enterprise architects to advise on best practices for Kafka & Elasticsearch both on prem and in the cloud.
Design and implement DR solutions for Confluent and Apache Kafka and Elasticsearch.
Modernizing ETL pipelines through the use of Kafka and Elastic.
Responsible for Kafka tuning, capacity planning, disaster recovery, replication, and troubleshooting.
Implementing Kafka security, limiting bandwidth usage, enforcing client quotas, backup and restoration.
Indepth understanding of the internals of Kafka cluster management, Zookeeper, partitioning, schema registry, topic replication and mirroring
Architect, configure, deploy and maintain Elasticsearch clusters.
Configure Logstash and Beats to collect data necessary to meet client requirements.
Configure XPack plugins to include Security, Watcher, Machine Learning, Monitoring, Graph, and Reporting.
Design, implement, and configure Kibana visualizations and dashboards.
Maintaining Kafka connectors to move data between systems.
Experience training, mentoring and leading an emerging team is beneficial
Knowledge of Cloud Formation/Terraform scripts.
Work closely with Big Data and AWS cloud technology groups.
Experience with open source Kafka distributions as well as enterprise Kafka products preferred.
Familiarity with both cloud native Kafka (on AWS) and onpremise architectures.
Managing Kafka & Elasticsearch clusters & creating tools to automate and improve clusters' reliability and performance.