Big Data Administrator

Description

Responsibilities As a Big Data Engineer, your responsibilities may include, but are not limited to, the following ● Build a scalable Big Data Platform designed to serve many different usecases and requirements● Build a highly scalable framework for ingesting, transforming and enhancing data at web scale● Develop data structures and processes using components of the Hadoop ecosystem such as Avro, Hive, Parquet, Impala, Hbase, Kudu, Tez, etc.● Establish automated build and deployment pipelines● Implement machine learning models that enable customers to glean hidden insights about their data● Implementing security and integrating with components such as LDAP, AD, Sentry, Kerberos.● Strong understanding of row level and role based security concepts such as inheritance● Establishing scalability benchmarks for predictable scalability thresholds.Qualifications ● Bachelor's degree in Computer Science or related field● 6+ years of system building experience● 4+ years of programming experience using JVM based languages● A passion for DevOps and an appreciation for continuous integration/deployment● A passion for QA and an understanding that testing is not someone elses responsibility● Experience automating infrastructure and build processes● Outstanding programming and problem solving skills● Strong passion for technology and building great systems● Excellent communication skills and ability to work using Agile methodologies● Ability to work quickly and collaboratively in a fastpaced, entrepreneurial environment● Experience with serviceoriented (SOA) and eventdriven (EDA) architectures● Experience using big data solutions in an AWS environment● Experience with noSQL data stores Cassandra, HDFS and/or Elasticsearch● Experience with javascript or associated frameworksPreferred skills We value these qualities, but theyre not required for this role ● Masters or Ph.D. in related field● Experience as an open source contributor● Experience with Akka, stream processing technologies and concurrency frameworks● Experience with Data modeling● Experience with Chef, Puppet, Ansible, Salt or equivalent● Experience with Docker, Mesos and Marathon● Experience with distributed messaging services, preferably Kafka● Experience with distributed data processors, preferably Spark● Experience with Angular, React, Redux, Immutable.js, Rx.js, Node.js or equivalent● Experience with Reactive and/or Functional programming● Understanding of Thrift, Avro or protocol buffers