Java, Python, SPARK Developer

Description

Strong experience in Java, Python, SPARK and KAFKA experience. Strong experience in developing Spring Boot microservices on Pivotal platform (using Pivotal Cloud Foundry, Pivotal Application Services). Experience with mediumtolarge data pipelines implementing, testing and deploying. Experience with stream processing using Kafka. Experience with automated unit and integration testing for Kafka and Py Spark. Experience with Kafka Connect. Execute strategic engineering proof of concepts and product evaluations. Develop monitoring strategies for infrastructure, platforms and applications aligning with enterprise strategy and overall industry trends Spark
HDFS, Spark, Hbase, Hive, Sqoop, Kafka, Storm, YARN, HiveQL, SparkSQL. Python experience is a plus. Experience in developing Orchestration / Data Pipelines in Py Spark. Strong Knowledge in developing PySpark connectors (to RDBMS, Mainframe applications, Web service, MQ, Kafka). Worked in large data processing, realtime streaming and scalable architectures. Kafka
Experience in developing large Kafka clusters Executing partitioning strategy Implementing security & authorization (permission based) on Kafka cluster Implementing ingestion services (to connect to RDBMS, Mainframe, Webservice, etc.) using Kafka producer APIs / Kafka Connect Spring Boot
Experience in developing Spring Boot Microservices Working experience on Pivotal platform (PCF and PAS) Working experience on developing pipelines using Spring Cloud Data Flow