Big Data Engineer + SCALA

Description

Build scalable data pipelines on top of Hive and Spark using the Airflow scheduler/executor framework
Collaborate with Payments Engineering, Data Science, FP&A, Accounting, and Product Management to produce accurate and timely financial data
Apply Machine Learning models to forecast and visualize key revenue metrics and trends
Anomaly detection and analysis for payments integrations, processing, and processor reconciliation
Build the infrastructure required for optimal extraction, transformation, and loading of data from a wide variety of data sources using SQL and AWS big data technologies.
Craft data tools for analytics and data scientist team members that assist them in building and optimizing our product into an innovative industry leader.
Work with data and analytics experts to strive for greater functionality in our data systems.
The following experience is meaningful to us Proficient understanding of distributed computing principles
Experience with building streamprocessing systems, using solutions such as Storm or SparkStreaming
ETL or Pipeline design/implementation with Large Distributed Databases (Hive/Spark/Scala preferred)
Experience in data validation, sampling & regression tests
Pipeline stability and performance tuning
Working knowledge of relational databases and query authoring (SQL)
Experience with Scala is Must
Love to use and develop open source technologies like Kafka, Hadoop, Hive, Presto, and Spark
Rigor in high code quality, automated testing, and other engineering standard methodologies
Payment data processing or system design knowledge is a plus (not required)
Experience with NoSQL databases, such as HBase or Cassandra is a plus (not required)
Outstanding interpersonal skills
BS/MS/PhD in Computer Science or a related field (ideal)