Data Engineer

Description

Job Description Senior Data Engineer As a Senior Data Engineer on the Analytics Systems team, youll work with a talented team of engineers to improve Credit Karmas data pipeline that powers our recommendation systems, enterprise tools, and data warehousing. Youll help to build a general, secure, scalable, fast, and high throughput data pipeline to process many terabytes of data a day. We are very passionate about performance, correctness, and data quality. The team spends their time day to day developing new pipelines or features that make it seamless for data to be moved throughout our infrastructure.This includes working with cutting edge tools such as Scala, Kafka, Spark, Akka and Google Cloud. We are looking for data engineers at all levels. If you enjoy working in a collaborative environment where everyone can have their say while still being able to set and hit deadlines, then this is the role for you. Responsibilities Work with analysts and data scientists to define processes and standards to inform system design, transform their needs into streaming or batch processingDesign infrastructure and systems to scale easily as data ingest growsImplement new data pipeline features with verified high quality from unit test coverage and production monitoringFocus on data quality! Detect data/analytics quality issues and implement bug fixes and data validation for preventionHelp understand our day to day operations for continuous improvement of production systems Our Ideal Candidate 5+ years experience with Big Data technologiesExperience with Scala/Java, Spark, Kafka, or demonstrated ability to pick up new technology quicklyFundamental knowledge about databases and strong SQL skillsFamiliar with Google Cloud ecosystem such as BigQuery, GCS, DataProc, etcEnjoys working collaboratively; CKs values include empathy and helpfulnessAble to estimate and meet deadlinesExcellent verbal and written communication skills Nice to have Experience working with cloud technologiesExperience scaling data throughput or building low latency streaming pipelinesExperience solving for data quality