Big Data Engineer

Skills

Description

Client is seeking a Senior Big Data Engineer to become part of the team implementing and supporting edge analytical solutions on a Google Cloud Ecosystem. You will find a great place to work if you are passionate about designing data ingestion jobs, learning new technologies, and proposing and adopting new technologies.

Responsibilities

• Extract, Transform and Load data from multiple sources and multiple formats using Big Data Technologies.
• Development, enhancement, and support of data ingestion jobs from various source systems following existing design patterns using GCP Services such as Apache Spark, Dataproc, Dataflow, BigQuery, Airflow, etc.
• Work across Teams and senior engineers to make Data more accessible to others within the organization.
• Modify data extraction pipelines into standardized approaches that can be repeatable and reusable with minimal supervision from senior engineers.
• Automation of manual processes, optimize data delivery, re-designing infrastructure for greater scalability, etc
• Work closely with senior engineers to optimize query and data access techniques.
• Apply modern software development practices (serverless computing, microservices architecture, CI/CD, infrastructure-as-code, etc.)
• Participate in a tight-knit engineering team employing agile software development practices