Medallion Data Engineer

Skills

Azure Databricks Databricks Workflows Databricks Delta Live Tables Databricks Unity Catalog Delta Lake Azure Data Factory (ingestion & orchestration) ADLS Gen 2 Azure SQL Database Azure CosmosDB Neo4j Apache Spark PySpark Spark SQL SQL PL/SQL Oracle Database Technologies (19c) Kafka REST APIs Alation Terraform

Description

Familiarity with ETL/ELT in Databricks and with Medallion architecture.
Self-motivated team player who can work with minimal supervision and can adapt to a quickly changing environment.
Ability to work on multiple tasks simultaneously and handle conflicting priorities.
Strong knowledge of Software Development Life Cycle including requirement analysis, design, development, testing, and implementation.
Hands-on experience in performance tuning of sources, targets, mappings, and sessions.

Responsibilities

• Implement & perform unit testing, system integration testing, regression testing, and assist with user acceptance testing.
• Adhere to established standards and best practices and provide input for improvement of those processes.
• Develop and implement data pipelines for movement of data through the layers based on team & enterprise standards.