Databricks Lead

Skills

Lead and mentor a team of developers in designing and implementing scalable and performant data engineering solutions using the Databricks platform. Collaborate closely with clients to understand their data engineering requirements business objectives and technical challenges. Design and develop end-to-end data pipelines ETL processes and data integrations leveraging the capabilities of the Databricks platform. Provide technical expertise and guidance to the team in leveraging Databricks technologies such as Databricks Runtime Delta Lake and Apache Spark effectively. Conduct code reviews ensure adherence to best practices and maintain high coding standards across the team. Collaborate with cross-functional teams including data scientists data analysts and business stakeholders to ensure seamless integration of data engineering solutions. Identify performance bottlenecks and optimize data engineering processes on the Databricks platform. Stay updated with the latest advancements in the Databricks platform and share knowledge with the team. Actively participate in technical discussions contribute to architectural decisions and drive continuous improvement in data engineering practices.

Description

We are looking for a talented Data Engineering Solution Lead Developer with expertise in the Databricks platform. In this role, you will be responsible for leading and guiding a team of developers in designing, implementing, and delivering robust data engineering solutions using the Databricks platform. You will collaborate closely with clients, understanding their business requirements, and translating them into effective technical solutions. As a technical leader, you will play a pivotal role in driving successful data engineering projects and ensuring the highest level of quality and efficiency in our deliverables.

Responsibilities

• Bachelor's or Master's degree in Computer Science, Information Technology, or a related field.
• Proven experience as a Data Engineer or Lead Developer with a strong focus on data engineering using the Databricks platform.
• In-depth understanding and hands-on experience with the Databricks platform, including Databricks Runtime, Delta Lake, and Apache Spark.
• Proficiency in programming languages such as Python, Scala, or SQL for data manipulation, transformation, and integration.
• Solid understanding of data modelling, data warehousing, and ETL principles.
• Strong leadership skills with the ability to lead, mentor, and inspire a team of developers.
• Excellent problem-solving and analytical skills, with the ability to identify and resolve complex technical issues.
• Effective communication skills, both verbal and written, with the ability to articulate technical concepts to both technical and non-technical stakeholders.
• Experience in cloud platforms such as AWS, Azure, or GCP is highly desirable.
• Relevant certifications in Databricks or related technologies would be a plus.