Data Engineer

Skills

Description

· Data modelling. Dimension Modelling in particular.
SQL/Python experience is a must to have.

· Spark is good to have.

· Should have done requirements gathering, created data models meeting the requirement , developed /tested data pipelines.

· Should have done data analysis/profiling. Needs minimal oversight Should be able to lead a team and be responsible for his/her and the team’s deliverables.

· Design, develop, implement, and maintain code, information architecture, and conceptual models to support data processing, and flows thru data lake

· Work with clients to get the requirements and build codes accordingly

· Translate client business requirements into data requirements

· Communicate the data requirements to other members of the team

· Conceptualizing and generating infrastructure that allows big data to be accessed and analyzed.

· Reformulating existing frameworks to optimize their functioning.

· Testing such structures to ensure that they are fit for use. Preparing raw data for manipulation by data scientists or for visualization teams.

· Proven experience as a data engineer, software developer, or similar.

· AWS: S3 Data Lake, Athena, Redshift, EMR, Glue, ECS, Data Bricks Proficiency with Python in a data engineering context.

· Ex: Pandas, PySpark Experience using VCS like GitHub, Gitlab Excellent analytical and problem-solving skills. A knack for independence and group work.