GCP Data Engineer

Description

· Must have 12 years plus of IT experience.

· Must have 4+ years of ‘Recent’ experience (With in last 6 years) working for a Major Bank or Brokerage house in the US working with GCP.

· Must have 8 plus years of experience as a Data Engineer designing, developing, and deploying data pipelines.

· Must have 5 plus years of experience with Google Cloud Platform (GCP).

· Must have 5 plus years of experience with GCP services Dataflow, Dataproc, and Pub/Sub,

· Must have 5 plus years of experience with core data services GCS, Big Query, Cloud Storage, and Dataflow.

· Must have 5 plus years’ experience managing and optimizing data storage Big Query, Cloud Storage, and Cloud SQL.

· Must have 5 plus years implementing data security and access controls using GCP's Identity and Access Management (IAM) and Cloud Security Command Center.

· Must have 5 plus years of experience with Python & SQL for data manipulation and querying

· Must have 5 plus years of experience with the distributed data processing framework Apache Beam and Apache Spark.

· Experience with data security and access control principles

Responsibilities

• ·  BGC wants Senior Google Cloud Platform (GCP) Data Engineer to play a critical role in designing, developing, and maintaining robust data pipelines that transform raw data into valuable insights for our organization.
• ·  Design, develop, and maintain data pipelines using GCP services like Dataflow, Dataproc, and Pub/Sub, in addition to developing and implementing data ingestion and transformation processes using tools like Apache Beam and Apache Spark.
• ·  Manage and optimize data storage solutions on GCP, including Big Query, Cloud Storage, and Cloud SQL, while implementing data security and access controls using GCP's Identity and Access Management (IAM) and Cloud Security Command Center.
• ·  Monitor and troubleshoot data pipelines and storage solutions using GCP's Stack driver and Cloud Monitoring tools.
• ·  Collaborate with data experts, analysts, and product teams to understand data needs and deliver effective solutions.
• ·  Automate data processing tasks using scripting languages like Python.