Data Engineer
Project44
Contract Chicago , Illinois, United States Posted 3 years ago
About Position
Data Engineer (Contract)
$90.00 / Hourly
Chicago , Illinois, United States
Data Engineer
Contract Chicago , Illinois, United States Posted 3 years ago
Skills
5+ years experience building data pipelines using Airflow Spark or Kafka Streams. Excellent problem solving skills. Good proficiency in Python Java or other similar languages. Familiarity with architecture of event collection pipelines and analytical data stores such as Snowflake. Bachelor's Degree in computer science or equivalent experience. Technical Skills Strong programming/scripting knowledge in building and maintaining ETL using Java SQL Python Bash Go. In-depth hands-on knowledge of public clouds - GCP(preferred)/AWS PostgreSQL (version 9.6+) ElasticSearch MongoDB MySQL/MariaDB Snowflake Participate in an on-call rotation to mitigate any data pipeline failures Strong experience with Kafka or equivalent event/streaming based systems Experience with Docker Kubernetes. Develop and deploy CICD pipelines for Data Engineering Experience and knowledge of optimizing database performance and capacity utilization to provide high availability and redundancy Proficiency with high volume OLTP Databases and large data warehouse environments. Ability to work in a fast-paced rapidly changing environment Understanding of Agile and its implementation for Data Warehouse DevelopmentDescription
Design, build and improve our ETL platform and pipelines utilizing Airflow/Argo Workflow and integrates with Data lake (Snowflake) and Postgres/Kafka.
Ensure standards for engineering excellence, scalability, reliability, and reusability
Debug production issues across services and multiple levels of the stack.
Partner with the insights and data science teams to automate processes to improve data sets for analytical and reporting needs.
Write test cases, code coverage, perform QA and participate with stakeholders on UAT
By applying to a job using PingJob.com you are agreeing to comply with and be subject to the PingJob.com Terms and Conditions for use of our website. To use our website, you must agree with the Terms and Conditions and both meet and comply with their provisions.
