Python Spark Developer
State of New Mexico
Contract Santa Fe, New Mexico, United States Posted 10 months ago
About Position
Python Spark Developer (Contract)
$55.00 / Hourly
Santa Fe, New Mexico, United States
Python Spark Developer
Contract Santa Fe, New Mexico, United States Posted 10 months ago
Skills
Design implement and maintain scalable data pipeline solutions using Spark Python AWS Glue and Snowflake Develop and deploy high-performance ETL processes using Spark and Python to ingest data from various sources and load it into Snowflake Write efficient maintainable and scalable data transformation code using Spark Python and SQL Collaborate with team members to develop test and maintain data pipeline solutions Optimize data processing performance through tuning caching partitioning and indexing techniques Monitor and troubleshoot issues related to data pipelines ensuring data quality and integrity Develop and maintain documentation on data pipeline architecture processes data flows and data dictionaryDescription
Solid experience with Spark is required
Strong expertise in Python is required
Expertise in AWS cloud platform is required
Knowledge of data pipeline concepts and best practices
Experience with AWS Glue and Snowflake is highly desirable
Proficient in SQL and database concepts, with the ability to optimize queries and data processing performance
Strong problem-solving and analytical skills
Excellent communication and collaboration skills, with the ability to work effectively in a team environment
Responsibilities
- Experience with other cloud platforms, such as Azure or Google Cloud
- Knowledge of Java/Scala is a plus
By applying to a job using PingJob.com you are agreeing to comply with and be subject to the PingJob.com Terms and Conditions for use of our website. To use our website, you must agree with the Terms and Conditions and both meet and comply with their provisions.
