Log Data Engineer
T Rowe Price Group Inc
Contract Owings Mills, Maryland, United States Posted 6 months ago
About Position
Log Data Engineer (Contract)
$70.00 / Hourly
Owings Mills, Maryland, United States
Log Data Engineer
Contract Owings Mills, Maryland, United States Posted 6 months ago
Skills
• Ability to troubleshoot and diagnose complex issues • Able to demonstrate experience supporting technical users and conduct requirements analysis • Can work independently with minimal guidance & oversight • Experience with IT Service Management and familiarity with Incident & Problem management • Highly skilled in identifying performance bottlenecks identifying anomalous system behavior and resolving root cause of service issues. • Demonstrated ability to effectively work across teams and functions to influence design operations and deployment of highly available software • Knowledge of standard methodologies related to security performance and disaster recoveryDescription
• Develop data processing pipelines using programming languages like Java and Python to extract, transform, and load log data
• Implement scalable, fault-tolerant solutions for data ingestion, processing, and storage.
• Support systems engineering lifecycle activities for data engineering deployments, including requirements gathering, design, testing, implementation, operations, and documentation.
• Automating platform management processes through Ansible or other scripting tools/languages
• Troubleshooting incidents impacting the log data platforms
• Collaborate with cross-functional teams to understand data requirements and design scalable solutions that meet business needs.
• Develop training and documentation materials
• Support log data platform upgrades including coordinating testing of upgrades with users of the platform
• Gather and process raw data from multiple disparate sources (including writing scripts, calling APIs, writing SQL queries, etc.) into a form suitable for analysis
• Enables log data, batch and real-time analytical processing solutions leveraging emerging technologies
• Participate in on-call rotations to address critical issues and ensure the reliability of data engineering systems
Responsibilities
- • Expertise in language such as Java, Python. Implementation knowledge in data processing pipelines using programming languages like Java and Python to extract, transform, and load (ETL) data
- • Create and maintain data models, ensuring efficient storage, retrieval, and analysis of large datasets
- • Troubleshoot and resolve issues related to data processing, storage, and retrieval.
- • 3-5 years’ Experience in designing, developing, and deploying data lakes using AWS native services (S3, Glue (Crawlers, ETL, Catalog), IAM, Terraform, Athena)
- • Experience in development of systems for data extraction, ingestion and processing of large volumes of data
- • Experience with data pipeline orchestration platforms
- • Experience in Ansible/Terraform/Cloud Formation scripts and Infrastructure as Code scripting is required
- • Implement version control and CI/CD practices for data engineering workflows to ensure reliable and efficient deployments
- • Proficiency in implementing monitoring, logging, and alerting solutions for data infrastructure (e.g., Prometheus, Grafana)
- • Proficiency in distributed Linux environments
Educational Requirements
- • Familiarity with data streaming technologies such as Kafka, Kinesis, spark streaming, etc
- • Knowledge of cloud platforms (prefer AWS) and container + orchestration technologies
- • Experience with AWS OpenSearch, Splunk
- • Experience with common scripting and query languages
By applying to a job using PingJob.com you are agreeing to comply with and be subject to the PingJob.com Terms and Conditions for use of our website. To use our website, you must agree with the Terms and Conditions and both meet and comply with their provisions.
