Snowflake Data Engineer

Northern Trust Corporation

Contract Chicago , Illinois, United States Posted 6 months ago

 Write a Review Add Vendor   Add Contact  

About Position

Snowflake Data Engineer (Contract)

$75.00 / Hourly

Chicago , Illinois, United States

Snowflake Data Engineer

Contract Chicago , Illinois, United States Posted 6 months ago

Skills
Data Pipeline Development: Design build and manage data pipelines for the ETL process using Airflow for orchestration and Python for scripting to transform raw data into a format suitable for our new Snowflake data model. Data Integration: Implement and maintain data synchronization between on-premises Oracle databases and Snowflake using CDC tools. Support Data Modeling: Assist in developing and optimizing the data model for Snowflake ensuring it supports our analytics and reporting requirements. Reporting Support: Collaborate with data architect to ensure the data within Snowflake is structured in a way that supports efficient and insightful reporting. Technical Documentation: Create and maintain comprehensive documentation of data pipelines ETL processes and data models to ensure best practices are followed and knowledge is shared within the team.
Description

This project aims to modernize our data architecture by leveraging cloud technologies, specifically Snowflake and Databricks, to enhance our data storage, processing, and analytics capabilities. It involves migrating application data from an on-premises Oracle database to a more scalable, flexible and secure cloud-based environment, optimizing data flows and analytics to support real-time decision-making and insights.

Responsibilities
  • Experience: Total 10+ years of experience as data engineer; proven experience as a Snowflake data engineer for 5+ years.
  • Skills: Strong knowledge of SQL writing complex queries, performance tuning, Strong experience with Oracle, Snowflake, ETL/ELT tools
  • Data engineering: proven track record of developing and maintaining data pipelines and data integration projects
  • Orchestration Tools: Experience in Airflow for managing data pipeline workflows.
  • Programming: proficiency in Python and SQL for data processing tasks.
  • Data Modeling: Understanding of data modeling principles and experience with data warehousing solutions.
  • Cloud Platforms: Knowledge of cloud infrastructure and services, preferably Azure, as it relates to Snowflake integration.
  • Collaboration Tools: Experience with version control systems (like Git) and collaboration platforms.
  • CI/CD Implementation: Utilize CI/CD tools to automate the deployment of data pipelines and infrastructure changes, ensuring high-quality data processing with minimal manual intervention.
  • Communication: Excellent communication and teamwork skills, with a detail-oriented mindset. Strong analytical skills, with the ability to work independently and solve complex problems.

By applying to a job using PingJob.com you are agreeing to comply with and be subject to the PingJob.com Terms and Conditions for use of our website. To use our website, you must agree with the Terms and Conditions and both meet and comply with their provisions.

Questions / Comments:

Display Questions / Comments:

No Questions / comments

Northern Trust Corporation Vendors

( Login to see all the 20 vendors)
TEK Systems
Write a Review

971 Corporate Boulevard
Linthicum
Maryland
www.teksystems.com ( 85 vendors)

Wipro Technologies
Write a Review

1300 Crittenden Lane
Mountain View
California
www.wipro.com ( 61 vendors)

Tata Consultancy Services
Write a Review

101 Park Avenue 26th Floor
New York
New York
www.usa-tcs.com ( 41 vendors)

Job Summary

$75.00 / Hourly

Contract

Chicago , Illinois, United States

Experience Level : Medium

Experience Required : 8 Year/s

Only Any

Posted : 6 months ago

Deadline : September 23, 2024 6 months ago

Job ID : Job0000002657

Northern Trust Corporation

50 South La Salle Street

312-630-6000

www.northerntrust.com

Share by E-Mail

Check all jobs of this employer

Flag this job

MessagesTimelineExceptionsViews3RouteQueries11Models140MailsGateSessionRequest
Request history8.1.32PHP Version2.3sRequest Duration695MBMemory UsageGET {slug}Route
    • Booting (2.02s)time
    • Application (277ms)time
    • 1 x Booting (87.97%)
      2.02s
      1 x Application (12.03%)
      277ms
      3 templates were rendered
      • frontend.job-viewjob-view.blade.php#?blade
      • admin.flash_msgflash_msg.blade.php#?blade
      • layouts.themetheme.blade.php#?blade
      uri
      GET {slug}
      middleware
      web
      controller
      App\Http\Controllers\JobController@view
      namespace
      prefix
      where
      as
      job_view
      file
      app/Http/Controllers/JobController.php:482-507
      11 statements were executed204ms
      • JobController.php#484clientfyi_final3.09msselect * from `jobs` where `job_slug` = 'snowflake-data-engineer-6' limit 1
        Bindings
        • 0: snowflake-data-engineer-6
        Backtrace
        • app/Http/Controllers/JobController.php:484
        • vendor/laravel/framework/src/Illuminate/Routing/Controller.php:54
        • vendor/laravel/framework/src/Illuminate/Routing/ControllerDispatcher.php:43
        • vendor/laravel/framework/src/Illuminate/Routing/Route.php:260
        • vendor/laravel/framework/src/Illuminate/Routing/Route.php:205
      • User.php#78clientfyi_final83.02msselect `id`, `company` from `users` where `user_type` in ('employer', 'agent') order by `id` desc
        Bindings
        • 0: employer
        • 1: agent
        Backtrace
        • app/Models/User.php:78
        • app/Http/Controllers/JobController.php:491
        • vendor/laravel/framework/src/Illuminate/Routing/Controller.php:54
        • vendor/laravel/framework/src/Illuminate/Routing/ControllerDispatcher.php:43
        • vendor/laravel/framework/src/Illuminate/Routing/Route.php:260
      • JobController.php#493clientfyi_final840μsselect * from `users` where `users`.`id` = 3986 limit 1
        Bindings
        • 0: 3986
        Backtrace
        • app/Http/Controllers/JobController.php:493
        • vendor/laravel/framework/src/Illuminate/Routing/Controller.php:54
        • vendor/laravel/framework/src/Illuminate/Routing/ControllerDispatcher.php:43
        • vendor/laravel/framework/src/Illuminate/Routing/Route.php:260
        • vendor/laravel/framework/src/Illuminate/Routing/Route.php:205
      • Vendor.php#57clientfyi_final760μsselect count(*) as aggregate from `vendors` inner join `users` on `vendors`.`vendor_id` = `users`.`id` inner join `states` on `users`.`state_id` = `states`.`id` inner join `countries` on `states`.`country_id` = `countries`.`id` where `vendors`.`approve_status` = 1 and `vendors`.`employer_id` = 3986
        Bindings
        • 0: 1
        • 1: 3986
        Backtrace
        • app/Models/Vendor.php:57
        • app/Http/Controllers/JobController.php:495
        • vendor/laravel/framework/src/Illuminate/Routing/Controller.php:54
        • vendor/laravel/framework/src/Illuminate/Routing/ControllerDispatcher.php:43
        • vendor/laravel/framework/src/Illuminate/Routing/Route.php:260
      • Vendor.php#57clientfyi_final1.83msselect `vendors`.*, `users`.`website`, `users`.`company`, `users`.`address`, `users`.`city`, `states`.`state_name`, `countries`.`country_name`, (SELECT (SUM(ratings)/COUNT(id)) FROM reviews WHERE reviews.employer_id = vendors.vendor_id AND reviews.approve_status = 1) as review_rating, (SELECT COUNT(id) FROM reviews WHERE reviews.employer_id = vendors.vendor_id AND reviews.approve_status = 1) as review_count, (SELECT COUNT(id) FROM vendors as b WHERE b.employer_id = vendors.vendor_id) as vendor_count from `vendors` inner join `users` on `vendors`.`vendor_id` = `users`.`id` inner join `states` on `users`.`state_id` = `states`.`id` inner join `countries` on `states`.`country_id` = `countries`.`id` where `vendors`.`approve_status` = 1 and `vendors`.`employer_id` = 3986 order by `vendor_count` desc limit 3 offset 0
        Bindings
        • 0: 1
        • 1: 3986
        Backtrace
        • app/Models/Vendor.php:57
        • app/Http/Controllers/JobController.php:495
        • vendor/laravel/framework/src/Illuminate/Routing/Controller.php:54
        • vendor/laravel/framework/src/Illuminate/Routing/ControllerDispatcher.php:43
        • vendor/laravel/framework/src/Illuminate/Routing/Route.php:260
      • JobController.php#500clientfyi_final370μsselect (sum(ratings)/count(id)) AS review_rating, count(id) as review_count from `reviews` where `approve_status` = 1 and `employer_id` = 3986
        Bindings
        • 0: 1
        • 1: 3986
        Backtrace
        • app/Http/Controllers/JobController.php:500
        • vendor/laravel/framework/src/Illuminate/Routing/Controller.php:54
        • vendor/laravel/framework/src/Illuminate/Routing/ControllerDispatcher.php:43
        • vendor/laravel/framework/src/Illuminate/Routing/Route.php:260
        • vendor/laravel/framework/src/Illuminate/Routing/Route.php:205
      • job-view.blade.php#17clientfyi_final8.51msselect DISTINCT email from `job_applications` where `category_id` = 187 and `status` = 1
        Bindings
        • 0: 187
        • 1: 1
        Backtrace
        • view::frontend.job-view:17
        • vendor/laravel/framework/src/Illuminate/Filesystem/Filesystem.php:124
        • vendor/laravel/framework/src/Illuminate/View/Engines/PhpEngine.php:58
        • vendor/laravel/framework/src/Illuminate/View/Engines/CompilerEngine.php:72
        • vendor/laravel/framework/src/Illuminate/View/View.php:207
      • job-view.blade.php#450clientfyi_final420μsselect * from `comments` where `comments`.`commentable_type` = 'App\\Models\\Job' and `comments`.`commentable_id` = 2657 and `comments`.`commentable_id` is not null and `parent_id` is null
        Bindings
        • 0: App\Models\Job
        • 1: 2657
        Backtrace
        • view::frontend.job-view:450
        • vendor/laravel/framework/src/Illuminate/Filesystem/Filesystem.php:124
        • vendor/laravel/framework/src/Illuminate/View/Engines/PhpEngine.php:58
        • vendor/laravel/framework/src/Illuminate/View/Engines/CompilerEngine.php:72
        • vendor/laravel/framework/src/Illuminate/View/View.php:207
      • theme.blade.php#101clientfyi_final960μsselect * from `categories` where `job_count` >= 1 order by `job_count` desc
        Bindings
        • 0: 1
        Backtrace
        • view::layouts.theme:101
        • vendor/laravel/framework/src/Illuminate/Filesystem/Filesystem.php:124
        • vendor/laravel/framework/src/Illuminate/View/Engines/PhpEngine.php:58
        • vendor/laravel/framework/src/Illuminate/View/Engines/CompilerEngine.php:72
        • vendor/laravel/framework/src/Illuminate/View/View.php:207
      • theme.blade.php#102clientfyi_final51.41msSELECT jobs.state_id,jobs.state_name, COUNT(*) AS numberOfSales FROM jobs JOIN states ON states.id = jobs.state_id GROUP BY jobs.state_id,jobs.state_name ORDER BY COUNT(*) DESC limit 15
        Backtrace
        • view::layouts.theme:102
        • vendor/laravel/framework/src/Illuminate/Filesystem/Filesystem.php:124
        • vendor/laravel/framework/src/Illuminate/View/Engines/PhpEngine.php:58
        • vendor/laravel/framework/src/Illuminate/View/Engines/CompilerEngine.php:72
        • vendor/laravel/framework/src/Illuminate/View/View.php:207
      • theme.blade.php#103clientfyi_final52.9msSELECT jobs.city_id,jobs.city_name, COUNT(*) AS numberOfcities FROM jobs JOIN cities ON cities.id = jobs.city_id GROUP BY jobs.city_id,jobs.city_name ORDER BY COUNT(*) DESC limit 15
        Backtrace
        • view::layouts.theme:103
        • vendor/laravel/framework/src/Illuminate/Filesystem/Filesystem.php:124
        • vendor/laravel/framework/src/Illuminate/View/Engines/PhpEngine.php:58
        • vendor/laravel/framework/src/Illuminate/View/Engines/CompilerEngine.php:72
        • vendor/laravel/framework/src/Illuminate/View/View.php:207
      App\Models\Category
      138Category.php#?
      App\Models\Job
      1Job.php#?
      App\Models\User
      1User.php#?
          _token
          0pyFVbCr3iNQtSsmkjyeG1qcJZSVbtzb5LeWdosL
          _previous
          array:1 [ "url" => "https://pingjob.com/snowflake-data-engineer-6" ]
          _flash
          array:2 [ "old" => [] "new" => [] ]
          path_info
          /snowflake-data-engineer-6
          status_code
          200
          status_text
          OK
          format
          html
          content_type
          text/html; charset=UTF-8
          request_query
          []
          request_request
          []
          request_headers
            0 of 0   
          array:25 [▼
  "sec-ch-ua-mobile" => array:1 [▶
    0 => "?0"
  ]
  "sec-ch-ua" => array:1 [▶
    0 => ""HeadlessChrome";v="129", "Not=A?Brand";v="8", "Chromium";v="129""
  ]
  "cache-control" => array:1 [▶
    0 => "no-cache"
  ]
  "pragma" => array:1 [▶
    0 => "no-cache"
  ]
  "cdn-loop" => array:1 [▶
    0 => "cloudflare; loops=1"
  ]
  "priority" => array:1 [▶
    0 => "u=0, i"
  ]
  "upgrade-insecure-requests" => array:1 [▶
    0 => "1"
  ]
  "user-agent" => array:1 [▶
    0 => "Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; ClaudeBot/1.0; +claudebot@anthropic.com)"
  ]
  "cf-connecting-ip" => array:1 [▶
    0 => "18.191.187.115"
  ]
  "accept" => array:1 [▶
    0 => "text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7"
  ]
  "sec-fetch-site" => array:1 [▶
    0 => "none"
  ]
  "cf-visitor" => array:1 [▶
    0 => "{"scheme":"https"}"
  ]
  "sec-fetch-mode" => array:1 [▶
    0 => "navigate"
  ]
  "sec-fetch-user" => array:1 [▶
    0 => "?1"
  ]
  "x-forwarded-proto" => array:1 [▶
    0 => "https"
  ]
  "cf-ipcountry" => array:1 [▶
    0 => "US"
  ]
  "accept-encoding" => array:1 [▶
    0 => "gzip, br"
  ]
  "sec-fetch-dest" => array:1 [▶
    0 => "document"
  ]
  "sec-ch-ua-platform" => array:1 [▶
    0 => ""Windows""
  ]
  "cf-ray" => array:1 [▶
    0 => "92ce4f3d3fcff161-ORD"
  ]
  "connection" => array:1 [▶
    0 => "close"
  ]
  "x-accel-internal" => array:1 [▶
    0 => "/internal-nginx-static-location"
  ]
  "x-forwarded-for" => array:1 [▶
    0 => "18.191.187.115"
  ]
  "x-real-ip" => array:1 [▶
    0 => "172.70.130.224"
  ]
  "host" => array:1 [▶
    0 => "pingjob.com"
  ]
]
          request_cookies
          []
          response_headers
            0 of 0   
          array:7 [▼
  "content-type" => array:1 [▶
    0 => "text/html; charset=UTF-8"
  ]
  "cache-control" => array:1 [▶
    0 => "private, must-revalidate"
  ]
  "date" => array:1 [▶
    0 => "Tue, 08 Apr 2025 02:24:40 GMT"
  ]
  "pragma" => array:1 [▶
    0 => "no-cache"
  ]
  "expires" => array:1 [▶
    0 => "-1"
  ]
  "set-cookie" => array:2 [▶
    0 => "XSRF-TOKEN=eyJpdiI6ImdoWm9qck1oampkdk5La3krQXpBbXc9PSIsInZhbHVlIjoiL1hiV2JyR3kvN3BXWWlncEFiU3hUbWZvRm1ZSGVMWWt6MjdyM2lPVkFvNXhtQytVS0M4cnpUNmt0OUwrQ0xnb0xFWUdURityQU9ROHJRcFVYdGMveXlhUFJtamRWU1l3WTJzRkl4TGFUdFZoWVVQY3BRM1ZFZlFrSGYzY2pIQTMiLCJtYWMiOiJjNmEwNTkwMzI1NDI1NzNhZjk0ZjA0OGY5NjcxMzc0MWRmN2RmYjQ5MDRiZGJmMjIyNGRkMjJhZGE5YzY5NGZhIiwidGFnIjoiIn0%3D; expires=Tue, 08 Apr 2025 04:24:40 GMT; Max-Age=7200; path=/; secure; samesite=lax ◀XSRF-TOKEN=eyJpdiI6ImdoWm9qck1oampkdk5La3krQXpBbXc9PSIsInZhbHVlIjoiL1hiV2JyR3kvN3BXWWlncEFiU3hUbWZvRm1ZSGVMWWt6MjdyM2lPVkFvNXhtQytVS0M4cnpUNmt0OUwrQ0xnb0xFWUdUR ▶"
    1 => "laravel_session=eyJpdiI6IjhVUWRSWUVGY2R2blUza2tJMVR3MWc9PSIsInZhbHVlIjoid2JRekRqT1FtM0pIeERJRXgyeHVWcEs4QVhvMFplOHFkUERwMW8vdFc3TXQ2bHdieEJPVzczRGQ4SVN1cXNWVklNcDFKdSt5QnpQZ2VqZE4rME1XU0t6S3dDakxhNmtuaWE5ajhyd2hsZ01zc1lOS3RrNnRnYkxoY0QxeVgzaGgiLCJtYWMiOiIyMjFjOTJhYmIyYWQ2YmY1YTM3MmIyMmE5N2IwMWE4ODg5YTU1ZWIwOWQyZDVhMTQ1MzZjN2RiMjkwNTk3YmZiIiwidGFnIjoiIn0%3D; expires=Tue, 08 Apr 2025 04:24:40 GMT; Max-Age=7200; path=/; httponly; samesite=lax ◀laravel_session=eyJpdiI6IjhVUWRSWUVGY2R2blUza2tJMVR3MWc9PSIsInZhbHVlIjoid2JRekRqT1FtM0pIeERJRXgyeHVWcEs4QVhvMFplOHFkUERwMW8vdFc3TXQ2bHdieEJPVzczRGQ4SVN1cXNWVklN ▶"
  ]
  "Set-Cookie" => array:2 [▶
    0 => "XSRF-TOKEN=eyJpdiI6ImdoWm9qck1oampkdk5La3krQXpBbXc9PSIsInZhbHVlIjoiL1hiV2JyR3kvN3BXWWlncEFiU3hUbWZvRm1ZSGVMWWt6MjdyM2lPVkFvNXhtQytVS0M4cnpUNmt0OUwrQ0xnb0xFWUdURityQU9ROHJRcFVYdGMveXlhUFJtamRWU1l3WTJzRkl4TGFUdFZoWVVQY3BRM1ZFZlFrSGYzY2pIQTMiLCJtYWMiOiJjNmEwNTkwMzI1NDI1NzNhZjk0ZjA0OGY5NjcxMzc0MWRmN2RmYjQ5MDRiZGJmMjIyNGRkMjJhZGE5YzY5NGZhIiwidGFnIjoiIn0%3D; expires=Tue, 08-Apr-2025 04:24:40 GMT; path=/; secure ◀XSRF-TOKEN=eyJpdiI6ImdoWm9qck1oampkdk5La3krQXpBbXc9PSIsInZhbHVlIjoiL1hiV2JyR3kvN3BXWWlncEFiU3hUbWZvRm1ZSGVMWWt6MjdyM2lPVkFvNXhtQytVS0M4cnpUNmt0OUwrQ0xnb0xFWUdUR ▶"
    1 => "laravel_session=eyJpdiI6IjhVUWRSWUVGY2R2blUza2tJMVR3MWc9PSIsInZhbHVlIjoid2JRekRqT1FtM0pIeERJRXgyeHVWcEs4QVhvMFplOHFkUERwMW8vdFc3TXQ2bHdieEJPVzczRGQ4SVN1cXNWVklNcDFKdSt5QnpQZ2VqZE4rME1XU0t6S3dDakxhNmtuaWE5ajhyd2hsZ01zc1lOS3RrNnRnYkxoY0QxeVgzaGgiLCJtYWMiOiIyMjFjOTJhYmIyYWQ2YmY1YTM3MmIyMmE5N2IwMWE4ODg5YTU1ZWIwOWQyZDVhMTQ1MzZjN2RiMjkwNTk3YmZiIiwidGFnIjoiIn0%3D; expires=Tue, 08-Apr-2025 04:24:40 GMT; path=/; httponly ◀laravel_session=eyJpdiI6IjhVUWRSWUVGY2R2blUza2tJMVR3MWc9PSIsInZhbHVlIjoid2JRekRqT1FtM0pIeERJRXgyeHVWcEs4QVhvMFplOHFkUERwMW8vdFc3TXQ2bHdieEJPVzczRGQ4SVN1cXNWVklN ▶"
  ]
]
          session_attributes
            0 of 0   
          array:3 [▼
  "_token" => "0pyFVbCr3iNQtSsmkjyeG1qcJZSVbtzb5LeWdosL"
  "_previous" => array:1 [▶
    "url" => "https://pingjob.com/snowflake-data-engineer-6"
  ]
  "_flash" => array:2 [▶
    "old" => []
    "new" => []
  ]
]
          ClearShow all
          Date ↕MethodURLData
          #12025-04-08 02:24:40GET/snowflake-data-engineer-6311140