Data Engineer

Crunchbase

Contract San Francisco, California, United States Posted 5 years ago

 Write a Review Add Vendor   Add Contact  

About Position

Data Engineer (Contract)

$90.00 / Hourly

San Francisco, California, United States

Data Engineer

Contract San Francisco, California, United States Posted 5 years ago

Description

We have a flexible model for team organization at Crunchbase, centered around the principles of agile software development, allowing us to fluidly adapt to new requirements and situations as they arise. We tend towards two main types of teams Featureoriented teams operate independently and own their projects from cradle to grave, building everything they need to execute. While they don't necessarily have to do all the work themselves, it's their responsibility to make sure the business needs are met all the way into production. Functionoriented teams provide support to feature teams and have deeper expertise on particular parts of the stack. It's their responsibility to build platforms that are so good the other teams want to use them, rather than being forced into it.We also ship code as early as we possibly can; getting betas into customer hands is a top priority. Our goal is to tune our process to customer needs as efficiently and quickly as possible. Crunchbase is built with a variety of tools a core part of our engineering philosophy is to use the right tool for the job Ruby, Scala, Java Node.js, Angular.js, Typescript Redis, PostgreSQL, ElasticSearch Git, Nginx, Docker, NewRelic, Jenkins AWS EC2, ELB, VPC, S3, R53, RDS, SQS, etc.Data Engineering at CrunchbaseOur data team is responsible for building and maintaining the infrastructure for our data needs. As Crunchbase grows, so too does the amount of data we process, the number of sources it comes from, and the number of ways that people want to slice and dice it. We are currently building out a robust pipeline for our core data and were continually expanding use cases for it, so it must be both scalable and flexible.At its core, Crunchbase is a data company, and data engineering is at the heart of our platform and will propel us into the future. The responsibilities of data engineers at Crunchbase include Architect and build new dimensional data models and schema designs to improve accessibility, efficiency, consistency and quality of both internal and production data. Build, monitor, and maintain analytics and production data ETL pipelines. Provide the foundation for a datadriven culture by empowering other engineers and the Product team to ask questions of the dataset in an easy, reliable way. Enable data scientists to implement NLP and ML algorithms at scale, in faulttolerant, highly available systems.Qualifications Solid understanding of computer science and software engineering fundamentals. Motivated to participate in ongoing learning and growth through pair programming, code reviews, application of new technologies and best practices. Excellent verbal and written communication skills. Familiarity with tools we presently use is a plus, but not required if you know something better, we may use that instead!

By applying to a job using PingJob.com you are agreeing to comply with and be subject to the PingJob.com Terms and Conditions for use of our website. To use our website, you must agree with the Terms and Conditions and both meet and comply with their provisions.

Questions / Comments:

Display Questions / Comments:

No Questions / comments

Job Summary

$90.00 / Hourly

Contract

San Francisco, California, United States

Experience Required : 5 Year/s

Posted : 5 years ago

Deadline : January 20, 2020 5 years ago

Job ID : Job0000017535

Crunchbase

564 Market Street Suite 700

www.Crunchbase.com

Request history8.1.32PHP Version2.99sRequest Duration695MBMemory UsageGET {slug}Route
    • Booting (2.53s)time
    • Application (462ms)time
    • 1 x Booting (84.57%)
      2.53s
      1 x Application (15.43%)
      462ms
      3 templates were rendered
      • frontend.job-viewjob-view.blade.php#?blade
      • admin.flash_msgflash_msg.blade.php#?blade
      • layouts.themetheme.blade.php#?blade
      uri
      GET {slug}
      middleware
      web
      controller
      App\Http\Controllers\JobController@view
      namespace
      prefix
      where
      as
      job_view
      file
      app/Http/Controllers/JobController.php:482-507
      10 statements were executed366ms
      • JobController.php#484clientfyi_final18.86msselect * from `jobs` where `job_slug` = 'Job0000017535' limit 1
        Bindings
        • 0: Job0000017535
        Backtrace
        • app/Http/Controllers/JobController.php:484
        • vendor/laravel/framework/src/Illuminate/Routing/Controller.php:54
        • vendor/laravel/framework/src/Illuminate/Routing/ControllerDispatcher.php:43
        • vendor/laravel/framework/src/Illuminate/Routing/Route.php:260
        • vendor/laravel/framework/src/Illuminate/Routing/Route.php:205
      • User.php#78clientfyi_final129msselect `id`, `company` from `users` where `user_type` in ('employer', 'agent') order by `id` desc
        Bindings
        • 0: employer
        • 1: agent
        Backtrace
        • app/Models/User.php:78
        • app/Http/Controllers/JobController.php:491
        • vendor/laravel/framework/src/Illuminate/Routing/Controller.php:54
        • vendor/laravel/framework/src/Illuminate/Routing/ControllerDispatcher.php:43
        • vendor/laravel/framework/src/Illuminate/Routing/Route.php:260
      • JobController.php#493clientfyi_final8.69msselect * from `users` where `users`.`id` = 52018 limit 1
        Bindings
        • 0: 52018
        Backtrace
        • app/Http/Controllers/JobController.php:493
        • vendor/laravel/framework/src/Illuminate/Routing/Controller.php:54
        • vendor/laravel/framework/src/Illuminate/Routing/ControllerDispatcher.php:43
        • vendor/laravel/framework/src/Illuminate/Routing/Route.php:260
        • vendor/laravel/framework/src/Illuminate/Routing/Route.php:205
      • Vendor.php#57clientfyi_final7.13msselect count(*) as aggregate from `vendors` inner join `users` on `vendors`.`vendor_id` = `users`.`id` inner join `states` on `users`.`state_id` = `states`.`id` inner join `countries` on `states`.`country_id` = `countries`.`id` where `vendors`.`approve_status` = 1 and `vendors`.`employer_id` = 52018
        Bindings
        • 0: 1
        • 1: 52018
        Backtrace
        • app/Models/Vendor.php:57
        • app/Http/Controllers/JobController.php:495
        • vendor/laravel/framework/src/Illuminate/Routing/Controller.php:54
        • vendor/laravel/framework/src/Illuminate/Routing/ControllerDispatcher.php:43
        • vendor/laravel/framework/src/Illuminate/Routing/Route.php:260
      • JobController.php#500clientfyi_final7.42msselect (sum(ratings)/count(id)) AS review_rating, count(id) as review_count from `reviews` where `approve_status` = 1 and `employer_id` = 52018
        Bindings
        • 0: 1
        • 1: 52018
        Backtrace
        • app/Http/Controllers/JobController.php:500
        • vendor/laravel/framework/src/Illuminate/Routing/Controller.php:54
        • vendor/laravel/framework/src/Illuminate/Routing/ControllerDispatcher.php:43
        • vendor/laravel/framework/src/Illuminate/Routing/Route.php:260
        • vendor/laravel/framework/src/Illuminate/Routing/Route.php:205
      • job-view.blade.php#17clientfyi_final14.95msselect DISTINCT email from `job_applications` where `category_id` = 211 and `status` = 1
        Bindings
        • 0: 211
        • 1: 1
        Backtrace
        • view::frontend.job-view:17
        • vendor/laravel/framework/src/Illuminate/Filesystem/Filesystem.php:124
        • vendor/laravel/framework/src/Illuminate/View/Engines/PhpEngine.php:58
        • vendor/laravel/framework/src/Illuminate/View/Engines/CompilerEngine.php:72
        • vendor/laravel/framework/src/Illuminate/View/View.php:207
      • job-view.blade.php#450clientfyi_final500μsselect * from `comments` where `comments`.`commentable_type` = 'App\\Models\\Job' and `comments`.`commentable_id` = 17535 and `comments`.`commentable_id` is not null and `parent_id` is null
        Bindings
        • 0: App\Models\Job
        • 1: 17535
        Backtrace
        • view::frontend.job-view:450
        • vendor/laravel/framework/src/Illuminate/Filesystem/Filesystem.php:124
        • vendor/laravel/framework/src/Illuminate/View/Engines/PhpEngine.php:58
        • vendor/laravel/framework/src/Illuminate/View/Engines/CompilerEngine.php:72
        • vendor/laravel/framework/src/Illuminate/View/View.php:207
      • theme.blade.php#101clientfyi_final840μsselect * from `categories` where `job_count` >= 1 order by `job_count` desc
        Bindings
        • 0: 1
        Backtrace
        • view::layouts.theme:101
        • vendor/laravel/framework/src/Illuminate/Filesystem/Filesystem.php:124
        • vendor/laravel/framework/src/Illuminate/View/Engines/PhpEngine.php:58
        • vendor/laravel/framework/src/Illuminate/View/Engines/CompilerEngine.php:72
        • vendor/laravel/framework/src/Illuminate/View/View.php:207
      • theme.blade.php#102clientfyi_final63.83msSELECT jobs.state_id,jobs.state_name, COUNT(*) AS numberOfSales FROM jobs JOIN states ON states.id = jobs.state_id GROUP BY jobs.state_id,jobs.state_name ORDER BY COUNT(*) DESC limit 15
        Backtrace
        • view::layouts.theme:102
        • vendor/laravel/framework/src/Illuminate/Filesystem/Filesystem.php:124
        • vendor/laravel/framework/src/Illuminate/View/Engines/PhpEngine.php:58
        • vendor/laravel/framework/src/Illuminate/View/Engines/CompilerEngine.php:72
        • vendor/laravel/framework/src/Illuminate/View/View.php:207
      • theme.blade.php#103clientfyi_final114msSELECT jobs.city_id,jobs.city_name, COUNT(*) AS numberOfcities FROM jobs JOIN cities ON cities.id = jobs.city_id GROUP BY jobs.city_id,jobs.city_name ORDER BY COUNT(*) DESC limit 15
        Backtrace
        • view::layouts.theme:103
        • vendor/laravel/framework/src/Illuminate/Filesystem/Filesystem.php:124
        • vendor/laravel/framework/src/Illuminate/View/Engines/PhpEngine.php:58
        • vendor/laravel/framework/src/Illuminate/View/Engines/CompilerEngine.php:72
        • vendor/laravel/framework/src/Illuminate/View/View.php:207
      App\Models\Category
      138Category.php#?
      App\Models\Job
      1Job.php#?
      App\Models\User
      1User.php#?
          _token
          v0khAn68ssevimGjWq3pyER1eXMiaU32UjNq1rf4
          _previous
          array:1 [ "url" => "https://pingjob.com/Job0000017535" ]
          _flash
          array:2 [ "old" => [] "new" => [] ]
          path_info
          /Job0000017535
          status_code
          200
          
          status_text
          OK
          format
          html
          content_type
          text/html; charset=UTF-8
          request_query
          []
          
          request_request
          []
          
          request_headers
          0 of 0
          array:25 [ "sec-ch-ua-mobile" => array:1 [ 0 => "?0" ] "sec-ch-ua" => array:1 [ 0 => ""HeadlessChrome";v="129", "Not=A?Brand";v="8", "Chromium";v="129"" ] "cache-control" => array:1 [ 0 => "no-cache" ] "pragma" => array:1 [ 0 => "no-cache" ] "cdn-loop" => array:1 [ 0 => "cloudflare; loops=1" ] "priority" => array:1 [ 0 => "u=0, i" ] "upgrade-insecure-requests" => array:1 [ 0 => "1" ] "user-agent" => array:1 [ 0 => "Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; ClaudeBot/1.0; +claudebot@anthropic.com)" ] "cf-connecting-ip" => array:1 [ 0 => "18.218.140.12" ] "accept" => array:1 [ 0 => "text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7" ] "sec-fetch-site" => array:1 [ 0 => "none" ] "cf-visitor" => array:1 [ 0 => "{"scheme":"https"}" ] "sec-fetch-mode" => array:1 [ 0 => "navigate" ] "sec-fetch-user" => array:1 [ 0 => "?1" ] "x-forwarded-proto" => array:1 [ 0 => "https" ] "cf-ipcountry" => array:1 [ 0 => "US" ] "accept-encoding" => array:1 [ 0 => "gzip, br" ] "sec-fetch-dest" => array:1 [ 0 => "document" ] "sec-ch-ua-platform" => array:1 [ 0 => ""Windows"" ] "cf-ray" => array:1 [ 0 => "92bfc2058caa10b7-ORD" ] "connection" => array:1 [ 0 => "close" ] "x-accel-internal" => array:1 [ 0 => "/internal-nginx-static-location" ] "x-forwarded-for" => array:1 [ 0 => "18.218.140.12" ] "x-real-ip" => array:1 [ 0 => "172.69.59.5" ] "host" => array:1 [ 0 => "pingjob.com" ] ]
          request_cookies
          []
          
          response_headers
          0 of 0
          array:7 [ "content-type" => array:1 [ 0 => "text/html; charset=UTF-8" ] "cache-control" => array:1 [ 0 => "private, must-revalidate" ] "date" => array:1 [ 0 => "Sun, 06 Apr 2025 08:01:35 GMT" ] "pragma" => array:1 [ 0 => "no-cache" ] "expires" => array:1 [ 0 => "-1" ] "set-cookie" => array:2 [ 0 => "XSRF-TOKEN=eyJpdiI6IlQrTWtudVBWTG9USUM1ZmZXUlVWL0E9PSIsInZhbHVlIjoiU0w2b1gvUzl2V2QvVkFKRWVNTkhCTXRMVU5ETmJNUGxWdy80aEVVb0dOcUNDYVRuZDVnbVFlemdzd0JiNTJNN1Q2aVhWNTNzemwxeEdwZXVLMEV1Tmd1ZnJLSUtGeGxtckRGWCtVeVkzMDBGSk1McXlCcWNkbGpYK0NEdXJSUUUiLCJtYWMiOiI5MTg4YTBjNDU2MzBhNTg4Y2ZhZmI4Y2Y4MmU3MmQ5YTA5NDY3ZGE3NTViMzk1NTViNjIxMGRjNzhjMmZiNDBjIiwidGFnIjoiIn0%3D; expires=Sun, 06 Apr 2025 10:01:36 GMT; Max-Age=7200; path=/; secure; samesite=laxXSRF-TOKEN=eyJpdiI6IlQrTWtudVBWTG9USUM1ZmZXUlVWL0E9PSIsInZhbHVlIjoiU0w2b1gvUzl2V2QvVkFKRWVNTkhCTXRMVU5ETmJNUGxWdy80aEVVb0dOcUNDYVRuZDVnbVFlemdzd0JiNTJNN1Q2aVhWN" 1 => "laravel_session=eyJpdiI6ImYxTExiTGhCRkx0MS9oeG1wR0NYbFE9PSIsInZhbHVlIjoiMlpBUTZSMzVMcjZsK0tIaUZ2U2dZc29Sb3BSamo2dkZCaHBVZ0J3OEhmRG5XNUdoSGxTUGEwVGZ3QkJscGVaTm9Ed3FpOEhXSzNsT0djdE5aTW9tVGpjQ1FkM1dQL1ZvczUwbEYxT3d0bDUzdmJGSVBMZVMwcW95cXRidVdmMjEiLCJtYWMiOiJkNzIxZjZjNDIyM2YwMGRlNjQ1NDc0YWM5NzM4NjQ3NzI5MWVlNDg3ZDQwZjU3MmE4ZjJmZTdjMGU2OTBiOTdkIiwidGFnIjoiIn0%3D; expires=Sun, 06 Apr 2025 10:01:36 GMT; Max-Age=7200; path=/; httponly; samesite=laxlaravel_session=eyJpdiI6ImYxTExiTGhCRkx0MS9oeG1wR0NYbFE9PSIsInZhbHVlIjoiMlpBUTZSMzVMcjZsK0tIaUZ2U2dZc29Sb3BSamo2dkZCaHBVZ0J3OEhmRG5XNUdoSGxTUGEwVGZ3QkJscGVaTm9E" ] "Set-Cookie" => array:2 [ 0 => "XSRF-TOKEN=eyJpdiI6IlQrTWtudVBWTG9USUM1ZmZXUlVWL0E9PSIsInZhbHVlIjoiU0w2b1gvUzl2V2QvVkFKRWVNTkhCTXRMVU5ETmJNUGxWdy80aEVVb0dOcUNDYVRuZDVnbVFlemdzd0JiNTJNN1Q2aVhWNTNzemwxeEdwZXVLMEV1Tmd1ZnJLSUtGeGxtckRGWCtVeVkzMDBGSk1McXlCcWNkbGpYK0NEdXJSUUUiLCJtYWMiOiI5MTg4YTBjNDU2MzBhNTg4Y2ZhZmI4Y2Y4MmU3MmQ5YTA5NDY3ZGE3NTViMzk1NTViNjIxMGRjNzhjMmZiNDBjIiwidGFnIjoiIn0%3D; expires=Sun, 06-Apr-2025 10:01:36 GMT; path=/; secureXSRF-TOKEN=eyJpdiI6IlQrTWtudVBWTG9USUM1ZmZXUlVWL0E9PSIsInZhbHVlIjoiU0w2b1gvUzl2V2QvVkFKRWVNTkhCTXRMVU5ETmJNUGxWdy80aEVVb0dOcUNDYVRuZDVnbVFlemdzd0JiNTJNN1Q2aVhWN" 1 => "laravel_session=eyJpdiI6ImYxTExiTGhCRkx0MS9oeG1wR0NYbFE9PSIsInZhbHVlIjoiMlpBUTZSMzVMcjZsK0tIaUZ2U2dZc29Sb3BSamo2dkZCaHBVZ0J3OEhmRG5XNUdoSGxTUGEwVGZ3QkJscGVaTm9Ed3FpOEhXSzNsT0djdE5aTW9tVGpjQ1FkM1dQL1ZvczUwbEYxT3d0bDUzdmJGSVBMZVMwcW95cXRidVdmMjEiLCJtYWMiOiJkNzIxZjZjNDIyM2YwMGRlNjQ1NDc0YWM5NzM4NjQ3NzI5MWVlNDg3ZDQwZjU3MmE4ZjJmZTdjMGU2OTBiOTdkIiwidGFnIjoiIn0%3D; expires=Sun, 06-Apr-2025 10:01:36 GMT; path=/; httponlylaravel_session=eyJpdiI6ImYxTExiTGhCRkx0MS9oeG1wR0NYbFE9PSIsInZhbHVlIjoiMlpBUTZSMzVMcjZsK0tIaUZ2U2dZc29Sb3BSamo2dkZCaHBVZ0J3OEhmRG5XNUdoSGxTUGEwVGZ3QkJscGVaTm9E" ] ]
          session_attributes
          0 of 0
          array:3 [ "_token" => "v0khAn68ssevimGjWq3pyER1eXMiaU32UjNq1rf4" "_previous" => array:1 [ "url" => "https://pingjob.com/Job0000017535" ] "_flash" => array:2 [ "old" => [] "new" => [] ] ]
          ClearShow all
          Date ↕MethodURLData
          #12025-04-06 08:01:36GET/Job0000017535310140