019790TS Big Data Engineer in Durham, NC (W2 ONLY, local preferred)

PMG Global

2019-09-05 00:36:19

Job location Durham, North Carolina, United States

Job type: fulltime

Job industry: I.T. & Communications

Job description

Primary Responsibilities:

  • You will be helping a team build data pipelines to feed our next generation cloud data & analytics platform
  • You will leverage your technical knowledge and experience to help design data ingestion and data preparation patterns which are reusable within and across teams
  • Daily you will be working a variety of tasks (analysis, development, DevOps) with a variety of technologies (Python, Airflow, AWS and many more)
  • You will be collaborating with squad resources in an agile framework to ensure work is planned and completed in the sprint cycle


  • Bachelor s degree or similar in a technology related field (e.g. Engineering, Computer Science, etc.) required.
  • 1+ years of hands-on experience developing and solutions in AWS, and hands-on knowledge of EC2, EBS, EKS, S3, IAM, CloudFormation, KMS, Route 53, RDS and other related AWS services.
  • 2+ years of advanced hands-on experience in implementing batch and near real-time Big Data integration frameworks and/or applications using various technologies (Hadoop, Spark, Impala, Sqoop, etc.)
  • Advanced experience with Python and SQL is required.
  • Experience building DAGs with Apache Airflow is a plus.
  • 3+ years of hands-on experience developing large-volume applications with various types of database systems: relational (Oracle, Postgres, MySQL, MSSQL) required, and knowledge of Snowflake is a plus.
  • Experience with Linux environments, ability to interface with the OS using system tools, scripting languages, integration frameworks, etc.
  • Experience with DevOps, Continuous Integration and Continuous Delivery (Maven, Jenkins, Stash, Docker).
  • Experience and comfort executing projects in Agile environments.
  • Excellent verbal and written communication skills.

- provided by Dice

Inform a friend!