Senior Data Engineer

Gravity IT Resources

To Apply for this Job Click Here

Job Title:  Sr Data Engineer
Location:  Salt Lake City, Utah (Remote)
Job-Type: Full Time
Salary Range: $130,000-$150,000
Referral Fee: $1000
Employment Eligibility: Gravity cannot transfer nor sponsor a work visa for this position. Applicants must be eligible to work in the U.S. for any employer directly (we are not open to contract or “corp to corp” agreements).
Position Overview:
We’re looking for an experienced data engineer to join our team. In this role, your primary focus will be on data automation and ensuring data quality. Utilizing Python and SQL to create data pipelines and quality assurance tools. This role involves working extensively with PySpark, Databricks pipelines, and other solutions within the Databricks ecosystem. Expect to write and refine Python code on a daily basis. Success in this position means being able to create and sustain environments that execute millions of lines of code accurately and reliably, with minimal errors.
Duties & Responsibilities:

  • Processing both streaming (Kinesis) and batch data at scale using Python.
  • Design, implement, and maintain the next generation data pipelines using industry-leading technologies such as Databricks.
  • Transitioning data ingestion strategy from utilizing Fivetran to implementing API based solutions and native data pipelines within Databricks.
  • Approach aims to replicate data efficiently into Snowflake, optimizing cost effectiveness by reducing ingestion expenses. Your role will involve setting up and managing these pipelines, ensuring their seamless replication and robust debugging capabilities.
  • In this capacity, Python will be employed extensively for developing PySpark applications within Databricks, as well as for API development to facilitate smooth data integration.

Required Experience & Skills:

  • 3-5 years of experience with Databricks and Snowflake
  • 3-5 years of experience with Python and writing APIs
  • 3-5 years of experience working with AWS/Azure
  • Bachelor’s or master’s Degree (computer science or other math-based technical field that requires significant programming.)

 

To Apply for this Job Click Here