Senior Data Engineer

To Apply for this Job Click Here

Job Title: Senior Data Engineer 

Location: Remote

Job-Type: Contract to Hire (must be a US Citizen or Green Card Holder)

Job Summary
Compass Technology is a dedicated internal team for Compass Group delivering enterprise-wide initiatives that support our diverse customer base and enhance our business operations.

Our domain encompasses a vast spectrum of opportunities, from hands-on desk support to Cybersecurity, Cloud Engineering, AI, and Modern Application development. We are committed to building robust IT infrastructures, driving digital transformation, and much more.

Compass Group is the leading foodservice management and support services company, with $26 billion in revenue in 2023.

In 2023, Compass Group was named one of Forbes’ America’s Best Large Employers along Springbuk’s Healthiest 100 Workplaces in America (since 2019).

Job Summary
We are looking for a hands-on Senior Data Engineer with expertise in developing data ingestion pipelines. This role is crucial in designing, building, and maintaining our data infrastructure, focusing on creating scalable pipelines, ensuring data integrity, and optimizing performance.

Key skills include strong Snowflake expertise, advanced SQL proficiency, data extraction from APIs using Python and AWS Lambda, and experience with ETL/ELT processes. Workflow automation using AWS Airflow is essential, and experience with Fivetran and DBT is a plus.

Job Responsibilities

• Design, build, test, and implement scalable data pipelines using Python and SQL.
• Maintain and optimize our Snowflake data warehouse’s performance, including data ingestion and query optimization.
• Extract data from APIs using Python and AWS Lambda and automate workflows with AWS Airflow.
• Perform analysis and critical thinking to troubleshoot data-related issues and implement checks/scripts to enhance data quality.
• Collaborate with other data engineers and architect to develop new pipelines and/or optimize existing ones.
• Maintain code via CI/CD processes as defined in our Azure DevOps platform.

   

Job Qualifications

• 7+ years of experience in Data Engineering roles, with a focus on building and implementing scalable data pipelines for data ingestion.
• Expertise in Snowflake, including data ingestion and performance optimization.
• Strong SQL skills for writing efficient queries and optimizing existing ones.
• Proficiency in Python for data extraction from APIs using AWS Lambda, Glue, etc.
• Experience with AWS services such as Lambda, Airflow, Glue, S3, SNS, etc.
• Highly self-motivated and detail-oriented with strong communication skills.
• Familiarity with ETL/ELT processes.

Experience with Fivetran and DBT is a plus.

We are seeking a hands-on Senior Data Engineer to join a growing data engineering team focused on building scalable, high-performing data ingestion pipelines and ensuring enterprise data quality. This individual will play a key role in designing, developing, optimizing, and maintaining modern cloud-based data infrastructure within a Snowflake ecosystem.

The ideal candidate will have strong expertise in Snowflake, advanced SQL and Python skills, and experience building ingestion frameworks using AWS services such as Lambda and Airflow. This role requires someone who is highly collaborative, detail-oriented, and comfortable working in a fast-paced Agile environment.

This is a remote contract-to-hire opportunity supporting teams operating in CST or EST time zones, with possible quarterly travel required for PI Planning sessions.

Key Responsibilities

• Design, build, test, and implement scalable data ingestion pipelines using Python and SQL
• Develop and optimize Snowflake data warehouse solutions, including ingestion performance and query tuning
• Extract and process data from APIs using Python and AWS services such as Lambda and Glue
• Automate workflows and orchestration using AWS Airflow
• Troubleshoot data-related issues and implement validation checks to improve data quality and reliability
• Collaborate closely with Data Engineers, Architects, and Data Transformation teams to develop and enhance enterprise pipelines
• Maintain and deploy code through CI/CD processes using Azure DevOps and modern deployment strategies
• Support documentation standards and maintain technical documentation for developed solutions
• Participate in Agile/SAFe ceremonies and quarterly PI Planning activities

Required Qualifications

• 7+ years of experience in Data Engineering with a strong focus on scalable data ingestion and ETL/ELT pipelines
• Deep expertise with Snowflake, including performance optimization and data warehouse administration
• Advanced SQL skills with experience writing and optimizing complex queries
• Strong Python development experience for API integrations and data processing
• Hands-on experience with AWS services including Lambda, Airflow, Glue, S3, SNS, and related cloud technologies
• Experience implementing and supporting CI/CD pipelines and deployment processes
• Strong understanding of data quality, monitoring, reconciliation, and validation strategies
• Experience working within Agile/SCRUM or SAFe environments
• Excellent communication skills with the ability to collaborate across technical and business teams
• Self-motivated, detail-oriented, and comfortable operating in a highly collaborative environment

Preferred Qualifications

• Experience with Fivetran for enterprise data ingestion
• Experience with DBT
• Prior experience supporting large-scale source-to-Snowflake replication initiatives
• Experience implementing CDC (Change Data Capture) solutions

Technical Areas of Focus

Candidates should be comfortable discussing and demonstrating experience in the following areas:

• Snowflake query optimization and performance tuning
• Large-scale data replication architectures
• Log-based CDC versus batch processing methodologies
• Batch reconciliation processes
• Data quality and monitoring strategies across pipeline stages
• API-based ingestion frameworks using Python

Interview Process

1. Initial Interview with Hiring Manager (30–45 minutes)
2. Technical Interview with Data Engineering Team
   • Live coding assessment focused on Python and SQL (60 minutes)
3. Final Interview with Senior Director (30 minutes)

Additional Information

• Contract-to-hire opportunity
• Conversion salary target: $140K–$160K
• Remote work environment (CST or EST hours preferred)
• Quarterly travel may be required for PI Planning
• Client cannot sponsor candidates now or in the future. Must be W-2 and able to hire on without any type of sponsorship 
• Please submit resume 2 pages maximum

Candidate Submission Requirement

As part of the submission process, candidates will complete a recorded video interview responding to three client-defined questions. The recording will be included with each submission package for client review.

To Apply for this Job Click Here

Equal Employment Opportunity Statement
Gravity IT Resources is an Equal Opportunity Employer. We are committed to creating an inclusive environment for all employees and applicants. We do not discriminate on the basis of race, color, religion, sex (including pregnancy, sexual orientation, or gender identity), national origin, age, disability, genetic information, veteran status, or any other legally protected characteristic. All employment decisions are based on qualifications, merit, and business needs.