Request ID: 79464-1
Title: Senior Data Engineer
Locations: Malvern, PA
Duration: 6 Months
Salary Range: $45 - $57/Hour on W2 (All inclusive)
Introduction
We are seeking a highly skilled and experienced Senior Data Engineer to join our team. The ideal candidate will have a strong background in building and maintaining data pipelines, with extensive experience in AWS and Python. This role is crucial for ensuring the performance, scalability, and reliability of our data infrastructure.
Required Skills & Qualifications
Preferred Skills & Qualifications
For immediate consideration please click APPLY to begin the screening process with Alex.
Title: Senior Data Engineer
Locations: Malvern, PA
Duration: 6 Months
Salary Range: $45 - $57/Hour on W2 (All inclusive)
Introduction
We are seeking a highly skilled and experienced Senior Data Engineer to join our team. The ideal candidate will have a strong background in building and maintaining data pipelines, with extensive experience in AWS and Python. This role is crucial for ensuring the performance, scalability, and reliability of our data infrastructure.
Required Skills & Qualifications
- Overall 10 years of experience in data engineering.
- 7 years of professional experience in Data Engineering.
- Strong expertise in Python and PySpark for large-scale data processing.
- Advanced hands-on experience with AWS services such as S3, Glue, EMR, Lambda, Step Functions, Kinesis/MSK, DynamoDB, Athena, and Redshift.
- Deep experience building event-driven and streaming data pipelines.
- Strong SQL experience for analytical and ETL workloads.
- Hands-on experience with workflow orchestration tools such as Airflow or Step Functions.
- Experience with CI/CD, Git, and Infrastructure-as-Code (Terraform or CloudFormation).
- Strong understanding of distributed systems, Spark performance tuning, data modeling, and cloud cost optimization.
- Knowledge of data security, encryption, networking, and compliance best practices in cloud environments.
- Prior work experience at client or in client's Industry.
Preferred Skills & Qualifications
- Experience of working on Java is an advantage.
- Knowledge of advanced Java concepts, Spring Boot, Angular 13, and Core Java.
- Build and maintain event-driven data pipelines using AWS services such as Kinesis, MSK/Kafka, Lambda, Step Functions, SQS/SNS, and Glue/EMR.
- Develop ETL/ELT workflows using Python and PySpark, ensuring performance, scalability, and cost efficiency.
- Implement and optimize Spark-based data transformations, partitioning strategies, and data processing frameworks.
- Design and manage data lake and warehouse structures using S3, Glue Catalog, Athena, and/or Redshift.
- Build streaming solutions with checkpointing, stateful transformations, idempotency, and schema evolution.
- Ensure high standards of data quality, observability, monitoring, and alerting (CloudWatch, Datadog, etc.).
- Implement data security best practices including IAM, encryption (KMS), networking, and governance.
- Create reusable frameworks, internal libraries, and CI/CD pipelines for automated deployments.
- Collaborate with data scientists, analysts, and business teams to deliver well-modeled, reliable datasets.
- Lead design reviews, mentor junior engineers, and contribute to engineering best practices.
- Competitive salary and comprehensive benefits package.
- Opportunities for professional growth and development.
- Collaborative and innovative work environment.
For immediate consideration please click APPLY to begin the screening process with Alex.