Senior Spark QA Engineer

Senior Spark QA Engineer – Functional & Performance Testing

Location: Remote

Experience: 5+ Years

Job Description -

We are looking for a Senior Spark QA Engineer with strong expertise in functional and performance testing of Apache Spark applications and distributed data platforms.

Key Responsibilities -

• Perform manual and automated testing of Spark jobs, Spark SQL queries, and ETL pipelines.

• Execute functional, scalability, and performance testing for Spark workloads.

• Set up and manage Spark clusters on Standalone, YARN, Kubernetes, Mesos, Databricks, EMR, and Dataproc.

• Conduct benchmarking, validation, and performance analysis of Spark applications.

• Identify bottlenecks and troubleshoot distributed system issues.

•

Lead QA initiatives and mentor team members.

Required Skills -

•

5+ years of QA/testing experience with strong hands-on expertise in Apache Spark.

• Experience in functional and performance testing of distributed systems.

• Strong understanding of Spark architecture and optimization techniques.

• Experience with Kubernetes and cloud-based Spark environments.

• Proficiency in Python or Java.

• Experience with automation frameworks and CI/CD pipelines.

Originally posted on Himalayas