Job Details

Lead Data Quality & Automation Engineer

  2025-10-20     Artmac     San Francisco,CA  
Description:

Who We Are

Artmac Soft is a technology consulting and service-oriented IT company dedicated to providing innovative technology solutions and services to customers.

Job Description

Job Title: Lead Data Quality & Automation Engineer

Job Type: W2/C2C

Experience: 8-14 years

Location: San Francisco, California

Responsibilities

  • 5 years of professional experience in software development, with at least 2 years in a lead or senior engineering role.
  • Strong proficiency in Python programming and experience building data processing applications.
  • Hands-on expertise with Apache Spark and Hadoop for big data processing.
  • Solid experience with workflow orchestration tools like Apache Airflow.
  • Experience designing and managing data ingestion and integration pipelines with Apache NiFi.
  • Understanding on Data Quality automation, CI/CD, Jenkins, Oracle, Power BI, Splunk
  • Deep understanding of data quality concepts, data validation techniques, and distributed data systems.
  • Experience with cloud platforms (AWS, GCP, or Azure) is a plus.
  • Lead the design and implementation of data quality frameworks and automated validation pipelines using Python, Apache Spark, and Hadoop ecosystem tools.
  • Develop, deploy, and maintain scalable ETL/ELT workflows using Apache Airflow and Apache NiFi to ensure seamless data ingestion, transformation, and quality checks.
  • Collaborate with cross-functional teams to understand data quality requirements and translate them into technical solutions.
  • Define and enforce data quality standards, rules, and monitoring processes.
  • Perform root cause analysis on data quality issues and implement effective fixes and enhancements.
  • Mentor and guide junior engineers, conducting code reviews and fostering best practices.
  • Continuously evaluate and integrate new tools and technologies to enhance data quality capabilities.
  • Ensure high code quality, performance, and reliability in all data processing pipelines.
  • Create comprehensive documentation and reports on data quality metrics and system architecture.

Qualifications

  • Bachelor's degree or equivalent combination of education and experience.
#J-18808-Ljbffr


Apply for this Job

Please use the APPLY HERE link below to view additional details and application instructions.

Apply Here

Back to Search