Senior Data Engineer

About CLARA

CLARA Analytics is the leading AI as a service (AIaaS) provider that improves casualty claims outcomes for commercial insurance carriers and self-insured organizations. The company’s product suite for workers comp, commercial auto and general liability insurance claims applies image recognition, natural language processing, and other AI-based techniques to unlock insights from medical notes, bills and other documents surrounding a claim. CLARA’s customers include companies from the top 25 global insurance carriers to large third-party administrators and self-insured organizations. Founded in 2017, CLARA Analytics is headquartered in California’s Silicon Valley. For more information, visit www.claraanalytics.com.

About the Role

Join CLARA Analytics as a Senior Data Engineer and help transform the insurance industry by solving complex data challenges at scale. You'll build next-generation data infrastructure using cutting-edge AWS technologies that power our revolutionary AI models - from ingesting claim documents to architecting pipelines that process medical claims. If you're passionate about leveraging modern tools like Spark, PySpark Glue, and the full AWS ecosystem to tackle problems that directly impact claims adjusters, this is your opportunity to be part of something truly transformative in an industry ripe for disruption.

What You’ll Do...

• Architect and implement modern, scalable ETL/ELT pipelines using modern AWS-native services to process insurance claims data.
• Build resilient, high-throughput data pipelines with an emphasis on quality and reliability to drive consistent, accurate data across the enterprise.
• Ingest and transform diverse data sources, structured, semi-structured and unstructured, into enterprise-level ETL/ELT solutions.
• Design and implement custom algorithms to solve complex data challenges and unlock new insights.
• Collaborate cross-functionally with Data Scientists, Analysts, and Product teams to deliver business-aligned solutions
• Ensure pipeline reliability, performance, and SLA adherence.
• Streamline operations through automation, CI/CD, infrastructure-as-code, and configuration management tools.

What We’re Looking For...

Required

• 6+ years building production data pipelines with Spark, PySpark, and Spark SQL, plus orchestration experience with Airflow, AWS Step Functions, or Prefect
• 4+ years deep AWS data engineering expertise including tools such as Glue, EMR, Spark, and Lake Formation
• 4+ years mastering Python (or Scala or Go) for data engineering with experience in modern frameworks like pandas, polars, and data validation libraries
• 3+ years architecting data solutions using AWS services, from S3 data lakes to Redshift warehouses, with Glue ETL and Lambda for serverless processing
• 3+ years with container orchestration (Docker, Kubernetes, EKS) for scalable data workloads and microservices
• 2+ years implementing data quality frameworks using AWS Glue Data Quality and Databrew.
• Expert-level SQL skills including advanced analytics, window functions, and query optimization for large-scale data processing
• Strong engineering practices including GitOps workflows, infrastructure-as-code (Terraform/CDK), automated testing, and DataOps methodologies
• Production containerization experience with Docker, Kubernetes, Helm, and AWS container services (EKS, ECS, ECR)
• Thrives in fast-paced environments with excellent collaboration skills and adaptability to evolving requirements
• Creative problem-solver with strong debugging skills and ability to architect innovative solutions for complex data challenges

Preferred

• Direct experience with insurance claims data systems, including claims management platforms (Guidewire, Duck Creek, or Majesco), or healthcare EDI standards
• Experience working with medical coding systems (ICD-10, CPT, NDC) and extracting insights from healthcare claims data
• AWS professional certifications (Solutions Architect, Data Analytics Specialty, or DevOps Professional) demonstrating cloud architecture expertise
• Background in insurance or healthcare analytics where you've built data solutions that improved underwriting accuracy, claims processing efficiency, or fraud detection

What We Offer...

• The opportunity to make a real impact on a growing company.
• Collaborative and supportive work environment.
• Competitive compensation package.
  • Salary + Bonus
  • Benefits: employer-provided health insurance and ancillary benefits (life, disability, etc.), flexible PTO, fully remote, 401k with match
• Be a part of a team that is passionate about what we do!