Big Data Engineer — Hadoop & Spark Migration

We run a large Hadoop cluster that's expensive, slow to maintain, and blocking our data team. We want to migrate our batch processing workloads to Databricks (Spark on Azure) over 4 months. We need a senior data engineer to lead the migration. You'll inventory our current jobs, rewrite the critical ones in PySpark, set up Databricks workflows, and validate output parity. You won't be working alone — we have two internal engineers supporting you — but you'll drive the technical decisions.