Data Platform Engineer - Azure Databricks
Salary: Up to £90,000 per annum
Location: London
Job type: Permanent
About the role
A well-established international B2B events and media business is looking for a Data Platform Engineer to build, run and improve its Azure Databricks platform, which supports analytics, customer insight and AI. You will own the infrastructure, tooling and integrations behind the platform, keeping it secure, reliable and easy for engineering teams to use.
Working with the Data Platform Lead and delivery partners, you will help take the platform into ongoing operation and improve its security, reliability and cost efficiency. You will support Data Engineers and provide the environments and tools that data scientists and AI engineers need to build and run models and applications.
Key responsibilities
- Manage Azure infrastructure through Terraform, including storage, Key Vault and Databricks workspaces across development, staging and production
- Configure and support Databricks workspaces, compute policies, SQL warehouses and Unity Catalog
- Operate networking and connectivity, including virtual networks, private endpoints, DNS and VPN
- Design, build and support integrations between the platform and business systems such as Salesforce, HubSpot and registration providers
- Maintain CI/CD with GitHub Actions and Databricks bundle deployments
- Implement least-privilege access, managed identities and secrets handling, working with IT on Entra ID
- Own monitoring, alerting and incident response, plus backup and recovery
- Improve performance and cost visibility, and document and automate platform operations
Essential skills
- Experience operating production Azure infrastructure (networking, storage, identity, monitoring)
- Hands-on Terraform, including remote state and drift detection
- Practical Azure Databricks administration
- Working knowledge of SQL, Delta Lake and lakehouse (Bronze/Silver/Gold) architecture
- CI/CD using Git and GitHub Actions or similar
- Production integration experience using REST APIs and managed connectors
- Cloud security fundamentals: private networking, service principals, managed identities and secrets management
- Python, PowerShell or similar for automation and troubleshooting
- Experience supporting production services and managing incidents
Desirable
- Unity Catalog permissions, masking and audit logging
- Databricks Asset Bundles and Lakeflow
- Salesforce, HubSpot or event registration integrations
- Azure Monitor, Log Analytics and Defender for Cloud
- Disaster recovery testing and cloud cost optimisation
What you'll bring
A methodical, ownership-driven approach, clear communication, and the confidence to work with data engineers, IT, security colleagues and delivery partners.
:quality(80))
:quality(80))
:quality(80))
