Senior Data Engineer/ PowerBI

Glasgow
1 day ago
Create job alert

Lead Data Engineer - Azure & Databricks Lakehouse

Glasgow (3/4 days onsite) | Exclusive Role with a Leading UK Consumer Business

A rapidly scaling UK consumer brand is undertaking a major data modernisation programme-moving away from legacy systems, manual Excel reporting and fragmented data sources into a fully automated Azure Enterprise Landing Zone + Databricks Lakehouse.
They are building a modern data platform from the ground up using Lakeflow Declarative Pipelines, Unity Catalog, and Azure Data Factory, and this role sits right at the heart of that transformation.
This is a rare opportunity to join early, influence architecture, and help define engineering standards, pipelines, curated layers and best practices that will support Operations, Finance, Sales, Logistics and Customer Care.
If you want to build a best-in-class Lakehouse from scratch-this is the one.

? What You'll Be Doing

Lakehouse Engineering (Azure + Databricks)

Engineer scalable ELT pipelines using Lakeflow Declarative Pipelines, PySpark, and Spark SQL across a full Medallion Architecture (Bronze ? Silver ? Gold).

Implement ingestion patterns for files, APIs, SaaS platforms (e.g. subscription billing), SQL sources, SharePoint and SFTP using ADF + metadata-driven frameworks.

Apply Lakeflow expectations for data quality, schema validation and operational reliability.

Curated Data Layers & Modelling

Build clean, conformed Silver/Gold models aligned to enterprise business domains (customers, subscriptions, deliveries, finance, credit, logistics, operations).

Deliver star schemas, harmonisation logic, SCDs and business marts to power high-performance Power BI datasets.

Apply governance, lineage and fine-grained permissions via Unity Catalog.

Orchestration & Observability

Design and optimise orchestration using Lakeflow Workflows and Azure Data Factory.

Implement monitoring, alerting, SLAs/SLIs, runbooks and cost-optimisation across the platform.

DevOps & Platform Engineering

Build CI/CD pipelines in Azure DevOps for notebooks, Lakeflow pipelines, SQL models and ADF artefacts.

Ensure secure, enterprise-grade platform operation across Dev ? Prod, using private endpoints, managed identities and Key Vault.

Contribute to platform standards, design patterns, code reviews and future roadmap.

Collaboration & Delivery

Work closely with BI/Analytics teams to deliver curated datasets powering dashboards across the organisation.

Influence architecture decisions and uplift engineering maturity within a growing data function.

? Tech Stack You'll Work With

Databricks: Lakeflow Declarative Pipelines, Workflows, Unity Catalog, SQL Warehouses

Azure: ADLS Gen2, Data Factory, Key Vault, vNets & Private Endpoints

Languages: PySpark, Spark SQL, Python, Git

DevOps: Azure DevOps Repos, Pipelines, CI/CD

Analytics: Power BI, Fabric

? What We're Looking For

Experience

5-8+ years of Data Engineering with 2-3+ years delivering production workloads on Azure + Databricks.

Strong PySpark/Spark SQL and distributed data processing expertise.

Proven Medallion/Lakehouse delivery experience using Delta Lake.

Solid dimensional modelling (Kimball) including surrogate keys, SCD types 1/2, and merge strategies.

Operational experience-SLAs, observability, idempotent pipelines, reprocessing, backfills.

Mindset

Strong grounding in secure Azure Landing Zone patterns.

Comfort with Git, CI/CD, automated deployments and modern engineering standards.

Clear communicator who can translate technical decisions into business outcomes.

Nice to Have

Databricks Certified Data Engineer Associate

Streaming ingestion experience (Auto Loader, structured streaming, watermarking)

Subscription/entitlement modelling experience

Advanced Unity Catalog security (RLS, ABAC, PII governance)

Terraform/Bicep for IaC

Fabric Semantic Model / Direct Lake optimisation

Related Jobs

View all jobs

Data Engineer

Senior Data Engineer - Azure | Permanent | Hybrid

Fabric Data Engineer

Senior Snowflake Data Engineer - Remote - £competitive

Lead Data Engineer - Microsoft Fabric - Hybrid - £75k

Lead Data Architect

Subscribe to Future Tech Insights for the latest jobs & insights, direct to your inbox.

By subscribing, you agree to our privacy policy and terms of service.

Industry Insights

Discover insightful articles, industry insights, expert tips, and curated resources.

Maths for Cloud Jobs: The Only Topics You Actually Need (& How to Learn Them)

If you are applying for cloud computing jobs in the UK you might have noticed something frustrating: job descriptions rarely ask for “maths” directly yet interviews often drift into capacity, performance, reliability, cost or security trade-offs that are maths in practice. The good news is you do not need degree-level theory to be job-ready. For most roles like Cloud Engineer, DevOps Engineer, Platform Engineer, SRE, Cloud Architect, FinOps Analyst or Cloud Security Engineer you keep coming back to a small set of practical skills: Units, rates & back-of-the-envelope estimation (requests per second, throughput, latency, storage growth) Statistics for reliability & observability (percentiles, error rates, SLOs, error budgets) Capacity planning & queueing intuition (utilisation, saturation, Little’s Law) Cost modelling & optimisation (right-sizing, break-even thinking, cost per transaction) Trade-off reasoning under constraints (performance vs cost vs reliability) This guide explains exactly what to learn plus a 6-week plan & portfolio projects you can publish to prove it.

Neurodiversity in Cloud Computing Careers: Turning Different Thinking into a Superpower

Cloud computing sits at the heart of modern tech. Almost every digital product runs on someone’s cloud platform – from banking apps & streaming services to AI tools & online shops. Behind those platforms are teams of cloud engineers, architects, SREs, security specialists & more. These roles demand problem-solvers who can think in systems, spot patterns, stay calm under pressure & imagine better ways to build & run infrastructure. That makes cloud computing a natural fit for many neurodivergent people – including those with ADHD, autism & dyslexia. If you are neurodivergent & considering a cloud career, you might have heard messages like “you’re too distracted for engineering”, “too literal for stakeholder work” or “too disorganised for operations”. In reality, many traits that come with ADHD, autism & dyslexia are exactly what cloud teams need. This guide is written for cloud computing job seekers in the UK. We will cover: What neurodiversity means in a cloud context How ADHD, autism & dyslexia strengths map to cloud roles Practical workplace adjustments you can ask for under UK law How to talk about neurodivergence in applications & interviews By the end, you should have a clearer sense of where you might thrive in cloud computing – & how to turn “different thinking” into a professional superpower.

Cloud Computing Hiring Trends 2026: What to Watch Out For (For Job Seekers & Recruiters)

As we move into 2026, the cloud computing jobs market in the UK is shifting again. The era of “lift & shift everything to the cloud” is giving way to a more mature, cost-conscious & security-focused phase. Many organisations are tightening budgets, some are rationalising cloud spend, yet demand for strong cloud talent remains high – especially around multi-cloud, FinOps, cloud security, data platforms & AI on cloud. Vendors are racing to integrate generative AI into their offerings, enterprises are modernising legacy estates, & regulators are asking tougher questions about resilience, sovereignty & risk. At the same time, some roles are being automated or commoditised, & the bar for cloud roles keeps rising. Whether you are a cloud job seeker planning your next move, or a recruiter building cloud teams, understanding the key cloud computing hiring trends for 2026 will help you stay ahead.