Senior Data Engineer (Big Data/ Hadoop/ Spark) (Banking)

London
1 week ago
Create job alert

Your new company

Working for a renowned financial services organisation

Your new role

We're looking for a Senior Data Engineer to design and deliver scalable on prem, high‑quality data solutions for low/ high-level data platforms that power analytical and business insights. This is a hands‑on role suited to someone with strong data engineering and big data expertise, ideally gained within financial services. Joining leading commodities, metals, trading, and exchange group, you will support a strategic metals initiative focused on reducing on‑prem platform costs and modernising legacy ETL processes.

You'll help design and build a new on‑prem data platform aligned to the metals strategy while developing and maintaining scalable data pipelines and analytics infrastructure. Using Hadoop, Big Data, and Spark technologies, you will ensure data quality through automated validation, monitoring, and testing. You will also enable seamless integration across data warehouses and data lakes, contributing to a robust, scalable, and resilient enterprise data ecosystem.

What you'll need to succeed

Vast Data Engineering expertise with Big Data technologies.
Experience designing and building on‑prem data platforms, from high‑level architecture to detailed technical design.
Hands‑on experience configuring multi‑node Hadoop clusters, including resource management, security, and performance tuning.
Strong Big Data engineering background using Apache Airflow, Spark, dbt, Kafka, and Hadoop ecosystem tools.
Knowledge of RDBMS systems (PostgreSQL, SQL Server) and familiarity with NoSQL/distributed databases such as MongoDB.
Proven delivery of streaming pipelines and real‑time data processing solutions.
Improved job efficiency and reduced runtimes through Apache Spark optimisation and development.
Some experience with containerisation (Docker, Kubernetes) and CI/CD pipelines.
Delivered streaming pipelines and real‑time data processing solutions.
Experience replacing legacy ETL tools (e.g., Informatica) with modern data engineering pipelines and platform builds.
Proven background working within financial services environments.
What you'll get in return
Flexible working options available.

What you need to do now
If you're interested in this role, click 'apply now' to forward an up-to-date copy of your CV, or call us now.

Hays Specialist Recruitment Limited acts as an employment agency for permanent recruitment and employment business for the supply of temporary workers. By applying for this job you accept the T&C's, Privacy Policy and Disclaimers which can be found at (url removed)

Related Jobs

View all jobs

Senior Data Engineer

Senior Data Engineer x1/ Data Engineer x1 (Financial Services)

Senior Data Engineer (Big Data/ Hadoop/ Spark) (Banking)

Senior Data Engineer/ PowerBI

Senior Data Engineer (AWS, Airflow, Python)

Senior Data Engineer (Microsoft Fabric)

Subscribe to Future Tech Insights for the latest jobs & insights, direct to your inbox.

By subscribing, you agree to our privacy policy and terms of service.

Industry Insights

Discover insightful articles, industry insights, expert tips, and curated resources.

What Hiring Managers Look for First in Cloud Computing Job Applications (UK Guide)

anding a job in cloud computing can be highly competitive — especially in the UK market where demand far outpaces supply in many segments. Whether you’re aiming for roles in Cloud Engineering, DevOps, Site Reliability, Cloud Architecture, Security, Data/Analytics, or Platform Operations, hiring managers screen applications quickly and with specific priorities in mind. Hiring managers don’t read every detail at first; they scan for critical signals in the first 10–20 seconds. These early signals determine whether your CV gets read more closely, whether your LinkedIn profile gets clicked, and whether you’re invited to interview. This guide breaks down, in practical terms, exactly what hiring managers look for first in cloud computing applications — and what you should emphasise in your CV, cover letter and portfolio to stand out on www.cloudcomputingjobs.co.uk .

The Skills Gap in Cloud Computing Jobs: What Universities Aren’t Teaching

Cloud computing underpins almost every modern digital service. From financial systems and healthcare platforms to AI, e-commerce, government infrastructure and cybersecurity, the cloud is now the default operating environment for UK organisations. Demand for cloud professionals has grown rapidly, with roles spanning architecture, engineering, security, DevOps, platform operations and cost optimisation. Salaries remain high, and vacancies remain stubbornly difficult to fill. Yet despite a growing number of graduates with computer science, IT and software engineering degrees, employers across the UK report a persistent problem: Too many candidates are not job-ready for real cloud computing roles. This is not a question of intelligence or motivation. It is a structural skills gap between what universities teach and what cloud jobs actually require. This article explores that gap in depth: what universities do well, what they consistently miss, why the gap exists, what employers genuinely want, and how jobseekers can bridge the divide to build sustainable careers in cloud computing.

Cloud Computing Jobs for Career Switchers in Their 30s, 40s & 50s (UK Reality Check)

Thinking about switching into cloud computing in your 30s, 40s or 50s? You aren’t alone. Across the UK, employers are hiring professionals from diverse backgrounds to help organisations adopt, manage & optimise cloud technology. But let’s cut through the buzzwords. This guide gives you a practical, UK-focused reality check on cloud computing careers for career switchers — what roles exist, what you actually need to learn, how long it takes to retrain and, importantly, whether age matters. If you’re exploring a move into cloud computing, this article lays out what’s realistic and how to get there without falling for hype.