AWS Data Engineer

Senior Data Engineer | Kafka | AWS Aurora | SQL | Kubernetes | Hybrid | Inside IR35

Location: UK (Hybrid)

Contract: Initial 3–4 Months (Likely Extension)

IR35: Inside IR35

Senior Data Engineer

We're recruiting for an experiencedSenior Data Engineer to join a leading organisation on an exciting data transformation programme. This is an excellent opportunity to play a key role in designing and delivering modern, cloud-native data platforms that power business-critical digital services. Working within a collaborative Agile environment, you'll lead the development of scalable data solutions, real-time streaming platforms, enterprise data warehouses, and API-driven integrations while mentoring junior engineers and driving engineering best practices.

Role Purpose

As a Senior Data Engineer, you will lead the design, build, and operation of scalable, secure, and high-performance data platforms. You'll own end-to-end data pipelines, streaming platforms, and enterprise data stores, ensuring they meet business requirements, DDaT standards, and non-functional requirements. You will also provide technical leadership, mentor junior engineers, and contribute to the continuous improvement of engineering standards.

Key Responsibilities

• Design and deliver robust data architectures covering data ingestion, streaming, storage, processing, and consumption.
• Architect, develop and support real-time data pipelines using Apache Kafka.
• Design and optimise relational databases using AWS Aurora (Postgres).
• Build and maintain enterprise-grade data warehouse solutions and analytical data models.
• Deploy, manage and optimise containerised data services using Docker and Kubernetes.
• Design and implement scalable API-based data ingestion solutions integrating operational systems, SaaS platforms, and partner applications.
• Define conceptual, logical and physical data models while ensuring data quality, governance, and lineage.
• Collaborate with development and integration teams to define API contracts, schemas, versioning strategies, and resilient integration patterns.
• Optimise SQL performance, pipeline throughput, reliability, and scalability.
• Implement monitoring, alerting, and operational support for data platforms.
• Mentor junior engineers and provide technical leadership across the engineering team.
• Produce and maintain high-quality technical documentation aligned with DDaT standards.

Essential Skills

• Expert SQL development and performance tuning
• Strong hands-on experience with Apache Kafka
• AWS Aurora (Postgres)
• Enterprise Data Warehousing
• Dimensional and Transactional Data Modelling
• Docker
• Kubernetes
• REST API Integration
• JSON and Schema Validation
• API resilience including retries, timeouts and idempotency
• Strong analytical, troubleshooting and problem-solving skills

Desirable Skills

• Experience with API Gateways or Integration Platforms
• OpenAPI / Swagger
• OAuth2 and token-based authentication
• Event-driven architectures combining APIs and Kafka
• Data observability and monitoring tools
• Experience working within Public Sector or regulated environments
• Knowledge of data governance, security and privacy principles
• Exposure to analytics and reporting platforms

Experience Required

• Significant experience in Senior Data Engineering roles
• Proven experience delivering scalable cloud-native data platforms
• Strong background in enterprise data architecture and engineering
• Experience leading technical delivery and mentoring engineers
• Experience working within Agile multidisciplinary teams
• Experience working within UK delivery environments

Ways of Working

• Agile, collaborative and multidisciplinary environment
• Strong ownership mindset with a focus on continuous improvement
• Active contribution to engineering standards and communities of practice

Security Clearance

Candidates must beeligible to obtain BPSS or SC Clearance, depending on client requirements.

Why Apply?

• Opportunity to work on a high-profile data transformation programme
• Modern technology stack including Kafka, AWS Aurora, Docker and Kubernetes
• Collaborative engineering culture with opportunities to influence technical decisions
• Initial 3–4 month contract with a strong likelihood of extension
• Hybrid working arrangement

If you're an experienced Senior Data Engineer looking to make an impact on large-scale cloud data platforms, we'd love to hear from you.

Apply today with your latest CV.