Senior Data Engineer

Reference: RP/DATA_1790950595

Senior Data Engineer Role (Python / Spark / Databricks) - Hybrid - Contract - Banking

Role - Senior Data Engineer

Rate - £850 p/d (Inside IR35)

Duration - 6 months with very likely extension

Location - Hybrid / Liverpool Street (London) - 3 days per week in a Liverpool Street office

About the Role

Global financial institution seeking a highly skilled Senior Data Engineer to join a growing data engineering function responsible for delivering critical data platforms and analytics capabilities across the business.

This is a genuinely hands-on engineering role, requiring strong expertise in Python, Databricks, Spark/PySpark and SQL. The successful candidate will be actively involved in the design, development, optimisation and support of enterprise-scale data platforms, working alongside engineers, architects and business stakeholders to deliver robust and scalable solutions.

The role offers the opportunity to work on complex data challenges within a modern cloud-based environment, leveraging the latest data engineering technologies and best practices.

Key Responsibilities

Data Engineering & Development

  • Design, develop and maintain scalable data pipelines using Python, Databricks and PySpark
  • Build and optimise ETL/ELT solutions processing large-scale datasets
  • Develop high-quality, reusable code and engineering frameworks
  • Deliver robust, performant and maintainable data solutions
  • Troubleshoot and resolve complex production and performance issues
  • Support the ongoing enhancement and modernisation of enterprise data platforms

Databricks & Data Platform Engineering

  • Develop and support Databricks-based data solutions and Lakehouse architectures
  • Utilise Delta Lake and Spark technologies to deliver scalable data processing capabilities
  • Optimise distributed data processing workloads for performance and reliability
  • Implement data quality controls and monitoring solutions
  • Contribute to platform stability, scalability and operational excellence

Solution Delivery

  • Work closely with business and technology stakeholders to understand requirements and deliver effective solutions
  • Participate in Agile delivery processes including sprint planning, refinement and estimation
  • Support the delivery of strategic data initiatives across multiple business areas
  • Collaborate with architects, engineers and product teams to deliver high-quality outcomes
  • Identify and resolve technical risks and implementation challenges

Engineering Best Practice

  • Promote software engineering best practices including testing, code reviews and version control
  • Support CI/CD adoption and deployment automation
  • Contribute to coding standards and engineering governance
  • Drive continuous improvement across development and support processes
  • Mentor junior engineers and contribute to knowledge sharing within the team

Required Skills & Experience

  • Strong commercial experience as a Data Engineer or Senior Data Engineer
  • Expert-level Python development skills
  • Extensive hands-on experience with Databricks in enterprise environments
  • Strong Apache Spark and PySpark expertise
  • Advanced SQL development and optimisation skills
  • Experience designing and building large-scale ETL and data integration solutions
  • Strong understanding of distributed computing concepts and data processing frameworks
  • Experience working with modern cloud platforms including Azure, AWS or GCP
  • Knowledge of software engineering principles, testing and code quality practices
  • Experience working within Agile development environments
  • Excellent problem-solving and troubleshooting capabilities

Desirable Experience

  • Experience within investment banking, financial services or other highly regulated environments
  • Exposure to risk, trading, treasury or regulatory data domains
  • Experience with Kafka, streaming technologies or real-time data processing
  • Databricks certifications
  • Experience with Airflow, Databricks Workflows or similar orchestration tools
  • CI/CD pipeline development and DevOps practices
  • Docker and Kubernetes
  • Experience supporting data science or machine learning workloads

This role will suit a Senior Data Engineer who enjoys remaining close to the technology and has strong recent experience building production-grade solutions using Python, Databricks, Spark/PySpark and SQL in enterprise environments.

GCS is acting as an Employment Business in relation to this vacancy.

£600.00 - £850.00
Per day
GBP600 - GBP850 per day + Inside IR35
Added 02/10/2026
Reference: RP/DATA_1790950595

Other similar jobs

Senior BI Engineer - Quicksight and Data Modelling

Added 20/10/2026

Hiring AlertSenior BI EngineerWe are looking for a Senior BI Engineer - Quicksight with strong data modeling experience, and owning and rebuilding the manufacturing reporting layer, ensuring accurate, trusted, and automated operational reporting. The ideal candidate will work closely with operations, software engineering, and data engineering teams to validate business metrics, redesign datasets, and deliver scalable QuickSight dashboards.Key ResponsibilitiesStrong data modeling experience, including designing fact and dimension schemas from business requirements and creating intuitive data models for business users.Designing the data model. Identify a sensible, simple model (fact and dimension tables) that would yield those dashboardsAssess existing QuickSight dashboards, Athena/Redshift...

Learn more

Senior Data Engineer

Added 24/09/2026

Senior Data Engineer Role (Python / Spark / Databricks) - Hybrid - Contract - Banking Role - Senior Data EngineerRate - £850 p/d (Inside IR35)Duration - 6 months with very likely extensionLocation - Hybrid / Liverpool Street (London) - 3 days per week in a Liverpool Street officeAbout the RoleGlobal financial institution seeking a highly skilled Senior Data Engineer to join a growing data engineering function responsible for delivering critical data platforms and analytics capabilities across the business.This is a genuinely hands-on engineering role, requiring strong expertise in Python, Databricks, Spark/PySpark and SQL. The successful candidate will be actively involved...

Learn more

Senior Software Engineer - Data, Machine Learning & Applications

Added 09/09/2026

Senior Software Engineer - Data, Machine Learning & ApplicationsJob SummaryWe are seeking a versatile Senior Software Engineer with experience in application development, databases, data processing, and Machine Learning systems. This role will focus on designing and building robust, scalable technical solutions while ensuring high standards for software quality and performance.The ideal candidate combines strong software engineering fundamentals with experience working with relational databases, SQL, data solutions, and modern application development technologies.ResponsibilitiesDesign, develop, test, and maintain scalable software and Machine Learning applications.Write clean, efficient, and well-documented code that meets performance and business requirements.Participate in application architecture, system design, and technical solution...

Learn more

Senior Software Engineer - Quality, Applications & Data

Added 08/09/2026

Senior Software Engineer - Quality, Applications & DataWe are seeking an experienced Senior Software Engineer to design, develop, test, and support high-quality software applications and data solutions. This individual will play a key role throughout the full software development lifecycle, with responsibilities spanning software development, quality assurance, test automation, database development, and Machine Learning applications.The ideal candidate will bring a strong technical foundation in modern programming languages and frameworks, software testing methodologies, relational databases, and system design.ResponsibilitiesDevelop and execute formal test plans to ensure the delivery of high-quality software applications.Participate in test planning, test case development, test automation, and test...

Learn more

Senior Data Engineer

Added 27/08/2026

Senior Data Engineer Location: Dublin, Ireland (Hybrid - 3 days in office) Employment Type: Full-TimeAbout UsWe're an innovative smart grid technology company building systems that connect and optimise energy assets as part of a Virtual Power Plant, helping accelerate the decarbonisation of the energy system. Our in-house software and hardware platform spans embedded controllers with rich sensor data collection deployed on customer sites, through to backend applications for control, automation, and data analytics - all the way to customer-facing visualisation and reporting tools.We're a leader in the evolving smart grid space, offering Load Shifting, Demand Response, and Frequency Response services....

Learn more

Senior Java Data Engineer

Added 29/07/2026

Senior Java Data Engineer (AWS | Databricks | Spring Boot) - ContractLocation: Philadelphia, PA (Hybrid) Duration: ContractBuild the Data Platform Behind Enterprise AnalyticsWe're looking for an experienced Java Data Engineer to join a growing Data Engineering team responsible for building the next generation of cloud-based data solutions. In this role, you'll help modernize enterprise data platforms by developing scalable ETL pipelines, enhancing legacy applications, and delivering reliable data assets that power business intelligence and analytics.This position is ideal for someone who enjoys working across software engineering and data engineering, building cloud-native solutions using AWS, Databricks, and Snowflake while leveraging Java,...

Learn more

Senior Data Analyst

Added 20/10/2026

Job Title: Senior Data Analyst (Payments & Scheme Reporting)The OpportunityWe are seeking a Senior Data Analyst with strong experience in Payments, Card Schemes, Acquiring/Issuing Operations, and Commercial Reporting to join a global sales operations and reporting function. This role will be responsible for delivering executive-level reporting, forecasting, dashboard development, and data-driven insights across regional and global stakeholders.The ideal candidate will have a blend of Salesforce reporting, Tableau/Snowflake expertise, advanced Excel modelling, and payments industry knowledge, particularly within Visa, Mastercard, acquiring, issuing, merchant, and transaction reporting environments.Key ResponsibilitiesDesign and maintain executive reporting and dashboards for Director and VP-level stakeholders.Build automated reporting...

Learn more

Senior Data Administrator

Added 14/09/2026

Senior Data Administrator | 3-Month Contract (Potential to Go Permanent)Are you an experienced Data Administrator with a passion for data, systems, and reporting? We're seeking a talented professional to join a growing education organisation, supporting data operations across multiple sites and helping drive data-led decision making.Key ResponsibilitiesManage and optimise MIS, reporting, and cloud-based data systems.Support statutory and external reporting requirements.Ensure GDPR compliance, data integrity, and governance standards.Develop dashboards, reports, and actionable insights for key stakeholders.Provide training and support to users across the organisation.Act as a senior escalation point for data-related queries and issues.Skills & Experience✅ Proven experience in educational data...

Learn more

Senior data architect

Added 07/09/2026

Snowflake Data ArchitectRole OverviewWe are seeking an experienced Snowflake Data Architect to design, develop, and govern scalable, secure, and high-performing cloud data platforms. The ideal candidate will have strong expertise in Snowflake, data architecture, cloud technologies, data modelling, and modern data engineering practices.The Snowflake Data Architect will work closely with data engineers, analysts, business stakeholders, and technology teams to define data architecture strategies and deliver robust data solutions that support analytics, reporting, data science, and business intelligence initiatives.Key ResponsibilitiesDesign and implement scalable, secure, and high-performance data architectures using Snowflake.Define enterprise data architecture, data models, data flows, integration patterns, and technology...

Learn more

databricks data engineer - MDM

Added 18/11/2026

Senior Data Engineer, data foundation (Databricks)Mission. Owns the data foundation the AI products run on. First the group client registry on Databricks: per-country resolver views, identity matching and numbering, history and event tables, data quality and the stewardship workflow. Then the financial data products the apps consume: ledgers, invoices, documents, time registration and tax data, published as versioned datasets with contracts that external software can rely on. Takes over the Belgian hub logic from the current data lead and extends it to NL and DE -> Builds a working MDM.Must have3+ years hands-on Databricks: Unity Catalog, Delta Lake, workflows and...

Learn more

Data Center Quality Engineer

Added 26/10/2026

Data Center Quality Engineer - Fiber / Racks / Structured CablingLocation: New Albany, OHSchedule: Full-time onsiteContract: Long-term projectWe're supporting a large-scale data center build and looking for hands-on infrastructure professionals with strong quality instincts.You do not necessarily need to come from a formal QA/QC title. We're interested in candidates who have worked directly with rack deployments, fiber/copper cabling, structured cabling, hardware installation, inspections, testing, troubleshooting, and technician oversight and understand what good installation quality looks like.The focus of the role is simple: inspect the work, catch defects early, document them clearly, make sure they're corrected, and confirm the infrastructure is...

Learn more

QC Engineer - Data Center Infrastructure

Added 19/10/2026

⚡ QA/QC Engineer - Data Center Infrastructure | New Albany, OHI'm looking for 2 QA/QC Engineers to join CommScope's data center team supporting Meta infrastructure deployments in New Albany.This role is ideal for someone who understands how data center infrastructure is actually installed in the field and can confidently identify when something does not match the drawing, standard, or expected workmanship.The work centers around:🧰 QA of rack deployments inside data halls🧵 Inspecting fiber + copper installs, labeling, polarity, cleanliness, and test results📐 Reading rack elevations, layouts, pathways, and infrastructure drawings🏗️ Inspecting prefabricated/offsite-built racks and modules before shipment📋 Building punch lists,...

Learn more

Data Center QA Engineer

Added 19/10/2026

🚨 Hiring: Data Center QA/QC Engineer | New Albany, OHI'm supporting CommScope on an urgent search for 2 QA/QC Engineers supporting large-scale Meta data center buildouts in New Albany, Ohio.This is a hands-on onsite quality role responsible for making sure infrastructure is built and installed correctly before final sign-off - across both prefabricated/offsite builds and onsite data hall deployments.You'll be involved with:🔹 Inspecting rack deployments, placement, labeling, cable management, and workmanship🔹 Reviewing data center layouts, rack elevations, pathways, and infrastructure drawings🔹 Verifying fiber and copper cabling, terminations, labeling, cleanliness, and test results🔹 Running QA checklists / punch lists and tracking...

Learn more

Data Engineer

Added 13/10/2026

Data Engineer (Python / Distributed Data Platforms)📍 Relocation to Malta Required | 💼 Permanent | 🏠 Remote from MaltaWe have partnered with a growing European technology company that is investing heavily in its data platform and engineering capabilities.As part of their expansion, they are seeking Data Engineers across multiple levels, including:Data Engineer II (3-5 years)Data Engineer III (5-8 years)Staff Data Engineer (10-12 years)This is an opportunity to build and scale modern distributed data platforms capable of handling large-scale data workloads while working with a cutting-edge open-source technology stack.What You'll Be DoingDesign and develop scalable batch and real-time data pipelinesBuild reliable...

Learn more

Data Engineer

Added 13/10/2026

Data Engineer (Python / Distributed Data Platforms)📍 Relocation to Malta Required | 💼 Permanent | 🏠 Remote from MaltaWe have partnered with a growing European technology company that is investing heavily in its data platform and engineering capabilities.As part of their expansion, they are seeking Data Engineers across multiple levels, including:Data Engineer II (3-5 years)Data Engineer III (5-8 years)Staff Data Engineer (10-12 years)This is an opportunity to build and scale modern distributed data platforms capable of handling large-scale data workloads while working with a cutting-edge open-source technology stack.What You'll Be DoingDesign and develop scalable batch and real-time data pipelinesBuild reliable...

Learn more
At least 8 characters, 1 uppercase, 1 lowercase and 1 special character or number
Your file must be a doc, docx or pdf. No larger than 5MB.