Data Engineer (Contract)
9+ Month Contract based in Sheffield
395 - 442 per day InsideIR35
BPSS clearance required - candidates must be eligible
My client is seeking a Data Engineer to design and operate large-scale telemetry and observability data pipelines within a modern OpenShift and Kafka ecosystem. This role is central to enabling proactive, Level 4 observability, delivering high-quality metrics, logs, and traces to support platform reliability, operational insight, and automation.
Responsibilities:
Design, implement and maintain scalable data pipelines to ingest and process
OpenShift telemetry (metrics, logs, traces)
Stream telemetry through Kafka (producers, topics, schemas) and build resilient
consumer services for enrichment and transformation
Engineer multi-tenant observability data models, ensuring data lineage, quality
controls and SLAs across streaming layers
Integrate processed telemetry into Splunk for dashboards, analytics, alerting and
operational insights
Implement schema management and governance using Avro/Protobuf, including versioning
and compatibility strategies
Build automated validation, replay and backfill mechanisms to ensure data
reliability and recovery
Instrument services using OpenTelemetry, standardising tracing, metrics and
structured logging
Apply LLMs to enhance observability, such as query assistance, anomaly summarisation
and runbook generation
Collaborate with Platform, SRE and Application teams to align telemetry, alerts and
SLOs
Ensure pipelines meet security, compliance and best-practice standards
Produce clear documentation covering data flows, schemas, dashboards and operational
runbooks
Skills & Experience:
Strong hands-on experience building streaming data pipelines with Kafka
(producers/consumers, schema registry, Kafka Connect, KSQL/KStreams)
Experience with OpenShift / Kubernetes telemetry, including OpenTelemetry and
Prometheus
Proven capability integrating telemetry into Splunk
(HEC, Universal Forwarders, sourcetypes, CIM, dashboards, alerting)
Solid data engineering skills in Python (or similar) for ETL/ELT, enrichment and
validation
Please apply for immediate interview!
CBSbutler is operating and advertising as an Employment Agency for permanent positions and as an Employment Business for interim / contract / temporary positions. CBSbutler is an Equal Opportunities employer and we encourage applicants from all backgrounds.