Skip to main content

18 June 2026 - EMEA1B Service Degradation - Restored

Issue of stuck journeys on EMEA1B Tenants

RCA

  1. Incident Summary

Incident Date

18 June 2026

Type of Event

Unplanned

Severity

Sev1

Service Affected

Journey Task Processing

Impact Start

18 June 2026 — 3:16pm

Service Restored

18 June 2026 — 4:21pm

Duration of Impact

Approximately 1 hour

Status

Resolved

  1. Detailed Event Narrative

On 18 June 2026, following a large load event, certain tasks within client journeys temporarily slowed progression across a number of production environments.

The investigation identified the root cause and the source of the additional processing was identified and stopped. Normal task processing resumed and all affected journeys progressed without any manual intervention or loss of data.

  1. Event Timeline

Time

Event

18 June 2026 — 3:16pm

Large load event triggered.

18 June 2026 — 3:20pm

Monitoring alerts triggered. Client Journey Processing (Task processing) slowed. Engineering investigation commenced.

18 June 2026 — 4:20pm

Recovery, incident resolved

  1. Services Impacted

The following services were impacted during this incident:

Service

Impact Description

Client Journey Task Processing

Certain tasks within client journeys temporarily delayed progressing across a number of production environments.

There was no impact on the accuracy or integrity of work completed prior to the incident. No data loss occurred.

  1. Affected Applications and Systems

The impact was contained to the client journey processing (tasks) area of the platform; all other areas of the Fenergo application continued to operate normally throughout the incident.

Recovery was completed across all impacted environments on the same day.

  1. Root Cause Analysis

The root cause of this incident was an unusually high volume of processing activity temporarily consumed shared system capacity that is also used for standard task processing.

The activity triggered a much larger volume of downstream system processing than is typical. This additional processing temporarily used up shared system resources, which in turn slowed the normal processing that drives task progression for other journeys.

An existing system behaviour meant that certain bulk actions could generate a larger amount of downstream processing than expected. Work to address this behaviour was already underway prior to this incident.

The final corrective deployment resolved the issue at its source. All affected tasks were subsequently reprocessed, and impacted client journeys were unblocked on the same day.

  1. Corrective Actions and Corrective Actions Timeline

Fenergo has identified the following corrective actions to prevent recurrence:

#

Corrective Action

Target Timeline

1

Introduce additional limits to prevent bulk actions from generating disproportionate downstream processing load.

Done

2

Continue migrating relevant configurations to an improved processing model that does not exhibit this behaviour.

In progress

  1. Closing Statement

Fenergo takes the reliability of its platform and the continuity of our clients' operations seriously. We are committed to delivering the corrective actions outlined above and to embedding the lessons learned from this incident into our engineering and deployment practices.

We apologise for the disruption caused and thank our clients for their patience and continued partnership with our support teams throughout the recovery process.

18 June 2026 17:00 GMT

System has recovered since 16.30. Full RCA will be published in coming days

18 June 2026 16:30 GMT

Fenergo are monitoring and observing system is recovered. We will continue to monitor and will provide RCA in coming days. All journeys will processed including those affected

18 June 2026 16:25 GMT

Fenergo are monitoring and observing system is recovering

18 June 2026 16:05 GMT

Fenergo monitoring has alerted us to an issue with the Tenants on EMEA1
This is resulting in service degradation and journeys becoming stuck on tenants within that region

Did this answer your question?