Incident Report 15/07/2025
Start: Aliens/3.3.5 (July 15, 2025, 11:13 AM UTC+2:00)
End: July 17, 2025, 3:39 PM UTC+2:00 (changes were reverted)
Reach: Limited to one customer. Their OAuth2 credentials were accessing authorize/token endpoints through the Cloudflare Relay Agent and were unavailable during this incident.
Impact: High – blocking
Description
Section titled “Description”Following the release of version Aliens/3.3.5, OAuth2 credentials accessing services through Cloudflare Relay Agent became unavailable, resulting in 403 Forbidden errors for authorize/token calls.
Causes
Section titled “Causes”The issue stemmed from a configuration error while setting up both timeout and mTLS (needed for relay agent to securely reach Cloudflare infrastructure) for OAuth2 clients.
The introduction of a timeout setting for OAuth2 authorize/token calls resulted in a configuration that overrode the mTLS settings, preventing our services from presenting an mTLS client certificate.
This caused two issues:
- New credentials could not exchange tokens with their OAuth2 server
- Existing credentials could not refresh their tokens or exchange new credentials
Resolution
Section titled “Resolution”The changes were reverted on July 17, 2025, 3:39 PM UTC+2:00.
A patch will be deployed to implement a properly functioning timeout configuration.
Follow-up
Section titled “Follow-up”The Mindflow engineering team is improving its testing mechanisms for these complex architectures with combined security add-ons to prevent such issues from occurring again.