How a 12 KB Audit Ring Buffer Killed My TLS Handshake (And I Chased the Wrong Ghost for an Hour)

I want to walk through a bug I shipped this afternoon, what I thought was wrong, what was actually wrong, and the hour and a half between the two. The whole thing was caused by a single static array declaration. The misdiagnosis cost real time, real production noise, and an unnecessary re-enrolment of a device that didn't need re-enrolling. If you ever ship anything to an ESP32 with mTLS in production, this is a category of bug you will eventually meet.

This is also a sequel to the memory engineering post I wrote earlier the same day. That post laid out the budgets. This post is what happens when you ignore them.

01What I shipped¶

Earlier in the same session I had added an audit-log forwarding feature to the Edge firmware. Each event (boot, register write, config change, ...) gets formatted to JSON and published via MQTT to a central back-end. When the broker is offline the events queue up in memory until the next successful connect.

The queue was a 16-slot ring buffer of 768-byte string slots:

#define AUDIT_RING_SIZE   16
#define AUDIT_PAYLOAD_MAX 768

typedef struct {
    char payload[AUDIT_PAYLOAD_MAX];
    bool used;
} audit_slot_t;

static audit_slot_t s_ring[AUDIT_RING_SIZE];

That is ~12.3 KB sitting in .bss, on the chip's internal SRAM. I shipped the commit, flashed it, watched the device boot, watched it publish a boot audit event over MQTT, watched it disconnect, watched the audit drain on reconnect. Every case worked locally. Pushed to git. Moved on.

A couple of hours later a production deploy happened (a different change, not this one) and the device couldn't reconnect to the broker. I spent the next hour and a half chasing a PKI / Mosquitto cause that didn't exist.

02What I thought was wrong¶

The Edge device's serial log showed this on every reconnect attempt:

W (43608) mqtt: error type=1 tls_last_err=0x801a sock_errno=0
W (43608) mqtt: disconnected

0x801a in mbedtls is MBEDTLS_ERR_SSL_FATAL_ALERT_MESSAGE - "received a TLS fatal alert from the peer." That is a server-side signal. So I went looking server-side.

I checked the broker's TLS config (mTLS, require_certificate true, CRL enabled). I looked at the broker logs and saw "Protocol error" on every connection attempt from the device's source IP. I verified the source PKI files on the server had not been touched - /data/pki/ca.crt mtime was 2026-03-07, three months old. I compared the CA fingerprint that the server was serving against the CA fingerprint embedded in the issued device certs in the database. Byte-for-byte identical: D0:83:D2:91:C1:34:05:95:....

To rule out PKI completely, I ran openssl s_client from my laptop against the broker hostname using the exact same client cert + key the device had been issued:

New, TLSv1.2, Cipher is ECDHE-ECDSA-AES256-GCM-SHA384
Verify return code: 0 (ok)

Works fine. PKI was healthy. So my theory became: the device's NVS-stored client cert had become corrupted somehow. I had stress-tested NVS earlier the same session with 15+ encrypted-flash cycles during UI iteration. The fix would be to re-enrol the device over USB - that writes a fresh cert into NVS from the Central enrolment endpoint, overwriting whatever was there.

I did the re-enrol. Generated a fresh provisioning token, pushed it over the USB serial protocol, watched the device reboot, watched the enrolment task POST to Central, fetch a new client cert and key, store them to NVS.

Then the device tried MQTT again. Same error. 0x801a. Same disconnect.

That was the moment I realised I had been chasing the wrong thing.

03What was actually wrong¶

The full serial log, the one I should have read in detail an hour earlier:

E (145056) Dynamic Impl: alloc(4770 bytes) failed
E (145057) esp-tls-mbedtls: mbedtls_ssl_handshake returned -0x7F00
I (145057) esp-tls-mbedtls: Certificate verified.
E (145057) esp-tls: Failed to open new connection
W (145058) mqtt: error type=1 tls_last_err=0x801a sock_errno=0
W (145059) mqtt: disconnected

-0x7F00 is MBEDTLS_ERR_SSL_ALLOC_FAILED. mbedtls had asked the heap for 4770 bytes during its handshake. The allocator returned NULL. mbedtls aborted the handshake. The TLS layer dropped the connection. The chip reported the failure upstream as tls_last_err=0x801a - because in this version of esp-tls, that is the generic "TLS broke" code that gets surfaced regardless of whether it was a remote alert or a local alloc fail.

The broker had not sent anything. The handshake never finished. The "Protocol error" entry in the broker logs was the broker noting that its peer hung up mid-handshake.

The 4770 bytes mbedtls could not find were 4770 bytes of contiguous internal SRAM. The PSRAM heap on this chip was 5+ MB free; that was not the issue. The issue was that the largest contiguous block of internal heap had dropped below 4770 bytes. And the reason was my 12 KB audit ring buffer sitting in .bss.

04Why this matters more than it looks¶

Internal SRAM on the ESP32-S3 is the chip's single most constrained resource. Of the 512 KB printed on the spec sheet, the actually usable amount is somewhere around 209 KB once the boot ROM, FreeRTOS structures, WiFi pool, and a handful of other consumers take their cuts. That 209 KB is rivalrous: every byte you give one consumer is a byte the next consumer cannot have.

In this firmware the consumers were roughly:

The text version of the same table, for reference:

LVGL static pool         72 KB   (bisected to this exact value earlier)
mbedtls TLS state        ~30 KB  (during handshake; cannot live in PSRAM)
WiFi + LWIP              ~50 KB
LCD bounce buffer        16 KB
FreeRTOS task stacks     24 KB
App static .bss          16 KB   ← my entire budget for static / .bss

The 16 KB at the bottom was my entire budget for compile-time static and .bss allocations across the whole app. I had spent the first half of the day bisecting CONFIG_LV_MEM_SIZE_KILOBYTES between 64 and 96 KB precisely because LVGL and mbedtls were already fighting over the same internal heap. I had the receipts. The right answer was 72 KB - any higher and mbedtls failed to parse a cert, any lower and the LVGL Settings tab crashed during lazy build. The seesaw had exactly one stable point.

And then a few hours later I added 12 KB to .bss without re-checking that bisect.

The first time mbedtls actually exercised its full handshake allocation - which is sporadic and load-dependent, it does not always need its maximum - the allocator ran out of contiguous internal SRAM and the chip stopped being able to talk to the broker. From the outside this looked exactly like a PKI failure on the server side.

05Why mbedtls cannot just live in PSRAM¶

You might reasonably ask: if PSRAM is huge and free, why doesn't mbedtls just live there?

I tried. CONFIG_MBEDTLS_EXTERNAL_MEM_ALLOC=y routes mbedtls allocations to PSRAM. The handshake completes. Then partway through the first MQTT publish, the chip reboots with a LoadProhibited exception on a PSRAM address.

Parts of mbedtls' session state cache are touched from inside interrupt-disabled critical sections. PSRAM reads can stall (cache miss into the external bus) and that is not allowed in an ISR-disabled context. The cache controller signals a fault, the CPU panics, the watchdog reboots us.

There is no software workaround on this chip. mbedtls is internal-SRAM-only. Anything that competes with it for internal SRAM has to be checked, every time, before merge. I knew this. I forgot it for one PR.

06The fix¶

One line. Move the ring buffer from .bss to PSRAM via heap_caps_calloc:

-static audit_slot_t s_ring[AUDIT_RING_SIZE];
+static audit_slot_t *s_ring;     // PSRAM-backed, allocated in init

And in the init path:

s_ring = heap_caps_calloc(AUDIT_RING_SIZE, sizeof(audit_slot_t),
                          MALLOC_CAP_SPIRAM | MALLOC_CAP_8BIT);
if (!s_ring) {
    ESP_LOGE(TAG, "audit ring PSRAM alloc failed - "
                  "audit will drop offline events");
}

The ring now lives in PSRAM. The audit logger's offline behaviour is unchanged because the ring is only touched from the regular audit_log() call site and from the drain on MQTT reconnect, never from an ISR or a cache-disabled critical section. Internal SRAM lost a 12 KB consumer. mbedtls got its handshake budget back. Device reconnected within 30 seconds of the flash. The next boot audit record landed in Central with full details:

{
  "fw": "v0.1.0-164-g4927060",
  "reset_reason": 11,
  "heap_int_free": 161707,
  "heap_int_largest": 65536,
  "heap_psram_free": 6731616
}

The interesting field is heap_int_largest: 65 KB after the fix, against around 13 KB during the broken state. That one number, published in every boot audit record going forward, would have made this bug visible to me as a regression on the very next boot.

The audit pipeline I shipped to detect this kind of problem was itself the cause. And now also reports the metric that would have caught it. Strong irony.

07The misdiagnosis cost, itemised¶

If the first thing I had done on seeing tls_last_err=0x801a was to scroll up one screen in the serial log, I would have hit Dynamic Impl: alloc(4770 bytes) failed and gone straight to the right fix in five minutes.

I did not. I assumed the upstream-most log line was the cause. It was the consequence. There is a discipline I keep teaching other engineers and somehow lost myself today: when an error has a chain, the chain matters and the deepest entry in it is the only one that diagnoses the bug.

08Lessons I am actually following from here¶

1. When debugging a TLS handshake failure, read the full serial trace before forming a server-side hypothesis. Dynamic Impl: alloc(...) failed is the actual signal. The downstream tls_last_err=0x801a is the misleading consequence. mbedtls reports its own alloc-fail as if it had received a fatal alert from the peer, because that is the most generic way for the TLS layer above it to express "broken."

2. Any allocation greater than 1 KB in .bss or static on this chip gets weighed against the internal SRAM budget before commit. That means: do the math, write it in the commit message, and mention which consumer's budget is being subtracted from. If the math doesn't add up, the buffer goes to PSRAM.

3. For buffers greater than 4 KB the default is PSRAM via heap_caps_calloc(MALLOC_CAP_SPIRAM | MALLOC_CAP_8BIT). The exceptions are buffers touched from ISR-disabled critical sections, buffers on the LCD bounce-buffer hot path, or the LVGL static pool itself. For application code - audit, telemetry queue, log buffers, JSON staging - the default is always PSRAM. Make me prove the buffer needs to be internal.

4. Container health is not deploy success. A deploy is not done until a real device has published a real message post-deploy. I declared a production deploy "complete" while the device was failing for a reason I had introduced two commits earlier. Production state can be wrong even when every check I had trained myself to run was green.

5. heap_caps_get_largest_free_block(MALLOC_CAP_INTERNAL) belongs in every boot record from now on. That single number telegraphs an entire class of regressions. 65 KB is healthy, 13 KB is your handshake about to die. Publish it on boot. Watch it on the Central side. Page when it crosses a threshold.

09Closing¶

This is the bug I shipped in the same session as the audit logger that would have detected it. There is no clean moral to extract beyond what I already knew, ignored, and rediscovered the hard way: on a constrained microcontroller every byte negotiates with every other byte, and .bss is the first lever you have to declare a permanent winner. Be sure before you do.

If you want the architectural side of the same story - the memory map, the budgets in detail, why the bisect was so tight in the first place - it is the memory engineering post from the morning of the same day.

10Update, October 2026¶

Where the five lessons stand, four months on.

• Lesson 3 lost one of its exceptions. I listed the LVGL static pool as a buffer that has to stay internal. The next day it moved to PSRAM as well, through CONFIG_SPIRAM_ALLOW_BSS_SEG_EXTERNAL_MEMORY and LVGL's EXT_RAM_BSS_ATTR hook, and free internal heap at boot went from 22 KB to 110 KB. The details are in the update to the memory engineering post. The rule itself stands: application buffers go to PSRAM unless they are touched from a cache-disabled path.
• Lesson 4 is now in the deploy script. Central's production deploy watches for a real message from a real device after the services are recreated, and warns if none arrives within 75 seconds. It also fingerprints the broker's PKI and access-control configuration before and after, and aborts if either changed without an explicit rotation flag. Green containers are no longer the definition of done.
• Lesson 5 shipped the same day. Every boot record now carries the free internal heap, the largest free internal block and free PSRAM, and every MQTT reconnect carries the client certificate's expiry. Central accepts the richer audit format from the ESP32 (actor and outcome on every event) and opens any event's full details with one click.

Built and broken on the same Waveshare ESP32-S3-Touch-LCD-4.3B BOX in the same session. esp-idf 5.4, LVGL 9, mbedtls.