NLTK: Corpus readers follow symlinks outside trusted roots despite pathsec enforcement

Summary

Several corpus readers still step outside NLTK's symlink-aware trusted-root model. They derive in-root paths from trusted corpus state, convert those paths back into plain strings, and reopen them with built-in open() rather than nltk.pathsec.open().

Details

• Vulnerability type: Path traversal and symlink boundary bypass
• Affected component: nltk.corpus.reader.ipipan, nltk.corpus.reader.crubadan, nltk.corpus.reader.lin
• Affected versions: Published 3.9.4 and current source v3.10.0-rc2 both reproduced.
• Patched versions: Not yet patched
• Root cause: Root-derived paths are reopened with raw open() without preserving the trusted-root boundary.

IPIPANCorpusReader opens header.xml derived from morph.xml, CrubadanCorpusReader opens table.txt directly, and LinThesaurusCorpusReader opens simN.lsp paths returned from its own root helpers. Under pathsec.ENFORCE=True, a symlink placed inside the trusted corpus root can point outside the root and still be parsed successfully. It was confirmed parsed outside-root content is returned through public methods such as channels(), domains(), categories(), langs(), crubadan_to_iso(), synonyms(), and scored_synonyms().

PoC

Preconditions

• The application processes attacker-influenced corpora inside a trusted NLTK data root or trusted corpus directory.

Steps

1. Create a trusted corpus root and keep pathsec.ENFORCE=True with that root allowlisted.
2. Place symlinked reader inputs such as header.xml, table.txt, or simN.lsp inside the root and point them to external files.
3. Instantiate the corresponding corpus reader and call its normal public methods.
4. Observe that parsed outside-root values are returned even though pathsec.open() blocks the same symlink targets.

Minimal reproducible excerpt

 1{'ipipan': ['LEAK', 'TOPSECRET', 'CLASSIFIED'], 'crubadan': ['LEAK'], 'lin': [('LEAK', 9.5)]}

Impact

An attacker who can stage corpus files or symlinks under a trusted data root can disclose outside-root content through normal corpus-reader results, defeating the boundary NLTK documents for shared and untrusted-input environments.

Remediation

Preserve PathPointer and required_root semantics end to end. Replace direct open() calls with nltk.pathsec.open() or a reader helper that keeps the trusted-root boundary intact.

References

• https://github.com/nltk/nltk/blob/3.9.4/nltk/corpus/reader/ipipan.py#L162-L192
• https://github.com/nltk/nltk/blob/3.9.4/nltk/corpus/reader/crubadan.py#L74-L98
• https://github.com/nltk/nltk/blob/3.9.4/nltk/corpus/reader/lin.py#L40-L43
• https://github.com/nltk/nltk/blob/v3.10.0-rc2/nltk/pathsec.py#L521-L545