feat(chunks): rilevamento universale has_math/has_table + dollarmath plugin

- parser.py: integra dollarmath_plugin (graceful fallback) — blocchi $$…$$ → token math_block atomico
- segmenter.py: gestisce math_block → Block(kind=math, atomic=True, plain_text=[formula])
- packer.py: has_table rileva anche <table> in blocchi html; has_math rileva $$ e \begin{ nel contenuto
- +11 test (parser×3, segmenter×3, packer×5) — totale 65 test

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
2026-06-09 08:40:41 +02:00
parent 1980efb0d6
commit 3b61df73d3
6 changed files with 121 additions and 3 deletions
+19
View File
@@ -191,6 +191,25 @@ def segment(tokens: list[Token], lines: list[str], config: ChunkerConfig) -> lis
i += 1
continue
# ── Math block (dollarmath plugin) ───────────────────────────────────
if tok.type == "math_block":
start = tok.map[0] if tok.map else 0
end = tok.map[1] if tok.map else start + 1
content = _lines_content(lines, start, end)
blocks.append(Block(
id=_make_id(),
kind="math",
content=content,
plain_text="[formula]",
atomic=True,
start_line=start,
end_line=end,
header_path=header_path,
chars=len(content),
))
i += 1
continue
# ── HTML block ────────────────────────────────────────────────────────
if tok.type == "html_block":
start = tok.map[0] if tok.map else 0