davide
f80b77ab93
chunker.py splits any revised Markdown (step-4) into RAG-ready chunks. Supports 4 strategies driven by structure_profile.json: h3_aware, h2_paragraph_split, paragraph, sliding_window. Respects MIN/MAX_CHARS and sentence-level overlap. Updates .gitignore and README paths.
43 lines
657 B
Plaintext
43 lines
657 B
Plaintext
# Virtual environment
|
|
.venv/
|
|
|
|
# PDF originali — file pesanti, non versionare
|
|
sources/
|
|
|
|
# Output pipeline — generati dagli script, non versionare
|
|
processed/
|
|
chroma_db/
|
|
|
|
# Python
|
|
__pycache__/
|
|
*.py[cod]
|
|
*.pyo
|
|
|
|
# Editor
|
|
.vscode/
|
|
.idea/
|
|
*.swp
|
|
*.swo
|
|
|
|
# OS
|
|
.DS_Store
|
|
Thumbs.db
|
|
|
|
# Report generati dagli script
|
|
step-0/*_step0_report.txt
|
|
step-1/*_step1_report.txt
|
|
|
|
# Output step-2 — MD grezzo generato da marker
|
|
step-2/*/
|
|
|
|
# Output step-3 — profilo struttura generato da detect_structure.py
|
|
step-3/*/
|
|
|
|
# Output step-4 — MD revisionato e log generati da revise.py
|
|
step-4/*/
|
|
step-4/revision_log.md
|
|
|
|
# Output step-5 — chunk generati da chunker.py
|
|
step-5/*/
|
|
|