{ "cells": [ { "cell_type": "markdown", "metadata": { "id": "ozeCGy0fwA4R" }, "source": [ "# Conversione PDF con MinerU" ] }, { "cell_type": "markdown", "metadata": { "id": "zyMHLIPh3sCk" }, "source": [ "[![stars](https://img.shields.io/github/stars/opendatalab/MinerU.svg)](https://github.com/opendatalab/MinerU)\n", "[![forks](https://img.shields.io/github/forks/opendatalab/MinerU.svg)](https://github.com/opendatalab/MinerU)\n", "[![open issues](https://img.shields.io/github/issues-raw/opendatalab/MinerU)](https://github.com/opendatalab/MinerU/issues)\n", "[![issue resolution](https://img.shields.io/github/issues-closed-raw/opendatalab/MinerU)](https://github.com/opendatalab/MinerU/issues)\n", "[![PyPI version](https://img.shields.io/pypi/v/mineru)](https://pypi.org/project/mineru/)\n", "[![PyPI - Python Version](https://img.shields.io/pypi/pyversions/mineru)](https://pypi.org/project/mineru/)\n", "[![Downloads](https://static.pepy.tech/badge/mineru)](https://pepy.tech/project/mineru)\n", "[![Downloads](https://static.pepy.tech/badge/mineru/month)](https://pepy.tech/project/mineru)\n", "[![OpenDataLab](https://img.shields.io/badge/webapp_on_mineru.net-blue?logo=data:image/svg+xml;base64,PHN2ZyB3aWR0aD0iMTM0IiBoZWlnaHQ9IjEzNCIgeG1sbnM9Imh0dHA6Ly93d3cudzMub3JnLzIwMDAvc3ZnIj48cGF0aCBkPSJtMTIyLDljMCw1LTQsOS05LDlzLTktNC05LTksNC05LDktOSw5LDQsOSw5eiIgZmlsbD0idXJsKCNhKSIvPjxwYXRoIGQ9Im0xMjIsOWMwLDUtNCw5LTksOXMtOS00LTktOSw0LTksOS05LDksNCw5LDl6IiBmaWxsPSIjMDEwMTAxIi8+PHBhdGggZD0ibTkxLDE4YzAsNS00LDktOSw5cy05LTQtOS05LDQtOSw5LTksOSw0LDksOXoiIGZpbGw9InVybCgjYikiLz48cGF0aCBkPSJtOTEsMThjMCw1LTQsOS05LDlzLTktNC05LTksNC05LDktOSw5LDQsOSw5eiIgZmlsbD0iIzAxMDEwMSIvPjxwYXRoIGZpbGwtcnVsZT0iZXZlbm9kZCIgY2xpcC1ydWxlPSJldmVub2RkIiBkPSJtMzksNjJjMCwxNiw4LDMwLDIwLDM4LDctNiwxMi0xNiwxMi0yNlY0OWMwLTQsMy03LDYtOGw0Ni0xMmM1LTEsMTEsMywxMSw4djMxYzAsMzctMzAsNjYtNjYsNjYtMzcsMC02Ni0zMC02Ni02NlY0NmMwLTQsMy03LDYtOGwyMC02YzUtMSwxMSwzLDExLDh2MjF6bS0yOSw2YzAsMTYsNiwzMCwxNyw0MCwzLDEsNSwxLDgsMSw1LDAsMTAtMSwxNS0zQzM3LDk1LDI5LDc5LDI5LDYyVjQybC0xOSw1djIweiIgZmlsbD0idXJsKCNjKSIvPjxwYXRoIGZpbGwtcnVsZT0iZXZlbm9kZCIgY2xpcC1ydWxlPSJldmVub2RkIiBkPSJtMzksNjJjMCwxNiw4LDMwLDIwLDM4LDctNiwxMi0xNiwxMi0yNlY0OWMwLTQsMy03LDYtOGw0Ni0xMmM1LTEsMTEsMywxMSw4djMxYzAsMzctMzAsNjYtNjYsNjYtMzcsMC02Ni0zMC02Ni02NlY0NmMwLTQsMy03LDYtOGwyMC02YzUtMSwxMSwzLDExLDh2MjF6bS0yOSw2YzAsMTYsNiwzMCwxNyw0MCwzLDEsNSwxLDgsMSw1LDAsMTAtMSwxNS0zQzM3LDk1LDI5LDc5LDI5LDYyVjQybC0xOSw1djIweiIgZmlsbD0iIzAxMDEwMSIvPjxkZWZzPjxsaW5lYXJHcmFkaWVudCBpZD0iYSIgeDE9Ijg0IiB5MT0iNDEiIHgyPSI3NSIgeTI9IjEyMCIgZ3JhZGllbnRVbml0cz0idXNlclNwYWNlT25Vc2UiPjxzdG9wIHN0b3AtY29sb3I9IiNmZmYiLz48c3RvcCBvZmZzZXQ9IjEiIHN0b3AtY29sb3I9IiMyZTJlMmUiLz48L2xpbmVhckdyYWRpZW50PjxsaW5lYXJHcmFkaWVudCBpZD0iYiIgeDE9Ijg0IiB5MT0iNDEiIHgyPSI3NSIgeTI9IjEyMCIgZ3JhZGllbnRVbml0cz0idXNlclNwYWNlT25Vc2UiPjxzdG9wIHN0b3AtY29sb3I9IiNmZmYiLz48c3RvcCBvZmZzZXQ9IjEiIHN0b3AtY29sb3I9IiMyZTJlMmUiLz48L2xpbmVhckdyYWRpZW50PjxsaW5lYXJHcmFkaWVudCBpZD0iYyIgeDE9Ijg0IiB5MT0iNDEiIHgyPSI3NSIgeTI9IjEyMCIgZ3JhZGllbnRVbml0cz0idXNlclNwYWNlT25Vc2UiPjxzdG9wIHN0b3AtY29sb3I9IiNmZmYiLz48c3RvcCBvZmZzZXQ9IjEiIHN0b3AtY29sb3I9IiMyZTJlMmUiLz48L2xpbmVhckdyYWRpZW50PjwvZGVmcz48L3N2Zz4=&labelColor=white)](https://mineru.net/OpenSourceTools/Extractor?source=github)\n", "[![HuggingFace](https://img.shields.io/badge/Demo_on_HuggingFace-yellow.svg?logo=data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAF8AAABYCAMAAACkl9t/AAAAk1BMVEVHcEz/nQv/nQv/nQr/nQv/nQr/nQv/nQv/nQr/wRf/txT/pg7/yRr/rBD/zRz/ngv/oAz/zhz/nwv/txT/ngv/0B3+zBz/nQv/0h7/wxn/vRb/thXkuiT/rxH/pxD/ogzcqyf/nQvTlSz/czCxky7/SjifdjT/Mj3+Mj3wMj15aTnDNz+DSD9RTUBsP0FRO0Q6O0WyIxEIAAAAGHRSTlMADB8zSWF3krDDw8TJ1NbX5efv8ff9/fxKDJ9uAAAGKklEQVR42u2Z63qjOAyGC4RwCOfB2JAGqrSb2WnTw/1f3UaWcSGYNKTdf/P+mOkTrE+yJBulvfvLT2A5ruenaVHyIks33npl/6C4s/ZLAM45SOi/1FtZPyFur1OYofBX3w7d54Bxm+E8db+nDr12ttmESZ4zludJEG5S7TO72YPlKZFyE+YCYUJTBZsMiNS5Sd7NlDmKM2Eg2JQg8awbglfqgbhArjxkS7dgp2RH6hc9AMLdZYUtZN5DJr4molC8BfKrEkPKEnEVjLbgW1fLy77ZVOJagoIcLIl+IxaQZGjiX597HopF5CkaXVMDO9Pyix3AFV3kw4lQLCbHuMovz8FallbcQIJ5Ta0vks9RnolbCK84BtjKRS5uA43hYoZcOBGIG2Epbv6CvFVQ8m8loh66WNySsnN7htL58LNp+NXT8/PhXiBXPMjLSxtwp8W9f/1AngRierBkA+kk/IpUSOeKByzn8y3kAAAfh//0oXgV4roHm/kz4E2z//zRc3/lgwBzbM2mJxQEa5pqgX7d1L0htrhx7LKxOZlKbwcAWyEOWqYSI8YPtgDQVjpB5nvaHaSnBaQSD6hweDi8PosxD6/PT09YY3xQA7LTCTKfYX+QHpA0GCcqmEHvr/cyfKQTEuwgbs2kPxJEB0iNjfJcCTPyocx+A0griHSmADiC91oNGVwJ69RudYe65vJmoqfpul0lrqXadW0jFKH5BKwAeCq+Den7s+3zfRJzA61/Uj/9H/VzLKTx9jFPPdXeeP+L7WEvDLAKAIoF8bPTKT0+TM7W8ePj3Rz/Yn3kOAp2f1Kf0Weony7pn/cPydvhQYV+eFOfmOu7VB/ViPe34/EN3RFHY/yRuT8ddCtMPH/McBAT5s+vRde/gf2c/sPsjLK+m5IBQF5tO+h2tTlBGnP6693JdsvofjOPnnEHkh2TnV/X1fBl9S5zrwuwF8NFrAVJVwCAPTe8gaJlomqlp0pv4Pjn98tJ/t/fL++6unpR1YGC2n/KCoa0tTLoKiEeUPDl94nj+5/Tv3/eT5vBQ60X1S0oZr+IWRR8Ldhu7AlLjPISlJcO9vrFotky9SpzDequlwEir5beYAc0R7D9KS1DXva0jhYRDXoExPdc6yw5GShkZXe9QdO/uOvHofxjrV/TNS6iMJS+4TcSTgk9n5agJdBQbB//IfF/HpvPt3Tbi7b6I6K0R72p6ajryEJrENW2bbeVUGjfgoals4L443c7BEE4mJO2SpbRngxQrAKRudRzGQ8jVOL2qDVjjI8K1gc3TIJ5KiFZ1q+gdsARPB4NQS4AjwVSt72DSoXNyOWUrU5mQ9nRYyjp89Xo7oRI6Bga9QNT1mQ/ptaJq5T/7WcgAZywR/XlPGAUDdet3LE+qS0TI+g+aJU8MIqjo0Kx8Ly+maxLjJmjQ18rA0YCkxLQbUZP1WqdmyQGJLUm7VnQFqodmXSqmRrdVpqdzk5LvmvgtEcW8PMGdaS23EOWyDVbACZzUJPaqMbjDxpA3Qrgl0AikimGDbqmyT8P8NOYiqrldF8rX+YN7TopX4UoHuSCYY7cgX4gHwclQKl1zhx0THf+tCAUValzjI7Wg9EhptrkIcfIJjA94evOn8B2eHaVzvBrnl2ig0So6hvPaz0IGcOvTHvUIlE2+prqAxLSQxZlU2stql1NqCCLdIiIN/i1DBEHUoElM9dBravbiAnKqgpi4IBkw+utSPIoBijDXJipSVV7MpOEJUAc5Qmm3BnUN+w3hteEieYKfRZSIUcXKMVf0u5wD4EwsUNVvZOtUT7A2GkffHjByWpHqvRBYrTV72a6j8zZ6W0DTE86Hn04bmyWX3Ri9WH7ZU6Q7h+ZHo0nHUAcsQvVhXRDZHChwiyi/hnPuOsSEF6Exk3o6Y9DT1eZ+6cASXk2Y9k+6EOQMDGm6WBK10wOQJCBwren86cPPWUcRAnTVjGcU1LBgs9FURiX/e6479yZcLwCBmTxiawEwrOcleuu12t3tbLv/N4RLYIBhYexm7Fcn4OJcn0+zc+s8/VfPeddZHAGN6TT8eGczHdR/Gts1/MzDkThr23zqrVfAMFT33Nx1RJsx1k5zuWILLnG/vsH+Fv5D4NTVcp1Gzo8AAAAAElFTkSuQmCC&labelColor=white)](https://huggingface.co/spaces/opendatalab/MinerU)\n", "[![ModelScope](https://img.shields.io/badge/Demo_on_ModelScope-purple?logo=data:image/svg+xml;base64,PHN2ZyB3aWR0aD0iMjIzIiBoZWlnaHQ9IjIwMCIgeG1sbnM9Imh0dHA6Ly93d3cudzMub3JnLzIwMDAvc3ZnIj4KCiA8Zz4KICA8dGl0bGU+TGF5ZXIgMTwvdGl0bGU+CiAgPHBhdGggaWQ9InN2Z18xNCIgZmlsbD0iIzYyNGFmZiIgZD0ibTAsODkuODRsMjUuNjUsMGwwLDI1LjY0OTk5bC0yNS42NSwwbDAsLTI1LjY0OTk5eiIvPgogIDxwYXRoIGlkPSJzdmdfMTUiIGZpbGw9IiM2MjRhZmYiIGQ9Im05OS4xNCwxMTUuNDlsMjUuNjUsMGwwLDI1LjY1bC0yNS42NSwwbDAsLTI1LjY1eiIvPgogIDxwYXRoIGlkPSJzdmdfMTYiIGZpbGw9IiM2MjRhZmYiIGQ9Im0xNzYuMDksMTQxLjE0bC0yNS42NDk5OSwwbDAsMjIuMTlsNDcuODQsMGwwLC00Ny44NGwtMjIuMTksMGwwLDI1LjY1eiIvPgogIDxwYXRoIGlkPSJzdmdfMTciIGZpbGw9IiMzNmNmZDEiIGQ9Im0xMjQuNzksODkuODRsMjUuNjUsMGwwLDI1LjY0OTk5bC0yNS42NSwwbDAsLTI1LjY0OTk5eiIvPgogIDxwYXRoIGlkPSJzdmdfMTgiIGZpbGw9IiMzNmNmZDEiIGQ9Im0wLDY0LjE5bDI1LjY1LDBsMCwyNS42NWwtMjUuNjUsMGwwLC0yNS42NXoiLz4KICA8cGF0aCBpZD0ic3ZnXzE5IiBmaWxsPSIjNjI0YWZmIiBkPSJtMTk4LjI4LDg5Ljg0bDI1LjY0OTk5LDBsMCwyNS42NDk5OWwtMjUuNjQ5OTksMGwwLC0yNS42NDk5OXoiLz4KICA8cGF0aCBpZD0ic3ZnXzIwIiBmaWxsPSIjMzZjZmQxIiBkPSJtMTk4LjI4LDY0LjE5bDI1LjY0OTk5LDBsMCwyNS42NWwtMjUuNjQ5OTksMGwwLC0yNS42NXoiLz4KICA8cGF0aCBpZD0ic3ZnXzIxIiBmaWxsPSIjNjI0YWZmIiBkPSJtMTUwLjQ0LDQybDAsMjIuMTlsMjUuNjQ5OTksMGwwLDI1LjY1bDIyLjE5LDBsMCwtNDcuODRsLTQ3Ljg0LDB6Ii8+CiAgPHBhdGggaWQ9InN2Z18yMiIgZmlsbD0iIzM2Y2ZkMSIgZD0ibTczLjQ5LDg5Ljg0bDI1LjY1LDBsMCwyNS42NDk5OWwtMjUuNjUsMGwwLC0yNS42NDk5OXoiLz4KICA8cGF0aCBpZD0ic3ZnXzIzIiBmaWxsPSIjNjI0YWZmIiBkPSJtNDcuODQsNjQuMTlsMjUuNjUsMGwwLC0yMi4xOWwtNDcuODQsMGwwLDQ3Ljg0bDIyLjE5LDBsMCwtMjUuNjV6Ii8+CiAgPHBhdGggaWQ9InN2Z18yNCIgZmlsbD0iIzYyNGFmZiIgZD0ibTQ3Ljg0LDExNS40OWwtMjIuMTksMGwwLDQ3Ljg0bDQ3Ljg0LDBsMCwtMjIuMTlsLTI1LjY1LDBsMCwtMjUuNjV6Ii8+CiA8L2c+Cjwvc3ZnPg==&labelColor=white)](https://www.modelscope.cn/studios/OpenDataLab/MinerU)\n", "[![Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/gist/myhloli/a3cb16570ab3cfeadf9d8f0ac91b4fca/mineru_demo.ipynb)\n", "[![arXiv](https://img.shields.io/badge/MinerU-Technical%20Report-b31b1b.svg?logo=arXiv)](https://arxiv.org/abs/2409.18839)\n", "[![arXiv](https://img.shields.io/badge/MinerU2.5-Technical%20Report-b31b1b.svg?logo=arXiv)](https://arxiv.org/abs/2509.22186)\n", "[![Ask DeepWiki](https://deepwiki.com/badge.svg)](https://deepwiki.com/opendatalab/MinerU)" ] }, { "cell_type": "markdown", "metadata": { "id": "F9dnYLSDwQyR" }, "source": [ "## 1.Install" ] }, { "cell_type": "markdown", "metadata": { "id": "Rw-hwakFwaeY" }, "source": [ "### Install via pip" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "colab": { "base_uri": "https://localhost:8080/", "height": 1000 }, "id": "_3EDplqlxKB_", "outputId": "a4c96ba2-6dea-4ef0-aac7-955dbfac0598" }, "outputs": [], "source": [ "!pip install --upgrade pip\n", "!pip install \"mineru[all]>=3.1.3\"" ] }, { "cell_type": "markdown", "metadata": { "id": "rSfS77AisXH8" }, "source": [ "### Check verison" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "colab": { "base_uri": "https://localhost:8080/" }, "id": "MSyCKHmqCp_j", "outputId": "eb0aad89-3261-4388-9dfc-2b6ce29a3030" }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "mineru, version 3.1.3\n" ] } ], "source": [ "!mineru -v" ] }, { "cell_type": "markdown", "metadata": { "id": "Qh8YKU19CrQY" }, "source": [ "## 2.Usage" ] }, { "cell_type": "markdown", "metadata": { "id": "3dZlkl3jDNKL" }, "source": [ "### 2.1 Command line" ] }, { "cell_type": "markdown", "metadata": { "id": "mVRMM_FaDi2d" }, "source": [ "#### **PDF**" ] }, { "cell_type": "markdown", "metadata": { "id": "Hyoz1B-tPXcM" }, "source": [ "#### **Converti un singolo PDF**\n", "\n", "Su Google Colab il file system è temporaneo: ogni volta che avvii una nuova sessione la cartella `/content/` è vuota.\n", "Devi quindi **caricare il tuo PDF prima di eseguire la conversione**.\n", "\n", "**Come caricare il file:**\n", "\n", "1. Nel pannello di sinistra clicca sull'icona **File**.\n", "2. Trascina il PDF nella cartella `/content/` (radice della sessione), oppure clicca il pulsante *Carica* (icona freccia su).\n", "3. Attendi che il caricamento sia completato: il file comparirà nell'elenco.\n", "\n", "In alternativa puoi caricare via codice:\n", "\n", "```python\n", "from google.colab import files\n", "uploaded = files.upload() # apre il selettore file\n", "```\n", "\n", "Una volta caricato, il PDF si trova in `/content/.pdf`.\n", "Sostituisci `mio_documento.pdf` nella cella seguente con il nome del tuo file:" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "PDF_NAME = \"mio_documento.pdf\" # ← sostituisci con il nome del file che hai caricato\n", "import os\n", "STEM = os.path.splitext(os.path.basename(PDF_NAME))[0]" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "colab": { "base_uri": "https://localhost:8080/" }, "collapsed": true, "id": "6RivAWteLeYM", "outputId": "dc56bcc0-6e84-432a-8511-e461ff605a51" }, "outputs": [], "source": [ "!mineru -p {PDF_NAME} -o ./output -b pipeline" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "colab": { "base_uri": "https://localhost:8080/", "height": 807 }, "id": "F6K4E8CfH6_V", "outputId": "136175e0-c1b0-4c80-84c4-ce7e7c78e656" }, "outputs": [], "source": [ "import os, zipfile, shutil\n", "\n", "# Cerca la cartella di output (backend auto o hybrid_auto)\n", "candidates = [\n", " f'./output/{STEM}/hybrid_auto',\n", " f'./output/{STEM}/auto',\n", " f'./output/{STEM}',\n", "]\n", "output_dir = next((p for p in candidates if os.path.isdir(p)), None)\n", "\n", "if output_dir is None:\n", " print(f'Cartella output non trovata per \"{STEM}\". Percorsi cercati:')\n", " for p in candidates:\n", " print(' ', p)\n", "else:\n", " zip_path = f'./{STEM}_output.zip'\n", " with zipfile.ZipFile(zip_path, 'w', zipfile.ZIP_DEFLATED) as zf:\n", " for root, dirs, files in os.walk(output_dir):\n", " for file in files:\n", " full_path = os.path.join(root, file)\n", " arcname = os.path.relpath(full_path, start=os.path.dirname(output_dir))\n", " zf.write(full_path, arcname)\n", " size_mb = os.path.getsize(zip_path) / (1024 * 1024)\n", " print(f'ZIP creato: {zip_path} ({size_mb:.1f} MB)')\n", "\n", " try:\n", " from google.colab import files\n", " files.download(zip_path)\n", " except ImportError:\n", " print('(download automatico disponibile solo su Google Colab)')\n" ] }, { "cell_type": "markdown", "metadata": { "id": "t53WETM90pKr" }, "source": [ "#### **Converti più PDF nella stessa cartella**\n", "\n", "Converte tutti i PDF presenti in `/content/` in un'unica esecuzione.\n", "\n", "Carica i file prima di eseguire la cella (trascina i PDF nel pannello File a sinistra oppure usa `files.upload()`).\n", "\n", "L'output di ogni PDF viene salvato in `./output//auto/`.\n", "\n", "> Usa questa cella al posto di quella precedente se hai più documenti da convertire." ] }, { "cell_type": "code", "execution_count": null, "metadata": { "colab": { "base_uri": "https://localhost:8080/" }, "id": "4dqBGZiz0orA", "outputId": "8233269d-d912-4dd3-9332-d26bd464db7c" }, "outputs": [], "source": [ "!mineru -p ./ -o ./output -b pipeline" ] }, { "cell_type": "markdown", "metadata": {}, "source": "## 3. Chunking\n\nScarica il repo RAG e suddivide il Markdown in chunk pronti per la vettorizzazione." }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": "!git clone https://santantonio.sytes.net/davide/rag-from-scratch.git /content/rag 2>/dev/null || (cd /content/rag && git pull)" }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": "# ── Parametri chunking — modifica qui prima di eseguire ──────────────────────\n\nMAX_CHARS = 1200 # lunghezza massima chunk (caratteri)\nMIN_CHARS = 80 # soglia minima (paragrafi più corti vengono fusi)\nCONTEXT_DEPTH = 3 # livelli heading inclusi nel prefisso (1–3)\nMERGE_SHORT_PARAGRAPHS = True\nSKIP_PRE_HEADING = True # salta contenuto prima del primo heading\nSKIP_HEADINGS = {\n \"indice\",\n \"sommario\",\n \"bibliografia\",\n \"ringraziamenti\",\n \"abbreviazioni\",\n}\n\n# ── Applica al config del repo ────────────────────────────────────────────────\n\nconfig_path = \"/content/rag/chunks/config.py\"\nwith open(config_path) as f:\n src = f.read()\n\nimport re\n\ndef _replace(src, name, value):\n if isinstance(value, bool):\n v = \"True\" if value else \"False\"\n return re.sub(rf\"^{name}\\s*=.*$\", f\"{name}: bool = {v}\", src, flags=re.MULTILINE)\n elif isinstance(value, int):\n return re.sub(rf\"^{name}\\s*=.*$\", f\"{name}: int = {value}\", src, flags=re.MULTILINE)\n elif isinstance(value, set):\n items = \",\\n \".join(f'\"{s}\"' for s in sorted(value))\n block = f\"{name}: set[str] = {{\\n {items},\\n}}\"\n return re.sub(rf\"^{name}.*?^\\}}\", block, src, flags=re.MULTILINE | re.DOTALL)\n return src\n\nfor name, val in [\n (\"MAX_CHARS\", MAX_CHARS),\n (\"MIN_CHARS\", MIN_CHARS),\n (\"CONTEXT_DEPTH\", CONTEXT_DEPTH),\n (\"MERGE_SHORT_PARAGRAPHS\", MERGE_SHORT_PARAGRAPHS),\n (\"SKIP_PRE_HEADING\", SKIP_PRE_HEADING),\n (\"SKIP_HEADINGS\", SKIP_HEADINGS),\n]:\n src = _replace(src, name, val)\n\nwith open(config_path, \"w\") as f:\n f.write(src)\n\nprint(\"Config applicato:\", config_path)\n" }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": "import os, sys, shutil\nsys.path.insert(0, \"/content/rag\")\n\n# Copia l'output MinerU dove il chunker si aspetta di trovarlo\nstem_output_src = f\"./output/{STEM}/auto\"\nstem_output_dst = f\"/content/rag/sources/{STEM}_output/auto\"\nif os.path.isdir(stem_output_src):\n os.makedirs(stem_output_dst, exist_ok=True)\n shutil.copytree(stem_output_src, stem_output_dst, dirs_exist_ok=True)\n\n!python /content/rag/chunks/chunker.py --stem {STEM}\n\n# Scarica chunks.json\nchunks_path = f\"/content/rag/chunks/{STEM}/chunks.json\"\nif os.path.exists(chunks_path):\n try:\n from google.colab import files\n files.download(chunks_path)\n except ImportError:\n print(f\"Chunk pronti in: {chunks_path}\")\nelse:\n print(\"chunks.json non trovato — controlla l'output sopra.\")\n" } ], "metadata": { "accelerator": "GPU", "colab": { "gpuType": "T4", "provenance": [] }, "kernelspec": { "display_name": "Python 3", "name": "python3" }, "language_info": { "name": "python" } }, "nbformat": 4, "nbformat_minor": 0 }