diff --git a/README.md b/README.md index a4d8b26..c9a2926 100644 --- a/README.md +++ b/README.md @@ -176,6 +176,32 @@ pip install -r requirements.txt --- +## Configurazione rapida + +Prima di usare la pipeline verifica i parametri nei due file di configurazione: + +**`chunks/config.py`** — parametri di chunking: + +| Parametro | Default | Quando cambiarlo | +|-----------|---------|-----------------| +| `MAX_CHARS` | 1200 | Aumenta per chunk più lunghi (es. testi narrativi) | +| `MIN_CHARS` | 80 | Abbassa se hai molti titoli brevi | +| `FRONTMATTER_HEADINGS` | set italiano | Svuota (`set()`) per documenti non italiani | +| `OVERLAP_SENTENCES` | 1 | Aumenta a 2 se il retrieval perde contesto tra chunk | + +**`config.py`** — parametri RAG: + +| Parametro | Default | Quando cambiarlo | +|-----------|---------|-----------------| +| `OLLAMA_MODEL` | `qwen3.5:4b` | Sostituisci con il modello Ollama che vuoi usare | +| `EMBED_MODEL` | `nomic-embed-text` | Deve corrispondere al modello usato in ingestion | +| `TOP_K` | 6 | Aumenta per recuperare più contesto per domanda | +| `OLLAMA_URL` | `localhost:11434` | Cambia se Ollama gira su un altro host/porta | + +> Se cambi `EMBED_MODEL` devi rieseguire l'ingestion con `--force`. + +--- + ## Flusso completo ``` @@ -205,15 +231,39 @@ sources//auto/ ### Passo 1 — Converti il PDF con MinerU -Usa MinerU per convertire il PDF e posiziona la cartella di output in `sources/`: +Per questo step è necessario **MinerU** ([github.com/opendatalab/MinerU](https://github.com/opendatalab/MinerU)). + +**Opzione A — Google Colab (consigliata, nessuna installazione locale)** + +Usa il notebook incluso in questo repository: + +``` +mineru_demo.ipynb +``` + +Aprilo su Google Colab, carica il tuo PDF nella sessione e segui le celle in ordine. Al termine scarica la cartella `output//` generata. + +**Opzione B — Installazione locale** + +```bash +pip install "mineru[all]" +mineru -p documento.pdf -o sources// +``` + +Al termine, **copia la cartella completa** prodotta da MinerU dentro `sources/`. La struttura attesa è: ``` sources/ └── / └── auto/ - ├── _content_list_v2.json - ├── _model.json - └── ... + ├── .md + ├── _content_list_v2.json ← richiesto + ├── _model.json ← raccomandato + ├── _middle.json + ├── _content_list.json + ├── _layout.pdf + ├── _span.pdf + └── images/ ``` ### Passo 2 — Chunking (Stage 1 + Stage 2) diff --git a/mineru.ipynb b/mineru.ipynb new file mode 100644 index 0000000..b7a9388 --- /dev/null +++ b/mineru.ipynb @@ -0,0 +1,338 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "metadata": { + "id": "ozeCGy0fwA4R" + }, + "source": [ + "# Conversione PDF con MinerU" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "zyMHLIPh3sCk" + }, + "source": [ + "[![stars](https://img.shields.io/github/stars/opendatalab/MinerU.svg)](https://github.com/opendatalab/MinerU)\n", + "[![forks](https://img.shields.io/github/forks/opendatalab/MinerU.svg)](https://github.com/opendatalab/MinerU)\n", + "[![open issues](https://img.shields.io/github/issues-raw/opendatalab/MinerU)](https://github.com/opendatalab/MinerU/issues)\n", + "[![issue resolution](https://img.shields.io/github/issues-closed-raw/opendatalab/MinerU)](https://github.com/opendatalab/MinerU/issues)\n", + "[![PyPI version](https://img.shields.io/pypi/v/mineru)](https://pypi.org/project/mineru/)\n", + "[![PyPI - Python Version](https://img.shields.io/pypi/pyversions/mineru)](https://pypi.org/project/mineru/)\n", + "[![Downloads](https://static.pepy.tech/badge/mineru)](https://pepy.tech/project/mineru)\n", + "[![Downloads](https://static.pepy.tech/badge/mineru/month)](https://pepy.tech/project/mineru)\n", + "[![OpenDataLab](https://img.shields.io/badge/webapp_on_mineru.net-blue?logo=data:image/svg+xml;base64,PHN2ZyB3aWR0aD0iMTM0IiBoZWlnaHQ9IjEzNCIgeG1sbnM9Imh0dHA6Ly93d3cudzMub3JnLzIwMDAvc3ZnIj48cGF0aCBkPSJtMTIyLDljMCw1LTQsOS05LDlzLTktNC05LTksNC05LDktOSw5LDQsOSw5eiIgZmlsbD0idXJsKCNhKSIvPjxwYXRoIGQ9Im0xMjIsOWMwLDUtNCw5LTksOXMtOS00LTktOSw0LTksOS05LDksNCw5LDl6IiBmaWxsPSIjMDEwMTAxIi8+PHBhdGggZD0ibTkxLDE4YzAsNS00LDktOSw5cy05LTQtOS05LDQtOSw5LTksOSw0LDksOXoiIGZpbGw9InVybCgjYikiLz48cGF0aCBkPSJtOTEsMThjMCw1LTQsOS05LDlzLTktNC05LTksNC05LDktOSw5LDQsOSw5eiIgZmlsbD0iIzAxMDEwMSIvPjxwYXRoIGZpbGwtcnVsZT0iZXZlbm9kZCIgY2xpcC1ydWxlPSJldmVub2RkIiBkPSJtMzksNjJjMCwxNiw4LDMwLDIwLDM4LDctNiwxMi0xNiwxMi0yNlY0OWMwLTQsMy03LDYtOGw0Ni0xMmM1LTEsMTEsMywxMSw4djMxYzAsMzctMzAsNjYtNjYsNjYtMzcsMC02Ni0zMC02Ni02NlY0NmMwLTQsMy03LDYtOGwyMC02YzUtMSwxMSwzLDExLDh2MjF6bS0yOSw2YzAsMTYsNiwzMCwxNyw0MCwzLDEsNSwxLDgsMSw1LDAsMTAtMSwxNS0zQzM3LDk1LDI5LDc5LDI5LDYyVjQybC0xOSw1djIweiIgZmlsbD0idXJsKCNjKSIvPjxwYXRoIGZpbGwtcnVsZT0iZXZlbm9kZCIgY2xpcC1ydWxlPSJldmVub2RkIiBkPSJtMzksNjJjMCwxNiw4LDMwLDIwLDM4LDctNiwxMi0xNiwxMi0yNlY0OWMwLTQsMy03LDYtOGw0Ni0xMmM1LTEsMTEsMywxMSw4djMxYzAsMzctMzAsNjYtNjYsNjYtMzcsMC02Ni0zMC02Ni02NlY0NmMwLTQsMy03LDYtOGwyMC02YzUtMSwxMSwzLDExLDh2MjF6bS0yOSw2YzAsMTYsNiwzMCwxNyw0MCwzLDEsNSwxLDgsMSw1LDAsMTAtMSwxNS0zQzM3LDk1LDI5LDc5LDI5LDYyVjQybC0xOSw1djIweiIgZmlsbD0iIzAxMDEwMSIvPjxkZWZzPjxsaW5lYXJHcmFkaWVudCBpZD0iYSIgeDE9Ijg0IiB5MT0iNDEiIHgyPSI3NSIgeTI9IjEyMCIgZ3JhZGllbnRVbml0cz0idXNlclNwYWNlT25Vc2UiPjxzdG9wIHN0b3AtY29sb3I9IiNmZmYiLz48c3RvcCBvZmZzZXQ9IjEiIHN0b3AtY29sb3I9IiMyZTJlMmUiLz48L2xpbmVhckdyYWRpZW50PjxsaW5lYXJHcmFkaWVudCBpZD0iYiIgeDE9Ijg0IiB5MT0iNDEiIHgyPSI3NSIgeTI9IjEyMCIgZ3JhZGllbnRVbml0cz0idXNlclNwYWNlT25Vc2UiPjxzdG9wIHN0b3AtY29sb3I9IiNmZmYiLz48c3RvcCBvZmZzZXQ9IjEiIHN0b3AtY29sb3I9IiMyZTJlMmUiLz48L2xpbmVhckdyYWRpZW50PjxsaW5lYXJHcmFkaWVudCBpZD0iYyIgeDE9Ijg0IiB5MT0iNDEiIHgyPSI3NSIgeTI9IjEyMCIgZ3JhZGllbnRVbml0cz0idXNlclNwYWNlT25Vc2UiPjxzdG9wIHN0b3AtY29sb3I9IiNmZmYiLz48c3RvcCBvZmZzZXQ9IjEiIHN0b3AtY29sb3I9IiMyZTJlMmUiLz48L2xpbmVhckdyYWRpZW50PjwvZGVmcz48L3N2Zz4=&labelColor=white)](https://mineru.net/OpenSourceTools/Extractor?source=github)\n", + "[![HuggingFace](https://img.shields.io/badge/Demo_on_HuggingFace-yellow.svg?logo=data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAF8AAABYCAMAAACkl9t/AAAAk1BMVEVHcEz/nQv/nQv/nQr/nQv/nQr/nQv/nQv/nQr/wRf/txT/pg7/yRr/rBD/zRz/ngv/oAz/zhz/nwv/txT/ngv/0B3+zBz/nQv/0h7/wxn/vRb/thXkuiT/rxH/pxD/ogzcqyf/nQvTlSz/czCxky7/SjifdjT/Mj3+Mj3wMj15aTnDNz+DSD9RTUBsP0FRO0Q6O0WyIxEIAAAAGHRSTlMADB8zSWF3krDDw8TJ1NbX5efv8ff9/fxKDJ9uAAAGKklEQVR42u2Z63qjOAyGC4RwCOfB2JAGqrSb2WnTw/1f3UaWcSGYNKTdf/P+mOkTrE+yJBulvfvLT2A5ruenaVHyIks33npl/6C4s/ZLAM45SOi/1FtZPyFur1OYofBX3w7d54Bxm+E8db+nDr12ttmESZ4zludJEG5S7TO72YPlKZFyE+YCYUJTBZsMiNS5Sd7NlDmKM2Eg2JQg8awbglfqgbhArjxkS7dgp2RH6hc9AMLdZYUtZN5DJr4molC8BfKrEkPKEnEVjLbgW1fLy77ZVOJagoIcLIl+IxaQZGjiX597HopF5CkaXVMDO9Pyix3AFV3kw4lQLCbHuMovz8FallbcQIJ5Ta0vks9RnolbCK84BtjKRS5uA43hYoZcOBGIG2Epbv6CvFVQ8m8loh66WNySsnN7htL58LNp+NXT8/PhXiBXPMjLSxtwp8W9f/1AngRierBkA+kk/IpUSOeKByzn8y3kAAAfh//0oXgV4roHm/kz4E2z//zRc3/lgwBzbM2mJxQEa5pqgX7d1L0htrhx7LKxOZlKbwcAWyEOWqYSI8YPtgDQVjpB5nvaHaSnBaQSD6hweDi8PosxD6/PT09YY3xQA7LTCTKfYX+QHpA0GCcqmEHvr/cyfKQTEuwgbs2kPxJEB0iNjfJcCTPyocx+A0griHSmADiC91oNGVwJ69RudYe65vJmoqfpul0lrqXadW0jFKH5BKwAeCq+Den7s+3zfRJzA61/Uj/9H/VzLKTx9jFPPdXeeP+L7WEvDLAKAIoF8bPTKT0+TM7W8ePj3Rz/Yn3kOAp2f1Kf0Weony7pn/cPydvhQYV+eFOfmOu7VB/ViPe34/EN3RFHY/yRuT8ddCtMPH/McBAT5s+vRde/gf2c/sPsjLK+m5IBQF5tO+h2tTlBGnP6693JdsvofjOPnnEHkh2TnV/X1fBl9S5zrwuwF8NFrAVJVwCAPTe8gaJlomqlp0pv4Pjn98tJ/t/fL++6unpR1YGC2n/KCoa0tTLoKiEeUPDl94nj+5/Tv3/eT5vBQ60X1S0oZr+IWRR8Ldhu7AlLjPISlJcO9vrFotky9SpzDequlwEir5beYAc0R7D9KS1DXva0jhYRDXoExPdc6yw5GShkZXe9QdO/uOvHofxjrV/TNS6iMJS+4TcSTgk9n5agJdBQbB//IfF/HpvPt3Tbi7b6I6K0R72p6ajryEJrENW2bbeVUGjfgoals4L443c7BEE4mJO2SpbRngxQrAKRudRzGQ8jVOL2qDVjjI8K1gc3TIJ5KiFZ1q+gdsARPB4NQS4AjwVSt72DSoXNyOWUrU5mQ9nRYyjp89Xo7oRI6Bga9QNT1mQ/ptaJq5T/7WcgAZywR/XlPGAUDdet3LE+qS0TI+g+aJU8MIqjo0Kx8Ly+maxLjJmjQ18rA0YCkxLQbUZP1WqdmyQGJLUm7VnQFqodmXSqmRrdVpqdzk5LvmvgtEcW8PMGdaS23EOWyDVbACZzUJPaqMbjDxpA3Qrgl0AikimGDbqmyT8P8NOYiqrldF8rX+YN7TopX4UoHuSCYY7cgX4gHwclQKl1zhx0THf+tCAUValzjI7Wg9EhptrkIcfIJjA94evOn8B2eHaVzvBrnl2ig0So6hvPaz0IGcOvTHvUIlE2+prqAxLSQxZlU2stql1NqCCLdIiIN/i1DBEHUoElM9dBravbiAnKqgpi4IBkw+utSPIoBijDXJipSVV7MpOEJUAc5Qmm3BnUN+w3hteEieYKfRZSIUcXKMVf0u5wD4EwsUNVvZOtUT7A2GkffHjByWpHqvRBYrTV72a6j8zZ6W0DTE86Hn04bmyWX3Ri9WH7ZU6Q7h+ZHo0nHUAcsQvVhXRDZHChwiyi/hnPuOsSEF6Exk3o6Y9DT1eZ+6cASXk2Y9k+6EOQMDGm6WBK10wOQJCBwren86cPPWUcRAnTVjGcU1LBgs9FURiX/e6479yZcLwCBmTxiawEwrOcleuu12t3tbLv/N4RLYIBhYexm7Fcn4OJcn0+zc+s8/VfPeddZHAGN6TT8eGczHdR/Gts1/MzDkThr23zqrVfAMFT33Nx1RJsx1k5zuWILLnG/vsH+Fv5D4NTVcp1Gzo8AAAAAElFTkSuQmCC&labelColor=white)](https://huggingface.co/spaces/opendatalab/MinerU)\n", + "[![ModelScope](https://img.shields.io/badge/Demo_on_ModelScope-purple?logo=data:image/svg+xml;base64,PHN2ZyB3aWR0aD0iMjIzIiBoZWlnaHQ9IjIwMCIgeG1sbnM9Imh0dHA6Ly93d3cudzMub3JnLzIwMDAvc3ZnIj4KCiA8Zz4KICA8dGl0bGU+TGF5ZXIgMTwvdGl0bGU+CiAgPHBhdGggaWQ9InN2Z18xNCIgZmlsbD0iIzYyNGFmZiIgZD0ibTAsODkuODRsMjUuNjUsMGwwLDI1LjY0OTk5bC0yNS42NSwwbDAsLTI1LjY0OTk5eiIvPgogIDxwYXRoIGlkPSJzdmdfMTUiIGZpbGw9IiM2MjRhZmYiIGQ9Im05OS4xNCwxMTUuNDlsMjUuNjUsMGwwLDI1LjY1bC0yNS42NSwwbDAsLTI1LjY1eiIvPgogIDxwYXRoIGlkPSJzdmdfMTYiIGZpbGw9IiM2MjRhZmYiIGQ9Im0xNzYuMDksMTQxLjE0bC0yNS42NDk5OSwwbDAsMjIuMTlsNDcuODQsMGwwLC00Ny44NGwtMjIuMTksMGwwLDI1LjY1eiIvPgogIDxwYXRoIGlkPSJzdmdfMTciIGZpbGw9IiMzNmNmZDEiIGQ9Im0xMjQuNzksODkuODRsMjUuNjUsMGwwLDI1LjY0OTk5bC0yNS42NSwwbDAsLTI1LjY0OTk5eiIvPgogIDxwYXRoIGlkPSJzdmdfMTgiIGZpbGw9IiMzNmNmZDEiIGQ9Im0wLDY0LjE5bDI1LjY1LDBsMCwyNS42NWwtMjUuNjUsMGwwLC0yNS42NXoiLz4KICA8cGF0aCBpZD0ic3ZnXzE5IiBmaWxsPSIjNjI0YWZmIiBkPSJtMTk4LjI4LDg5Ljg0bDI1LjY0OTk5LDBsMCwyNS42NDk5OWwtMjUuNjQ5OTksMGwwLC0yNS42NDk5OXoiLz4KICA8cGF0aCBpZD0ic3ZnXzIwIiBmaWxsPSIjMzZjZmQxIiBkPSJtMTk4LjI4LDY0LjE5bDI1LjY0OTk5LDBsMCwyNS42NWwtMjUuNjQ5OTksMGwwLC0yNS42NXoiLz4KICA8cGF0aCBpZD0ic3ZnXzIxIiBmaWxsPSIjNjI0YWZmIiBkPSJtMTUwLjQ0LDQybDAsMjIuMTlsMjUuNjQ5OTksMGwwLDI1LjY1bDIyLjE5LDBsMCwtNDcuODRsLTQ3Ljg0LDB6Ii8+CiAgPHBhdGggaWQ9InN2Z18yMiIgZmlsbD0iIzM2Y2ZkMSIgZD0ibTczLjQ5LDg5Ljg0bDI1LjY1LDBsMCwyNS42NDk5OWwtMjUuNjUsMGwwLC0yNS42NDk5OXoiLz4KICA8cGF0aCBpZD0ic3ZnXzIzIiBmaWxsPSIjNjI0YWZmIiBkPSJtNDcuODQsNjQuMTlsMjUuNjUsMGwwLC0yMi4xOWwtNDcuODQsMGwwLDQ3Ljg0bDIyLjE5LDBsMCwtMjUuNjV6Ii8+CiAgPHBhdGggaWQ9InN2Z18yNCIgZmlsbD0iIzYyNGFmZiIgZD0ibTQ3Ljg0LDExNS40OWwtMjIuMTksMGwwLDQ3Ljg0bDQ3Ljg0LDBsMCwtMjIuMTlsLTI1LjY1LDBsMCwtMjUuNjV6Ii8+CiA8L2c+Cjwvc3ZnPg==&labelColor=white)](https://www.modelscope.cn/studios/OpenDataLab/MinerU)\n", + "[![Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/gist/myhloli/a3cb16570ab3cfeadf9d8f0ac91b4fca/mineru_demo.ipynb)\n", + "[![arXiv](https://img.shields.io/badge/MinerU-Technical%20Report-b31b1b.svg?logo=arXiv)](https://arxiv.org/abs/2409.18839)\n", + "[![arXiv](https://img.shields.io/badge/MinerU2.5-Technical%20Report-b31b1b.svg?logo=arXiv)](https://arxiv.org/abs/2509.22186)\n", + "[![Ask DeepWiki](https://deepwiki.com/badge.svg)](https://deepwiki.com/opendatalab/MinerU)" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "F9dnYLSDwQyR" + }, + "source": [ + "## 1.Install" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "Rw-hwakFwaeY" + }, + "source": "### Install via pip" + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/", + "height": 1000 + }, + "id": "_3EDplqlxKB_", + "outputId": "a4c96ba2-6dea-4ef0-aac7-955dbfac0598" + }, + "outputs": [], + "source": "!pip install --upgrade pip\n!pip install \"mineru[all]>=3.1.3\"" + }, + { + "cell_type": "markdown", + "metadata": { + "id": "rSfS77AisXH8" + }, + "source": [ + "### Check verison" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "MSyCKHmqCp_j", + "outputId": "eb0aad89-3261-4388-9dfc-2b6ce29a3030" + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "mineru, version 3.1.3\n" + ] + } + ], + "source": [ + "!mineru -v" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "Qh8YKU19CrQY" + }, + "source": [ + "## 2.Usage" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "3dZlkl3jDNKL" + }, + "source": [ + "### 2.1 Command line" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "mVRMM_FaDi2d" + }, + "source": [ + "#### **PDF**" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "Hyoz1B-tPXcM" + }, + "source": "#### **Converti un singolo PDF**\n\nSu Google Colab il file system è temporaneo: ogni volta che avvii una nuova sessione la cartella `/content/` è vuota.\nDevi quindi **caricare il tuo PDF prima di eseguire la conversione**.\n\n**Come caricare il file:**\n\n1. Nel pannello di sinistra clicca sull'icona **File**.\n2. Trascina il PDF nella cartella `/content/` (radice della sessione), oppure clicca il pulsante *Carica* (icona freccia su).\n3. Attendi che il caricamento sia completato: il file comparirà nell'elenco.\n\nIn alternativa puoi caricare via codice:\n\n```python\nfrom google.colab import files\nuploaded = files.upload() # apre il selettore file\n```\n\nUna volta caricato, il PDF si trova in `/content/.pdf`.\nSostituisci `mio_documento.pdf` nella cella seguente con il nome del tuo file:" + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "PDF_NAME = \"mio_documento.pdf\" # ← sostituisci con il nome del file che hai caricato\n", + "import os\n", + "STEM = os.path.splitext(os.path.basename(PDF_NAME))[0]" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "collapsed": true, + "id": "6RivAWteLeYM", + "outputId": "dc56bcc0-6e84-432a-8511-e461ff605a51" + }, + "outputs": [], + "source": "!mineru -p {PDF_NAME} -o ./output -b pipeline" + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/", + "height": 807 + }, + "id": "F6K4E8CfH6_V", + "outputId": "136175e0-c1b0-4c80-84c4-ce7e7c78e656" + }, + "outputs": [], + "source": [ + "import os, io\n", + "import fitz\n", + "import matplotlib.pyplot as plt\n", + "import numpy as np\n", + "from IPython.display import display\n", + "\n", + "os.system('pip install -q pymupdf')\n", + "\n", + "# Cerca il layout PDF nell'output di MinerU (backend auto o hybrid_auto)\n", + "candidates = [\n", + " f'./output/{STEM}/hybrid_auto/{STEM}_layout.pdf',\n", + " f'./output/{STEM}/auto/{STEM}_layout.pdf',\n", + "]\n", + "pdf_path = next((p for p in candidates if os.path.exists(p)), None)\n", + "\n", + "if pdf_path is None:\n", + " print('Layout PDF non trovato. Percorsi cercati:')\n", + " for p in candidates:\n", + " print(' ', p)\n", + "else:\n", + " doc = fitz.open(pdf_path)\n", + " n_pages = min(len(doc), 2) # mostra al massimo 2 pagine\n", + " fig, axes = plt.subplots(1, n_pages, figsize=(12 if n_pages > 1 else 6, 8))\n", + " if n_pages == 1:\n", + " axes = [axes]\n", + " for idx in range(n_pages):\n", + " pix = doc.load_page(idx).get_pixmap()\n", + " img = np.frombuffer(pix.samples, dtype=np.uint8).reshape(pix.height, pix.width, pix.n)\n", + " axes[idx].imshow(img)\n", + " axes[idx].axis('off')\n", + " axes[idx].set_title(f'Pagina {idx + 1}')\n", + " plt.tight_layout()\n", + " plt.show()\n", + " doc.close()\n", + " print(f'Layout da: {pdf_path}')\n" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "t53WETM90pKr" + }, + "source": "#### **Converti più PDF nella stessa cartella**\n\nConverte tutti i PDF presenti in `/content/` in un'unica esecuzione.\n\nCarica i file prima di eseguire la cella (trascina i PDF nel pannello File a sinistra oppure usa `files.upload()`).\n\nL'output di ogni PDF viene salvato in `./output//auto/`.\n\n> Usa questa cella al posto di quella precedente se hai più documenti da convertire." + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "4dqBGZiz0orA", + "outputId": "8233269d-d912-4dd3-9332-d26bd464db7c" + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "\u001b[32m2026-04-23 11:36:57.400\u001b[0m | \u001b[1mINFO \u001b[0m | \u001b[36mmineru.cli.client\u001b[0m:\u001b[36mrun_orchestrated_cli\u001b[0m:\u001b[36m874\u001b[0m - \u001b[1mStarted local mineru-api at http://127.0.0.1:37453\u001b[0m\n", + "\u001b[32m2026-04-23 11:37:00.858\u001b[0m | \u001b[1mINFO \u001b[0m | \u001b[36m__main__\u001b[0m:\u001b[36mcreate_app\u001b[0m:\u001b[36m260\u001b[0m - \u001b[1mRequest concurrency limited to 3\u001b[0m\n", + "Start MinerU FastAPI Service: http://127.0.0.1:37453\n", + "API documentation: http://127.0.0.1:37453/docs\n", + "\u001b[32mINFO\u001b[0m: Started server process [\u001b[36m10876\u001b[0m]\n", + "\u001b[32mINFO\u001b[0m: Waiting for application startup.\n", + "\u001b[32mINFO\u001b[0m: Application startup complete.\n", + "\u001b[32mINFO\u001b[0m: Uvicorn running on \u001b[1mhttp://127.0.0.1:37453\u001b[0m (Press CTRL+C to quit)\n", + "\u001b[32m2026-04-23 11:37:01.423\u001b[0m | \u001b[1mINFO \u001b[0m | \u001b[36mmineru.cli.client\u001b[0m:\u001b[36mrun_planned_task\u001b[0m:\u001b[36m771\u001b[0m - \u001b[1mSubmitting batch 1/1 | 2 documents, 19 pages in this batch | 19 pages total | task#1 [demo1, demo2]\u001b[0m\n", + "2026-04-23 11:37:06.518957: E external/local_xla/xla/stream_executor/cuda/cuda_fft.cc:467] Unable to register cuFFT factory: Attempting to register factory for plugin cuFFT when one has already been registered\n", + "WARNING: All log messages before absl::InitializeLog() is called are written to STDERR\n", + "E0000 00:00:1776944226.544180 10908 cuda_dnn.cc:8579] Unable to register cuDNN factory: Attempting to register factory for plugin cuDNN when one has already been registered\n", + "E0000 00:00:1776944226.552343 10908 cuda_blas.cc:1407] Unable to register cuBLAS factory: Attempting to register factory for plugin cuBLAS when one has already been registered\n", + "W0000 00:00:1776944226.572768 10908 computation_placer.cc:177] computation placer already registered. Please check linkage and avoid linking the same target more than once.\n", + "W0000 00:00:1776944226.572813 10908 computation_placer.cc:177] computation placer already registered. Please check linkage and avoid linking the same target more than once.\n", + "W0000 00:00:1776944226.572818 10908 computation_placer.cc:177] computation placer already registered. Please check linkage and avoid linking the same target more than once.\n", + "W0000 00:00:1776944226.572821 10908 computation_placer.cc:177] computation placer already registered. Please check linkage and avoid linking the same target more than once.\n", + "2026-04-23 11:37:06.578247: I tensorflow/core/platform/cpu_feature_guard.cc:210] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.\n", + "To enable the following instructions: AVX2 AVX512F FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags.\n", + "\u001b[32m2026-04-23 11:37:11.673\u001b[0m | \u001b[1mINFO \u001b[0m | \u001b[36mmineru.backend.pipeline.pipeline_analyze\u001b[0m:\u001b[36mdoc_analyze_streaming\u001b[0m:\u001b[36m183\u001b[0m - \u001b[1mPipeline processing-window multi-file run. doc_count=2, total_pages=19, window_size=64, total_batches=1\u001b[0m\n", + "\u001b[32m2026-04-23 11:37:17.154\u001b[0m | \u001b[1mINFO \u001b[0m | \u001b[36mmineru.backend.pipeline.pipeline_analyze\u001b[0m:\u001b[36mdoc_analyze_streaming\u001b[0m:\u001b[36m235\u001b[0m - \u001b[1mPipeline processing window batch 1/1: 19/19 pages, batch_pages=19, doc_slices=doc0:1-13,doc1:1-6\u001b[0m\n", + "\u001b[32m2026-04-23 11:37:17.158\u001b[0m | \u001b[1mINFO \u001b[0m | \u001b[36mmineru.backend.pipeline.pipeline_analyze\u001b[0m:\u001b[36mbatch_image_analyze\u001b[0m:\u001b[36m328\u001b[0m - \u001b[1mGPU Memory: 15 GB, Batch Ratio: 4. \u001b[0m\n", + "\u001b[32m2026-04-23 11:37:17.160\u001b[0m | \u001b[1mINFO \u001b[0m | \u001b[36mmineru.backend.pipeline.model_init\u001b[0m:\u001b[36m__init__\u001b[0m:\u001b[36m207\u001b[0m - \u001b[1mDocAnalysis init, this may take some times......\u001b[0m\n", + "Fetching 7 files: 100% 7/7 [00:00<00:00, 72494.14it/s]\n", + "Fetching 3 files: 100% 3/3 [00:00<00:00, 20004.63it/s]\n", + "Fetching 1 files: 100% 1/1 [00:00<00:00, 7244.05it/s]\n", + "Fetching 1 files: 100% 1/1 [00:00<00:00, 8160.12it/s]\n", + "Fetching 1 files: 100% 1/1 [00:00<00:00, 7653.84it/s]\n", + "Fetching 1 files: 100% 1/1 [00:00<00:00, 21183.35it/s]\n", + "Fetching 1 files: 0% 0/1 [00:00