feature(extract-python:docling): avoid locking the GIL by using page batches and result buffering

[ClemDoum]

Description

When running Docling parsing the GIL was locked trigger heartbeat timeouts when processing long documents.
Indeed, while some docling are async calls from the Python interpreter pov (torch or numpy calls), other are truely blocking like calling the pdf backend or other pure python CPU ops. Call to_thread(converter.convert_all) is blocking in these case because we're processing potentially hundred of pages at the same time.

This PR aims at avoid this GIL lock and optimize GPU inference by processing batches of constant size in terms of pages.
All documents are split into pages and convert_all is called on a constant size batch of pages.

More precisely, batching operates at 2 differente level:

• page_batch_size is the number of pages concurrent sent to the GPU (or other processing unit) and processed in parallel
• max_page_batches is the number of consecutive batch which are sent to the Docling converter and processed sequentially

The second level of batching avoids calling the converter for a single batch of pages. The underlying reason is that calling converter.convert_all as a fixed cost that we'd like to mutualize between batches (calling the PDF backend for instance).

Additionnally some custom logic had to be implement in order to deal with the fact that converter.convert_all(docs, page_range) takes a single page_range arguments for all processed documents.

The choice made here was to split processing of small and large docs (n_pages < (or >) max_pages = page_batch_size * max_page_batches):

• small documents have less than max_pages pages, we hence try to group their pages in batches of max_pages pages using bin filling
• largs documents have more than max_pages pages, we split them in chunks of max_pages and process them by chunk. For their last chunk of pages which is usually less than max_pages pages, we buffer them by page range in order to process the remainder of different documents together

Because this batching strategy process documents unordered, a result buffer was implemented to aggregate partial results form pages. Since results can be large in size, we offload buffering to the filesystem when necessary.

Changes

extract-core

Added

• added ResultBufferConfig
• added BatchConcurrencySettings to the DoclingSettings to set docling page concurrency (and other options)

Changed

extract-python

Added

• implemented page parallelism and results buffering in the DoclingPipeline