watsonx.ai

watsonx.ai

A one-stop, integrated, end- to-end AI development studio

 View Only
Back to discussions
Expand all | Collapse all

LLMs "The PDF Problem" - can LLM deliver deterministic answers bases on deterministic data structures stored in PDFs?

  • 1.  LLMs "The PDF Problem" - can LLM deliver deterministic answers bases on deterministic data structures stored in PDFs?

    Posted 03/30/26 10:56 AM

    Guy, I am not sure if this is good thread for me, nevertheless let me ask you this:

    • PDF was designed to look the same across devices, was not designed for to carry semantic information
    • all LLMs have a problem with this PDFs "disability" as they are not machine readable

    The question:

    • imagine I attach PDF tags or structural JSON file that houses sematic structure of the document and they tell LLMs what is a table, list, header, footer, note etc.
    • imagine these documents are corporate data so there a thousand of them (so RAG is the only way to I believe)
    • can I expect that if I provide deterministic data alongside with visual PDF content to LLM so literally no ORC recognition, no vision-capable LLM processes are needed (as everything is in machine readable PDF tags or JSON), can I expect deterministic results or output? 

    Thanks much.

    Dusan Baranec-


    #GenerativeAI

    ------------------------------
    DusanBaranec Foxit
    ------------------------------


  • 2.  RE: LLMs "The PDF Problem" - can LLM deliver deterministic answers bases on deterministic data structures stored in PDFs?

    Posted 03/31/26 07:39 AM

    Hi Dusan,

    Fundamentally not... LLMs are statistical by nature.😉

    Note, however, that what you describe is usually no longer needed today. Tools like Docling (from IBM Research) can reliably extract (using VLM and ML) most of what you describe and make that directly available to your RAG pipeline.
     If you really want to stick to your initial approach, you would need the LLM to generate "signals" of where to extract data, and then use a deterministic "data extractor" algorithm.

    Cheers,



    ------------------------------
    Lionel Clavien, PhD
    IBM Champion for Power
    CTO & Co-founder
    InnoBoost SA
    Lausanne, Switzerland
    ------------------------------



  • 3.  RE: LLMs "The PDF Problem" - can LLM deliver deterministic answers bases on deterministic data structures stored in PDFs?

    Posted 03/31/26 12:23 PM

    Thank you Lionel for you reply and time. Yes, all you wrote makes sense, thanks again!



    ------------------------------
    DusanBaranec Foxit
    ------------------------------

    Original Message


  • 4.  RE: LLMs "The PDF Problem" - can LLM deliver deterministic answers bases on deterministic data structures stored in PDFs?

    Posted 04/01/26 02:14 AM

    You're welcome.

    If you need any help working with Docling/RAG, DM me. ;)



    ------------------------------
    Lionel Clavien, PhD
    IBM Champion for Power
    CTO & Co-founder
    InnoBoost SA
    Lausanne, Switzerland
    ------------------------------

    Original Message


  • 5.  RE: LLMs "The PDF Problem" - can LLM deliver deterministic answers bases on deterministic data structures stored in PDFs?

    Posted 03/31/26 10:27 AM

    Even with perfect input, LLMs generate probabilistically-temperature >0 introduces variation, and reasoning over retrieved context can lead to paraphrasing or edge-case interpretations.  Outputs become "deterministic enough" (e.g., 95%+ consistency) via techniques like grammar-constrained decoding (llama.cpp JSON mode), structured prompting, or verification loops, but never 100% without rule-based post-processing.



    ------------------------------
    Ton Hoang
    ------------------------------



  • 6.  RE: LLMs "The PDF Problem" - can LLM deliver deterministic answers bases on deterministic data structures stored in PDFs?

    Posted 03/31/26 12:22 PM

    Thanks for the answer and your time Ton! As many times before, one problem evolved into several ones :)



    ------------------------------
    DusanBaranec Foxit
    ------------------------------

    Original Message


  • 7.  RE: LLMs "The PDF Problem" - can LLM deliver deterministic answers bases on deterministic data structures stored in PDFs?

    Posted 04/01/26 08:56 AM

    My two cents: in one of my projects, we upload the PDF files to COS and store vectors plus metadata in a vector database, with the link to the raw file fro each chunk. This makes it possible to search the content and return answers along with the link and page number of the original document. From there, you can show the exact excerpt from the source document in a deterministic way, the LLM-generated answer, or both together, with a disclaimer for the user.

    RAG searching Vector DB with metadata with link of PDF document part from the chunk retrieve


    ------------------------------
    Sergio Gama
    Sr AI Solution Architect
    TD Synnex
    São Paulo
    +55 11 94581-7571
    ------------------------------



  • 8.  RE: LLMs "The PDF Problem" - can LLM deliver deterministic answers bases on deterministic data structures stored in PDFs?

    Posted 04/01/26 10:15 AM

    Thanks much Sergio :) it seems like NotebookLM atchitecture to me. Am I right?



    ------------------------------
    DusanBaranec Foxit
    ------------------------------

    Original Message


  • 9.  RE: LLMs "The PDF Problem" - can LLM deliver deterministic answers bases on deterministic data structures stored in PDFs?

    Posted 05/05/26 09:42 AM

    Hi Dusan

    This may not be exactly what you are after but might guide you. Papyrus Software has an AI Agentic solution platform called Converse which utilises watsonx and can understand semantic information from pdfs.... (see pic below)  It is an ontology-scoped retrieval solution from document understanding. RAG is brittle in specialized domains: chunking splits context, retrievers miss jargon, and models hallucinate. Adding a structured layer helps, but reliability hinges on knowledge-graph quality. A joined up approach-IBM watsonx.ai together with the Papyrus Converse Ontology-binds retrieval to a domain model and orchestrates ontology-driven queries with semantic search and explicit provenance. In combination, they reduce fragmentation and hallucination, improve recall and explainability, and deliver traceable, verifiable document answers.

    Papyrus Converse Platform
    Rupert Fallows
    ISIS Papyrus UK Ltd
    Watership Barn, Kingsclere Park,
    Kingsclere, Hampshire England


    ------------------------------
    Rupert Fallows
    Client Partner
    ISIS Papyrus UK Ltd
    Kingsclere
    +44 (0)1635299849
    ------------------------------



  • 10.  RE: LLMs "The PDF Problem" - can LLM deliver deterministic answers bases on deterministic data structures stored in PDFs?

    Posted 05/07/26 03:40 AM

    Thanks Rupert for your time and effort. It's really complex topic with no "golden bullet" solution and with too many variables in play.

    Dusan-



    ------------------------------
    DusanBaranec Foxit
    ------------------------------

    Original Message