Watson Discovery

 View Only
  • 1.  How to solve: Excerpt unavailable. in Watson Discovery V2 (beta) ?

    Posted Fri June 11, 2021 11:37 AM
    Hi,

    When uploading JSON files in my collection, I get Excerpt unavailable. in the search results (both for passages and fields)
    The text is analyzed and entities are extracted.

    This does not happen when I put the same text  in a .txt or docx file.

    What can I do to get an excerpt in my search results ?

    Thanks 


    ------------------------------
    Michel Jonker
    ------------------------------

    #WatsonDiscovery


  • 2.  RE: How to solve: Excerpt unavailable. in Watson Discovery V2 (beta) ?

    Posted Mon June 14, 2021 05:07 PM
    Hi Michel,

    Excerpt unavailable is seen when the field you have chosen to display returned passages or results from does not contain a match to the search term.   
    For example, if you ingest a json file that has three fields and you set up the UI to display results from `field 1`, but the search term used matches a term in `field 3`, you'll see `excerpt unavailable`because there is no passage or highlight that matches the term in `field 1`.

    You are not likely to see this issue occur in a .txt or .docx file as you mention because the text from these types of files is converted into a field called `text`and the UI searches this `text` field by default.

    ------------------------------
    JOHN PECORARI
    ------------------------------



  • 3.  RE: How to solve: Excerpt unavailable. in Watson Discovery V2 (beta) ?

    Posted Mon June 14, 2021 05:08 PM
    Hi Michel,

    Excerpt unavailable is seen when the field you have chosen to display returned passages or results from does not contain a match to the search term.   
    For example, if you ingest a json file that has three fields and you set up the UI to display results from `field 1`, but the search term used matches a term in `field 3`, you'll see `excerpt unavailable`because there is no passage or highlight that matches the term in `field 1`.

    You are not likely to see this issue occur in a .txt or .docx file as you mention because the text from these types of files is converted into a field called `text`and the UI searches this `text` field by default.

    ------------------------------
    JOHN PECORARI
    ------------------------------



  • 4.  RE: How to solve: Excerpt unavailable. in Watson Discovery V2 (beta) ?

    Posted Tue June 15, 2021 03:03 AM

    Thanks @JOHN PECORARI.

    Where can I specify this field ?
    I want to display passages and not the entire field (which does not make a difference by the way if I choose to display the content of the field)
    When I change the title field to the field that contains the content, I do get the content as the title, so the field is available.


    ​​​​

    ------------------------------
    Michel Jonker
    ------------------------------



  • 5.  RE: How to solve: Excerpt unavailable. in Watson Discovery V2 (beta) ?

    Posted Tue June 15, 2021 03:18 AM
    @JOHN PECORARI I have also renamed the field in the JSON file (from "Abstract" to "text"). But this has no effect.​

    ------------------------------
    Michel Jonker
    ------------------------------



  • 6.  RE: How to solve: Excerpt unavailable. in Watson Discovery V2 (beta) ?

    Posted Tue June 15, 2021 08:31 AM
    Edited by System Fri January 20, 2023 04:46 PM
    When I inspect the document (JSON-view) the text field has [  ] , as if it is a JSONArray. Other fields like the title, don't have these square brackets



    ------------------------------
    Michel Jonker
    ------------------------------



  • 7.  RE: How to solve: Excerpt unavailable. in Watson Discovery V2 (beta) ?

    Posted Tue June 15, 2021 09:12 AM
    Edited by System Fri January 20, 2023 04:20 PM
    Hi.  I just tested passage retrieval on one of my collections that has an array valued field like the one you show, and passage retrieval does work fine on mine -- so I don't think the arrays (brackets) are causing a problem in your case.  I would have expected that renaming the field in the JSON file (from "Abstract" to "text") would solve your problem, so I would take a closer look at the schema in the project where you have the JSON field named "text" to make sure it looks just like the one in your screen shot above with "Abstract" as the filed name -- maybe something when wrong in that conversion.

    Incidentally, the reason why this is so clunky is that there is an implicit assumption in the tooling UX that people who depend heavily on the tooling are ingesting ordinary document files like .txt, .docx, .pdf, .html, etc.  For those kinds of documents, Discovery controls the structure and can ensure that the main field for text is named "text".  In general, we tend to assume that people who have pre-structured their content in JSON are more likely to do testing via the API either using a programming language or an API testing tool like Postman or Insomnia.  If you test your projects directly via the API, you can set the value of the fields array of the passage block of your query directly, e.g.:

    POST /v2/projects/{project_id}/query
    {"count":10,
    "natural_language_query":"What causes hypertension",
    "passages": {"enabled": true, "fields": ["JournalTitle", "Abstract", "Title"]}
    }

    ------------------------------
    Bill Murdock
    Principal Research Staff Member
    IBM Watson
    ------------------------------



  • 8.  RE: How to solve: Excerpt unavailable. in Watson Discovery V2 (beta) ?

    Posted Tue June 15, 2021 11:43 AM

    Thanks @Bill Murdock, it is indeed our intention to use the API to upload content and query​ the results.

    And behold:

    "passages": [
    {
    "passage_text": "A Consensus on the Diagnosis and Treatment of Acromegaly Comorbidities: An Update.",
    "passage_score": 6.7850114453105079,
    "document_id": "8cb1d6bc91b2bcc1b5f32e3326b467e6",
    "collection_id": "ff746713-cd4e-9383-0000-0179eba23f5b",
    "start_offset": 0,
    "end_offset": 82,
    "field": "Title"
    },
    {
    "passage_text": "Acromegaly pathogenesis and treatment.",
    "passage_score": 6.1693145079515,
    "document_id": "be4a3bdcc01625f6c36a14d0212b110b",
    "collection_id": "ff746713-cd4e-9383-0000-0179eba23f5b",
    "start_offset": 0,
    "end_offset": 38,
    "field": "Title"
    },
    {
    "passage_text": "Pediatric Hypothyroidism: Diagnosis and Treatment.",
    "passage_score": 6.136437798609899,
    "document_id": "2f3ac59fde23ad65ba8237dd7189df53",
    "collection_id": "ff746713-cd4e-9383-0000-0179eba23f5b",
    "start_offset": 0,
    "end_offset": 50,
    "field": "Title"
    },
    {
    "passage_text": "Most of these comorbidities can be prevented or delayed with adequate disease treatment and, more recent studies with the use of modern treatments of acromegaly, have shown a change in the severity and prevalence of these complications. In addition",
    "passage_score": 5.992064038038074,
    "document_id": "0f76b923965f525483a71956a5a8b165",
    "collection_id": "ff746713-cd4e-9383-0000-0179eba23f5b",
    "start_offset": 178,
    "end_offset": 426,
    "field": "Abstract"
    },
    {
    "passage_text": "Careful evaluation will usually reveal the etiology of congenital hypothyroidism, which may inform treatment and prognosis. Early and adequate treatment with levothyroxine results in excellent neurodevelopmental outcomes for most patients with congenital hypothyroidism.",
    "passage_score": 5.1689006025044599,
    "document_id": "b8dce7a05c54b1e3e5658b815f1a7170",
    "collection_id": "ff746713-cd4e-9383-0000-0179eba23f5b",
    "start_offset": 458,
    "end_offset": 728,
    "field": "Abstract"
    }



    ------------------------------
    Michel Jonker
    ------------------------------