Results 1 to 3 of 3

Thread: Can I use Trove API to extract search string results from Newspapers?

  1. #1
    Trove user
    Join Date
    Nov 2010
    Location
    Canberra
    Posts
    44

    Can I use Trove API to extract search string results from Newspapers?

    You may take it from the following question I know nothing about Trove API. But, ever the naive optimist, I want to know if the following scenario is possible, and how do I do it. So here goes:

    I want to pull all the results from a text search of Trove newspapers into a text file for later reference. For example, I search for the name of a ship, let's say "Port Bowen", and get 4,233 hits. I have these hits (search results) sorted by ascending date. Now I want to extract all these hits for "Port Bowen", plus the next 10 words in each article that occur directly after the mention of "Port Bowen". What I would hope, at the end of the process, is to have a text file, with each of the hits now consisting of a 1-line header of newspaper title and date, along with the in-article data. This text file would be a lot easier to scan through for useful chronological info than having to click into each newspaper article 4,233 times.

    So I imagine a text dump with my 4,233 records, a snippet of which might look roughly like:

    Mercury, Wed 27 Oct 1920 Page 4
    Port Bowen departed Sydney for Hobart at 8 a.m. yesterday after loading ...

    Mercury, Thu 28 Oct 1920 Page 4
    Port Bowen is due to arrive at Hobart on ocr jibberish jibberish ...

    Mercury, Fri 29 Oct 1920 Page 5
    Port Bowen will leave for London next week etc 10 words etc ...

  2. #2
    Trove user
    Join Date
    Nov 2010
    Location
    Canberra
    Posts
    44
    Well, I've got a basic API search going for the above, which I've constructed using various documentation found by Googling "Trove API". It's getting results.

    However, I cannot see a way yet to return a snippet that begins with my query term. Is that possible?

    Actually, the snippets returned from my API call are different from the snippets returned from a search within Trove newspapers itself. The API snippet starts with the first words of the article following the title, whereas the Trove search snippet is centred on the search term.

    I've tried searching the documentation, and it seems so far that the behaviour of the snippets cannot to be controlled by any sort of command
    Last edited by stringybark; 29-05-2019 at 04:48 PM.

  3. #3
    Trove user
    Join Date
    Nov 2010
    Location
    Canberra
    Posts
    44
    Oh well, I'm talking to myself.

    So, best I can see how to solve my particular problem, is that I set up my api call to have an &include=articletext command,

    and that gives me the whole article text for all articles that include my search term. Then it seem the wheels fall off the API, so it's time to jump to a different process to finish off.

    What I'm doing is saving the API output as a txt file, going to Notepad++ (which is opensource text editor), and within that, use regex expressions to get rid of all the text I don't need, so that I'm just down to what I envisaged in my first post above.

    So if anyone reads this, and has a better idea, i.e. a more efficient process, feel free to make your suggestion.

    And, geez, I'm going to be sitting here a loooong time, getting 4000+ records off the API server just 100 at a time, readjust the call string, run again, select-all-copy, paste into the master text file, and repeat. Are there any circumstances the 100 limit can be relaxed for a one-off query?

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •