Page 1 of 2 12 LastLast
Results 1 to 10 of 20

Thread: Graphing your newspaper search results

  1. #1
    Trove user wragge's Avatar
    Join Date
    Jun 2010
    Location
    Canberra
    Posts
    63

    Graphing your newspaper search results

    I've added a little script to my TroveTools Python library that takes a newspapers search query and then extracts for each year both the number of matching articles and the proportion this represents of the total number of articles.

    The results are saved as JSON allowing them to be easily graphed using tools like jqPlot

    Here's an example, a search for articles containing any of the following keywords: 'chinese', 'chinaman', 'chinamen'. (The historian of Chinese Australia in my house successfully predicted the 1888 peak!)



    Or what about 'floods':



    I put out a call on Twitter for suggested searches and I've been adding them to a Trove Experiments Flickr set.

    I might do a few more, so feel free to add some requests to this thread and I'll try and run them.

    I'll be adding a blog post with more details soon to discontents.

    Have fun!
    Tim Sherratt
    @wragge on Twitter
    Words at discontents.com.au
    Experiments at wraggelabs.com and labs.nma.gov.au

  2. #2
    Trove user wragge's Avatar
    Join Date
    Jun 2010
    Location
    Canberra
    Posts
    63
    Just a follow-up, I can also do comparisons of keyword frequency. Here's a case where I compare the occurance of the word 'Australian' versus 'British' or 'Briton' (as proportions of the total number of articles):



    If you have any ideas for interesting graphs, post them below.
    Tim Sherratt
    @wragge on Twitter
    Words at discontents.com.au
    Experiments at wraggelabs.com and labs.nma.gov.au

  3. #3
    Trove user wragge's Avatar
    Join Date
    Jun 2010
    Location
    Canberra
    Posts
    63
    Another update...

    I've created dynamic versions of the graphs developed so far. This means that you can zoom in and. more importantly, click on any of the date points to bring up the first 20 results from Trove for that year.

    http://wraggelabs.com/shed/trove/graphs/

    This allows you to explore the peaks and troughs, indeed the graphs become a whole new interface to Trove itself.

    I'm still taking requests...
    Tim Sherratt
    @wragge on Twitter
    Words at discontents.com.au
    Experiments at wraggelabs.com and labs.nma.gov.au

  4. #4

    occurrence of 'snow' in South Australian newspapers

    You invited some suggestions (especially meteorological) for further graphs:
    The occurrence of snow here in South Australia is a relatively unusual event. Nevertheless there are many occasions when snow has fallen in our past 175 years. A graph of the frequency of SA newspaper articles mentioning snow might be enlightening. Of course, there's the risk of finding people called Snow, or events at Snowtown, but I suppose we could eliminate words such as snowtown from the search.
    Regards, Frank

  5. #5
    Wow!! You have the historians all in a lather — can I suggest a comparative search on COFFEE and TEA. This should map an interesting cultural and consumer shift from the late nineteenth century to the postwar years.
    Thanks and congrats on a fab tool.
    Andy

  6. #6
    Member
    Join Date
    Mar 2011
    Location
    Wentworth @ Murray-Darling Junction
    Posts
    3
    I'm also interested in snow in dry places - have an oral history source for snow at Tarella (near White Cliffs, NSW) ca 1910-1916.

  7. #7
    Trove user wragge's Avatar
    Join Date
    Jun 2010
    Location
    Canberra
    Posts
    63
    @Jayma Ok, here's Beverage wars - tea versus coffee.

    It doesn't really show any major shifts (the lines are pretty much in sync). Of course, if we had post 1954 data it could be different!
    Last edited by wragge; 17-03-2011 at 11:27 AM.
    Tim Sherratt
    @wragge on Twitter
    Words at discontents.com.au
    Experiments at wraggelabs.com and labs.nma.gov.au

  8. #8
    Trove user wragge's Avatar
    Join Date
    Jun 2010
    Location
    Canberra
    Posts
    63
    @Frankwll - I need to make some changes to the script to make it easy to do state by state breakdowns. I'll get on to it shortly.
    Tim Sherratt
    @wragge on Twitter
    Words at discontents.com.au
    Experiments at wraggelabs.com and labs.nma.gov.au

  9. #9
    Trove user wragge's Avatar
    Join Date
    Jun 2010
    Location
    Canberra
    Posts
    63

    Thumbs up

    Yvonne Perkins has posted a nice account of using my graph tools on her blog.

    It's great to see the tools in use, and even better to see them raising questions about the use of newspapers as a primary source. As Yvonne says:

    I see this type of analysis as a starting point for research not an end in itself.
    Tim Sherratt
    @wragge on Twitter
    Words at discontents.com.au
    Experiments at wraggelabs.com and labs.nma.gov.au

  10. #10
    Trove user mogedon's Avatar
    Join Date
    Jan 2011
    Location
    Sydney
    Posts
    41
    Hi wragge, thanks for all the hard work—this a fantastic tool that I'm still trying to get my head around. Any chance of a mac version for your scraper soon??

    By the way, I'd love to see a graph for "Australian Federation" or "Australia" and "Federation". It would be interesting to see if the phrase deimply deflated after 1901 or continued to carry on in the Australia lexicon.

Tags for this Thread

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •