User-agent: AhrefsBot Disallow: / User-agent: Twitterbot Disallow: User-agent: * Disallow: /collection/zone Disallow: /collection/category #Disallow: /newspaper/browse #Disallow: /newspaper/annotations/search #Disallow: /newspaper/fulltext/history #Disallow: /newspaper/fulltext/diff # Disabled as this causes Google to classify newpspaper articles as soft 404s - AO 20210624 #Disallow: /newspaper/image/info Disallow: /newspaper/rendition Disallow: /newspaper/lists/search Disallow: /workHistory Disallow: /goto Disallow: /work/*/item/*/bookshops/* Disallow: /people/924238 Disallow: /newspaper/fulltext/history/paginated Disallow: /newspaper/article/126890100 Disallow: /newspaper/article/247066760 Disallow: /newspaper/article/230435926 Disallow: /newspaper/article/122273608 Disallow: /work/161916990 Disallow: /newspaper/article/122281910 Disallow: /work/115016956 Disallow: /newspaper/article/240722057 Disallow: /newspaper/article/240703111 Disallow: /newspaper/article/240703109 Disallow: /newspaper/article/122275843 Sitemap: https://trove.nla.gov.au/sitemap-index.xml Sitemap: https://trove.nla.gov.au/newspaper-sitemap-index.xml Sitemap: https://trove.nla.gov.au/cms/sitemap.xml