Building a prototype for quality information retrieval from the World Wide Web

Author
Kc, Milly Wei-Tsen
Subjects
Internet searching.; Information organization.; Web search engines.
Summary
Given the phenomenal rate by which the World Wide Web is changing, retrieval methods and quality assurance have become bottleneck issues for many information retrieval services on the Internet, e.g. Web search engine designs. In this thesis, approaches that increase the efficiency of information retrieval methods, and provide for quality assurance of information obtained from the Web, are developed through the implementation of a quality-focused information retrieval system. A novel approach to the retrieval of quality information from the Internet is introduced. Implemented as a component of a vertical search application, this results in a focused crawler which is capable of retrieving quality information from the Internet. The three main contributions of this research are: (1) An effective and flexible crawling application that is well-suited for information retrieving tasks on the dynamic World Wide Web (WWW) is implemented. The resulting crawling application (crawler) is designed after having observed the dynamics of the web evolution through regular monitoring of the WWW; it also addresses the shortcomings of some existing crawlers, therefore presenting itself as a practical implementation. (2) A mechanism that converts human quality judgement through user surveys into an algorithm is developed, so that user perceptions of a set of criteria which may lead to determination of the quality content on the web page concerned, can be applied to a large amount of Web documents with minimal manual effort. This was obtained through a relatively large user survey which was conducted in a collaborative research work with Dr Shirlee-Ann Knight of Edith Cowan University. The survey was conducted to determine what criteria Web documents are perceived to meet to qualify as a quality document. This results in an aggregate numeric score for each web page between 0 and 1 respectively indicating that it does not meet any quality criteria, or that it meets all quality criteria perfectly. (3) This research proposes an approach to predict the quality of a web page before it is retrieved by a crawler. The approach allows its incorporation into a vertical search application which focuses on the retrieval of quality information. Experimental results on real world data show that the proposed approach is more effective than any other brute force approaches which have been published so far. The proposed methods produce a numerical quality score for any text based Web document. This thesis will show that such score can also be used as a web page ranking criterion for horizontal search engines. As part of this research project, this ranking scheme has been implemented and embedded into a working search engine. The observed user feedback confirms that search results when ranked by quality score, satisfy user needs more satisfactorily than when ranked by other popular ranking schemes such as PageRank or relevancy ranking. It is also investigated whether the combination of quality score with existing ranking schemes can further enhance the user experience with search engines.
Bookmark
http://trove.nla.gov.au/work/28866070
Work ID
28866070
Editions
2 versions of this work are listed below
Actions

A version is available from...

How do I enable "My libraries"?

In order to set up a list of libraries that you have access to, you must first login or sign up. Then set up a personal list of libraries from your profile page by clicking on your user name at the top right of any screen.

This single location:

Show 0 more libraries...

This single location in New South Wales:

Show 0 more libraries...

Found at these bookshops

Searching - please wait...

No Comments or reviews for this work

 

Add a comment



Work history

Versions / Editions

versions:
2 (out of 2)
Expand all / Collapse all
Title, Author, Edition Date Language Format Libraries
Building a prototype for quality information retrieval from the World Wide Web / by Milly Wei-Tsen Kc.

258 pages
2009 English Thesis 1 & Possibly online
Building a prototype for quality information retrieval from the World Wide Web
Faculty of Informatics 2009-01-01 English Thesis 1 & Online

Possibly related

Digitised newspapers and more

Searching...

Minimise

Diaries, letters, archives

Searching...

Minimise

Archived websites (1996 – now)

Searching...

Minimise

Pictures, photos, objects

Searching...

Minimise

Books

Searching...

Minimise

Music, sound and video

Searching...

Minimise

People and organisations

Searching...

Minimise

Journals, articles and data sets

Searching...

Minimise

Maps

Searching...

Minimise

Lists

Searching...

Minimise
  1. Internet Directories by Australian Librarians
    Created by asearchinglibrarian, 26 items.

    Looking for a reliable Australian Internet Directory, but can't find one? Have you tried the library? Librarians are creating selected lists of internet sites for inclusion on their websites, and th...

    Keywords: Australian Internet Directories, Directories, Internet directories, Library Subject Guides, Subject Guides, Websites
  2. Library Studies
    Created by asearchinglibrarian, 179 items.

    Primarily, this list is for items available on the topic of library studies from the Canberra Institute of Technology, the ...

    Keywords: Information science, Library science, LIS
  3. Digitised newspaper search engines, archives, and media directories, on the net
    Created by asearchinglibrarian, 16 items.

    A list of search engines for free access news items all over the world...
    Also includes links to media associations, and directories of media sites.

    Keywords: Digitisation, free, Journalism, Journalist, Library, New Zealand, Newspaper, Open access, Reference
  1. World Zionist Organization Organization and Information Dept
    ... In 1968 or 9 the Information Dept. and the Organization Dept. of the World Zionist Organization ...
    This resource may be of limited relevance to your query (score: 0.008)

    This resource may be of limited relevance to your query (score: 0.008)

  2. World Zionist Organization Information Dept
    ... In 1968 or 9 the Information Dept. and the Organization Dept. of the World Zionist Organization ...
    This resource may be of limited relevance to your query (score: 0.008)

    This resource may be of limited relevance to your query (score: 0.008)

  3. North Atlantic Treaty Organization Information Directorate
    This resource may be of limited relevance to your query (score: 0.007)

    This resource may be of limited relevance to your query (score: 0.007)

  1. IRE transactions on information theory
    IRE Professional Group on Information Theory
    [ Periodical, Map, Book : 1955-1962 ]
    View online (access conditions)
    At 12 libraries
    This resource may only be vaguely related to your query: matching terms may only have been found in the full text contents of the resource (score: 0.024)

    This resource may only be vaguely related to your query: matching terms may only have been found in the full text contents of the resource (score: 0.024)

  2. INFOMINE scholarly Internet resource collections
    University of California (System). Regents
    [ Map : 1990 ]
    Possibly online
    This resource may only be vaguely related to your query: matching terms may only have been found in the full text contents of the resource (score: 0.013)

    This resource may only be vaguely related to your query: matching terms may only have been found in the full text contents of the resource (score: 0.013)

  3. WHO global atlas of traditional, complementary and alternative medicine
    Ong, C. K
    [ Book, Map : 2005 ]
    At 19 libraries
    This resource may only be vaguely related to your query: matching terms may only have been found in the full text contents of the resource (score: 0.011)

    This resource may only be vaguely related to your query: matching terms may only have been found in the full text contents of the resource (score: 0.011)

  1. Net search : http:www
    [ Poster, chart, other : 2000 ]
  2. Searching for Cancer Information on...
    Bader, Judith L...
    [ Image : 2003-2005 ]
  3. Some Benchmark Searches for Testing...
    Johnson, E Dian...
    [ Image : 2000-2005 ]
  4. Adolescents Searching for Health In...
    Hansen, Derek L...
    [ Image : 2003-2005 ]
  5. Improving Web Searches: Case Study ...
    Koo, Malcolm; S...
    [ Image : 2003-2005 ]
  6. Search hints : Boolean operators, p...
    [ Poster, chart, other : 1999 ]
  1. What color is your parachute? : a practical manual for job-hunters & careerchangers / by Richard Nelson Bolles
    Bolles, Richard Nelson
    [ Book, Periodical, Audio book : 1970-2012 ]
    Languages: English, Chinese, French, Undetermined
    read online at Open Library/Internet Archive
    View online
    At 279 libraries
    This resource may only be vaguely related to your query: matching terms may only have been found in the full text contents of the resource (score: 0.033)

    This resource may only be vaguely related to your query: matching terms may only have been found in the full text contents of the resource (score: 0.033)

  2. R 261 Historic Records Search Committee Records
    Historic Records Search Committee
    [ Unpublished : 1987 ]
    Possibly online
    At State Library of QLD
    This resource may only be vaguely related to your query: matching terms may only have been found in the full text contents of the resource (score: 0.017)

    This resource may only be vaguely related to your query: matching terms may only have been found in the full text contents of the resource (score: 0.017)

  3. Archives of the National Information Standards Organization (NISO)
    Avram, Henriette D.
    [ Unpublished : 1901-1949 ]
    View online
    This resource may only be vaguely related to your query: matching terms may only have been found in the full text contents of the resource (score: 0.013)

    This resource may only be vaguely related to your query: matching terms may only have been found in the full text contents of the resource (score: 0.013)

  1. The Invisible Web : uncovering information sources search engines can't see / Chris Sherman and Gary Price
    Sherman, Chris
    [ Book : 2001 ]
    Possibly online
    At 103 libraries
    This resource may be of limited relevance to your query (score: 0.293)

    This resource may be of limited relevance to your query (score: 0.293)

  2. Search engines for the World Wide Web / Alfred and Emily Glossbrenner
    Glossbrenner, Alfred
    [ Book, Audio book : 1998-2001 ]
    read online at Open Library/Internet Archive
    View online
    At 69 libraries
    This resource may be of limited relevance to your query (score: 0.261)

    This resource may be of limited relevance to your query (score: 0.261)

  3. Website indexing : enhancing access to information within websites / Glenda Browne, Jonathan Jermey
    Browne, Glenda
    [ Book : 2001-2004 ]
    View online (access conditions)
    At 136 libraries
    This resource may be of limited relevance to your query (score: 0.207)

    This resource may be of limited relevance to your query (score: 0.207)

  4. Information architecture for the World Wide Web / Louis Rosenfeld and Peter Morville
    Rosenfeld, Louis
    [ Book : 1998-2007 ]
    read online at Open Library/Internet Archive
    View online
    At 90 libraries
    This resource may be of limited relevance to your query (score: 0.202)

    This resource may be of limited relevance to your query (score: 0.202)

  1. Search engines for the World Wide Web / Alfred and Emily Glossbrenner
    Glossbrenner, Alfred
    [ Book, Audio book : 1998-2001 ]
    read online at Open Library/Internet Archive
    View online
    At 69 libraries
    This resource may be of limited relevance to your query (score: 0.261)

    This resource may be of limited relevance to your query (score: 0.261)

  2. The Google story / David A. Vise with Mark Malseed
    Vise, David A
    [ Book, Audio book : 2005-2008 ]
    Languages: English, Chinese, Vietnamese
    read online at Open Library/Internet Archive
    View online
    At 126 libraries
    This resource may be of limited relevance to your query (score: 0.125)

    This resource may be of limited relevance to your query (score: 0.125)

  3. Googled : the end of the world as we know it / by Ken Auletta
    Auletta, Ken
    [ Book, Audio book : 2009-2011 ]
    At 63 libraries
    This resource may only be vaguely related to your query: matching terms may only have been found in the full text contents of the resource (score: 0.074)

    This resource may only be vaguely related to your query: matching terms may only have been found in the full text contents of the resource (score: 0.074)

  4. The shallows : how the internet is changing the way we think, read and remember / Nicholas Carr
    Carr, Nicholas G., 1959-
    [ Book, Audio book : 2010-2011 ]
    At 115 libraries
    This resource may only be vaguely related to your query: matching terms may only have been found in the full text contents of the resource (score: 0.072)

    This resource may only be vaguely related to your query: matching terms may only have been found in the full text contents of the resource (score: 0.072)

  1. THE UNITED STATES. DISCOVERY OF A "SUICIDE CLUB." FIVE SUICIDES THIS YEAR. London, June 17.
    South Australian Register (Adelaide, SA : 1839 - 1900) Tuesday 18 June 1895 p 5 Article
    ... THE UNITED STATES. DISCOVERY OF A 'SUICIDE CLUB.' FIVE SUICIDES THIS YEAR. London, June 17. Information from New York reports the discovery of a 'Suicide Club.' It is stated that five members of the organization have already committed suicide during the present year. ... 43 words
    This resource may have relevance to your query (score: 0.004)

    This resource may have relevance to your query (score: 0.004)

  2. CRIMINAL COURT. Sentence Day in Brisbane.
    The West Australian (Perth, WA : 1879 - 1954) Friday 13 March 1931 p 20 Article
    ... ... 124 words
    This resource may have relevance to your query (score: 0.003)

    This resource may have relevance to your query (score: 0.003)

  3. LABOUR TROUBLES. EMPLOYERS ORGANIZING. London, March 6.
    South Australian Register (Adelaide, SA : 1839 - 1900) Wednesday 8 March 1893 p 5 Article
    ... LABOUR TROUBLES. EMPLOYERS ORGANIZING. London, March 6. The unsettled state of the labour market, particularly in shipping circles, is leading to further elaborate organization tion on the part of the employers, the latest information in this connection being that great free labour federations are ... 53 words
    This resource may have relevance to your query (score: 0.003)

    This resource may have relevance to your query (score: 0.003)

  1. Web Searching, Search Engines and Information Retrieval
    Lewandowski, Dirk
    [ Article : 2005 ]
    View online
    This resource may be of limited relevance to your query (score: 0.075)

    This resource may be of limited relevance to your query (score: 0.075)

  2. More strategies for searching the Web.(multiple Internet search engines)
    Lankford, Jefferson
    Arizona Attorney
    [ Article : 2000 ]
    This resource may be of limited relevance to your query (score: 0.07)

    This resource may be of limited relevance to your query (score: 0.07)

  3. A Machine Learning Architecture for Optimizing Web Search Engines
    Justin Boyan,Dayne Freitag,Thorsten Joachims
    [ Article : 1970-1999 ]
    View online
    This resource may be of limited relevance to your query (score: 0.069)

    This resource may be of limited relevance to your query (score: 0.069)

  4. Clustering search engines.(Searching the Web)
    Gunn, Holly
    Teacher Librarian
    [ Article : 2003 ]
    View online (conditions apply)
    This resource may be of limited relevance to your query (score: 0.065)

    This resource may be of limited relevance to your query (score: 0.065)

  1. Website: SEARCH Foundation
    www.search.org.au
    Matching pages:
    • Internet Links - 2/11/1999
    • ... Internet Links [LINKS] Internet Links A collection of useful Web sites Environmental Trade Union...
    View 190 matching archived pages
    This resource may be of limited relevance to your query (score: 0.008)

    This resource may be of limited relevance to your query (score: 0.008)

  2. Website: Caslon Analytics : internet research, analysis, strategies
    www.caslon.com.au
    Matching pages:
    View 858 matching archived pages
    This resource may be of limited relevance to your query (score: 0.007)

    This resource may be of limited relevance to your query (score: 0.007)

  3. Website: Providing information and services using the internet guideline
    www.oit.nsw.gov.au
    Matching pages:
    This resource may be of limited relevance to your query (score: 0.006)

    This resource may be of limited relevance to your query (score: 0.006)