English, Article edition: DISTRIBUTION OF MULTI-WORDS IN CHINESE AND ENGLISH DOCUMENTS WEN ZHANG; TAKETOSHI YOSHIDA; XIJIN TANG

User activity

Share to:
 
Bookmark: http://trove.nla.gov.au/version/15720
Physical Description
  • article
Language
  • English

Edition details

Title
  • DISTRIBUTION OF MULTI-WORDS IN CHINESE AND ENGLISH DOCUMENTS
Author
  • WEN ZHANG
  • TAKETOSHI YOSHIDA
  • XIJIN TANG
Physical Description
  • article
Notes
  • As a hybrid of N-gram in natural language processing and collocation in statistical linguistics, multi-word is becoming a hot topic in area of text mining and information retrieval. In this paper, a study concerning distribution of multi-words is carried out to explore a theoretical basis for probabilistic term-weighting scheme. Specifically, the Poisson distribution, zero-inflated binomial distribution, and G-distribution are comparatively studied on a task of predicting probabilities of multi-words' occurrences using these distributions, for both technical multi-words and nontechnical multi-words. In addition, a rule-based multi-word extraction algorithm is proposed to extract multi-words from texts based on words' occurring patterns and syntactical structures. Our experimental results demonstrate that G-distribution has the best capability to predict probabilities of frequency of multi-words' occurrence and the Poisson distribution is comparable to zero-inflated binomial distribution in estimation of multi-word distribution. The outcome of this study validates that burstiness is a universal phenomenon in linguistic count data, which is applicable not only for individual content words but also for multi-words.
  • Multi-word, term distribution, Poisson distribution, zero-inflated distribution, G-distribution
  • RePEc:wsi:ijitdm:v:08:y:2009:i:02:p:249-265
Language
  • English
Contributed by
OAIster

Get this edition

  • Set up My libraries

    How do I set up "My libraries"?

    In order to set up a list of libraries that you have access to, you must first login or sign up. Then set up a personal list of libraries from your profile page by clicking on your user name at the top right of any screen.

  • All (1)
  • Unknown (1)
None of your libraries hold this item.
None of your libraries hold this item.
None of your libraries hold this item.
None of your libraries hold this item.
None of your libraries hold this item.
None of your libraries hold this item.
None of your libraries hold this item.
None of your libraries hold this item.

User activity


e.g. test cricket, Perth (WA), "Parkes, Henry"

Separate different tags with a comma. To include a comma in your tag, surround the tag with double quotes.

Be the first to add a tag for this edition

Be the first to add this to a list

Comments and reviews

What are comments? Add a comment

No user comments or reviews for this version

Add a comment