1999 DiscoveringChineseWordsFromUnsegText

Jump to: navigation, search

Subject Headings: Surface Word Segmentation Task.


Cited By

~69 http://scholar.google.com/scholar?cites=5555599676164448189



  • In English written text, words are separated by spaces, but in written Chinese text, there are no such separators between words. (See Figure 1.) Thus, effective information retrieval of Chinese text first requires good word segmentation. In this paper, we investigate an efficient algorithm to discover the words and their occurrence probabilities from a corpus of unsegmented text without using a dictionary. Using the probabilities of the words, word segmentation is done according to the maximum likelihood principle. Comparing the segmentation output by the algorithm with the correct segmentation, recall/precision of 65:65%=71:91% is achieved. If some simple post-processing is performed, recall/precision can be boosted up to 97:72%=91:05%.



 AuthorvolumeDate ValuetitletypejournaltitleUrldoinoteyear
1999 DiscoveringChineseWordsFromUnsegTextXianping Ge
Wanda Pratt
Padhraic Smyth
Discovering Chinese Words from Unsegmented TextProceedings of SIGIR-1999http://ictclas.cn/otherdocs/Discovering Chinese words from unsegmented text.pdf10.1145/312624.3134721999