- (Ge et al, 1999) ⇒ Xianping Ge, Wanda Pratt, Padhraic Smyth. (1999). "Discovering Chinese Words from Unsegmented Text." In: Proceedings of SIGIR-1999. doi:10.1145/312624.313472
Subject Headings: Surface Word Segmentation Task.
- In English written text, words are separated by spaces, but in written Chinese text, there are no such separators between words. (See Figure 1.) Thus, effective information retrieval of Chinese text first requires good word segmentation. In this paper, we investigate an efficient algorithm to discover the words and their occurrence probabilities from a corpus of unsegmented text without using a dictionary. Using the probabilities of the words, word segmentation is done according to the maximum likelihood principle. Comparing the segmentation output by the algorithm with the correct segmentation, recall/precision of 65:65%=71:91% is achieved. If some simple post-processing is performed, recall/precision can be boosted up to 97:72%=91:05%.
Facts about "1999 DiscoveringChineseWordsFromUnsegText"
|author||Xianping Ge +, Wanda Pratt + and Padhraic Smyth +|
|journal||Proceedings of SIGIR-1999 +|
|title||Discovering Chinese Words from Unsegmented Text +|