基于半CRF模型的百科全书文本段落划分
A Semi-Markov CRF Model Approach to Encyclopedia Text Topic Segmentation
-
摘要: 介绍了基于半条件随机域(semi-Markov conditional random fields,简称semi-CRFs)模型的百科全书文本段落划分方法.为了克服单纯的HMM模型和CRF模型的段落类型重复问题,以经过整理的HMM模型状态的后验分布为基本依据,使用了基于词汇语义本体知识库的段落开始特征以及针对特定段落类型的提示性特征来进一步适应目标文本的特点.实验结果表明,该划分方法可以综合利用各种不同类型的信息,比较适合百科全书文本的段落结构,可以取得比单纯的HMM模型和CRF模型更好的性能.Abstract: This paper introduced the semi-markov Conditional Random Fields(semi-CRFs)model based method for Chinese Encyclopedia text topic segmentation.The authors adopted HMM model state posterior as the basic segmentation clue which was adjusted to each text instance to overcome the topic duplication problem of fully connected state HMM model and CRF model.The authors also used several segment level word semantic features derived from domain thesaurus,and additional topic specific cue phrases to make the method more adapted to target domain.The experiment result showed that this method was suitable for Chinese Encyclopedia text topic structure and achieved better performance than HMM model and CRF model.