Abstract: A image annotation algorithm based on Cross Media Relevance Model was proposed to bridge the semantic gap of content-based image retrieval. The algorithm reduced the word bias observed in probabilistic models by converting the word-image joint probability to image probability conditioned on annotation words and estimated the probability of a set of annotation words by measuring the semantic similarities of each annotation word to all other word. It used a contextual term vector to represent a annotation word,and implemented image annotation by estimating maximum correlation between an image and a set of annotation words. Compared with image annotation algorithm based on Cross Media Relevance Model, the proposed algorithm stopped making the assumption that each keyword was independent to each other,instead,took contextual relevance information of annotation words into account. Experimental results on the typical corel dataset demonstrated the effectiveness and the increasing annotation precision of the proposed algorithm.