Entity Image Collection Based on Multi-Modality Pattern Transfer

The core of constructing multi-modality knowledge graph is to ensure the correct and appropriate images match the entities in the knowledge graph.Existing entity image collection methods mainly use encyclopedias and image search engines as the source of images to serve as entity candidates;however, their application of image data elements is relatively simple in that they cannot accurately grasp the characteristics of image data sources, and their scalability is poor.Here, an entity image collection method based on multi-modality pattern transfer is proposed.The method extracts the corresponding semantic template from different types of head entities and transfers the visual mode to the image acquisition process of similar non-head entities.Semantic templates are used to build search engine search keywords, and visual modes are used to denoise the search results.Ultimately, the method collects 1.8×10⁶ images for 1.278×10⁵ entities in 25 categories of WikiData.The experimental results show that, compared with IMGpedia, VisualSem, Richpedia, and MMKG, the images corresponding to entities in the multi-modality knowledge graph constructed by the proposed method are more accurate with greater diversity.The accuracy of the link prediction in downstream task can be significantly improved by introducing the images collected by this method.In Hits@10, the accuracy of the index is 59.74%, which is at least 12.7 percentage points higher than that of the methods used for comparison.