A SEMANTIC INFORMATION RETRIEVAL APPROACH TO SOLVING PAPER-REVIEWER ASSIGNMENT PROBLEM USING A NEURAL NETWORK LANGUAGE MODEL
No Thumbnail Available
Date
Journal Title
Journal ISSN
Volume Title
Publisher
Abstract
Description
The task of assigning papers to reviewers is crucial to the realisation of an effective peer review of academic conferences and journal articles. For an excellent performance of the reviewers, it is important that the papers assigned to them are related to their knowledge
domain. The manual process of ensuring paper submissions assigned to reviewers is related to the reviewers’ knowledge domain can be very cumbersome and inefficient.
Besides, low quality and unfair reviews can result from an inefficient assignment of papers to the invited reviewers. From extant literature, automated reviewer assignment systems have been built to address this challenge as an information retrieval problem. In a bid to leverage on the recent advancement of artificial neural networks in solving natural language problems in the society, a neural network language model, Word2Vec was used to derive suitability
scores based on the semantic relatedness between a submitted paper meant for review and
a reviewer’s representation. The Integer Linear Programming model used suitability
scores to optimise the assignments to ensure the workload on the reviewers is balanced.
To test our model and compare with other Information retrieval models used in solving
paper-reviewer assignment problem, a system was implemented in Python programming
language. Python libraries such as Natural Language Toolkit was used to perform natural
language processing on the experimental datasets and the Gensim Python library were
used to implement the models. ORTools, a python library for operation research, was
used to develop the Integer Linear Programming exact optimisation used for the
assignments. Django framework was used to make the project portable for the web.
MySQL was the database management system used in managing the database. Celery was
used to make the large tasks run on the background.
From our experiment, we explored whether Word2Vec could be used in paper-reviewer
assignments and our results indicate that Word2Vec had approximately the same
performance with Latent Semantic Indexing. Evaluation was carried out to test the
efficiency of the assignment system on an on-ground truth basis. The results show that
Word2Vec provided more accuracy in semantic assignments than Latent Semantic
Indexing.
Keywords
Q Science (General), QA75 Electronic computers. Computer science