Learning to Rank Answers to Closed-Domain Questions by using Fuzzy Logic

Question answering (QA) is a challenging task and has received considerable attention in the last years. Answer selection among candidate answers is one of the main phases for QA and the best answer to be returned is determined in this phase. A common approach consists in considering the selection of the final answer(s) as a ranking problem. So far, different methods have been proposed, mainly oriented to produce a single best ranking model operating in the same way on all the question types. Differently, this paper proposes a fuzzy approach for ranking and selecting the correct answer among a list of candidates in a state-of-the-art QA system operating with factoid and description questions on Italian corpora pertaining a closed domain. Starting from the consideration that this ranking problem can be reduced to a classification one, the proposed approach is based on the Likelihood-Fuzzy Analysis (LFA), applied in this case for mining fuzzy rule-based models able to discern correct (True) from incorrect answers (False). Such fuzzy models are mined as specifically tailored to each question type, and, thus, can be individually applied to produce a more robust and accurate final ranking. An experimental session over a collection of questions pertaining the Cultural Heritage domain, using a manually annotated gold-standard dataset, shows that considering specific fuzzy ranking models for each question type improves the accuracy of the best answer returned back to the user.