Effect of dynamic pruning safety on learning to rank effectiveness

A dynamic pruning strategy, such as Wand, enhances retrieval efficiency without degrading effectiveness to a given rank K, known as safe-to-rank-K. However, it is also possible for Wand to obtain more efficient but unsafe retrieval without actually significantly degrading effectiveness. On the other hand, in a modern search engine setting, dynamic pruning strategies can be used to efficiently obtain the set of documents to be re-ranked by the application of a learned model in a learning to rank setting. No work has examined the impact of safeness on the effectiveness of the learned model. In this work, we investigate the impact of Wand safeness through experiments using 150 TREC Web track topics. We find that unsafe Wand is biased towards documents with lower docids, thereby impacting effectiveness