Move-to-front, distance coding, and inversion frequencies revisited

Move-to-Front, Distance Coding and Inversion Frequencies are three simple and effective techniques used to process the output of the BurrowsWheeler Transform. In this paper we provide the first complete comparative analyses of these techniques, establishing upper and lower bounds on their compression ratios. We describe simple variants of these three techniques that compress any string up to a constant factor of its kth-order empirical entropy for any k>=0. At the same time we prove lower bounds for the compression of arbitrary strings which show these variants to be nearly optimal. The bounds we establish are "entropy-only" bounds in the sense that they do not involve non-constant overheads. Our analyses provide new insights into the inner workings of these techniques, partially explain their good behavior in practice, and suggest strategies for improving their performance. © 2010 Elsevier B.V. All rights reserved.