Mott metal-insulator transition in the half-filled Hubbard model on the triangular lattice

We investigate the metal-insulator transition in the half-filled Hubbard model on the two-dimensional triangular lattice using both the Kotliar-Ruckenstein slave-boson technique and an exact numerical diagonalization of finite clusters. Contrary to the case of a square lattice, where a perfect nesting of the Fermi surface leads to a metal-insulator transition at arbitrarily small values of U, always accompanied by antiferromagnetic ordering, on a triangular lattice, due to the lack of perfect nesting, the transition takes place at a finite value of U, and frustration induces a nontrivial competition among different magnetic phases. Indeed, within the mean-field approximation in the slave-boson approach, as the interaction grows the paramagnetic metal turns into a metallic phase with incommensurate spiral ordering. Increasing the interaction further, a linear spin density wave is stabilized, and finally for strong coupling the latter phase undergoes a first-order transition toward an antiferromagnetic insulator. No trace of the intermediate phases is seen in the exact diagonalization results, indicating a transition between a paramagnetic metal and an antiferromagnetic insulator.