Recently, we have witnessed generative retrieval increasingly gaining
attention in the information retrieval (IR) field, which retrieves documents by
directly generating their identifiers. So far, much effort has been devoted to
developing effective generative retrieval models. There has been less attention
paid to the robustness perspective. When a new retrieval paradigm enters into
the real-world application, it is also critical to measure the
out-of-distribution (OOD) generalization, i.e., how would generative retrieval
models generalize to new distributions. To answer this question, firstly, we
define OOD robustness from three perspectives in retrieval problems: 1) The
query variations; 2) The unforeseen query types; and 3) The unforeseen tasks.
Based on this taxonomy, we conduct empirical studies to analyze the OOD
robustness of several representative generative retrieval models against dense
retrieval models. The empirical results indicate that the OOD robustness of
generative retrieval models requires enhancement. We hope studying the OOD
robustness of generative retrieval models would be advantageous to the IR
community.