Olaf seems bored with the conversation. Perhaps he's wanting to ask, "Where are the kids?"