Can GPT Dream of a Pink Elephant?