The major source for this claim is Suetonius, a Roman historian, who said that Caligula was planning to do so. Suetonius lived after Caligula, and could only rely on the testimony of others, which was likely to be politically biased. It is very possible that it was a rumor to discredit Caligula; but unless we find new evidence, we might never know.
