The origin of the name is unknown, as there are no representations of deer or other animals on the building.