The building housing the Cologne city archives collapsed on Tuesday and officials said they feared that people could be trapped inside.
Fire department spokesman Daniel Leupold said rescue officials found several lightly injured people at the scene, and it was likely that there were some people inside — though it was unclear how many. The building was open to the public at the time of the collapse.
Work was being carried out on a subway line under the street on which the building stands, but the cause of the collapse was unclear, Leupold said.
About 180 firefighters were sent to the scene.
A photograph on German television showed the building reduced to a pile of rubble.
Police spokesman Wolfgang Baldes said the rubble spilled out over 50 to 70 yards (meters).
Alfred Hoovestedt, who works in a building nearby, told n-tv television that he was sitting with colleagues when "suddenly there was a rumbling, as if a train were going past."
"Then the noise got louder and louder, and we heard vibration," he said. "Then, in seconds, the whole area was covered in a haze."