Flames shot 20 to 30ft into the air and smoke from the blaze could be seen across the city.
About 100 firefighters were called to the site in Twelvetrees Crescent in Bow on Wednesday night but say it needs to burn itself out.
Iron Mountain, the firm that owns the data storage unit, said it held "archived, inactive business records".
It is thought the building will be alight for between two and three days.
A spokesman for Iron Mountain, which stores and manages records for firms around the world, said: "Mostly it is old archives that were not required currently, business records.
"In most cases we wouldn't even know because most people they use paper documents in boxes - commercial business records."
He added: "We are just interested as anybody else to know what could have caused it. It was a fair-sized warehouse but it was full."
Four delivery lorries also caught fire, but no-one is thought to have been injured.
On Thursday, an Iron Mountain warehouse in Ottawa, Canada, was also hit by fire.
The fire at a warehouse in the Cyrville industrial area, believed to contain thousands of boxes of inactive records, was designated as accidental, the Ottawa Sun newspaper reported.