The castle, with a male central and two lateral towers that are round, scalloped and have slits, was built starting in1501 by Count Gian Carlo Tramontano, feudal lord of Matera. The count was quickly unpopular with the people because with the passage of time he became steeped in debt and to meet such debt he began heavily taxing the population. At that time construction of the castle on the hill began in order to control the "feudal" lands surrounding it more so than for the defense of the city. It appears that the building should have included other defensive towers, one of which was found in the central Piazza Vittorio Veneto of Matera along with other underground sites.
It was that some citizens, tired of constant harassment, gathered hidden behind a boulder, which was thereafter called the Stone of Bad Advice, organized the murder of the Count of Matera. On 29 December 1514, the Count, just out of the Cathedral, was assassinated on a side street, which was subsequently eloquently named Way of Redemption.
The castle currently has three large towers, while twelve were probably included in the original design. During some restoration work in the main square of the town, workers came across what was believed to be the main footings of another castle tower. However, on further excavation large Roman cisterns were unearthed. Whole house structures were discovered where one can see how the people of that era lived. Found under the main square of the modern city was a large underground reservoir, complete with columns and a vaulted ceiling.
In 2008, the castle, together with the surrounding park, closed for restoration. The restoration is expected to take many years.