A first castle was the seat of the medieval dynasty of the viscounts of Peralada, started by Berenguer, son of Ponce I, count of Ampurias. During the French invasion of the Ampurdán, in the course of the crusade against Catalonia led by Philip III of France, the castle and the nearby buildings were set on fire and destroyed (1285). Remains of these original structure are in the upper part of the town.
A new castle was built in the mid-13th century outside the line of the new walls. The current edifice received a new Renaissance façade, while the building was enlarged in the 19th century.