The history of the castle dates back to 1078 and was owned by Counts in the Empúries region. In 1472, during the Catalan Civil War, the castle is occupied by forces loyal to Joan II. In 1808, the castle was captured by Napoleonic troops, occupied for several years, and was even fortified as a keep for armaments. Despite the addition fortifications, an explosion ordered by Marshal Suchet during the French withdrawal in 1814 left the castle in ruins, which have degraded further in to what is now the current state of the castle.
The surrealist painter, Salvador Dalí had a deep affection for the castle, who was very familiar with it as it is positioned on the road at nearly the halfway between his home town of Figueres and his coastal home in Cadaqués. He had several ideas for the castle including: creating a natural pipe organ in the castle which would have been "played" by the tramuntana wind that constantly blows in the area, another was to house a rhinoceros in the basement purely for tourists to come and view, and the other was to acquire the ruins of the castle and place them in the residence of his wife, Gala.
Moderate work has been performed on the castle to maintain its current state, but visitors will note that it is on privately owned land. Also of note is that the interior has been locked off to the public in order to thwart vandals, so only the exterior is viewable to public visitors.