In the late 17th century John Lowther, 1st Viscount Lonsdale rebuilt the family home, then known as Lowther Hall, on a grand scale. The current building is a castellated mansion which was built by Robert Smirke between 1806 and 1814, and it was only at that time that Lowther was designated a "castle".