Oxford Castle, located in Oxford city centre, was built by a Norman baron, Robert D'Oyly, in 1071 (shortly after the Norman Conquest of England in 1066). It was originally an earth mound with a wooden keep on the top, which was later replaced with a stone keep and later a fifty foot wall with towers was built around the structure.