Frederick William Vanderbilt (1856-1938) purchased the property in 1895 for use as a seasonal country residence.
The property entered the National Park Service in 1939. The site includes 211 acres (85 ha) of the original larger property historically called Hyde Park. Situated on the east bank of the Hudson River, the property includes pleasure grounds with views of the river and the distant Catskill Mountains, formal gardens, natural woodlands, and numerous support structures. The grounds also include Italian Gardens that have been restored by the volunteer Frederick W. Vanderbilt Garden Association.