The site for the building of Floors Castle is a natural terrace overlooking the River Tweed and facing the Cheviot Hills, the heartland of the family's Estate. It lies on the opposite river bank to Roxburgh Castle, once the strongest fortress along the former march with England. In 1721, William Adam was commissioned by the 1st Duke to make additions to the eastern end of an existing tower-house to create a plain, but symmetrical, Georgian country house.

The 6th Duke invited the leading architect in Edinburgh, William Playfair, to remodel the castle between 1837 and 1847.

He drew his inspiration for Floors from the highly ornamented picturesque style of Heriot's Hospital in Edinburgh. The result is a romantic fairytale castle with its roofscape of turrets, pinnacles and cupolas.