There has been a settlement here since the 12th century. The town has many notable 17th and 18th century buildings, an ancient mercat cross, and an attractive harbour dating from 1770.
A burgh since 1550, Portsoy has a most attractive harbour dating from 1692, with fine 18th century warehouses. From here a greenish serpentine, known as Portsoy Marble, was once exported.