There has been a fortified tower on this site since the 11th century. To the north of Fenton Tower there is a medieval burial site and the remains of the foundations of one of the earliest Christian chapels in Scotland.

Patrick Whytelaw, son of Lord Ruthven, constructed the current building around 1550, a typical fortified tower of the time. But in 1587, ownership of the tower was forfeited to Sir John Carmichael. No doubt Fenton Tower was the reward for being the Scottish Ambassador to Denmark and a loyal subject of the crown.

In 1591, King James VI of Scotland, son of Mary Queen of Scots and Lord Henry Darnley, was surrounded by a rebel army in Fife, north of Edinburgh. Fortunately the local townspeople rallied to his support and helped him to escape. James took the ferry across the Firth of Forth to North Berwick and took refuge with the Carmichaels at Fenton Tower.

James VI later became James I of England and Scotland, and upon the death of Carmichael he granted Fenton Tower to Sir Thomas Erskine who became Lord Dirleton, Viscount Fenton and later Earl of Kellie.

In 1631, the Tower passed to Sir John Maxwell of Innerwick, who became Earl of Dirleton in 1646. However his enjoyment of the castle was shortlived. In 1650, Oliver Cromwell invaded Scotland and he sacked Fenton Tower along with the nearby Dirleton Castle. With Fenton Tower now just a ruin, the surrounding land passed to Sir John Nisbet in 1663. In the 1850s the land and ruins passed to the Simpson family.

Ian Simpson and his life-long friend John Macaskill, decided to start the Tower’s restoration in 1998. As it is a listed ancient monument and a Category A-listed building, Historic Scotland supervised the meticulous restoration. Fenton Tower has been two years in the planning and two years in restoration. It has opened its doors once again as a great Scottish house.