The history of Caer Beris is long and varied. The name itself could either be from the Caer Peris of the nennius manuscript or from the name Brewys, the Welsh spelling of the first Norman Lord.

Pasgent ap Gwrtheyrn, King of Beallt and Gwrtheyrn, established his court here in 406 A.D.. After he was killed at the Battle of St. David's by Uthyr Pendragon, his son Bragaid was allowed to keep the family lands and stayed with them until at least 745 A.D. when Ffernfael ap Tewdwr was King. 

The Vikings ransacked the area in 896 A.D. but with the coming of the Normans another castle was built at Caer Beris in 1093 A.D. by Philip Brewys. The castle was re-fortified in 1210 A.D.. It was besieged many times over the years and all that now remains are the earthworks and approaches. These are now classified as an Ancient Monument.

Many centuries later in 1896, Captain Charles Gam Harcourt-Wood of the 15th Hussars, turned Caer Beris into a sporting estate. He had married Lina Styleman and this was to be the family home. An extension, housing a library and billiard room with master bedrooms above was built using adze cut elm. The dining room was rearranged to accommodate the oak paneling which dates from about 1570.


The third Lord Swansea bought the house in 1923 and developed the grounds and gardens. Many of the trees and shrubs growing on the property were brought from their estate at Singleton Park. Caer Beris estate was broken up in the 1970's with the cottages and stables being sold off separately. 

The main Manor was being used as a clubhouse when in the 80s, Peter and Katharine Smith, began the task of restoring the building and making it into a country house hotel. Rooms that had once been the butler's pantry, servants' hall, boot room and orangery are now letting bedrooms. The original swing bridge over the river has been rebuilt and an orchard of old English varieties of apples has been planted. More importantly, a historic house which had fallen into a bad state of repair was well on the road to recovery.

The Manor is now a grade 2 listed building.