Lordship in the March of Wales, on the River Wye, west of Hereford and today just in England.
The castle was built by William Fitzosborn, Earl of Hereford in the early years after the Norman Conquest. The Lordship was held by the great house of Tosni and in the 12th century passed by marriage to Walter, the son of Richard FitzPons, who had built a castle at Llandovery and had been lord of Cantref Bychan until the Welsh reclaimed the land.
Walter took the name of Clifford. His descendants in the north were Lords of Skipton and Appleby and Earls of Cumberland; his descendants in Devon still survive as the family of Lord Clifford of Chudleigh, but Clifford itself passed to the Giffards in 1263 through the heiress of the senior line.
Clifford became part of Herefordshire in 1536.