Sir Joseph Bailey (1783-1858), wealthy baronet, iron-master, landowner and MP, resided in Hay Castle and later became Lord of the Manor of Hay. In 1826 he bought the Glanusk Estate, where his descendants still live.

In 1833 he commissioned the rebuilding of the Town Hall, described as a ‘neat and substantial building … which consists of an upper room with an open space below, supported on pillars, used as a cheese market’. Bailey also bequeathed to Hay in 1887 a riverside walk, still popular today, following the Wye upstream from Hay Bridge to the conservation area at The Warren.