The Cheese Market was built around 1835 on the site of a pre-17th century guildhall. At that time, the castle steps ran from the Norman tower to the street and people walked through the market hall down into High Town. Cottages and shops extended up Castle Street from the hall and a tavern dominated what is now Memorial Square.

Hay was founded by Bernard de Neufmarché as a fortified town, taking its’ name from the Norman French ‘La Haie’ meaning hedged enclosure; the Welsh Y Gelli means copse or wooded enclosure. The borderlands were ruled by independent lords until Wales and England united in 1536. The town of Hay stands within ancient walls extant in several places, built circa 1236, after two centuries of border battles.