Clues?:
ABERFORD Yorkshire 4432 4372. 1334 Subsidy £9.75.
M (Charter) Wed; gr 7 Mar 1251, by K Hen III to Richard de Grammary. To be held at the manor (CChR, 1226-57, p. 354). On 22 Feb 1307, the Wed market was granted to Hugh Despenser, to be held at the manor (CChR, 1300-26, p. 81). On 18 Sept 1335, the Wed market was granted to William de Gramary, to be held at the manor (CChR, 1327-41, p. 345).
F (Charter) vfm, Richerius [Riquier] (9 Oct); gr 7 Mar 1251, by K Hen III to Richard de Grammayr. To be held at the manor (CChR, 1226-57, p. 354). On 22 Feb 1307, a fair on vfm Denis (9 Oct) was granted to Hugh Despenser, to be held at the manor (CChR, 1300-26, p. 81). On 18 Sept 1335, a fair on vfm Ricarius in the winter (9 Oct) was granted to William de Gramary, to be held at the manor (CChR, 1327-41, p. 345).
Presumably the manor was forfeited by and subsequently returned to the Gramary family.
---
Aberford is close to Middleton.

1. Eliza de Gramary

1. Text: Gazeteer of Markets and Fairs to 1516
   http://www.history.ac.uk/cmh/gaz