The sorting office would not enter into any discussion about charges, which are levied by HMRC. At the sorting office your choice is simple - pay for the goods to be released to you, or decline to pay, and the package will be returned to sender.
Royal Mail also charge an £8 'handling fee', in addition to any Import duty and VAT levied, which is to cover their costs for collecting and forwarding the HMRC charges.
Import duty is not the same as VAT, and dependent on value of goods, you may get charged Import duty on non VAT items.
Having said all that, I don't think I have been leathered for charges for some years now - most of my fabric splurges have come from within the EU, where they don't currently attract any extra fees.
https://www.gov.uk/goods-sent-from-abroad/tax-and-duty