The eye-watering fee being lost each year because of Brexit