Around a fifth of manufacturers thought they might gain from customers bringing work back to Britain from the EU.