Hi Andrew,
By "public", I assume you mean ones that you don't have to be a member of BN or a club in order to attend?
You'll find that the majority are only open to members of BN. In the Midlands, the ones that I am aware of are (in no particular order):
A weekly swim at Northfield Baths, Birmingham, from 18:30 - 20:30 each Saturday
A weekly swim at Bramcote Leisure Centre (Nottingham way), also from 18:30 - 20:30 each Saturday
A weekly swim at Erdington Leisure Centre, Birmingham, from 20:30 - 22:00 each Thursday
For ones that are more open, your closest ones are probably going to be those put on by Club Soleil. They hold swims in Whittlesey and Bedford (I have been to the Bedford one - quite a nice facility!)
You can go to Soleil's swims three times before you have to join... you do need to show them some form of ID if you're not a BN member though (driving licence or the like will do fine). Next swim at Bedford is this Saturday (10 November) from 19:00 - 22:00. I think it costs £6.50 for non-members, but I can't quite remember.
All the details are here, if you're interested:
http://soleil.org.uk/