He has to select other days when women are not allowed to come. Swimming is not wajib because of that anyone can fall in a haram act. It’s a good exercise but it doesn’t mean that we have to do it anyway as securing our modesty is wajib and looking to non mahram is haram.