Can men wear silk apart from namaz?

If it is pure silk then not.