Why there is a need to initialize weights randomly in a Deep Learning Models? What is Symmetry breaking in Deep Learning model? So this is what is explained here. I have used cleaned dataset for titanic problem. And ran a two layer deep learning model with this data to classify the class as survived or not. the observation is that when weights are initialized to zero or same values then a symmetery pattern is observed while this is not the case when weights are initialized randomly.