The extra parameters are useful for optimization. Deep-learning relies on highly singular solutions.