gradient vanishing1

• 

  Gradient vanishing Information Gradient vanishing refers to a problem that occurs during the training of deep neural networks where the gradients used to update the model's parameters become extremely small as they propagate through the layers of the network. This happens because gradients are calculated using the chain rule of differentiation, and the chain rule involves multiplying many small gradients together,.. Zettelkasten/Terminology Information 2023. 2. 24.