Code and BeyondCode😋
Posts Tools Tags Categories BeyondCode😋
Code and BeyondCode😋

Contents

Lec9-Normalization, Dropout, + Implementation

 included in CMU-10-414-714
   24 words   One minute 

Normalization and Regularization

Normalization and Initialization

/lec9-normalization-dropout--implementation/image.png 注意看weight variance的曲线，几乎不变

norm的思想来源 /lec9-normalization-dropout--implementation/image-1.png

  • layer normalization
  • batch normalization /lec9-normalization-dropout--implementation/image-2.png 这么看来batch_norm确实很奇怪, odd! 😢 /lec9-normalization-dropout--implementation/image-3.png

Regularization

L2 Regularization

针对的是过拟合?但是只要是减少function class的操作都是regularization的一种

/lec9-normalization-dropout--implementation/image-4.png 然后发现weight decay和regularization有联系！

dropout

/lec9-normalization-dropout--implementation/image-5.png /lec9-normalization-dropout--implementation/image-6.png

Updated on 2024-09-28
Back | Home