Linear vs ReLU activation
It seems ReLU is often used as an activation function in neural networks? But could not quite understand why. What advantages does in have compared to regular Linear Activation?
Answers
This is a quite vague question, I would recommend reading more about activation functions and ReLU in particular. But here is a short answer: Linear activation function gives you linear relationship between input and output, and you generally want your model to be non-linear. You can achieve this with several other activation functions, including ReLU.
Comments
Making all layers in your model linear will essentially collapse your model to two layers, making other layers unefective over time. You really need non-linear layers in your network.
Are you shure you want to close this post?
Are you shure you want to pin this post?
Report comment
Thank you for reporting this comment, the Imaginghub Community Management will verify your request.
Log in with Google