In this post we look deep into activation function.
If you remember from my previous blog post. Activation function receives a weighted sum of weights from the nodes and it decides if the node will get activated or not.
To add activation to our NN its very simple
from keras.layers import Activation
model.add(Activation('relu'))
model.summary()
It is very simple to implement activation function using Keras as we see above, but its important that we understand in detail different kind of activation layers and their pro/cons
https://colab.research.google.com/drive/1IOjSB1PF85t9qEhCWgDbWxuCRSDz71H6
Activation Functions
This is a blog which post which explores in detail pro/cons of different activation functions
It’s highly recommended to read this.
In summary Relu is a very simple and efficient activation function which solves the “varnishing gradient problem” effectively. You can use this as a default activation function in most cases and also experiment with sigmoid/tahn in case of logical regression problems.
Further most read links to understand the above
https://deeplizard.com/learn/video/m0pIlLfpXWE
All these blogs are quite technical and explain activation functions in depth.
Keras Activation Functions
Keras supports many different activation functions out of the box which you can see here https://keras.io/activations/
