The sigmoid function is a unique function where the value of the derivative of the sigmoid function includes the value of the sigmoid function. You may be asking what's the big deal. However, since the sigmoid function is already calculated it allows for simpler and more efficient processing when performing backpropagation over many layers. Additionally, it is the derivative of the sigmoid function that is used in the calculation to derive the optimal w1
, w2
, and b
values to derive the most accurate predicted output.
A cursory understanding of derivatives from calculus will assist in understanding the sigmoid derivative function.
This section walks through the steps to create a sigmoid derivative function.
- Just like the
sigmoid
function, create the derivative of thesigmoid
function can with Python using the following script:
def sigmoid_derivative(x): return sigmoid(x) * (1-sigmoid(x))
- Plot the derivative of the
sigmoid...