Skip to main content

Posts

Showing posts from June, 2021

Stochastic gradient descent with multiple variables

 The idea is to understand how u can create a sample for stochastic gradient descent using python , numpy and some basic maths. so what is gradient descent  u might be bored with the term and it is always boring with visualization here i will run through a sample using python and plot that in mat plot lib for visual. matplot lib is another library u can install on the go and it is like a simple x,y graph that we used in our school days.don't worry this is pretty simple. 1.install python - https://www.python.org/downloads/ if you are using windows it will be a exe run that . once installed type in command line   since the function is  y = w1* x1 +w2 * x2 +b we will use the know  mean square error loss function  mae  = (y - y_hat)**2 next step would be to calculate partial derivatives for w1 , w2 , b  with respect to y dy/dw1 = x1 dy/dw2 = x2  dy/db = 1 h = (y - y_hat) ** 2  h = u **2 , u = y - y_hat dh/du = 2u = 2 (y - y_hat) du/dy = 1 by chain rule dh/dy = dh/du * du/dy dh/dy = 2