Skip to main content

Posts

Showing posts with the label partial derivative

Partial Derivative - Chain Rule

what is the chain rule ? consider the function y = m * x +b  i am going to work on the function which we can use for our machine learning problems . consider a loss function h = [y - y_hat] ^2 which is also called mean square error y - dependent on x y_hat - predicted value  h = called loss    so why again two functions , ok h = [m * x +b - y_hat]^2  looks complicated right ,this is where we will use the  chain rule ,how then ? say partial derivative of m , b are [1]  dy /dm  = x , dy/db = 1 do a substitution for h  = u^2  , u = [y- y_hat] [2]  dh/du = 2u = 2 * (y - y_hat) du/dy = 1 so dh/dy  = dh/du * du /dy    --------[ chain rule] = 2 * (y - y_hat) *1 dh/dy = 2 * (y - y_hat) so what is partial derivative of  dh/dm and dh/db  then ?   chain rule again from 1 and 2  dh/dm = dh/dy * dy/dm [3] dh/dm =  2 * (y - y_hat) * x  dh/db = dh/dy * dy/db [4] dh/db   = 2 * (y - y_hat) * 1 values 3 and 4 are used in values corecction for m and b  new value of  m = m - dh/dm new value of b = b

Partial Derivative - Introduction

  So what is partial derivatives ,don't worry i am not going to take a boring maths class will go straight to the point 1.partial derivative are used in machine learning back propagation. so all of a sudden what is this back propagation  before that there must be a forward prop or forward pass , ok got it don't worry we will talk about a example 1.consider data points x =[1, 2, 3] some values for our understanding 2. consider a function y =x so the values of y will be [1, 2 , 3] 3.consider a function y= x^2 so the values of y will be [1, 4, 9]  4. In real life the data points could be different and the dependent function y could be like y =m*x +b or y = w1* x1 +w2  * x2 +b as u can see the data plotted and the line we are trying to fit could be either the green or blue line , so what are tryin to achieve with this  1. plot your data as X 2. have a function Y as dependant of X 3.use of function is to predict values of y for x since we have introduced variables like m , b , w1, w