Linear Regression


title: Linear Regression

Linear Regression

Linear Regression is a discriminative machine learning algorithm to relate the dependent variable, i.e ‘Y’ on the independent variables i.e ‘Xi‘. Unlike generative models, it does not allow the user to generate samples from the distribution of X and Y. Linear regression is a supervised learning algorithm, i.e labelled dataset is needed to train it for the model to be able to make predictions on its own.

Linear regression is a type of regression, or one of the several regression techniques which are used to find the best fitting line for the given set of points in the given dataset.
Linear regression helps us predict score of a variable X from the scores on other variables Y. When the variables Y are plotted, linear regression finds the best-fitting straight line through the points. The best-fitting line is called a regression line.
This is done by taking a line equation and comparing it with the points and the required result and then calibrated in such a way that the difference/distance between the points and the line, or simply error, is kept to the minimum. This way of calibrating is called Least Squared Error method.

Online linear regression simulator

In Python:

#Price of wheat/kg and the average price of bread wheat_and_bread = [[0.5,5],[0.6,5.5],[0.8,6],[1.1,6.8],[1.4,7]] def step_gradient(b_current, m_current, points, learningRate): b_gradient = 0 m_gradient = 0 N = float(len(points)) for i in range(0, len(points)): x = points[i][0] y = points[i][1] b_gradient += -(2/N) * (y - ((m_current * x) + b_current)) m_gradient += -(2/N) * x * (y - ((m_current * x) + b_current)) new_b = b_current - (learningRate * b_gradient) new_m = m_current - (learningRate * m_gradient) return [new_b, new_m] def gradient_descent_runner(points, starting_b, starting_m, learning_rate, num_iterations): b = starting_b m = starting_m for i in range(num_iterations): b, m = step_gradient(b, m, points, learning_rate) return [b, m] gradient_descent_runner(wheat_and_bread, 1, 1, 0.01, 100)

Code example is from this article. It also explains gradient descent and other essential concepts for deep learning.

It is important to note that not all linear regression is done with gradient descent. The normal equation can also be used for finding the linear regression coefficients, however, this uses matrix multiplication, and therefore can be very time consuming to use for more than 100,000 or 1,000,000 instances.

In Python:
Apply directly by using scikit library, thus making linear regression easy to use even on large datasets.

import pandas as pd from sklearn.cross_validation import train_test_split from sklearn.linear_model import LinearRegression as lr from sklearn import metrics # Load the data train = pd.read_csv('../input/train.csv') test = pd.read_csv('../input/test.csv') # Separate the features and labels X = train.iloc[:, 0:4].values y = train.iloc[:, 4].values # Split the data into train and test X_train, X_test, y_train, y_test = train_test_split(X, y, test_size = 0.2, random_state = 0) # Build the model model = lr() model.fit(X_train, y_train) print("Training score: ", model.score(X_train,y_train)) print("Gradient: ", model.coef_) print("y-intercept: ", model.intercept_) # Evaluating the model on test data y_pred_class = model.predict(X_test) print(metrics.accuracy_score(y_test, y_pred_class))

Metrics

How would you know if your regression model is accurate or not? Well, one method is to calculate the R-squared score of your model. R-squared score (also known as coefficient of determination) is a statistical measure of how close the data are to the fitted regression line.

More Information:

scikit learn
Article on regression
Medium article – Implementing Linear Regression with sklearn Tutorial

This article needs improvement. You can help improve this article. You can also write similar articles and help the community.