Lesson 17 of 27By Simplilearn
Last updated on Jan 28, 202184533Machine learning has revolutionized the world of business and is helping us build sophisticated applications to solve tough business problems. Using supervised and unsupervised machine learning models, you can solve problems using classification, regression, and clustering algorithms. In this article, we’ll discuss a supervised machine learning algorithm known as logistic regression in Python. Logistic regression can be used to solve both classification and regression problems.
The following are topics that this article will cover:
Supervised machine learning algorithms derive insights, patterns, and relationships from a labeled training dataset. It means the dataset already contains a known value for the target variable for each record. It is called supervised learning because the process of an algorithm learning from the training dataset is like an instructor supervising the learning process. You know the correct answers, the algorithm iteratively makes predictions on the training data and the instructor corrects it. Learning ends when the algorithm achieves the desired level of performance and accuracy.
Supervised learning problems can be further classified into regression and classification problems.
The following is an example of a supervised learning method where we have labeled data to identify dogs and cats. The algorithm learns from this data and trains a model to predict the new input.
Now that we learned the basics of supervised learning, let's have a look at a popular supervised machine learning algorithm: logistic regression.
Logistic regression is a statistical method that is used for building machine learning models where the dependent variable is dichotomous: i.e. binary. Logistic regression is used to describe data and the relationship between one dependent variable and one or more independent variables. The independent variables can be nominal, ordinal, or of interval type.
The name “logistic regression” is derived from the concept of the logistic function that it uses. The logistic function is also known as the sigmoid function. The value of this logistic function lies between zero and one.
The following is an example of a logistic function we can use to find the probability of a vehicle breaking down, depending on how many years it has been since it was serviced last.
Here is how you can interpret the results from the graph to decide whether the vehicle will break down or not.
Consider the following example: An organization wants to determine an employee’s salary increase based on their performance.
For this purpose, a linear regression algorithm will help them decide. Plotting a regression line by considering the employee’s performance as the independent variable, and the salary increase as the dependent variable will make their task easier.
Now, what if the organization wants to know whether an employee would get a promotion or not based on their performance? The above linear graph won’t be suitable in this case. As such, we clip the line at zero and one, and convert it into a sigmoid curve (S curve).
Based on the threshold values, the organization can decide whether an employee will get a salary increase or not.
To understand logistic regression, let’s go over the odds of success.
Odds (𝜃) = Probability of an event happening / Probability of an event not happening
𝜃 = p / 1 - p
The values of odds range from zero to ∞ and the values of probability lies between zero and one.
Consider the equation of a straight line:
𝑦 = 𝛽0 + 𝛽1* 𝑥
Here, 𝛽0 is the y-intercept
𝛽1 is the slope of the line
x is the value of the x coordinate
y is the value of the prediction
Now to predict the odds of success, we use the following formula:
Exponentiating both the sides, we have:
Let Y = e 𝛽0+𝛽1 * 𝑥
Then p(x) / 1 - p(x) = Y
p(x) = Y(1 - p(x))
p(x) = Y - Y(p(x))
p(x) + Y(p(x)) = Y
p(x)(1+Y) = Y
p(x) = Y / 1+Y
The equation of the sigmoid function is:
The sigmoid curve obtained from the above equation is as follows:
Now that you know more about logistic regression algorithms, let’s look at the difference between linear regression and logistic regression.
Linear Regression |
Logistic Regression |
Used to solve regression problems |
Used to solve classification problems |
The response variables are continuous in nature |
The response variable is categorical in nature |
It helps estimate the dependent variable when there is a change in the independent variable |
It helps to calculate the possibility of a particular event taking place |
It is a straight line |
It is an S-curve (S = Sigmoid) |
Now, let’s look at some logistic regression algorithm examples.
Now, let’s look at the assumptions you need to take to build a logistic regression model.
Let’s now jump into understanding the logistics Regression algorithm in Python.
We’ll be using the digits dataset in the scikit learn library to predict digit values from images using the logistic regression model in Python.
The images above depict the actual numbers and the predicted digit values from our logistic regression model.
Are you an AI and Machine Learning enthusiast? If yes, the Post Graduate Program in AI and Machine Learning is a perfect fit for your career growth.
We hoped that this article has helped you get acquainted with the basics of supervised learning and logistic regression. We covered the logistic regression algorithm and went into detail with an elaborate example. Then, we looked at the different applications of logistic regression, followed by the list of assumptions you should make to create a logistic regression model. Finally, we built a model using the logistic regression algorithm to predict the digits in images.
If you want to kick off a career in this exciting field, check out Simplilearn’s Artificial Intelligence Engineer Master’s Program, offered in collaboration with IBM. The program enables you to dive much deeper into the concepts and technologies used in AI, machine learning, and deep learning. You will also get to work on an awesome Capstone Project and earn a certificate in all disciplines in this exciting and lucrative field.
Simplilearn is one of the world’s leading providers of online training for Digital Marketing, Cloud Computing, Project Management, Data Science, IT, Software Development, and many other emerging technologies.
Post Graduate Program in AI and Machine Learning
Machine Learning
*Lifetime access to high-quality, self-paced e-learning content.
Linear Regression in Python
Machine Learning Career Guide: A complete playbook to becoming a Machine Learning Engineer
The Best Guide to Understand Regression Analysis in Excel
Logistic Regression in R: The Ultimate Tutorial with Examples
Getting Started with Linear Regression in R
Introduction to Machine Learning: A Beginner's Guide