3 - K-Nearest Neighbor(KNN) Algorithm for Machine Learning - Javatpoint

09/01/2023, 23:46 K-Nearest Neighbor(KNN) Algorithm for Machine Learning - Javatpoint https://www.javatpoint.com/k-nearest-neighbor-algorithm-for-machine-learning 1/18 Home AI Machine Learning DBMS Java Blockchain Control System Selenium HTML

09/01/2023, 23:46 K-Nearest Neighbor(KNN) Algorithm for Machine Learning - Javatpoint https://www.javatpoint.com/k-nearest-neighbor-algorithm-for-machine-learning 2/18 K-Nearest Neighbor(KNN) Algorithm for Machine Learning K-Nearest Neighbour is one of the simplest Machine Learning algorithms based on Supervised Learning technique. K-NN algorithm assumes the similarity between the new case/data and available cases and put the new case into the category that is most similar to the available categories. K-NN algorithm stores all the available data and classifies a new data point based on the similarity. This means when new data appears then it can be easily classified into a well suite category by using K- NN algorithm. K-NN algorithm can be used for Regression as well as for Classification but mostly it is used for the Classification problems. K-NN is a non-parametric algorithm , which means it does not make any assumption on underlying data. It is also called a lazy learner algorithm because it does not learn from the training set immediately instead it stores the dataset and at the time of classification, it performs an action on the dataset. KNN algorithm at the training phase just stores the dataset and when it gets new data, then it classifies that data into a category that is much similar to the new data. Example: Suppose, we have an image of a creature that looks similar to cat and dog, but we want to know either it is a cat or dog. So for this identification, we can use the KNN algorithm, as it works on a similarity measure. Our KNN model will find the similar features of the new data set to the cats and dogs images and based on the most similar features it will put it in either cat or dog category.

09/01/2023, 23:46 K-Nearest Neighbor(KNN) Algorithm for Machine Learning - Javatpoint https://www.javatpoint.com/k-nearest-neighbor-algorithm-for-machine-learning 4/18 Firstly, we will choose the number of neighbors, so we will choose the k=5. Next, we will calculate the Euclidean distance between the data points. The Euclidean distance is the distance between two points, which we have already studied in geometry. It can be calculated as: By calculating the Euclidean distance we got the nearest neighbors, as three nearest neighbors in category A and two nearest neighbors in category B. Consider the below image:

09/01/2023, 23:46 K-Nearest Neighbor(KNN) Algorithm for Machine Learning - Javatpoint https://www.javatpoint.com/k-nearest-neighbor-algorithm-for-machine-learning 5/18 As we can see the 3 nearest neighbors are from category A, hence this new data point must belong to category A. How to select the value of K in the K-NN Algorithm? Below are some points to remember while selecting the value of K in the K-NN algorithm: There is no particular way to determine the best value for "K", so we need to try some values to find the best out of them. The most preferred value for K is 5. A very low value for K such as K=1 or K=2, can be noisy and lead to the effects of outliers in the model. Large values for K are good, but it may find some difficulties.

09/01/2023, 23:46 K-Nearest Neighbor(KNN) Algorithm for Machine Learning - Javatpoint https://www.javatpoint.com/k-nearest-neighbor-algorithm-for-machine-learning 7/18 Steps to implement the K-NN algorithm: Data Pre-processing step Fitting the K-NN algorithm to the Training set Predicting the test result Test accuracy of the result(Creation of Confusion matrix) Visualizing the test set result. Data Pre-Processing Step: The Data Pre-processing step will remain exactly the same as Logistic Regression. Below is the code for it: # importing libraries import numpy as nm import matplotlib.pyplot as mtp import pandas as pd #importing datasets data_set= pd.read_csv( 'user_data.csv' )

09/01/2023, 23:46 K-Nearest Neighbor(KNN) Algorithm for Machine Learning - Javatpoint https://www.javatpoint.com/k-nearest-neighbor-algorithm-for-machine-learning 8/18 #Extracting Independent and dependent Variable x= data_set.iloc[:, [ 2 , 3 ]].values y= data_set.iloc[:, 4 ].values # Splitting the dataset into training and test set. from sklearn.model_selection import train_test_split x_train, x_test, y_train, y_test= train_test_split(x, y, test_size= 0.25 , random_state= 0 ) #feature Scaling from sklearn.preprocessing import StandardScaler st_x= StandardScaler() x_train= st_x.fit_transform(x_train) x_test= st_x.transform(x_test) By executing the above code, our dataset is imported to our program and well pre-processed. After feature scaling our test dataset will look like: From the above output image, we can see that our data is successfully scaled. Fitting K-NN classifier to the Training data: Now we will fit the K-NN classifier to the training data. To do this we will import the

09/01/2023, 23:46 K-Nearest Neighbor(KNN) Algorithm for Machine Learning - Javatpoint https://www.javatpoint.com/k-nearest-neighbor-algorithm-for-machine-learning 10/18 The output for the above code will be: Creating the Confusion Matrix: Now we will create the Confusion Matrix for our K-NN model to see the accuracy of the classifier. Below is the code for it: #Creating the Confusion matrix from sklearn.metrics import confusion_matrix cm= confusion_matrix(y_test, y_pred) In above code, we have imported the confusion_matrix function and called it using the variable cm. Output: By executing the above code, we will get the matrix as below:

09/01/2023, 23:46 K-Nearest Neighbor(KNN) Algorithm for Machine Learning - Javatpoint https://www.javatpoint.com/k-nearest-neighbor-algorithm-for-machine-learning 11/18 In the above image, we can see there are 64+29= 93 correct predictions and 3+4= 7 incorrect predictions, whereas, in Logistic Regression, there were 11 incorrect predictions. So we can say that the performance of the model is improved by using the K-NN algorithm. Visualizing the Training set result: Now, we will visualize the training set result for K-NN model. The code will remain same as we did in Logistic Regression, except the name of the graph. Below is the code for it: #Visulaizing the trianing set result from matplotlib.colors import ListedColormap x_set, y_set = x_train, y_train x1, x2 = nm.meshgrid(nm.arange(start = x_set[:, 0 ].min() - 1 , stop = x_set[:, 0 ].max() + 1 , step = 0.01 ) nm.arange(start = x_set[:, 1 ].min() - 1 , stop = x_set[:, 1 ].max() + 1 , step = 0.01 )) mtp.contourf(x1, x2, classifier.predict(nm.array([x1.ravel(), x2.ravel()]).T).reshape(x1.shape), alpha = 0.75 , cmap = ListedColormap(( 'red' , 'green' ))) mtp.xlim(x1.min(), x1.max())

09/01/2023, 23:46 K-Nearest Neighbor(KNN) Algorithm for Machine Learning - Javatpoint https://www.javatpoint.com/k-nearest-neighbor-algorithm-for-machine-learning 13/18 The graph is showing good result but still, there are some green points in the red region and red points in the green region. But this is no big issue as by doing this model is prevented from overfitting issues. Hence our model is well trained. Visualizing the Test set result: After the training of the model, we will now test the result by putting a new dataset, i.e., Test dataset. Code remains the same except some minor changes: such as x_train and y_train will be replaced by x_test and y_test . Below is the code for it: #Visualizing the test set result from matplotlib.colors import ListedColormap x_set, y_set = x_test, y_test x1, x2 = nm.meshgrid(nm.arange(start = x_set[:, 0 ].min() - 1 , stop = x_set[:, 0 ].max() + 1 , step = 0.01 ) nm.arange(start = x_set[:, 1 ].min() - 1 , stop = x_set[:, 1 ].max() + 1 , step = 0.01 )) mtp.contourf(x1, x2, classifier.predict(nm.array([x1.ravel(), x2.ravel()]).T).reshape(x1.shape), alpha = 0.75 , cmap = ListedColormap(( 'red' , 'green' ))) mtp.xlim(x1.min(), x1.max()) mtp.ylim(x2.min(), x2.max()) for i, j in enumerate(nm.unique(y_set)): mtp.scatter(x_set[y_set == j, 0 ], x_set[y_set == j, 1 ], c = ListedColormap(( 'red' , 'green' ))(i), label = j) mtp.title( 'K-NN algorithm(Test set)' ) mtp.xlabel( 'Age' ) mtp.ylabel( 'Estimated Salary' ) mtp.legend() mtp.show() Output:

09/01/2023, 23:46 K-Nearest Neighbor(KNN) Algorithm for Machine Learning - Javatpoint https://www.javatpoint.com/k-nearest-neighbor-algorithm-for-machine-learning 14/18 ← Prev Next → The above graph is showing the output for the test data set. As we can see in the graph, the predicted output is well good as most of the red points are in the red region and most of the green points are in the green region. However, there are few green points in the red region and a few red points in the green region. So these are the incorrect observations that we have observed in the confusion matrix(7 Incorrect output). Youtube For Videos Join Our Youtube Channel: Join Now

09/01/2023, 23:46 K-Nearest Neighbor(KNN) Algorithm for Machine Learning - Javatpoint https://www.javatpoint.com/k-nearest-neighbor-algorithm-for-machine-learning 16/18 Aptitude Aptitude Logical Reasoning Reasoning Verbal Ability Verbal Ability Interview Questions Interview Questions Company Interview Questions Company Questions Artificial Intelligence Artificial Intelligence AWS Tutorial AWS Selenium tutorial Selenium Cloud Computing Cloud Computing Hadoop tutorial Hadoop ReactJS Tutorial ReactJS Data Science Tutorial Data Science Angular 7 Tutorial Angular 7 Blockchain Tutorial Blockchain Git Tutorial Git Machine Learning Tutorial Machine Learning DevOps Tutorial DevOps Preparation Trending Technologies

09/01/2023, 23:46 K-Nearest Neighbor(KNN) Algorithm for Machine Learning - Javatpoint https://www.javatpoint.com/k-nearest-neighbor-algorithm-for-machine-learning 17/18 DBMS tutorial DBMS Data Structures tutorial Data Structures DAA tutorial DAA Operating System Operating System Computer Network tutorial Computer Network Compiler Design tutorial Compiler Design Computer Organization and Architecture Computer Organization Discrete Mathematics Tutorial Discrete Mathematics Ethical Hacking Ethical Hacking Computer Graphics Tutorial Computer Graphics Software Engineering Software Engineering html tutorial Web Technology Cyber Security tutorial Cyber Security Automata Tutorial Automata C Language tutorial C Programming C++ tutorial C++ Java tutorial Java .Net Framework tutorial .Net Python tutorial Python List of Programs Programs Control Systems tutorial Control System Data Mining Tutorial Data Mining Data Warehouse Tutorial Data Warehouse B.Tech / MCA