{"id":9428,"date":"2016-02-26T05:50:04","date_gmt":"2016-02-26T05:50:04","guid":{"rendered":"https:\/\/www.bridge-global.com\/blog\/blog\/\/?p=9428"},"modified":"2019-09-17T10:43:21","modified_gmt":"2019-09-17T10:43:21","slug":"bayes-theorem-implementation-in-python","status":"publish","type":"post","link":"https:\/\/www.bridge-global.com\/blog\/bayes-theorem-implementation-in-python\/","title":{"rendered":"Bayes’ theorem implementation in python"},"content":{"rendered":"

\n<\/p>\n

Machine learning is a method of data analysis that automates analytical model building of data set. Using the implemented algorithms that iteratively learn from data, machine learning allows computers to find hidden insights without being explicitly programmed where to look. Naive bayes algorithm is one of the most popular machine learning technique. In this article we will look how to implement Naive bayes algorithm using python.\u00a0<\/p>\n

Before someone can understand Bayes\u2019 theorem, they need to know a couple of related concepts first, namely, the idea of Conditional Probability, and Bayes\u2019 Rule.<\/p>\n

Conditional Probability is just What is the probability that something will happen, given that something else has already happened.<\/p>\n

Let say we have a collection of people. Some of them are singers. They are either male or female. If we select a random sample, what is the probability that this person is a male? what is the probability that this person is a male and singer? Conditional Probability is the best option here. We can calculate probability like,<\/p>\n

P(Singer & Male) = P(Male) x P(Singer \/ Male)<\/p>\n

What is Bayes rule ?<\/strong><\/p>\n

We can simply define Bayes rule like this. Let A1, A2, … , An be a set of mutually exclusive events that together form the sample space S. Let B be any event from the same sample space, such that P(B) > 0. Then, P( Ak | B ) = P( Ak \u2229 B ) \/ P( A1 \u2229 B ) + P( A2 \u2229 B ) + . . . + P( An \u2229 B )<\/p>\n

What is Bayes classifier ?<\/strong><\/p>\n

Naive Bayes classifiers are a family of simple probabilistic classifiers based on applying Bayes’ theorem with strong (naive) independence assumptions between the features in machine learning. Basically we can use above theories and equations for classification problem.<\/p>\n

Bayes classifier implementation in python<\/strong><\/p>\n

Now we have to implement this great theorem in python. Fortunately we have amazing library called scikit-learn in python.In this example we are going to create some random points in three dimensional space. We classified these points onto RED and BLUE. Our task is classify new points in this three dimensional space into either BLUE or RED. Lets start with importing required modules.\u00a0<\/p>\n

import<\/span>\u00a0warnings<\/span><\/strong><\/span><\/p>\n

warnings.filterwarnings(‘ignore’<\/span>)<\/span><\/p>\n

import<\/span>\u00a0numpy<\/span>\u00a0as<\/span>\u00a0np<\/span><\/strong><\/span><\/p>\n

import<\/span>\u00a0matplotlib.pyplot<\/span>\u00a0as<\/span>\u00a0plt<\/span><\/strong><\/span><\/p>\n

from<\/span>\u00a0sklearn.naive_bayes\u00a0import<\/span><\/span><\/strong>\u00a0GaussianNB<\/span><\/p>\n

from<\/span>\u00a0IPython.display\u00a0import<\/span><\/span><\/strong>\u00a0Image<\/span><\/p>\n

Now we are going to create sample three dimensional data for training<\/p>\n

x_blue = np.array([1,2,1,5,1.5,2.4,4.9,4.5])<\/span><\/p>\n

y_blue = np.array([5,6.3,6.1,4,3.5,2,4.1,3])<\/span><\/p>\n

z_blue = np.array([5,1.3,1.1,1,3.5,2,4.1,3])<\/span><\/p>\n

 <\/p>\n

x_red = np.array([5,7,7,8,5.5,6,6.1,7.7])<\/span><\/p>\n

y_red = np.array([5,7.7,7,9,5,4,8.5,5.5])<\/span><\/p>\n

z_red = np.array([5,6.7,7,9,1,4,6.5,5.5])<\/span><\/p>\n

We have to format this data to train with sklearn<\/p>\n

red_points = np.array(zip<\/span>(x_red,y_red,z_red))<\/span><\/p>\n

blue_points = np.array(zip<\/span>(x_blue,y_blue,z_blue))<\/span><\/p>\n

points = np.concatenate([red_points,blue_points])<\/span><\/p>\n

output = np.concatenate([np.ones(x_red.size),np.zeros(x_blue.size)])<\/span><\/p>\n

Now we want to classify following points<\/p>\n

predictor = np.array([5.3,4.2,3.3])<\/span><\/p>\n

We are going to apply Bays classification theorem<\/p>\n

classifier = GaussianNB()<\/span><\/p>\n

classifier.fit(points,output)<\/span><\/p>\n

print<\/span>\u00a0classifier.predict([predictor])<\/span><\/p>\n

Lets move into more real world example. Suppose we have a list of name. We want to classify this names into Male and Female categories . Our classification process is as show in below.<\/p>\n

Image(filename=‘classification.png’<\/span>)<\/span><\/p>\n

\"\"<\/a><\/p>\n

So first step is feature extraction. I am going to observe (extract) following evidance from a name last letter, last two letter and last_is_vowel<\/p>\n

Next step is machine leraning algorithm. of cource we are going to use Naive Bayes classification.<\/p>\n

Lets start implimentation in python. As it is more a NLP problem, We could use NLTK module from python. We have two csv files traing file names.txt and predict.txt which contains name to be predicted.<\/p>\n

Lets import required module,<\/p>\n

import<\/span>\u00a0numpy<\/span>\u00a0as<\/span>\u00a0np<\/span><\/strong><\/span><\/p>\n

import<\/span>\u00a0pandas<\/span>\u00a0as<\/span>\u00a0pd<\/span><\/strong><\/span><\/p>\n

import<\/span>\u00a0nltk<\/span><\/strong><\/span><\/p>\n

Define a function that parse csv file and return feature sets. We are using panda for parsing csv file.<\/p>\n

def<\/strong><\/span>\u00a0get_data<\/span>(name, result=“gender”<\/span>):\u00a0<\/span><\/p>\n

\u00a0 \u00a0 df = pd.read_csv(name)<\/span><\/p>\n

\u00a0 \u00a0 df[‘last_letter’<\/span>] = df.apply (lambda<\/strong><\/span>\u00a0row: row[0][-1],axis=1)<\/span><\/p>\n

\u00a0 \u00a0 df[‘last_two_letter’<\/span>] = df.apply (lambda<\/strong><\/span>\u00a0row: row[0][-2:],axis=1)<\/span><\/p>\n

\u00a0 \u00a0 df[‘last_is_vowel’<\/span>] = df.apply (lambda<\/strong><\/span>\u00a0row:\u00a0int(<\/span><\/span>row[0][-1]\u00a0in<\/strong><\/span>\u00a0“aeiouy”<\/span>),axis=1)<\/span><\/p>\n

\u00a0 \u00a0 train = df.loc[:,[‘last_letter’,’last_two_letter’,’last_is_vowel’]<\/span><\/span>]<\/span><\/p>\n

\u00a0 \u00a0 train_dicts = train.T.to_dict().values()<\/span><\/p>\n

\u00a0 \u00a0 genders = df.loc[:,[result]][result]<\/span><\/p>\n

\u00a0 \u00a0 return<\/strong><\/span>\u00a0[(train_dict, gender)\u00a0for<\/strong><\/span>\u00a0train_dict,gender\u00a0in<\/strong><\/span>\u00a0zip<\/span>(train_dicts,genders)]<\/span><\/p>\n

our names.txt is looks like,<\/p>\n

df = pd.read_csv(“names.txt”<\/span>)<\/span><\/p>\n

print<\/strong><\/span>\u00a0df<\/span><\/p>\n

name gender<\/p>\n

0 ebin M<\/p>\n

1 leekas M<\/p>\n

2 jinesh M<\/p>\n

3 neethu F<\/p>\n

4 mary F<\/p>\n

5 neenu F<\/p>\n

6 sanitha F<\/p>\n

7 lekha F<\/p>\n

df[‘last_letter’<\/span>] = df.apply (lambda<\/strong><\/span>\u00a0row: row[0][-1],axis=1)<\/span><\/p>\n

df[‘last_two_letter’<\/span>] = df.apply (lambda<\/strong><\/span>\u00a0row: row[0][-2:],axis=1)<\/span><\/p>\n

df[‘last_is_vowel’<\/span>] = df.apply (lambda<\/strong><\/span>\u00a0row:\u00a0int<\/span>(row[0][-1]\u00a0in<\/strong><\/span>\u00a0“aeiouy”<\/span>),axis=1)<\/span><\/p>\n

The extracted features are like<\/p>\n

print<\/strong><\/span>\u00a0df<\/span><\/p>\n

\u00a0<\/p>\n\n\n\n\n\n\n\n\n\n\n
 <\/td>\n\u00a0name<\/em><\/td>\ngender<\/em><\/td>\nlast_letter<\/em><\/td>\nlast_two_letter<\/em><\/td>\nlast_is_vowel<\/em><\/td>\n<\/tr>\n
0<\/td>\nebin<\/td>\nM<\/td>\nn<\/td>\nin<\/td>\n0<\/td>\n<\/tr>\n
1<\/td>\nleekas<\/td>\nM<\/td>\ns<\/td>\nas<\/td>\n0<\/td>\n<\/tr>\n
2<\/td>\njinesh<\/td>\nM<\/td>\nh<\/td>\nsh<\/td>\n0<\/td>\n<\/tr>\n
3<\/td>\nneethu<\/td>\nF<\/td>\nu<\/td>\nhu<\/td>\n1<\/td>\n<\/tr>\n
4<\/td>\nmary<\/td>\nF<\/td>\ny<\/td>\nry<\/td>\n1<\/td>\n<\/tr>\n
5<\/td>\nneenu<\/td>\nF<\/td>\nu<\/td>\nnu<\/td>\n1<\/td>\n<\/tr>\n
6<\/td>\nsanitha<\/td>\nF<\/td>\na<\/td>\nha<\/td>\n1<\/td>\n<\/tr>\n
7<\/td>\nlekha<\/td>\nF<\/td>\na<\/td>\nha<\/td>\n1<\/td>\n<\/tr>\n<\/table>\n<\/div>\n

Now we want to train with data from names.txt<\/p>\n

train_set = get_data(“names.txt”<\/span>)<\/span><\/p>\n

classifier = nltk.NaiveBayesClassifier.train(train_set)<\/span><\/p>\n

Finally we want to test our model. We can use names from predict.txt file to test the created model<\/p>\n

for<\/strong><\/span>\u00a0name_and_feature\u00a0in<\/strong><\/span>\u00a0get_data(“predict.txt”,”name”<\/span>):<\/span><\/p>\n

\u00a0 \u00a0 print<\/strong>\u00a0<\/span>name_and_feature[1],“==”<\/span>, classifier.classify(name_and_feature[0])<\/span><\/p>\n

sukesh == M<\/p>\n

jithil == M<\/p>\n

sijith == M<\/p>\n

maria == F<\/p>\n

soumya == F<\/p>\n

neethu == F<\/p>\n

<\/p>\n","protected":false},"excerpt":{"rendered":"

Machine learning is a method of data analysis that automates analytical model building of data set. Using the implemented algorithms that iteratively learn from data, machine learning allows computers to find hidden insights without being explicitly programmed where to look. …<\/p>\n","protected":false},"author":88,"featured_media":9431,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[14],"tags":[],"class_list":["post-9428","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-offshoring"],"featured_image_src":"https:\/\/www.bridge-global.com\/blog\/wp-content\/uploads\/2016\/02\/df1.png","author_info":{"display_name":"Ebin","author_link":"https:\/\/www.bridge-global.com\/blog\/author\/ebin\/"},"_links":{"self":[{"href":"https:\/\/www.bridge-global.com\/blog\/wp-json\/wp\/v2\/posts\/9428","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.bridge-global.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.bridge-global.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.bridge-global.com\/blog\/wp-json\/wp\/v2\/users\/88"}],"replies":[{"embeddable":true,"href":"https:\/\/www.bridge-global.com\/blog\/wp-json\/wp\/v2\/comments?post=9428"}],"version-history":[{"count":6,"href":"https:\/\/www.bridge-global.com\/blog\/wp-json\/wp\/v2\/posts\/9428\/revisions"}],"predecessor-version":[{"id":28339,"href":"https:\/\/www.bridge-global.com\/blog\/wp-json\/wp\/v2\/posts\/9428\/revisions\/28339"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.bridge-global.com\/blog\/wp-json\/wp\/v2\/media\/9431"}],"wp:attachment":[{"href":"https:\/\/www.bridge-global.com\/blog\/wp-json\/wp\/v2\/media?parent=9428"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.bridge-global.com\/blog\/wp-json\/wp\/v2\/categories?post=9428"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.bridge-global.com\/blog\/wp-json\/wp\/v2\/tags?post=9428"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}