Hot questions for Using Neural networks in sklearn pandas

Question:

I have created a very simple Artificial Neural Network in Python. In the example below I get an accuracy based off of values in a confusion matrix. These are results of the confusion matrix:

array([[3990,    2],
       [  56,  172]])

I am interested in finding the rows where it was marked as false positive(2) and false negative(56).

The following is my code:

#Import the dataset
X = DBF2.iloc[:, 1:2].values
y = DBF2.iloc[:, 2].values

#Encoding categorical data
from sklearn.preprocessing import LabelEncoder, OneHotEncoder
labelencoder_X_1 = LabelEncoder()
X[:, 0] = labelencoder_X_1.fit_transform(X[:, 0])

#Create dummy variables
onehotencoder = OneHotEncoder(categorical_features = [0])
X = onehotencoder.fit_transform(X).toarray()
#Remove 2 variables to avoid falling into the dummy variable trap
X = np.delete(X, [0], axis=1)

#Splitting the dataset
from sklearn.model_selection import train_test_split
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size = 0.2, 
random_state = 0)

#Feature Scaling
from sklearn.preprocessing import StandardScaler
sc = StandardScaler()
X_train = sc.fit_transform(X_train)
X_test = sc.transform(X_test)

#Make the ANN
import keras
from keras.models import Sequential
from keras.layers import Dense

#Initialising the ANN
classifier = Sequential()

#Adding the input layer and the first hidden layer
classifier.add(Dense(units=200, kernel_initializer='uniform', activation='relu', input_dim=400))

#Adding a second hidden layer
classifier.add(Dense(units=200, kernel_initializer='uniform', 
activation='relu'))

#Adding the output layer
classifier.add(Dense(units=1, kernel_initializer='uniform', activation='sigmoid'))

#Compiling the ANN
classifier.compile(optimizer='adam', loss='binary_crossentropy', metrics=['accuracy'])

#Fitting the ANN to the training set
classifier.fit(X_train, y_train, batch_size=10, epochs=20)                                

#Predicting the Test set results
y_pred = classifier.predict(X_test)
y_pred = (y_pred > 0.5)

#Making the confusion matrix
from sklearn.metrics import confusion_matrix
cm = confusion_matrix(y_test, y_pred)

#Find accuracy of test set
TruePos = cm[0,0]
FalsePos = cm[0,1]
TrueNeg = cm[1,1]
FalseNeg = cm[1,0]

accuracy = float(TruePos + TrueNeg) / float(TruePos + FalsePos + TrueNeg + FalseNeg)
accuracy = accuracy*100
print "Test Accuracy: " ,accuracy

Answer:

In order to do that, you can use a mask on ypred and ytest:

X_test[(y_test == 1) & (y_pred[:,0].T == 0)]
X_test[(y_test == 0) & (y_pred[:,0].T == 1)]

Or if you don't care about separating FN from FP:

X_test[(y_test != y_pred[:,0].T).T]

Question:

I am building a Neural Network Code on Python. When I executed the code on my laptop, everything was smooth. When I sent it to my friend: it gave the error shown in the picture. I tried uninstalling python and reinstalling it, unistalling pip, uninstalling sklearn, and then installing them. Nothing worked.

ImportError:DLL load failed.

Can anyone please help ?

The complete error is the following:

RESTART: C:\D-drive\AUB 2019-2020\Fall\CHEN499\Mohamad Ibrahim\499 research\neural networks\example 6\NN1.py 
Traceback (most recent call last):
  File "C:\D-drive\AUB 2019-2020\Fall\CHEN499\Mohamad Ibrahim\499 research\neural networks\example 6\NN1.py", line 1, in <module>
    from sklearn.neural_network import MLPRegressor
  File "C:\Users\jz08\AppData\Local\Programs\Python\Python37-32\lib\site-packages\sklearn\__init__.py", line 75, in <module>
    from .utils._show_versions import show_versions
  File "C:\Users\jz08\AppData\Local\Programs\Python\Python37-32\lib\site-packages\sklearn\utils\_show_versions.py", line 12, in <module>
    from ._openmp_helpers import _openmp_parallelism_enabled
ImportError: DLL load failed: The specified module could not be found.

Thank you.


Answer:

I had the same problem and resolved it only through getting back to one of the previous versions of scikit-learn (0.20.2) Try downgrading sklearn. I'm aware that this is a sub-optimal solution, but hopefully, you'll be able to advance in whatever project you are currently working one. Hope this helps !

Question:

I'm new to Machine Learning and I'm working on a python application that classifies poker hands using a dataset which I will post snippets. It does not seem to work well. And I am getting the following error:

Traceback (most recent call last):
  File "C:\Users\Anaconda3\lib\site-packages\IPython\core\interactiveshell.py", line 2881, in run_code
    exec(code_obj, self.user_global_ns, self.user_ns)
  File "<ipython-input-62-0d21cd839ce4>", line 1, in <module>
    mlp.fit(X_test, y_train.values.reshape(len(y_train), 1))
  File "C:\Users\Anaconda3\lib\site-packages\sklearn\neural_network\multilayer_perceptron.py", line 618, in fit
    return self._fit(X, y, incremental=False)
  File "C:\Users\Anaconda3\lib\site-packages\sklearn\neural_network\multilayer_perceptron.py", line 330, in _fit
    X, y = self._validate_input(X, y, incremental)
  File "C:\Users\Anaconda3\lib\site-packages\sklearn\neural_network\multilayer_perceptron.py", line 902, in _validate_input
    multi_output=True)
  File "C:\Users\Anaconda3\lib\site-packages\sklearn\utils\validation.py", line 531, in check_X_y
    check_consistent_length(X, y)
  File "C:\Users\Anaconda3\lib\site-packages\sklearn\utils\validation.py", line 181, in check_consistent_length
    " samples: %r" % [int(l) for l in lengths])
ValueError: Found input variables with inconsistent numbers of samples: [6253, 18757]

here is the code I am trying to produce:

import pandas as pnd
from sklearn.model_selection import train_test_split
from sklearn.preprocessing import StandardScaler
from sklearn.neural_network import MLPClassifier
from sklearn.metrics import classification_report, confusion_matrix

training_data = pnd.read_csv("train.csv")
training_data['id'] = range(1, len(training_data) + 1)  # For 1-base index
training_datafile = training_data
target = training_datafile['hand']
data = training_datafile.drop(['id', 'hand'], axis=1)
X = data
y = target
X_train, X_test, y_train, y_test = train_test_split(X, y)
X_train.shape
y_train.shape
scaler = StandardScaler()
scaler.fit(X_train)
X_train = scaler.transform(X_train)
X_test = scaler.transform(X_test)
mlp = MLPClassifier(hidden_layer_sizes=(100, 100, 100))
mlp.fit(X_test, y_train.values.reshape(len(y_train), 1))
predictions = mlp.predict(X_test)
len(mlp.coefs_)
len(mlp.coefs_[0])
len(mlp.intercepts_[0])
print(confusion_matrix(y_test, predictions))
print(classification_report(y_test, predictions))

The shape of X_train.shape is (18757, 10) and the shape of y_train.shape is (18757,) I have tried using following previous post

y_train.values.reshape(len(y_train), 1)

But I still get the same error. Some guidance would be of much help since I am not sure of what the shape has wrong.

Data snippet:


Answer:

You are fiting X_test instead of X_train.

mlp.fit(X_train, y_train.values.reshape(len(y_train), 1))

Question:

I am doing a homework assignment and I've been stomped for hours and hours looking up how to set the dimensions correctly for random number generated weights for a neural network. No matter how many articles I read or Google search's I do, I can't find a solution. Every time I change the dimensions, based off of the dimensions of the incoming X_train set, the program ends up having a "ValueError: operands could not be broadcast together with shapes (X,X) (y,y)". The major problem is the convoluted way in which dot product does it's math with 2d arrays. I don't know where else to turn so here I am. I will post cost and sample output to give as much information as possible, and check back every hour to see if anyone could figure this problem out. What I really need is a cookie cutter way to say...if you are pushing a n-dimensional array, weight one should be these dimensions, and weight two these dimensions, and weight three, and so on...so there are not compatibility errors in the calculations.

I've tried looking and looking on the internet for a way to decipher weights based off incoming dimensionality of data structure(s). I.E. rows and columns.

THIS IS THE CODE BELOW IN IT'S ENTIRETY SO FAR:
import numpy as np
import pandas as pd
from numpy import tanh
from sklearn.preprocessing import StandardScaler
from sklearn.model_selection import train_test_split

class NeuralNetwork():

    def __init__(self):
        print('constructor working...')
        self.inputsize = 4
        self.outputsize = 4
        self.hiddensize = 1

        self.W1 = np.random.randn(self.inputsize, self.hiddensize)
        self.W2 = np.random.randn(self.hiddensize, self.outputsize)

    def forward(self, X):
        #print('forward - X:\n', X)
        self.z = np.dot(X, self.W1)
        #print('forward - self.z:\n', self.z)
        self.z2 = self.sigmoid(self.z)
        #print('forward - self.z2:\n', self.z2)
        self.z3 = np.dot(self.z2, self.W2)
        #print('forward - self.z3:\n', self.z3)
        o = self.sigmoid(self.z3)
        print('forward - o:\n', o)
        print('forward shape of o:\n', o.shape)
        print('forward shape of X:\n', X.shape)
        return o

    def sigmoid(self, s):
        #print('sigmoid:\n', (1/(1+np.exp(-s))))
        return(1/(1+np.exp(-s)))

    def sigmoidPrime(self, s):
        return(s * (1 - s))

    def backward(self, X, y, o):
        print('backward - X:\n',X,'\ny:\n',y,'\no:\n',o)
        self.o_error = y - o
        print('backward - o_error:\n', self.o_error)
        self.o_delta = self.o_error * self.sigmoidPrime(o)
        print('backward - o_delta:\n', self.o_delta)
        self.z2_error = self.o_delta.dot(self.W2.T)
        print('backward - z2_error:\n', self.z2_error)
        self.z2_delta = self.z2_error * self.sigmoidPrime(self.z2)
        print('backward - z2_delta:\n', self.z2_delta)
        self.W1 += X.T.dot(self.z2_delta)
        print('backward - W1:\n', self.W1)
        self.W2 += self.z2.T.dot(self.o_delta)
        print('backward - W2:\n', self.W2)

    def train(self, X, y):
        o = self.forward(X)
        self.backward(X, y, o)

    def saveWeights(self):
        np.savetxt('w1.txt', self.W1, fmt='%s')
        np.savetxt('w2.txt', self.W2, fmt='%s')

    def predict(self):
        print("Predicted data based on trained weights: ")
        print("Input (scaled): \n" + str(X_test))
        print("Output: \n" + str(self.forward(X_test)))

if __name__ == "__main__":

    nn = NeuralNetwork()
    titanic_original_df = pd.read_csv(r'./titanic_data.csv')
    titanic_df = titanic_original_df.copy()
    print('titanic data shape:', titanic_df.shape)
    #print('titanic data head:\n', titanic_df.head(3))
    '''
    for col in titanic_df:
        print(col,': ',titanic_df[col].dtypes)
    for col in titanic_df:
        print(col,'- value counts:\n', titanic_df[col].value_counts())
    for col in titanic_df:
        print(col,'- null data:', titanic_df[col].isnull().sum())
    '''
    titanic_df['Age'] = titanic_df['Age'].interpolate().round()
    #print('after interpolation, Age null counts:\n', titanic_df['Age'].isnull().sum())
    titanic_df['Sex'] = pd.get_dummies(titanic_df['Sex'])
    #print('after dummy encoding Sex:\n', titanic_df['Sex'].value_counts())
    for col in titanic_df:
        print(col,'- null data:', titanic_df[col].dtypes)
    titanic_df[['Pclass','Sex']] = titanic_df[['Pclass','Sex']].astype(np.float64)
    sc = StandardScaler()
    #scaled_data = sc.fit(titanic_df[['Age','Fare']])
    #titanic_df[['Age','Fare']] = sc.transform(titanic_df[['Age','Fare']])
    #print('after scaling, Age column:\n', titanic_df['Age'].value_counts())
    #print('after scaling, Fare column:\n', titanic_df['Fare'].value_counts())
    y = titanic_df.Survived
    X = titanic_df.drop(['PassengerId','Survived','Name','SibSp','Parch','Ticket','Cabin','Embarked'], axis=1)
    X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.3, random_state=124)
    print('X_train shape:', X_train.shape)
    print('X_test shape:', X_test.shape)
    print('y_train shape:', y_train.shape)
    print('y_test shape:', y_test.shape)
    #print('X_train:\n', X_train['Sex'].value_counts())
    for i in range(1):
        print('# '+str(i)+'\n')
        #print('Input (scaled):\n'+str(X_train))
        #print('Actual output:\n'+str(y_train))
        print('Predicted output:\n'+str(nn.forward(X_train)))
        print('shape of X_train:',X_train.shape)
        print('shape of y_train:',y_train.shape)
        print('Loss:\n'+str(np.mean(np.square(y_train - nn.forward(X_train)))))
        print('\n')
        nn.train(X_train, y_train)

    nn.saveWeights()
    nn.predict()

In [55]: runfile('C:/Users/John/.spyder-py3/ProgrammingAssignment#9.py', wdir='C:/Users/John/.spyder-py3')
constructor working...
titanic data shape: (891, 12)
PassengerId - null data: int64
Survived - null data: int64
Pclass - null data: int64
Name - null data: object
Sex - null data: uint8
Age - null data: float64
SibSp - null data: int64
Parch - null data: int64
Ticket - null data: object
Fare - null data: float64
Cabin - null data: object
Embarked - null data: object
X_train shape: (623, 4)
X_test shape: (268, 4)
y_train shape: (623,)
y_test shape: (268,)
# 0

forward - o:
 [[0.50384373 0.4961504  0.50183024 0.49790133]
 [0.5001908  0.49980891 0.50009085 0.49989583]
 [0.51753819 0.48243502 0.50835355 0.49042155]
 ...
 [0.51554828 0.48442797 0.50740524 0.49150886]
 [0.50025489 0.49974472 0.50012137 0.49986083]
 [0.50000075 0.49999925 0.50000036 0.49999959]]
forward shape of o:
 (623, 4)
forward shape of X:
 (623, 4)
Predicted output:
[[0.50384373 0.4961504  0.50183024 0.49790133]
 [0.5001908  0.49980891 0.50009085 0.49989583]
 [0.51753819 0.48243502 0.50835355 0.49042155]
 ...
 [0.51554828 0.48442797 0.50740524 0.49150886]
 [0.50025489 0.49974472 0.50012137 0.49986083]
 [0.50000075 0.49999925 0.50000036 0.49999959]]
shape of X_train: (623, 4)
shape of y_train: (623,)
forward - o:
 [[0.50384373 0.4961504  0.50183024 0.49790133]
 [0.5001908  0.49980891 0.50009085 0.49989583]
 [0.51753819 0.48243502 0.50835355 0.49042155]
 ...
 [0.51554828 0.48442797 0.50740524 0.49150886]
 [0.50025489 0.49974472 0.50012137 0.49986083]
 [0.50000075 0.49999925 0.50000036 0.49999959]]
forward shape of o:
 (623, 4)
forward shape of X:
 (623, 4)
Traceback (most recent call last):

  File "<ipython-input-55-52d7c067a2dd>", line 1, in <module>
    runfile('C:/Users/John/.spyder-py3/ProgrammingAssignment#9.py', wdir='C:/Users/John/.spyder-py3')

  File "C:\Users\John\Anaconda3\lib\site-packages\spyder_kernels\customize\spydercustomize.py", line 786, in runfile
    execfile(filename, namespace)

  File "C:\Users\John\Anaconda3\lib\site-packages\spyder_kernels\customize\spydercustomize.py", line 110, in execfile
    exec(compile(f.read(), filename, 'exec'), namespace)

  File "C:/Users/John/.spyder-py3/ProgrammingAssignment#9.py", line 117, in <module>
    print('Loss:\n'+str(np.mean(np.square(y_train - nn.forward(X_train)))))

  File "C:\Users\John\Anaconda3\lib\site-packages\pandas\core\ops.py", line 1583, in wrapper
    result = safe_na_op(lvalues, rvalues)

  File "C:\Users\John\Anaconda3\lib\site-packages\pandas\core\ops.py", line 1529, in safe_na_op
    return na_op(lvalues, rvalues)

  File "C:\Users\John\Anaconda3\lib\site-packages\pandas\core\ops.py", line 1505, in na_op
    result = expressions.evaluate(op, str_rep, x, y, **eval_kwargs)

  File "C:\Users\John\Anaconda3\lib\site-packages\pandas\core\computation\expressions.py", line 208, in evaluate
    return _evaluate(op, op_str, a, b, **eval_kwargs)

  File "C:\Users\John\Anaconda3\lib\site-packages\pandas\core\computation\expressions.py", line 123, in _evaluate_numexpr
    result = _evaluate_standard(op, op_str, a, b)

  File "C:\Users\John\Anaconda3\lib\site-packages\pandas\core\computation\expressions.py", line 68, in _evaluate_standard
    return op(a, b)

ValueError: operands could not be broadcast together with shapes (623,) (623,4)

Answer:

Without going in too much in Python programming. The main thing about forwarding the data is that the weights must "fit" into the neurons.

In a mathematical expression you could say for an easy dot product eg: [2 x 3] Matrix dot multiply [3 x 1] results in a [2 x 1] Matrix, please note that the direction of a matrix multiplication is important.

You can then split it up to a Matrix A with n-rows and m-columns dot multiplied with a Matrix B which must have m-columns (!!) and an optional number of columns, lets say z-columns. The result A x B >> Results in a Matrix C with the shape of [n x z].

Looking in your code, you might have a typo in the sizes of the array, missing transpose etc.