Hour 18 Neural Networks

#### Concept

Neural Networks are a set of algorithms, modeled loosely after the human brain, designed to recognize patterns. They interpret sensory data through a kind of machine perception, labeling, or clustering of raw input. The patterns they recognize are numerical, contained in vectors, into which all real-world data, be it images, sound, text, or time series, must be translated.

#### Key Features of Neural Networks

1. Layers: Composed of an input layer, hidden layers, and an output layer.

2. Neurons: Basic units that take inputs, apply weights, add a bias, and pass through an activation function.

3. Activation Functions: Functions applied to the neurons' output, introducing non-linearity (e.g., ReLU, sigmoid, tanh).

4. Backpropagation: Learning algorithm for training the network by minimizing the error.

5. Training: Adjusts weights based on the error calculated from the output and the expected output.

#### Key Steps

1. Initialize Weights and Biases: Start with small random values.

2. Forward Propagation: Pass inputs through the network layers to get predictions.

3. Calculate Loss: Measure the difference between predictions and actual values.

4. Backward Propagation: Compute the gradient of the loss function and update weights.

5. Iteration: Repeat forward and backward propagation for a set number of epochs or until the loss converges.

#### Implementation

Let's implement a simple Neural Network using Keras on the Breast Cancer dataset.

##### Example

import numpy as np

from sklearn.datasets import load_breast_cancer

from sklearn.model_selection import train_test_split

from sklearn.preprocessing import StandardScaler

from sklearn.metrics import accuracy_score, confusion_matrix,

     classification_report

import tensorflow as tf

from tensorflow.keras.models import Sequential

from tensorflow.keras.layers import Dense

# Load the Breast Cancer dataset

data = load_breast_cancer()

X = data.data

y = data.target

# Splitting the data into training and testing sets

X_train, X_test, y_train, y_test =

train_test_split(X, y, test_size=0.2, random_state=42)

# Standardizing the data

scaler = StandardScaler()

X_train = scaler.fit_transform(X_train)

X_test = scaler.transform(X_test)

# Creating the Neural Network model

model = Sequential([

    Dense(30, input_shape=(X_train.shape[1],), activation='relu'),

    Dense(15, activation='relu'),

    Dense(1, activation='sigmoid')

])

# Compiling the model

model.compile(optimizer='adam', loss='binary_crossentropy',

metrics=['accuracy'])

# Training the model

model.fit(X_train, y_train, epochs=50, batch_size=10,

validation_split=0.2, verbose=1)

# Making predictions

y_pred = (model.predict(X_test) > 0.5).astype("int32")

# Evaluating the model

accuracy = accuracy_score(y_test, y_pred)

conf_matrix = confusion_matrix(y_test, y_pred)

class_report = classification_report(y_test, y_pred)

print(f"Accuracy: {accuracy}")

print(f"Confusion Matrix:\n{conf_matrix}")

print(f"Classification Report:\n{class_report}")

Result

Epoch 50/50

37/37 ━━━━━━━━━━━━━━━━━━━━ 0s 4ms/step - accuracy: 1.0000 - loss: 0.0051 - val_accuracy: 0.9560 - val_loss: 0.1234

4/4 ━━━━━━━━━━━━━━━━━━━━ 0s 15ms/step

Accuracy: 0.9649122807017544

Confusion Matrix:

[[41  2]

 [ 2 69]]

Classification Report:

              precision    recall  f1-score   support

           0       0.95      0.95      0.95        43

           1       0.97      0.97      0.97        71

    accuracy                           0.96       114

   macro avg       0.96      0.96      0.96       114

weighted avg       0.96      0.96      0.96       114

#### Explanation of the Code

1. Libraries: We import necessary libraries like numpy, sklearn, and tensorflow.keras.

2. Data Preparation: We load the Breast Cancer dataset with features and the target variable (malignant or benign).

3. Train-Test Split: We split the data into training and testing sets.

4. Data Standardization: We standardize the data for better convergence of the neural network.

5. Model Creation: We create a sequential neural network with an input layer, two hidden layers, and an output layer.

6. Model Compilation: We compile the model with the Adam optimizer and binary cross-entropy loss function.

7. Model Training: We train the model for 50 epochs with a batch size of 10 and validate on 20% of the training data.

8. Predictions: We make predictions on the test set and convert them to binary values.

9. Evaluation:

    - Accuracy: Measures the proportion of correctly classified instances.

    - Confusion Matrix: Shows the counts of true positive, true negative, false positive, and false negative predictions.

    - Classification Report: Provides precision, recall, F1-score, and support for each class.

print(f"Accuracy: {accuracy}")

print(f"Confusion Matrix:\n{conf_matrix}")

print(f"Classification Report:\n{class_report}")

#### Advanced Features of Neural Networks

1. Hyperparameter Tuning: Tuning the number of layers, neurons, learning rate, batch size, and epochs for optimal performance.

2. Regularization Techniques: 

   - Dropout: Randomly drops neurons during training to prevent overfitting.

   - L1/L2 Regularization: Adds penalties to the loss function for large weights to prevent overfitting.

3. Early Stopping: Stops training when the validation loss stops improving.

4. Batch Normalization: Normalizes inputs of each layer to stabilize and accelerate training.

# Example with Dropout and Batch Normalization  (as per Modify the above from line 33 in vscode onwards)

# # Training the model

# model.fit(X_train, y_train, epochs=50, batch_size=10,

validation_split=0.2, verbose=1)

# # Making predictions

# y_pred = (model.predict(X_test) > 0.5).astype("int32")

# # Evaluating the model

# accuracy = accuracy_score(y_test, y_pred)

# conf_matrix = confusion_matrix(y_test, y_pred)

# class_report = classification_report(y_test, y_pred)

# print(f"Accuracy: {accuracy}")

# print(f"Confusion Matrix:\n{conf_matrix}")

# print(f"Classification Report:\n{class_report}")

model = Sequential([

    Dense(30, input_shape=(X_train.shape[1],), activation='relu'),

    BatchNormalization(),

    Dropout(0.5),

    Dense(15, activation='relu'),

    BatchNormalization(),

    Dropout(0.5),

    Dense(1, activation='sigmoid')

])

# Compiling and training remain the same as before

model.compile(optimizer='adam', loss='binary_crossentropy',

metrics=['accuracy'])

model.fit(X_train, y_train, epochs=50, batch_size=10,

validation_split=0.2, verbose=1)

y_pred = (model.predict(X_test) > 0.5).astype("int32")

# Evaluating the model

accuracy = accuracy_score(y_test, y_pred)

conf_matrix = confusion_matrix(y_test, y_pred)

class_report = classification_report(y_test, y_pred)

print(f"Accuracy: {accuracy}")

print(f"Confusion Matrix:\n{conf_matrix}")

print(f"Classification Report:\n{class_report}")

Results

Epoch 50/50

37/37 ━━━━━━━━━━━━━━━━━━━━ 0s 4ms/step - accuracy: 0.9634 -

loss: 0.1376 - val_accuracy: 0.9670 - val_loss: 0.1026

4/4 ━━━━━━━━━━━━━━━━━━━━ 0s 24ms/step

Accuracy: 0.9824561403508771

Confusion Matrix:

[[41  2]

 [ 0 71]]

Classification Report:

              precision    recall  f1-score   support

           0       1.00      0.95      0.98        43

           1       0.97      1.00      0.99        71

    accuracy                           0.98       114

   macro avg       0.99      0.98      0.98       114

weighted avg       0.98      0.98      0.98       114

#### Applications

Neural Networks are widely used in various fields such as:

- Computer Vision: Image classification, object detection, facial recognition.

- Natural Language Processing: Sentiment analysis, language translation, text generation.

- Healthcare: Disease prediction, medical image analysis, drug discovery.

- Finance: Stock price prediction, fraud detection, credit scoring.

- Robotics: Autonomous driving, robotic control, gesture recognition.

Neural Networks' ability to learn from data and recognize complex patterns makes them suitable for a wide range of applications.

Best Data Science & Machine Learning Resources: https://topmate.io/coding/914624

ENJOY LEARNING 👍👍