Skip to content

Commit

Permalink
Linting
Browse files Browse the repository at this point in the history
  • Loading branch information
@javidahmed64592
javidahmed64592 committed Oct 9, 2024
1 parent 1d03573 commit 39bd6f6
Show file tree
Hide file tree
Showing 5 changed files with 45 additions and 50 deletions.
51 changes: 24 additions & 27 deletions example_training.ipynb
View file
Original file line number Diff line number Diff line change
Expand Up @@ -4,36 +4,36 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"# Neural Network - Training on Binary Lists"
"# Neural Network - Training on Binary Lists\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"This notebook shows how to set up a training scenario using a neural network.\n",
"It will be used to feedforward a list of bits and convert it to a number."
"It will be used to feedforward a list of bits and convert it to a number.\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Code Implementation"
"## Code Implementation\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Libraries and Helper Functions"
"### Libraries and Helper Functions\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"First, we need to import `NeuralNetwork`."
"First, we need to import `NeuralNetwork`.\n"
]
},
{
Expand All @@ -46,16 +46,18 @@
"\n",
"import numpy as np\n",
"\n",
"from neural_network.layer import HiddenLayer, InputLayer, OutputLayer\n",
"from neural_network.math.activation_functions import LinearActivation, SigmoidActivation\n",
"from neural_network.neural_network import NeuralNetwork\n",
"from neural_network.nn.layer import HiddenLayer, InputLayer, OutputLayer"
"\n",
"rng = np.random.default_rng()\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"We can define some helper functions to analyse the results from the training."
"We can define some helper functions to analyse the results from the training.\n"
]
},
{
Expand All @@ -68,8 +70,7 @@
" \"\"\"\n",
" Get message prefix with current datetime.\n",
" \"\"\"\n",
" time_msg = f\"[{datetime.now().strftime('%d-%m-%G | %H:%M:%S')}]\"\n",
" return time_msg\n",
" return f\"[{datetime.now().strftime('%d-%m-%G | %H:%M:%S')}]\"\n",
"\n",
"\n",
"def print_system_msg(msg: str) -> None:\n",
Expand Down Expand Up @@ -104,23 +105,22 @@
" \"\"\"\n",
" squared = np.square(errors)\n",
" mean = np.average(squared)\n",
" rms = np.sqrt(mean)\n",
" return rms"
" return np.sqrt(mean)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Creating Methods to Generate Training Data"
"### Creating Methods to Generate Training Data\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"We will be using 8-bit numbers to train the neural network.\n",
"We can use the following bit map to convert numbers between integers and byte lists."
"We can use the following bit map to convert numbers between integers and byte lists.\n"
]
},
{
Expand Down Expand Up @@ -155,16 +155,15 @@
" _num_bin = bin(num)\n",
" _num_bytes = _num_bin[2:]\n",
" _padding = [0] * (NUM_BITS - len(_num_bytes))\n",
" byte_list = _padding + [int(b) for b in _num_bytes]\n",
" return byte_list"
" return _padding + [int(b) for b in _num_bytes]"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"With 8 bits, we can generate numbers between 0-255.\n",
"The neural network outputs numbers between 0-1 and therefore we need to map the values accordingly."
"The neural network outputs numbers between 0-1 and therefore we need to map the values accordingly.\n"
]
},
{
Expand All @@ -191,8 +190,7 @@
" Returns:\n",
" y (float): Number mapped to new range\n",
" \"\"\"\n",
" y = (x - in_min) * (out_max - out_min) / (in_max - in_min) + out_min\n",
" return y\n",
" return (x - in_min) * (out_max - out_min) / (in_max - in_min) + out_min\n",
"\n",
"\n",
"def training_data_from_num(num: int) -> tuple[list[int], list[float]]:\n",
Expand All @@ -207,15 +205,14 @@
" \"\"\"\n",
" _byte_list = np.array(num_to_byte_list(num))\n",
" _mapped_num = map_val(num, IN_LIMS[0], IN_LIMS[1], OUT_LIMS[0], OUT_LIMS[1])\n",
" training_data = (_byte_list, _mapped_num)\n",
" return training_data"
" return (_byte_list, _mapped_num)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Creating the Neural Network"
"### Creating the Neural Network\n"
]
},
{
Expand Down Expand Up @@ -249,7 +246,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"### Running the Algorithm"
"### Running the Algorithm\n"
]
},
{
Expand All @@ -260,7 +257,7 @@
"\n",
"First, we need to create the dataset.\n",
"We will generate a training dataset with the specified size.\n",
"We will then allocate a subset of this data to be used for testing."
"We will then allocate a subset of this data to be used for testing.\n"
]
},
{
Expand All @@ -286,7 +283,7 @@
"dataset_outputs = []\n",
"\n",
"for _ in range(dataset_size):\n",
" random_num = np.random.randint(low=IN_LIMS[0], high=(IN_LIMS[1] + 1))\n",
" random_num = rng.integers(low=IN_LIMS[0], high=(IN_LIMS[1] + 1))\n",
" training_input, expected_output = training_data_from_num(random_num)\n",
"\n",
" dataset_inputs.append(training_input)\n",
Expand All @@ -304,7 +301,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"Now we can begin the training process with the training inputs and expected outputs."
"Now we can begin the training process with the training inputs and expected outputs.\n"
]
},
{
Expand Down Expand Up @@ -344,7 +341,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"We can then test the neural network against some inputs and expected outputs to check its accuracy."
"We can then test the neural network against some inputs and expected outputs to check its accuracy.\n"
]
},
{
Expand Down Expand Up @@ -422,7 +419,7 @@
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.11.4"
"version": "3.12.6"
}
},
"nbformat": 4,
Expand Down
4 changes: 3 additions & 1 deletion neural_network/layer.py
View file
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,8 @@
from neural_network.math.activation_functions import ActivationFunction
from neural_network.math.matrix import Matrix

rng = np.random.default_rng()


class Layer:
"""
Expand Down Expand Up @@ -61,7 +63,7 @@ def weights(self, new_weights: Matrix) -> None:

@property
def random_weight(self) -> float:
return np.random.uniform(low=self._weights_range[0], high=self._weights_range[1])
return rng.uniform(low=self._weights_range[0], high=self._weights_range[1])

@property
def bias(self) -> Matrix:
Expand Down
19 changes: 9 additions & 10 deletions neural_network/math/matrix.py
View file
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,8 @@

from neural_network.math.activation_functions import ActivationFunction

rng = np.random.default_rng()


class Matrix:
"""
Expand Down Expand Up @@ -62,8 +64,7 @@ def from_array(cls, matrix_array: NDArray | list[list[float]] | list[float]) ->
matrix_array = np.expand_dims(matrix_array, axis=1)
_rows, _cols = matrix_array.shape

matrix = cls(_rows, _cols, matrix_array)
return matrix
return cls(_rows, _cols, matrix_array)

@classmethod
def random_matrix(cls, rows: int, cols: int, low: float, high: float) -> Matrix:
Expand All @@ -79,9 +80,8 @@ def random_matrix(cls, rows: int, cols: int, low: float, high: float) -> Matrix:
Returns:
matrix (Matrix): Matrix with random values
"""
_vals = np.random.uniform(low=low, high=high, size=(rows, cols))
matrix = cls.from_array(_vals)
return matrix
_vals = rng.uniform(low=low, high=high, size=(rows, cols))
return cls.from_array(_vals)

@classmethod
def random_column(cls, rows: int, low: float, high: float) -> Matrix:
Expand All @@ -96,8 +96,7 @@ def random_column(cls, rows: int, low: float, high: float) -> Matrix:
Returns:
matrix (Matrix): Column Matrix with random values
"""
matrix = cls.random_matrix(rows=rows, cols=1, low=low, high=high)
return matrix
return cls.random_matrix(rows=rows, cols=1, low=low, high=high)

@staticmethod
def add(matrix: Matrix, other_matrix: Matrix) -> Matrix:
Expand Down Expand Up @@ -218,9 +217,9 @@ def mutated_matrix(matrix: Matrix, mutation_rate: float, random_range: list[floa
Returns:
new_matrix (Matrix): Mutated Matrix
"""
_mutation_matrix = np.random.uniform(low=0, high=1, size=matrix.shape)
_mutation_matrix = rng.uniform(low=0, high=1, size=matrix.shape)
new_matrix = np.where(
_mutation_matrix < mutation_rate, np.random.uniform(low=random_range[0], high=random_range[1]), matrix.vals
_mutation_matrix < mutation_rate, rng.uniform(low=random_range[0], high=random_range[1]), matrix.vals
)
return Matrix.from_array(new_matrix)

Expand Down Expand Up @@ -284,5 +283,5 @@ def shift_vals(self, shift: float) -> None:
Parameters:
shift (float): Factor to shift values by
"""
_mult_array = np.random.uniform(low=(1 - shift), high=(1 + shift), size=self.shape)
_mult_array = rng.uniform(low=(1 - shift), high=(1 + shift), size=self.shape)
self._vals = _mult_array
15 changes: 5 additions & 10 deletions neural_network/math/nn_math.py
View file
Original file line number Diff line number Diff line change
Expand Up @@ -21,8 +21,7 @@ def feedforward_through_layer(
"""
output_vals = Matrix.multiply(weights, input_vals)
output_vals = Matrix.add(output_vals, bias)
output_vals = Matrix.map(output_vals, activation)
return output_vals
return Matrix.map(output_vals, activation)


def calculate_gradient(layer_vals: Matrix, errors: Matrix, activation: ActivationFunction, lr: float) -> Matrix:
Expand All @@ -40,8 +39,7 @@ def calculate_gradient(layer_vals: Matrix, errors: Matrix, activation: Activatio
"""
gradient = Matrix.from_array(np.vectorize(activation.derivative)(layer_vals.vals))
gradient = Matrix.multiply_element_wise(gradient, errors)
gradient = Matrix.multiply(gradient, lr)
return gradient
return Matrix.multiply(gradient, lr)


def calculate_delta(layer_vals: Matrix, gradients: Matrix) -> Matrix:
Expand All @@ -56,8 +54,7 @@ def calculate_delta(layer_vals: Matrix, gradients: Matrix) -> Matrix:
delta (Matrix): Delta factors
"""
incoming_transposed = Matrix.transpose(layer_vals)
delta = Matrix.multiply(gradients, incoming_transposed)
return delta
return Matrix.multiply(gradients, incoming_transposed)


def calculate_error_from_expected(expected_outputs: Matrix, actual_outputs: Matrix) -> Matrix:
Expand All @@ -71,8 +68,7 @@ def calculate_error_from_expected(expected_outputs: Matrix, actual_outputs: Matr
Returns:
errors (Matrix): Difference between expected and actual outputs
"""
errors = Matrix.subtract(expected_outputs, actual_outputs)
return errors
return Matrix.subtract(expected_outputs, actual_outputs)


def calculate_next_errors(weights: Matrix, calculated_errors: Matrix) -> Matrix:
Expand All @@ -87,5 +83,4 @@ def calculate_next_errors(weights: Matrix, calculated_errors: Matrix) -> Matrix:
errors (Matrix): Next errors
"""
weights_t = Matrix.transpose(weights)
errors = Matrix.multiply(weights_t, calculated_errors)
return errors
return Matrix.multiply(weights_t, calculated_errors)
6 changes: 4 additions & 2 deletions tests/math/test_matrix.py
View file
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,8 @@
from neural_network.math.activation_functions import ActivationFunction
from neural_network.math.matrix import Matrix

rng = np.random.default_rng()


class TestMatrix:
def test_given_no_vals_when_creating_matrix_then_check_matrix_has_zero_vals(
Expand Down Expand Up @@ -36,7 +38,7 @@ def test_given_shape_when_creating_random_column_then_check_matrix_has_correct_s
def test_given_2d_array_when_creating_matrix_then_check_matrix_has_correct_shape(
self, mock_weights_range: list[float], mock_len_inputs: int, mock_len_hidden: list[int]
) -> None:
test_array = np.random.uniform(
test_array = rng.uniform(
low=mock_weights_range[0], high=mock_weights_range[1], size=(mock_len_inputs, mock_len_hidden[0])
)
test_matrix = Matrix.from_array(matrix_array=test_array)
Expand All @@ -49,7 +51,7 @@ def test_given_2d_array_when_creating_matrix_then_check_matrix_has_correct_shape
def test_given_1d_array_when_creating_matrix_then_check_matrix_has_correct_shape(
self, mock_weights_range: list[float], mock_len_hidden: list[int]
) -> None:
test_array = np.random.uniform(low=mock_weights_range[0], high=mock_weights_range[1], size=(mock_len_hidden[0]))
test_array = rng.uniform(low=mock_weights_range[0], high=mock_weights_range[1], size=(mock_len_hidden[0]))
test_matrix = Matrix.from_array(matrix_array=test_array)

expected_shape = (mock_len_hidden[0], 1)
Expand Down

0 comments on commit 39bd6f6

Please sign in to comment.