PyTorch is an Artificial Intelligence library that has been created by Facebook’s artificial intelligence research group . The source code is accessible on GitHub and it becomes more popular day after day with more than 33.4kstars and 8.3k. This PyTorch is getting a lot of consideration since 2017 and is in constant adoption increase. Now let’s see this in action on how to create a neural network with PyTorch:
Define The network
PyTorch has an official style for you to design and build your neural network. The complete explanation or definition should stay inside an object (OOP) that is a child of the class nn.Module. And inside this class, you can see that there are just two methods or functions that need to be implemented. These functions are __init__ and forward. To provide a fundamental idea on how to utilize these, I display the following example below of an implementation of a real neural network that has 3 linear layers combined by 2 RELU layers.
import torch
import torch.nn as nn
import torch.nn.functional as F
class Net(nn.Module):
def __init__(self):
super(Net, self).__init__()
# Ceating the 3 linear layers of our neural network
# Gets the array 240 in length and outputs one with 120 in length
self.fc1 = nn.Linear(240, 120)
# Gets the array 120 in length and outputs one with 60 in length
self.fc2 = nn.Linear(120, 60)
# Gets the array 60 in length and outputs one with 10 in length
self.fc3 = nn.Linear(60, 10)
def forward(self, x):
# Setting how the model layers must be connected
# Apply the RELU activation function on the layer output 'self.fc1 = nn.Linear(240, 120)'
x = F.relu(self.fc1(x))
# Apply the RELU activation function on the layer output 'self.fc2 = nn.Linear(120, 60)'
x = F.relu(self.fc2(x))
# Passing the array within the lastest linear layer 'self.fc3 = nn.Linear(60, 10)'
x = self.fc3(x)
return x
net = Net()I will provide you with a few more description on each of the methods or functions is as follows:
- __init__
like in other python classes, the __init__ function is utilized to determine the class’s characteristics and populate each value that you want in the instantiation.
In the PyTorch setting, you must always request the super() function to initialize and start the parent or the father class. Behind that, you can determine all the layers that have parameters and it can be optimizable. The meaning of layers does not require to be in the order of which they will be utilized in the network. That is because you are just defining every layer and not how it is attached to others at this time.
- forward
This is the process where you do the big job which is on how the layers are connected. You can recognize from the model above that it is where you request the layers that you specified inside the __init__then return a value that represents your network output.
Notice that there are some other methods or functions implemented inside the forward function that is not specified in the __init__function but can traditionally be considered as layers. See at the F.relu method. We didn’t specify that in the __init__function because it doesn’t have any trainable setting. In other words, the F.relu function will always return the equal output if you give it the same input. Training the neural network doesn’t modify its performance. So you can set inside the forward function or method all the layers that don’t have any weights to be refreshed or in other words updated. On the opposite side, you must update all the layers that have weights in the __init__.
Load The Data: The Data Loaders and The Dataset
Data loaders and Dataset are the tools in the PyTorch library can determine how to reach your data. This is especially impressive when your data is scattered over various files. For example, if you have some pictures in some directory construction, you also able to personalize the way you reach it by the Dataset class.
The following program provides a primary example of how to set your Dataset class and then loop over the data reaching the elements.
import torch
import pandas as pd
from torch.utils.data import Dataset, DataLoader
class ExampleDataset(Dataset):
def __init__(self, csv_file):
self.data_frame = pd.read_csv(csv_file) # Here you will import your dataset
def __len__(self):
return len(self.data_frame)
def __getitem__(self, idx):
return self.data_frame[idx]
# create your dataset
example_dataset = ExampleDataset('file.csv')
example_data_loader = DataLoader(example_dataset, , batch_size=4, shuffle=True, num_workers=4)
for batch_index, batch in enumerate(example_data_loader):
print(batch_index, batch)There are other 3 needs functions we want to apply inside the Dataset class.
- __init__
At the start, you must put your directories information and other items that would enable you to access it. In my model, this data is loaded from a CSV file. You can easily load a file names list, where every filename describes a data point.
- __len__
In this function, you must perform a way to receive the whole size of your dataset. For example, if you do have a collection of pictures in any directories, you have to achieve such a way of counting the total number of files that make your data. In my simple model, I only obtain the length of my data frame. Nevertheless, this can be arbitrarily confused and complicated depending on how the data is saved.
- __getitem__
Here is where you perform how to obtain a single piece from the dataset. For example, if you have some pictures, this is where you want to load your picture from disk in memory and assist it in what the method returns.
For extra efficiently reach your dataset, you would work with the DataLoader class. It is a class that simply reads data in a batch at a moment in parallel while optionally shuffling the data. All those actions are extremely essential for effective training.
The Training Phase: Update the Neural Weights
Once you have set your neural network and dataset. Now, we are required to build something that enables your model to learn. In PyTorch library, this is done utilizing what is named an optimizer. This optimizer operates over all the weights and updates them. That indicates that it automatically reaches all the layers you specified in the __init__ function of your neural network class and updates them utilizing a specific learning rate and algorithm depending on the type of optimizer you prefer or choose.
The optimizer needs to have a standard to work for the optimization process. Here is where you want to determine the loss function. There are many various loss types you can perform. You can refer to this link here.
Here, I provide a fundamental sample on how to instantiate your neural network, first pick an optimizer and then loop over your data and update the weights.
import torch.optim as optim
import torch.nn as nn
# Create the neural network
net = Net()
# create an optimizer (Choose any optimazer you want)
optimizer = optim.SGD(net.parameters(), lr=0.01)
# specify the standard for the optimization
criterion = nn.MSELoss()
# The dat_set taken from someplace (any data you are working with)
for data in data_set:
# zero the gradient buffers
optimizer.zero_grad()
# Passing your data into the neural network model
output = net.forward(data)
# measuring the loss function
loss = criterion(output, target)
# Here we are propagiting the loss back
loss.backward()
# Finally, we gonna Update all the weights of your neural network model
optimizer.step()A word about Gradients and Tensors
Everything that occurs in this NN model is tensor processes. Tensors are the same as arrays but it can be in any dimension. That indicates that a one-dimensional array is a tensor. The PyTorch library is all on building processes on tensors and measuring the gradients on these processes.
Hence, PyTorch retains track of every process executed on your tensors and makes it simple for us to calculate theirs. An essential point to mention is that it will just record the leaves gradients and not the intermediate tensors. That occurs because of the idea that the gradients must be calculated for the Neural Networks model.
Summing
Whenever you are operating with the PyTorch library, the measures you must follow are these:
- Describe your Neural Network model class by putting the layers with weights that can be refreshed or updated in the __init__ method. Then specify how the flows of data through the layers inside the forward method.
- Specify how the data must be loaded by utilizing the Dataset class. After that use DataLoader class to loop over the data.
- Pick loss function and an optimizer. Loop over the training data and allow the optimizer to update the weights of the neural network model.
Note: This is a guest post, and opinion in this article is of the guest writer. If you have any issues with any of the articles posted at www.marktechpost.com please contact at asif@marktechpost.com
