2 Classes you need to self define when you implement Graph Neural Network(GNN):
MyNet(pytorch.nn.Moduel) MyGraphModel(torch_geometric.nn.MessagePassing) MyNet(pytorch.nn.Moduel) In your overall model structure, you should implement:
(in __init__): call a MessagePassing child class to build massage-passing model (in forward): make sure the data follows the requirement of MessagePassing child class do the “iterative massage passing"(K-times) in forward, the final output will be the node embedding you need.
To train a model, we need:
loss function Optimizer model data Some facts testing is very expensive, since you have to look at whole testing set.
| First first, the forward of nn.Module is the one single most important part of the entire implementation. It may effect:
other part of the nn.Module customized dataloader optimizer and loss function.
PyTorch has two primitives to work with data: torch.utils.data.DataLoader and torch.utils.data.Dataset.
Dataset is an object that stores the samples and their corresponding labels there are two types of Dataset: https://pytorch.
Single Machine Data Parallel When data is too large to feed in a single GPU data is scatterred across GPUs Model is replicated on each GPUs Pytorch Gather the ouput from each GPUs, and compute loss, gradient, updates weights Code is simple, just one line:
Training Streams Recall the Typical Training Process:
from torch.optim import SGD loader = ... model = MyNet() criterion = torch.nn.CrossEntropyLoss() optimizer = SGD(model.parameters) for epoch in range(10): for batch, labels in loader: outputs = model(batch) loss = criterion(outputs, labels) optimizer.