Link
Highlights&&Note
torch.utils.data.DataLoaderandtorch.utils.data.Dataset.Datasetstores the samples and their corresponding labels, andDataLoaderwraps an iterable around theDataset.
Dataset 用于定义和存储数据及标签，而 DataLoader 则将 Dataset 包装成一个可迭代对象，方便进行批量数据处理和多线程数据加载。 ^40d33fcfPyTorch offers domain-specific libraries such as TorchText,TorchVision, and TorchAudio
^03b16e1aTo define a neural network in PyTorch, we create a class that inherits from nn.Module. We define the layers of the network in the
__init__function and specify how data will pass through the network in theforwardfunction. To accelerate operations in the neural network, we move it to the GPU or MPS if available.
Using cuda device
NeuralNetwork(
(flatten): Flatten(startdim=1, end_dim=-1)
(linear_relu_stack): Sequential(
(0): Linear(in_features=784, out_features=512, bias=True)
(1): ReLU()
(2): Linear(in_features=512, out_features=512, bias=True)
(3): ReLU()
(4): Linear(in_features=512, out_features=10, bias=True)
)
)
继承自nn.Module的类来定义一个新的神经网络_ ^60509680
Content
Note
Click hereto download the full example code
Learn the Basics ||Quickstart ||Tensors ||Datasets & DataLoaders ||Transforms ||Build Model ||Autograd ||Optimization ||Save & Load Model
Quickstart
This section runs through the API for common tasks in machine learning. Refer to the links in each section to dive deeper.
Working with data
PyTorch has two primitives to work with data:==torch====.utils====.data====.DataLoader== ==and== ==torch====.utils====.data====.Dataset====.====Dataset== ==stores the samples and their corresponding labels, and== ==DataLoader== ==wraps an iterable around
the== ==Dataset====.==
import torch
from torch import nn
from torch.utils.data import DataLoader
from torchvision import datasets
from torchvision.transforms import ToTensor
==PyTorch offers domain-specific libraries such as== ==TorchText====,====TorchVision====, and== ==TorchAudio==, all of which include datasets. For this tutorial, we will be using a TorchVision dataset.
The torchvision.datasets module contains Dataset objects for many real-world vision data like CIFAR, COCO (full list here). In this tutorial, we use the FashionMNIST dataset. Every TorchVision Dataset includes two arguments: transform andtarget_transform to modify the samples and labels respectively.
Downloading http://fashion-mnist.s3-website.eu-central-1.amazonaws.com/train-images-idx3-ubyte.gz
Downloading http://fashion-mnist.s3-website.eu-central-1.amazonaws.com/train-images-idx3-ubyte.gz to data/FashionMNIST/raw/train-images-idx3-ubyte.gz
0%| | 0/26421880 [00:00<?, ?it/s]
0%| | 65536/26421880 [00:00<01:11, 367533.95it/s]
1%| | 229376/26421880 [00:00<00:38, 687510.81it/s]
3%|3 | 917504/26421880 [00:00<00:12, 2122627.18it/s]
12%|#1 | 3145728/26421880 [00:00<00:03, 6173840.09it/s]
32%|###1 | 8323072/26421880 [00:00<00:01, 14317420.15it/s]
52%|#####2 | 13860864/26421880 [00:01<00:00, 19911191.27it/s]
73%|#######2 | 19169280/26421880 [00:01<00:00, 23027294.46it/s]
95%|#########4| 25001984/26421880 [00:01<00:00, 26015038.29it/s]
100%|##########| 26421880/26421880 [00:01<00:00, 18290784.99it/s]
Extracting data/FashionMNIST/raw/train-images-idx3-ubyte.gz to data/FashionMNIST/raw
Downloading http://fashion-mnist.s3-website.eu-central-1.amazonaws.com/train-labels-idx1-ubyte.gz
Downloading http://fashion-mnist.s3-website.eu-central-1.amazonaws.com/train-labels-idx1-ubyte.gz to data/FashionMNIST/raw/train-labels-idx1-ubyte.gz
0%| | 0/29515 [00:00<?, ?it/s]
100%|##########| 29515/29515 [00:00<00:00, 325111.24it/s]
Extracting data/FashionMNIST/raw/train-labels-idx1-ubyte.gz to data/FashionMNIST/raw
Downloading http://fashion-mnist.s3-website.eu-central-1.amazonaws.com/t10k-images-idx3-ubyte.gz
Downloading http://fashion-mnist.s3-website.eu-central-1.amazonaws.com/t10k-images-idx3-ubyte.gz to data/FashionMNIST/raw/t10k-images-idx3-ubyte.gz
0%| | 0/4422102 [00:00<?, ?it/s]
1%|1 | 65536/4422102 [00:00<00:12, 362694.62it/s]
5%|5 | 229376/4422102 [00:00<00:06, 682437.48it/s]
16%|#5 | 688128/4422102 [00:00<00:02, 1525993.86it/s]
27%|##6 | 1179648/4422102 [00:00<00:01, 1995322.62it/s]
39%|###9 | 1736704/4422102 [00:00<00:01, 2382528.91it/s]
53%|#####2 | 2326528/4422102 [00:01<00:00, 2683842.12it/s]
67%|######7 | 2981888/4422102 [00:01<00:00, 2990009.04it/s]
84%|########4 | 3735552/4422102 [00:01<00:00, 3358051.05it/s]
100%|##########| 4422102/4422102 [00:01<00:00, 2867549.97it/s]
Extracting data/FashionMNIST/raw/t10k-images-idx3-ubyte.gz to data/FashionMNIST/raw
Downloading http://fashion-mnist.s3-website.eu-central-1.amazonaws.com/t10k-labels-idx1-ubyte.gz
Downloading http://fashion-mnist.s3-website.eu-central-1.amazonaws.com/t10k-labels-idx1-ubyte.gz to data/FashionMNIST/raw/t10k-labels-idx1-ubyte.gz
0%| | 0/5148 [00:00<?, ?it/s]
100%|##########| 5148/5148 [00:00<00:00, 38765308.78it/s]
Extracting data/FashionMNIST/raw/t10k-labels-idx1-ubyte.gz to data/FashionMNIST/raw
We pass the Dataset as an argument to DataLoader. This wraps an iterable over our dataset, and supports automatic batching, sampling, shuffling and multiprocess data loading. Here we define a batch size of 64, i.e. each element in the dataloader iterable will return a batch of 64 features and labels.
Shape of X [N, C, H, W]: torch.Size([64, 1, 28, 28])
Shape of y: torch.Size([64]) torch.int64
Read more about loading data in PyTorch.
Creating Models
==To define a neural network in PyTorch, we create a class that inherits
from== ==nn.Module====. We define the layers of the network
in the== ==__init__== ==function and specify how data will pass through the network in the== ==forward== ==function. To accelerate
operations in the neural network, we move it to the GPU or MPS if available.==
==Using cuda device
NeuralNetwork(
(flatten): Flatten(start_dim=1, end_dim=-1)
(linear_relu_stack): Sequential(
(0): Linear(in_features=784, out_features=512, bias=True)
(1): ReLU()
(2): Linear(in_features=512, out_features=512, bias=True)
(3): ReLU()
(4): Linear(in_features=512, out_features=10, bias=True)
)
)==
Read more about building neural networks in PyTorch.
Optimizing the Model Parameters
To train a model, we need a loss functionand an optimizer.
In a single training loop, the model makes predictions on the training dataset (fed to it in batches), and backpropagates the prediction error to adjust the model’s parameters.
def train(dataloader, model, loss_fn, optimizer):
size = len(dataloader.dataset)
model.train()
for batch, (X, y) in enumerate(dataloader):
X, y = X.to(device), y.to(device)
# Compute prediction error
[pred](https://pytorch.org/docs/stable/tensors.html#torch.Tensor "torch.Tensor") = model([X](https://pytorch.org/docs/stable/tensors.html#torch.Tensor "torch.Tensor"))
loss = [loss_fn](https://pytorch.org/docs/stable/generated/torch.nn.CrossEntropyLoss.html#torch.nn.CrossEntropyLoss "torch.nn.CrossEntropyLoss")([pred](https://pytorch.org/docs/stable/tensors.html#torch.Tensor "torch.Tensor"), y)
# Backpropagation
loss.backward()
[optimizer.step](https://pytorch.org/docs/stable/generated/torch.optim.SGD.html#torch.optim.SGD.step "torch.optim.SGD.step")()
[optimizer.zero_grad](https://pytorch.org/docs/stable/generated/torch.optim.SGD.html#torch.optim.SGD.zero%5Fgrad "torch.optim.SGD.zero_grad")()
if batch % 100 == 0:
loss, current = loss.item(), (batch + 1) * len([X](https://pytorch.org/docs/stable/tensors.html#torch.Tensor "torch.Tensor"))
print(f"loss: {loss:>7f} [{current:>5d}/{size:>5d}]")
We also check the model’s performance against the test dataset to ensure it is learning.
def test(dataloader, model, loss_fn):
size = len(dataloader.dataset)
num_batches = len(dataloader)
model.eval()
test_loss, correct = 0, 0
with torch.no_grad():
for X, y in dataloader:
X, y = X.to(device), y.to(device)
pred = model(X)
test_loss += loss_fn(pred, y).item()
correct += (pred.argmax(1) == y).type(torch.float).sum().item()
test_loss /= num_batches
correct /= size
print(f”Test Error: \n Accuracy: {(100*correct):>0.1f}%, Avg loss: {test_loss:>8f} \n”)
The training process is conducted over several iterations (epochs). During each epoch, the model learns parameters to make better predictions. We print the model’s accuracy and loss at each epoch; we’d like to see the accuracy increase and the loss decrease with every epoch.
Epoch 1
loss: 2.303494 [ 64/60000]
loss: 2.294637 [ 6464/60000]
loss: 2.277102 [12864/60000]
loss: 2.269977 [19264/60000]
loss: 2.254235 [25664/60000]
loss: 2.237146 [32064/60000]
loss: 2.231055 [38464/60000]
loss: 2.205037 [44864/60000]
loss: 2.203240 [51264/60000]
loss: 2.170889 [57664/60000]
Test Error:
Accuracy: 53.9%, Avg loss: 2.168588
Epoch 2
loss: 2.177787 [ 64/60000]
loss: 2.168083 [ 6464/60000]
loss: 2.114910 [12864/60000]
loss: 2.130412 [19264/60000]
loss: 2.087473 [25664/60000]
loss: 2.039670 [32064/60000]
loss: 2.054274 [38464/60000]
loss: 1.985457 [44864/60000]
loss: 1.996023 [51264/60000]
loss: 1.917241 [57664/60000]
Test Error:
Accuracy: 60.2%, Avg loss: 1.920374
Epoch 3
loss: 1.951705 [ 64/60000]
loss: 1.919516 [ 6464/60000]
loss: 1.808730 [12864/60000]
loss: 1.846550 [19264/60000]
loss: 1.740618 [25664/60000]
loss: 1.698733 [32064/60000]
loss: 1.708889 [38464/60000]
loss: 1.614436 [44864/60000]
loss: 1.646475 [51264/60000]
loss: 1.524308 [57664/60000]
Test Error:
Accuracy: 61.4%, Avg loss: 1.547092
Epoch 4
loss: 1.612695 [ 64/60000]
loss: 1.570870 [ 6464/60000]
loss: 1.424730 [12864/60000]
loss: 1.489542 [19264/60000]
loss: 1.367256 [25664/60000]
loss: 1.373464 [32064/60000]
loss: 1.376744 [38464/60000]
loss: 1.304962 [44864/60000]
loss: 1.347154 [51264/60000]
loss: 1.230661 [57664/60000]
Test Error:
Accuracy: 62.7%, Avg loss: 1.260891
Epoch 5
loss: 1.337803 [ 64/60000]
loss: 1.313278 [ 6464/60000]
loss: 1.151837 [12864/60000]
loss: 1.252142 [19264/60000]
loss: 1.123048 [25664/60000]
loss: 1.159531 [32064/60000]
loss: 1.175011 [38464/60000]
loss: 1.115554 [44864/60000]
loss: 1.160974 [51264/60000]
loss: 1.062730 [57664/60000]
Test Error:
Accuracy: 64.6%, Avg loss: 1.087374
Done!
Read more about Training your model.
Saving Models
A common way to save a model is to serialize the internal state dictionary (containing the model parameters).
Saved PyTorch Model State to model.pth
Loading Models
The process for loading a model includes re-creating the model structure and loading the state dictionary into it.
This model can now be used to make predictions.
classes = [
“T-shirt/top”,
“Trouser”,
“Pullover”,
“Dress”,
“Coat”,
“Sandal”,
“Shirt”,
“Sneaker”,
“Bag”,
“Ankle boot”,
]
model.eval()
x, y = test_data[0][0], test_data[0][1]
with torch.no_grad():
x = x.to(device)
pred = model(x)
predicted, actual = classes[pred[0].argmax(0)], classes[y]
print(f’Predicted: “{predicted}”, Actual: “{actual}”‘)
Predicted: “Ankle boot”, Actual: “Ankle boot”
Read more about Saving & Loading your model.
Total running time of the script: ( 1 minutes 5.911 seconds)