Link
Highlights&&Note
Tensors are similar to NumPy’s ndarrays, except that tensors can run on GPUs or other hardware accelerators. In fact, tensors and NumPy arrays can often share the same underlying memory, eliminating the need to copy data (see Bridge with NumPy).
Tensor与np.ndarray非常相似 ^baaea2f8Tensors can be created from NumPy arrays (and vice versa
^60681efcBridge with NumPy
Tensors on the CPU and NumPy arrays can share their underlying memory locations, and changing one will change the other.
Tensor to NumPy array
t = torch.ones(5)
print(f”t: {t}”)
n = t.numpy()
print(f”n: {n}”)
t: tensor([1., 1., 1., 1., 1.])
n: [1. 1. 1. 1. 1.]
A change in the tensor reflects in the NumPy array.
t.add_(1)
print(f”t: {t}”)
print(f”n: {n}”)
t: tensor([2., 2., 2., 2., 2.])
n: [2. 2. 2. 2. 2.]
tensor与np.ndarray实在是太相似了
两个甚至使用同一个memory location
修改tensor,对应的array也会改变 ^5d6d8dd1
Bridge with NumPy
^86073604
Content
Note
Click hereto download the full example code
Learn the Basics ||Quickstart ||Tensors ||Datasets & DataLoaders ||Transforms ||Build Model ||Autograd ||Optimization ||Save & Load Model
Tensors
Tensors are a specialized data structure that are very similar to arrays and matrices. In PyTorch, we use tensors to encode the inputs and outputs of a model, as well as the model’s parameters.
Tensors are similar to NumPy’s ndarrays, except that tensors can run on GPUs or other hardware accelerators. In fact, tensors and NumPy arrays can often share the same underlying memory, eliminating the need to copy data (see Bridge with NumPy). Tensors are also optimized for automatic differentiation (we’ll see more about that later in the Autogradsection). If you’re familiar with ndarrays, you’ll be right at home with the Tensor API. If not, follow along!
import torch
import numpy as np
Initializing a Tensor
Tensors can be initialized in various ways. Take a look at the following examples:
Directly from data
Tensors can be created directly from data. The data type is automatically inferred.
From a NumPy array
==Tensors can be created from NumPy arrays (and vice versa== - see Bridge with NumPy).
From another tensor:
The new tensor retains the properties (shape, datatype) of the argument tensor, unless explicitly overridden.
Ones Tensor:
tensor([[1, 1],
[1, 1]])
Random Tensor:
tensor([[0.8823, 0.9150],
[0.3829, 0.9593]])
With random or constant values:
shape is a tuple of tensor dimensions. In the functions below, it determines the dimensionality of the output tensor.
Random Tensor:
tensor([[0.3904, 0.6009, 0.2566],
[0.7936, 0.9408, 0.1332]])
Ones Tensor:
tensor([[1., 1., 1.],
[1., 1., 1.]])
Zeros Tensor:
tensor([[0., 0., 0.],
[0., 0., 0.]])
Attributes of a Tensor
Tensor attributes describe their shape, datatype, and the device on which they are stored.
Shape of tensor: torch.Size([3, 4])
Datatype of tensor: torch.float32
Device tensor is stored on: cpu
Operations on Tensors
Over 100 tensor operations, including arithmetic, linear algebra, matrix manipulation (transposing, indexing, slicing), sampling and more are comprehensively described here.
Each of these operations can be run on the GPU (at typically higher speeds than on a CPU). If you’re using Colab, allocate a GPU by going to Runtime > Change runtime type > GPU.
By default, tensors are created on the CPU. We need to explicitly move tensors to the GPU using.to method (after checking for GPU availability). Keep in mind that copying large tensors across devices can be expensive in terms of time and memory!
Try out some of the operations from the list. If you’re familiar with the NumPy API, you’ll find the Tensor API a breeze to use.
Standard numpy-like indexing and slicing:
First row: tensor([1., 1., 1., 1.])
First column: tensor([1., 1., 1., 1.])
Last column: tensor([1., 1., 1., 1.])
tensor([[1., 0., 1., 1.],
[1., 0., 1., 1.],
[1., 0., 1., 1.],
[1., 0., 1., 1.]])
Joining tensors You can use torch.cat to concatenate a sequence of tensors along a given dimension. See also torch.stack, another tensor joining operator that is subtly different from torch.cat.
tensor([[1., 0., 1., 1., 1., 0., 1., 1., 1., 0., 1., 1.],
[1., 0., 1., 1., 1., 0., 1., 1., 1., 0., 1., 1.],
[1., 0., 1., 1., 1., 0., 1., 1., 1., 0., 1., 1.],
[1., 0., 1., 1., 1., 0., 1., 1., 1., 0., 1., 1.]])
Arithmetic operations
tensor([[1., 0., 1., 1.],
[1., 0., 1., 1.],
[1., 0., 1., 1.],
[1., 0., 1., 1.]])
Single-element tensors If you have a one-element tensor, for example by aggregating all values of a tensor into one value, you can convert it to a Python numerical value using item():
agg = tensor.sum()
agg_item = agg.item()
print(agg_item, type(agg_item))
In-place operationsOperations that store the result into the operand are called in-place. They are denoted by a _ suffix. For example: x.copy_(y), x.t_(), will change x.
tensor([[1., 0., 1., 1.],
[1., 0., 1., 1.],
[1., 0., 1., 1.],
[1., 0., 1., 1.]])
tensor([[6., 5., 6., 6.],
[6., 5., 6., 6.],
[6., 5., 6., 6.],
[6., 5., 6., 6.]])
Note
In-place operations save some memory, but can be problematic when computing derivatives because of an immediate loss of history. Hence, their use is discouraged.
==Bridge with NumPy==
==Tensors on the CPU and NumPy arrays can share their underlying memory
locations, and changing one will change the other.==
==Tensor to NumPy array==
t ===== torch.ones==(====5====)==
==print====(====f====”t:== =={==t==}====”====)==
==n== ===== t==.====numpy====()==
==print====(====f====”n:== =={====n====}====”====)==
==t: tensor([1., 1., 1., 1., 1.])
n: [1. 1. 1. 1. 1.]==
==A change in the tensor reflects in the NumPy array.==
t==.====add_====(====1====)==
==print====(====f====”t:== =={==t==}====”====)==
==print====(====f====”n:== =={====n====}====”====)==
==t: tensor([2., 2., 2., 2., 2.])
n: [2. 2. 2. 2. 2.]==
NumPy array to Tensor
Changes in the NumPy array reflects in the tensor.
np.add(n, 1, out=n)
print(f”t: {t}”)
print(f”n: {n}”)
t: tensor([2., 2., 2., 2., 2.], dtype=torch.float64)
n: [2. 2. 2. 2. 2.]
Total running time of the script: ( 0 minutes 0.023 seconds)