Creates a tensor from data like lists, arrays, or scalars. It's the most common way to create tensors in PyTorch and the foundation for all neural network operations.
torch.tensor(data, dtype=None, device=None, requires_grad=False)Converting Python lists/arrays to PyTorch tensors for model input
The main tensor class in PyTorch. Represents a multi-dimensional array that can store data and gradients, supporting both CPU and GPU operations with automatic differentiation.
torch.Tensor(*sizes) or torch.Tensor(data)Base class for all tensor operations in neural networks
Creates a tensor filled with zeros. Essential for initializing tensors with known values, commonly used for padding, masking, and weight initialization in neural networks.
torch.zeros(*size, dtype=None, device=None, requires_grad=False)Initializing weight matrices and bias vectors
Creates a tensor filled with ones. Useful for initialization, creating masks, and mathematical operations where you need a tensor of all ones.
torch.ones(*size, dtype=None, device=None, requires_grad=False)Creating bias initialization in neural networks
Creates a tensor with random values from standard normal distribution (mean=0, std=1). Essential for weight initialization and adding noise in neural networks.
torch.randn(*size, dtype=None, device=None, requires_grad=False)Initializing neural network weights (Xavier/He initialization)
Creates a 1D tensor with evenly spaced values within a given range. Similar to Python's range() but returns a tensor, useful for indexing and creating sequences.
torch.arange(start=0, end, step=1, dtype=None, device=None)Creating position indices for positional encoding in transformers
Concatenates tensors along a specified dimension. Essential for combining embeddings, stacking sequences, and building complex tensor structures in neural networks.
torch.cat(tensors, dim=0)Combining outputs from multiple attention heads
Stacks tensors along a new dimension. Unlike concat, it creates a new dimension and requires all tensors to have identical shapes. Essential for batching and creating higher-dimensional tensors.
torch.stack(tensors, dim=0)Creating batches from individual samples
Removes dimensions of size 1 from a tensor. Essential for reshaping tensors and removing unnecessary singleton dimensions that can cause broadcasting issues.
torch.squeeze(input, dim=None)Removing batch dimensions of size 1 from model outputs
Adds a dimension of size 1 at the specified position. Essential for broadcasting operations and preparing tensors for operations that require specific dimensionality.
torch.unsqueeze(input, dim)Adding batch dimensions to single samples
Performs matrix multiplication between tensors. Supports broadcasting and handles various dimensional cases automatically. The go-to function for most matrix operations in neural networks.
torch.matmul(input, other)Computing attention scores in transformer models
Performs batch matrix multiplication on 3D tensors. Specifically designed for batched operations where you have multiple matrices to multiply in parallel.
torch.bmm(input, mat2)Multi-head attention computations in transformers