Neural Ninjas · PyTorch Internals Lab
Same blocks moving = zero-copy. New blocks appearing = a real memory allocation. Watch which one each operation actually does.
Click an operation above. Watch the memory panel on the right — that's where the real story is.
| Concept | Visual analogy |
|---|---|
| Strides | Page numbers in a book — how far to skip to reach the next item |
| Contiguous | A bookshelf with books in order — no gaps, no jumps |
| .permute() / .transpose() | Relabeling which shelf is "row" vs "column" — same books, new labels, shelf now out of natural order |
| .view() | Changing book labels without moving a single book — fails if the shelf's already out of order |
| .reshape() | Relabels if it can — reorganizes the whole shelf (copies) if it can't |