مطالعهٔ NumPy Docs
نامپای برای تازهکارها، شروع سریع، اندیسگذاری
باز کردن مرجع کامل دانلود آرشیو مستندات ناشر
گزیدهای از مستندات NumPy Docs · نسخهٔ 2.5 · BSD-3-Clause
نامپای برای تازهکارها
NumPy: the absolute basics for beginners#
Welcome to the absolute beginner’s guide to NumPy!
NumPy (Numerical Python) is an open source Python library that’s
widely used in science and engineering. The NumPy library contains
multidimensional array data structures, such as the homogeneous, N-dimensional
ndarray, and a large library of functions that operate efficiently on these
data structures. Learn more about NumPy at What is NumPy,
and if you have comments or suggestions, please
reach out!
How to import NumPy#
After installing NumPy, it may be imported into Python code like:
import numpy as np
This widespread convention allows access to NumPy features with a short,
recognizable prefix (np.) while distinguishing NumPy features from others
that have the same name.
Reading the example code#
Throughout the NumPy documentation, you will find blocks that look like:
>>> a = np.array([[1, 2, 3], ... [4, 5, 6]]) >>> a.shape (2, 3)
Text preceded by >>> or ... is input, the code that you would
enter in a script or at a Python prompt. Everything else is output, the
results of running your code. Note that >>> and ... are not part of the
code and may cause an error if entered at a Python prompt.
To run the code in the examples, you can copy and paste it into a Python script or REPL, or use the experimental interactive examples in the browser provided in various locations in the documentation.
Why use NumPy?#
Python lists are excellent, general-purpose containers. They can be “heterogeneous”, meaning that they can contain elements of a variety of types, and they are quite fast when used to perform individual operations on a handful of elements.
Depending on the characteristics of the data and the types of operations that need to be performed, other containers may be more appropriate; by exploiting these characteristics, we can improve speed, reduce memory consumption, and offer a high-level syntax for performing a variety of common processing tasks. NumPy shines when there are large quantities of “homogeneous” (same-type) data to be processed on the CPU.
What is an “array”?#
In computer programming, an array is a structure for storing and retrieving data. We often talk about an array as if it were a grid in space, with each cell storing one element of the data. For instance, if each element of the data were a number, we might visualize a “one-dimensional” array like a list:
\[\begin{split}\begin{array}{|c||c|c|c|} \hline 1 & 5 & 2 & 0 \\ \hline \end{array}\end{split}\]A two-dimensional array would be like a table:
\[\begin{split}\begin{array}{|c||c|c|c|} \hline 1 & 5 & 2 & 0 \\ \hline 8 & 3 & 6 & 1 \\ \hline 1 & 7 & 2 & 9 \\ \hline \end{array}\end{split}\]A three-dimensional array would be like a set of tables, perhaps stacked
as though they were printed on separate pages. In NumPy, this idea is
generalized to an arbitrary number of dimensions, and so the fundamental
array class is called ndarray: it represents an “N-dimensional
array”.
Most NumPy arrays have some restrictions. For instance:
All elements of the array must be of the same type of data.
Once created, the total size of the array can’t change.
The shape must be “rectangular”, not “jagged”; e.g., each row of a two-dimensional array must have the same number of columns.
When these conditions are met, NumPy exploits these characteristics to make the array faster, more memory efficient, and more convenient to use than less restrictive data structures.
For the remainder of this document, we will use the word “array” to refer to
an instance of ndarray.
Array fundamentals#
One way to initialize an array is using a Python sequence, such as a list. For example:
>>> a = np.array([1, 2, 3, 4, 5, 6]) >>> a array([1, 2, 3, 4, 5, 6])
Elements of an array can be accessed in various ways. For instance, we can access an individual element of this array as we would access an element in the original list: using the integer index of the element within square brackets.
>>> a[0] 1
Note
As with built-in Python sequences, NumPy arrays are “0-indexed”: the first
element of the array is accessed using index 0, not 1.
Like the original list, the array is mutable.
>>> a[0] = 10 >>> a array([10, 2, 3, 4, 5, 6])
Also like the original list, Python slice notation can be used for indexing.
>>> a[:3] array([10, 2, 3])
One major difference is that slice indexing of a list copies the elements into a new list, but slicing an array returns a view: an object that refers to the data in the original array. The original array can be mutated using the view.
>>> b = a[3:] >>> b array([4, 5, 6]) >>> b[0] = 40 >>> a array([ 10, 2, 3, 40, 5, 6])
See Copies and views for a more comprehensive explanation of when array operations return views rather than copies.
Two- and higher-dimensional arrays can be initialized from nested Python sequences:
>>> a = np.array([[1, 2, 3, 4], [5, 6, 7, 8], [9, 10, 11, 12]])
>>> a
array([[ 1, 2, 3, 4],
[ 5, 6, 7, 8],
[ 9, 10, 11, 12]])
In NumPy, a dimension of an array is sometimes referred to as an “axis”. This
terminology may be useful to disambiguate between the dimensionality of an
array and the dimensionality of the data represented by the array. For
instance, the array a could represent three points, each lying within a
four-dimensional space, but a has only two “axes”.
Another difference between an array and a list of lists is that an
element of the array can be accessed by specifying the index along each
axis within a single set of square brackets, separated by commas.
For instance, the element 8 is in row 1 and column 3:
>>> a[1, 3] 8
Note
It is familiar practice in mathematics to refer to elements of a matrix by the row index first and the column index second. This happens to be true for two-dimensional arrays, but a better mental model is to think of the column index as coming last and the row index as second to last. This generalizes to arrays with any number of dimensions.
Note
You might hear of a 0-D (zero-dimensional) array referred to as a “scalar”, a 1-D (one-dimensional) array as a “vector”, a 2-D (two-dimensional) array as a “matrix”, or an N-D (N-dimensional, where “N” is typically an integer greater than 2) array as a “tensor”. For clarity, it is best to avoid the mathematical terms when referring to an array because the mathematical objects with these names behave differently than arrays (e.g. “matrix” multiplication is fundamentally different from “array” multiplication), and there are other objects in the scientific Python ecosystem that have these names (e.g. the fundamental data structure of PyTorch is the “tensor”).
Array attributes#
This section covers the ndim, shape, size, and dtype
attributes of an array.
The number of dimensions of an array is contained in the ndim attribute.
>>> a.ndim 2
The shape of an array is a tuple of non-negative integers that specify the number of elements along each dimension.
>>> a.shape (3, 4) >>> len(a.shape) == a.ndim True
The fixed, total number of elements in array is contained in the size
attribute.
>>> a.size 12 >>> import math >>> a.size == math.prod(a.shape) True
Arrays are typically “homogeneous”, meaning that they contain elements of
only one “data type”. The data type is recorded in the dtype attribute.
>>> a.dtype
dtype('int64') # "int" for integer, "64" for 64-bit
Read more about array attributes here and learn about array objects here.
How to create a basic array#
This section covers np.zeros(), np.ones(),
np.empty(), np.arange(), np.linspace()
Besides creating an array from a sequence of elements, you can easily create an
array filled with 0’s:
>>> np.zeros(2) array([0., 0.])
Or an array filled with 1’s:
>>> np.ones(2) array([1., 1.])
Or even an empty array! The function empty creates an array whose initial
content is random and depends on the state of the memory. The reason to use
empty over zeros (or something similar) is speed - just make sure to
fill every element afterwards!
>>> # Create an empty array with 2 elements >>> np.empty(2) array([3.14, 42. ]) # may vary
You can create an array with a range of elements:
>>> np.arange(4) array([0, 1, 2, 3])
And even an array that contains a range of evenly spaced intervals. To do this, you will specify the first number, last number, and the step size.
>>> np.arange(2, 9, 2) array([2, 4, 6, 8])
You can also use np.linspace() to create an array with values that are
spaced linearly in a specified interval:
>>> np.linspace(0, 10, num=5) array([ 0. , 2.5, 5. , 7.5, 10. ])
Specifying your data type
While the default data type is floating point (np.float64), you can explicitly
specify which data type you want using the dtype keyword.
>>> x = np.ones(2, dtype=np.int64) >>> x array([1, 1])
Learn more about creating arrays here
Adding, removing, and sorting elements#
This section covers np.sort(), np.concatenate()
Sorting an array is simple with np.sort(). You can specify the axis, kind,
and order when you call the function.
If you start with this array:
>>> arr = np.array([2, 1, 5, 3, 7, 4, 6, 8])
You can quickly sort the numbers in ascending order with:
>>> np.sort(arr) array([1, 2, 3, 4, 5, 6, 7, 8])
In addition to sort, which returns a sorted copy of an array, you can use:
argsort, which is an indirect sort along a specified axis,lexsort, which is an indirect stable sort on multiple keys,searchsorted, which will find elements in a sorted array, andpartition, which is a partial sort.
To read more about sorting an array, see: sort.
If you start with these arrays:
>>> a = np.array([1, 2, 3, 4]) >>> b = np.array([5, 6, 7, 8])
You can concatenate them with np.concatenate().
>>> np.concatenate((a, b)) array([1, 2, 3, 4, 5, 6, 7, 8])
Or, if you start with these arrays:
>>> x = np.array([[1, 2], [3, 4]]) >>> y = np.array([[5, 6]])
You can concatenate them with:
>>> np.concatenate((x, y), axis=0)
array([[1, 2],
[3, 4],
[5, 6]])
In order to remove elements from an array, it’s simple to use indexing to select the elements that you want to keep.
To read more about concatenate, see: concatenate.
How do you know the shape and size of an array?#
This section covers ndarray.ndim, ndarray.size, ndarray.shape
ndarray.ndim will tell you the number of axes, or dimensions, of the array.
ndarray.size will tell you the total number of elements of the array. This
is the product of the elements of the array’s shape.
ndarray.shape will display a tuple of integers that indicate the number of
elements stored along each dimension of the array. If, for example, you have a
2-D array with 2 rows and 3 columns, the shape of your array is (2, 3).
For example, if you create this array:
>>> array_example = np.array([[[0, 1, 2, 3], ... [4, 5, 6, 7]], ... ... [[0, 1, 2, 3], ... [4, 5, 6, 7]], ... ... [[0 ,1 ,2, 3], ... [4, 5, 6, 7]]])
To find the number of dimensions of the array, run:
>>> array_example.ndim 3
To find the total number of elements in the array, run:
>>> array_example.size 24
And to find the shape of your array, run:
>>> array_example.shape (3, 2, 4)
Can you reshape an array?#
This section covers arr.reshape()
Yes!
Using arr.reshape() will return a reshaped array without changing the
data. Just remember that when you use the reshape method, the array you want to
produce needs to have the same number of elements as the original array. If you
start with an array with 12 elements, you’ll need to make sure that your new
array also has a total of 12 elements.
If you start with this array:
>>> a = np.arange(6) >>> print(a) [0 1 2 3 4 5]
You can use reshape() to reshape your array. For example, you can reshape
this array to an array with three rows and two columns:
>>> b = a.reshape(3, 2) >>> print(b) [[0 1] [2 3] [4 5]]
With np.reshape, you can specify a few optional parameters:
>>> np.reshape(a, shape=(1, 6), order='C') array([[0, 1, 2, 3, 4, 5]])
a is the array to be reshaped.
shape is the new shape you want. You can specify an integer or a tuple of
integers. If you specify an integer, the result will be an array of that length.
The shape should be compatible with the original shape.
order: C means to read/write the elements using C-like index order,
F means to read/write the elements using Fortran-like index order, A
means to read/write the elements in Fortran-like index order if a is Fortran
contiguous in memory, C-like order otherwise. (This is an optional parameter and
doesn’t need to be specified.)
If you want to learn more about C and Fortran order, you can read more about the internal organization of NumPy arrays here. Essentially, C and Fortran orders have to do with how indices correspond to the order the array is stored in memory. In Fortran, when moving through the elements of a two-dimensional array as it is stored in memory, the first index is the most rapidly varying index. As the first index moves to the next row as it changes, the matrix is stored one column at a time. This is why Fortran is thought of as a Column-major language. In C on the other hand, the last index changes the most rapidly. The matrix is stored by rows, making it a Row-major language. What you do for C or Fortran depends on whether it’s more important to preserve the indexing convention or not reorder the data.
Learn more about shape manipulation here.
How to convert a 1D array into a 2D array (how to add a new axis to an array)#
This section covers np.newaxis, np.expand_dims
You can use np.newaxis and np.expand_dims to increase the dimensions of
your existing array.
Using np.newaxis will increase the dimensions of your array by one dimension
when used once. This means that a 1D array will become a 2D array, a
2D array will become a 3D array, and so on.
For example, if you start with this array:
>>> a = np.array([1, 2, 3, 4, 5, 6]) >>> a.shape (6,)
You can use np.newaxis to add a new axis:
>>> a2 = a[np.newaxis, :] >>> a2.shape (1, 6)
You can explicitly convert a 1D array to either a row vector or a column
vector using np.newaxis. For example, you can convert a 1D array to a row
vector by inserting an axis along the first dimension:
>>> row_vector = a[np.newaxis, :] >>> row_vector.shape (1, 6)
Or, for a column vector, you can insert an axis along the second dimension:
>>> col_vector = a[:, np.newaxis] >>> col_vector.shape (6, 1)
You can also expand an array by inserting a new axis at a specified position
with np.expand_dims.
For example, if you start with this array:
>>> a = np.array([1, 2, 3, 4, 5, 6]) >>> a.shape (6,)
You can use np.expand_dims to add an axis at index position 1 with:
>>> b = np.expand_dims(a, axis=1) >>> b.shape (6, 1)
You can add an axis at index position 0 with:
>>> c = np.expand_dims(a, axis=0) >>> c.shape (1, 6)
Find more information about newaxis here and
expand_dims at expand_dims.
Indexing and slicing#
You can index and slice NumPy arrays in the same ways you can slice Python lists.
>>> data = np.array([1, 2, 3]) >>> data[1] 2 >>> data[0:2] array([1, 2]) >>> data[1:] array([2, 3]) >>> data[-2:] array([2, 3])
You can visualize it this way:
You may want to take a section of your array or specific array elements to use in further analysis or additional operations. To do that, you’ll need to subset, slice, and/or index your arrays.
If you want to select values from your array that fulfill certain conditions, it’s straightforward with NumPy.
For example, if you start with this array:
>>> a = np.array([[1, 2, 3, 4], [5, 6, 7, 8], [9, 10, 11, 12]])
You can easily print all of the values in the array that are less than 5.
>>> print(a[a < 5]) [1 2 3 4]
You can also select, for example, numbers that are equal to or greater than 5, and use that condition to index an array.
>>> five_up = (a >= 5) >>> print(a[five_up]) [ 5 6 7 8 9 10 11 12]
You can select elements that are divisible by 2:
>>> divisible_by_2 = a[a%2==0] >>> print(divisible_by_2) [ 2 4 6 8 10 12]
Or you can select elements that satisfy two conditions using the & and |
operators:
>>> c = a[(a > 2) & (a < 11)] >>> print(c) [ 3 4 5 6 7 8 9 10]
You can also make use of the logical operators & and | in order to return boolean values that specify whether or not the values in an array fulfill a certain condition. This can be useful with arrays that contain names or other categorical values.
>>> five_up = (a > 5) | (a == 5) >>> print(five_up) [[False False False False] [ True True True True] [ True True True True]]
You can also use np.nonzero() to select elements or indices from an array.
Starting with this array:
>>> a = np.array([[1, 2, 3, 4], [5, 6, 7, 8], [9, 10, 11, 12]])
You can use np.nonzero() to print the indices of elements that are, for
example, less than 5:
>>> b = np.nonzero(a < 5) >>> print(b) (array([0, 0, 0, 0]), array([0, 1, 2, 3]))
In this example, a tuple of arrays was returned: one for each dimension. The first array represents the row indices where these values are found, and the second array represents the column indices where the values are found.
If you want to generate a list of coordinates where the elements exist, you can zip the arrays, iterate over the list of coordinates, and print them. For example:
>>> list_of_coordinates= list(zip(b[0], b[1])) >>> for coord in list_of_coordinates: ... print(coord) (np.int64(0), np.int64(0)) (np.int64(0), np.int64(1)) (np.int64(0), np.int64(2)) (np.int64(0), np.int64(3))
You can also use np.nonzero() to print the elements in an array that are less
than 5 with:
>>> print(a[b]) [1 2 3 4]
If the element you’re looking for doesn’t exist in the array, then the returned array of indices will be empty. For example:
>>> not_there = np.nonzero(a == 42) >>> print(not_there) (array([], dtype=int64), array([], dtype=int64))
Learn more about indexing and slicing here and here.
Read more about using the nonzero function at: nonzero.
How to create an array from existing data#
This section covers slicing and indexing, np.vstack(), np.hstack(),
np.hsplit(), .view(), copy()
You can easily create a new array from a section of an existing array.
Let’s say you have this array:
>>> a = np.array([1, 2, 3, 4, 5, 6, 7, 8, 9, 10])
You can create a new array from a section of your array any time by specifying where you want to slice your array.
>>> arr1 = a[3:8] >>> arr1 array([4, 5, 6, 7, 8])
Here, you grabbed a section of your array from index position 3 through index position 8 but not including position 8 itself.
Reminder: Array indexes begin at 0. This means the first element of the array is at index 0, the second element is at index 1, and so on.
You can also stack two existing arrays, both vertically and horizontally. Let’s
say you have two arrays, a1 and a2:
>>> a1 = np.array([[1, 1], ... [2, 2]]) >>> a2 = np.array([[3, 3], ... [4, 4]])
You can stack them vertically with vstack:
>>> np.vstack((a1, a2))
array([[1, 1],
[2, 2],
[3, 3],
[4, 4]])
Or stack them horizontally with hstack:
>>> np.hstack((a1, a2))
array([[1, 1, 3, 3],
[2, 2, 4, 4]])
You can split an array into several smaller arrays using hsplit. You can
specify either the number of equally shaped arrays to return or the columns
after which the division should occur.
Let’s say you have this array:
>>> x = np.arange(1, 25).reshape(2, 12)
>>> x
array([[ 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12],
[13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24]])
If you wanted to split this array into three equally shaped arrays, you would run:
>>> np.hsplit(x, 3)
[array([[ 1, 2, 3, 4],
[13, 14, 15, 16]]), array([[ 5, 6, 7, 8],
[17, 18, 19, 20]]), array([[ 9, 10, 11, 12],
[21, 22, 23, 24]])]
If you wanted to split your array after the third and fourth column, you’d run:
>>> np.hsplit(x, (3, 4))
[array([[ 1, 2, 3],
[13, 14, 15]]), array([[ 4],
[16]]), array([[ 5, 6, 7, 8, 9, 10, 11, 12],
[17, 18, 19, 20, 21, 22, 23, 24]])]
Learn more about stacking and splitting arrays here.
You can use the view method to create a new array object that looks at the
same data as the original array (a shallow copy).
Views are an important NumPy concept! NumPy functions, as well as operations like indexing and slicing, will return views whenever possible. This saves memory and is faster (no copy of the data has to be made). However it’s important to be aware of this - modifying data in a view also modifies the original array!
Let’s say you create this array:
>>> a = np.array([[1, 2, 3, 4], [5, 6, 7, 8], [9, 10, 11, 12]])
Now we create an array b1 by slicing a and modify the first element of
b1. This will modify the corresponding element in a as well!
>>> b1 = a[0, :]
>>> b1
array([1, 2, 3, 4])
>>> b1[0] = 99
>>> b1
array([99, 2, 3, 4])
>>> a
array([[99, 2, 3, 4],
[ 5, 6, 7, 8],
[ 9, 10, 11, 12]])
Using the copy method will make a complete copy of the array and its data (a
deep copy). To use this on your array, you could run:
>>> b2 = a.copy()
Learn more about copies and views here.
Basic array operations#
This section covers addition, subtraction, multiplication, division, and more
Once you’ve created your arrays, you can start to work with them. Let’s say, for example, that you’ve created two arrays, one called “data” and one called “ones”
You can add the arrays together with the plus sign.
>>> data = np.array([1, 2]) >>> ones = np.ones(2, dtype=np.int_) >>> data + ones array([2, 3])
You can, of course, do more than just addition!
>>> data - ones array([0, 1]) >>> data * data array([1, 4]) >>> data / data array([1., 1.])
Basic operations are simple with NumPy. If you want to find the sum of the
elements in an array, you’d use sum(). This works for 1D arrays, 2D arrays,
and arrays in higher dimensions.
>>> a = np.array([1, 2, 3, 4]) >>> a.sum() 10
To add the rows or the columns in a 2D array, you would specify the axis.
If you start with this array:
>>> b = np.array([[1, 1], [2, 2]])
You can sum over the axis of rows with:
>>> b.sum(axis=0) array([3, 3])
You can sum over the axis of columns with:
>>> b.sum(axis=1) array([2, 4])
Learn more about basic operations here.
Broadcasting#
There are times when you might want to carry out an operation between an array and a single number (also called an operation between a vector and a scalar) or between arrays of two different sizes. For example, your array (we’ll call it “data”) might contain information about distance in miles but you want to convert the information to kilometers. You can perform this operation with:
>>> data = np.array([1.0, 2.0]) >>> data * 1.6 array([1.6, 3.2])
NumPy understands that the multiplication should happen with each cell. That
concept is called broadcasting. Broadcasting is a mechanism that allows
NumPy to perform operations on arrays of different shapes. The dimensions of
your array must be compatible, for example, when the dimensions of both arrays
are equal or when one of them is 1. If the dimensions are not compatible, you
will get a ValueError.
Learn more about broadcasting here.
More useful array operations#
This section covers maximum, minimum, sum, mean, product, standard deviation, and more
NumPy also performs aggregation functions. In addition to min, max, and
sum, you can easily run mean to get the average, prod to get the
result of multiplying the elements together, std to get the standard
deviation, and more.
>>> data = np.array([1, 2, 3]) >>> data.max() 3 >>> data.min() 1 >>> data.sum() 6
Let’s start with this array, called “a”
>>> a = np.array([[0.45053314, 0.17296777, 0.34376245, 0.5510652], ... [0.54627315, 0.05093587, 0.40067661, 0.55645993], ... [0.12697628, 0.82485143, 0.26590556, 0.56917101]])
It’s very common to want to aggregate along a row or column. By default, every NumPy aggregation function will return the aggregate of the entire array. To find the sum or the minimum of the elements in your array, run:
>>> a.sum() 4.8595784
Or:
>>> a.min() 0.05093587
You can specify on which axis you want the aggregation function to be computed.
For example, you can find the minimum value within each column by specifying
axis=0.
>>> a.min(axis=0) array([0.12697628, 0.05093587, 0.26590556, 0.5510652 ])
The four values listed above correspond to the number of columns in your array. With a four-column array, you will get four values as your result.
Read more about array methods here.
Creating matrices#
You can pass Python lists of lists to create a 2-D array (or “matrix”) to represent them in NumPy.
>>> data = np.array([[1, 2], [3, 4], [5, 6]])
>>> data
array([[1, 2],
[3, 4],
[5, 6]])
Indexing and slicing operations are useful when you’re manipulating matrices:
>>> data[0, 1]
2
>>> data[1:3]
array([[3, 4],
[5, 6]])
>>> data[0:2, 0]
array([1, 3])
You can aggregate matrices the same way you aggregated vectors:
>>> data.max() 6 >>> data.min() 1 >>> data.sum() 21
You can aggregate all the values in a matrix and you can aggregate them across
columns or rows using the axis parameter. To illustrate this point, let’s
look at a slightly modified dataset:
>>> data = np.array([[1, 2], [5, 3], [4, 6]])
>>> data
array([[1, 2],
[5, 3],
[4, 6]])
>>> data.max(axis=0)
array([5, 6])
>>> data.max(axis=1)
array([2, 5, 6])
Once you’ve created your matrices, you can add and multiply them using arithmetic operators if you have two matrices that are the same size.
>>> data = np.array([[1, 2], [3, 4]])
>>> ones = np.array([[1, 1], [1, 1]])
>>> data + ones
array([[2, 3],
[4, 5]])
You can do these arithmetic operations on matrices of different sizes, but only if one matrix has only one column or one row. In this case, NumPy will use its broadcast rules for the operation.
>>> data = np.array([[1, 2], [3, 4], [5, 6]])
>>> ones_row = np.array([[1, 1]])
>>> data + ones_row
array([[2, 3],
[4, 5],
[6, 7]])
Be aware that when NumPy prints N-dimensional arrays, the last axis is looped over the fastest while the first axis is the slowest. For instance:
>>> np.ones((4, 3, 2))
array([[[1., 1.],
[1., 1.],
[1., 1.]],
[[1., 1.],
[1., 1.],
[1., 1.]],
[[1., 1.],
[1., 1.],
[1., 1.]],
[[1., 1.],
[1., 1.],
[1., 1.]]])
There are often instances where we want NumPy to initialize the values of an
array. NumPy offers functions like ones() and zeros(), and the
random.Generator class for random number generation for that.
All you need to do is pass in the number of elements you want it to generate:
>>> np.ones(3) array([1., 1., 1.]) >>> np.zeros(3) array([0., 0., 0.]) >>> rng = np.random.default_rng() # the simplest way to generate random numbers >>> rng.random(3) array([0.63696169, 0.26978671, 0.04097352])
You can also use ones(), zeros(), and random() to create
a 2D array if you give them a tuple describing the dimensions of the matrix:
>>> np.ones((3, 2))
array([[1., 1.],
[1., 1.],
[1., 1.]])
>>> np.zeros((3, 2))
array([[0., 0.],
[0., 0.],
[0., 0.]])
>>> rng.random((3, 2))
array([[0.01652764, 0.81327024],
[0.91275558, 0.60663578],
[0.72949656, 0.54362499]]) # may vary
Read more about creating arrays, filled with 0’s, 1’s, other values or
uninitialized, at array creation routines.
Generating random numbers#
The use of random number generation is an important part of the configuration and evaluation of many numerical and machine learning algorithms. Whether you need to randomly initialize weights in an artificial neural network, split data into random sets, or randomly shuffle your dataset, being able to generate random numbers (actually, repeatable pseudo-random numbers) is essential.
With Generator.integers, you can generate random integers from low (remember
that this is inclusive with NumPy) to high (exclusive). You can set
endpoint=True to make the high number inclusive.
You can generate a 2 x 4 array of random integers between 0 and 4 with:
>>> rng.integers(5, size=(2, 4))
array([[2, 1, 1, 0],
[0, 0, 0, 4]]) # may vary
Read more about random number generation here.
How to get unique items and counts#
This section covers np.unique()
You can find the unique elements in an array easily with np.unique.
For example, if you start with this array:
>>> a = np.array([11, 11, 12, 13, 14, 15, 16, 17, 12, 13, 11, 14, 18, 19, 20])
you can use np.unique to print the unique values in your array:
>>> unique_values = np.unique(a) >>> print(unique_values) [11 12 13 14 15 16 17 18 19 20]
To get the indices of unique values in a NumPy array (an array of first index
positions of unique values in the array), just pass the return_index
argument in np.unique() as well as your array.
>>> unique_values, indices_list = np.unique(a, return_index=True) >>> print(indices_list) [ 0 2 3 4 5 6 7 12 13 14]
You can pass the return_counts argument in np.unique() along with your
array to get the frequency count of unique values in a NumPy array.
>>> unique_values, occurrence_count = np.unique(a, return_counts=True) >>> print(occurrence_count) [3 2 2 2 1 1 1 1 1 1]
This also works with 2D arrays! If you start with this array:
>>> a_2d = np.array([[1, 2, 3, 4], [5, 6, 7, 8], [9, 10, 11, 12], [1, 2, 3, 4]])
You can find unique values with:
>>> unique_values = np.unique(a_2d) >>> print(unique_values) [ 1 2 3 4 5 6 7 8 9 10 11 12]
If the axis argument isn’t passed, your 2D array will be flattened.
If you want to get the unique rows or columns, make sure to pass the axis
argument. To find the unique rows, specify axis=0 and for columns, specify
axis=1.
>>> unique_rows = np.unique(a_2d, axis=0) >>> print(unique_rows) [[ 1 2 3 4] [ 5 6 7 8] [ 9 10 11 12]]
To get the unique rows, index position, and occurrence count, you can use:
>>> unique_rows, indices, occurrence_count = np.unique( ... a_2d, axis=0, return_counts=True, return_index=True) >>> print(unique_rows) [[ 1 2 3 4] [ 5 6 7 8] [ 9 10 11 12]] >>> print(indices) [0 1 2] >>> print(occurrence_count) [2 1 1]
To learn more about finding the unique elements in an array, see unique.
Transposing and reshaping a matrix#
This section covers arr.reshape(), arr.transpose(), arr.T
It’s common to need to transpose your matrices. NumPy arrays have the property
T that allows you to transpose a matrix.
You may also need to switch the dimensions of a matrix. This can happen when,
for example, you have a model that expects a certain input shape that is
different from your dataset. This is where the reshape method can be useful.
You simply need to pass in the new dimensions that you want for the matrix.
>>> data.reshape(2, 3)
array([[1, 2, 3],
[4, 5, 6]])
>>> data.reshape(3, 2)
array([[1, 2],
[3, 4],
[5, 6]])
You can also use .transpose() to reverse or change the axes of an array
according to the values you specify.
If you start with this array:
>>> arr = np.arange(6).reshape((2, 3))
>>> arr
array([[0, 1, 2],
[3, 4, 5]])
You can transpose your array with arr.transpose().
>>> arr.transpose()
array([[0, 3],
[1, 4],
[2, 5]])
You can also use arr.T:
>>> arr.T
array([[0, 3],
[1, 4],
[2, 5]])
To learn more about transposing and reshaping arrays, see transpose and
reshape.
How to reverse an array#
This section covers np.flip()
NumPy’s np.flip() function allows you to flip, or reverse, the contents of
an array along an axis. When using np.flip(), specify the array you would like
to reverse and the axis. If you don’t specify the axis, NumPy will reverse the
contents along all of the axes of your input array.
Reversing a 1D array
If you begin with a 1D array like this one:
>>> arr = np.array([1, 2, 3, 4, 5, 6, 7, 8])
You can reverse it with:
>>> reversed_arr = np.flip(arr)
If you want to print your reversed array, you can run:
>>> print('Reversed Array: ', reversed_arr)
Reversed Array: [8 7 6 5 4 3 2 1]
Reversing a 2D array
A 2D array works much the same way.
If you start with this array:
>>> arr_2d = np.array([[1, 2, 3, 4], [5, 6, 7, 8], [9, 10, 11, 12]])
You can reverse the content in all of the rows and all of the columns with:
>>> reversed_arr = np.flip(arr_2d) >>> print(reversed_arr) [[12 11 10 9] [ 8 7 6 5] [ 4 3 2 1]]
You can easily reverse only the rows with:
>>> reversed_arr_rows = np.flip(arr_2d, axis=0) >>> print(reversed_arr_rows) [[ 9 10 11 12] [ 5 6 7 8] [ 1 2 3 4]]
Or reverse only the columns with:
>>> reversed_arr_columns = np.flip(arr_2d, axis=1) >>> print(reversed_arr_columns) [[ 4 3 2 1] [ 8 7 6 5] [12 11 10 9]]
You can also reverse the contents of only one column or row. For example, you can reverse the contents of the row at index position 1 (the second row):
>>> arr_2d[1] = np.flip(arr_2d[1]) >>> print(arr_2d) [[ 1 2 3 4] [ 8 7 6 5] [ 9 10 11 12]]
You can also reverse the column at index position 1 (the second column):
>>> arr_2d[:,1] = np.flip(arr_2d[:,1]) >>> print(arr_2d) [[ 1 10 3 4] [ 8 7 6 5] [ 9 2 11 12]]
Read more about reversing arrays at flip.
Reshaping and flattening multidimensional arrays#
This section covers .flatten(), ravel()
There are two popular ways to flatten an array: .flatten() and .ravel().
The primary difference between the two is that the new array created using
ravel() is actually a reference to the parent array (i.e., a “view”). This
means that any changes to the new array will affect the parent array as well.
Since ravel does not create a copy, it’s memory efficient.
If you start with this array:
>>> x = np.array([[1, 2, 3, 4], [5, 6, 7, 8], [9, 10, 11, 12]])
You can use flatten to flatten your array into a 1D array.
>>> x.flatten() array([ 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12])
When you use flatten, changes to your new array won’t change the parent
array.
For example:
>>> a1 = x.flatten() >>> a1[0] = 99 >>> print(x) # Original array [[ 1 2 3 4] [ 5 6 7 8] [ 9 10 11 12]] >>> print(a1) # New array [99 2 3 4 5 6 7 8 9 10 11 12]
But when you use ravel, the changes you make to the new array will affect
the parent array.
For example:
>>> a2 = x.ravel() >>> a2[0] = 98 >>> print(x) # Original array [[98 2 3 4] [ 5 6 7 8] [ 9 10 11 12]] >>> print(a2) # New array [98 2 3 4 5 6 7 8 9 10 11 12]
Read more about flatten at ndarray.flatten and ravel at ravel.
How to access the docstring for more information#
This section covers help(), ?, ??
When it comes to the data science ecosystem, Python and NumPy are built with the
user in mind. One of the best examples of this is the built-in access to
documentation. Every object contains the reference to a string, which is known
as the docstring. In most cases, this docstring contains a quick and concise
summary of the object and how to use it. Python has a built-in help()
function that can help you access this information. This means that nearly any
time you need more information, you can use help() to quickly find the
information that you need.
For example:
>>> help(max)
Help on built-in function max in module builtins:
max(...)
max(iterable, *[, default=obj, key=func]) -> value
max(arg1, arg2, *args, *[, key=func]) -> value
With a single iterable argument, return its biggest item. The
default keyword-only argument specifies an object to return if
the provided iterable is empty.
With two or more ...arguments, return the largest argument.
Because access to additional information is so useful, IPython uses the ?
character as a shorthand for accessing this documentation along with other
relevant information. IPython is a command shell for interactive computing in
multiple languages.
You can find more information about IPython here.
For example:
In [0]: max? max(iterable, *[, default=obj, key=func]) -> value max(arg1, arg2, *args, *[, key=func]) -> value With a single iterable argument, return its biggest item. The default keyword-only argument specifies an object to return if the provided iterable is empty. With two or more arguments, return the largest argument. Type: builtin_function_or_method
You can even use this notation for object methods and objects themselves.
Let’s say you create this array:
>>> a = np.array([1, 2, 3, 4, 5, 6])
Then you can obtain a lot of useful information (first details about a itself,
followed by the docstring of ndarray of which a is an instance):
In [1]: a?
Type: ndarray
String form: [1 2 3 4 5 6]
Length: 6
File: ~/anaconda3/lib/python3.9/site-packages/numpy/__init__.py
Docstring: <no docstring>
Class docstring:
ndarray(shape, dtype=float, buffer=None, offset=0,
strides=None, order=None)
An array object represents a multidimensional, homogeneous array
of fixed-size items. An associated data-type object describes the
format of each element in the array (its byte-order, how many bytes it
occupies in memory, whether it is an integer, a floating point number,
or something else, etc.)
Arrays should be constructed using `array`, `zeros` or `empty` (refer
to the See Also section below). The parameters given here refer to
a low-level method (`ndarray(...)`) for instantiating an array.
For more information, refer to the `numpy` module and examine the
methods and attributes of an array.
Parameters
----------
(for the __new__ method; see Notes below)
shape : tuple of ints
Shape of created array.
...
This also works for functions and other objects that you create. Just
remember to include a docstring with your function using a string literal
(""" """ or ''' ''' around your documentation).
For example, if you create this function:
>>> def double(a): ... '''Return a * 2''' ... return a * 2
You can obtain information about the function:
In [2]: double? Signature: double(a) Docstring: Return a * 2 File: ~/Desktop/<ipython-input-23-b5adf20be596> Type: function
You can reach another level of information by reading the source code of the
object you’re interested in. Using a double question mark (??) allows you to
access the source code.
For example:
In [3]: double??
Signature: double(a)
Source:
def double(a):
'''Return a * 2'''
return a * 2
File: ~/Desktop/<ipython-input-23-b5adf20be596>
Type: function
If the object in question is compiled in a language other than Python, using
?? will return the same information as ?. You’ll find this with a lot of
built-in objects and types, for example:
In [4]: len? Signature: len(obj, /) Docstring: Return the number of items in a container. Type: builtin_function_or_method
and :
In [5]: len?? Signature: len(obj, /) Docstring: Return the number of items in a container. Type: builtin_function_or_method
have the same output because they were compiled in a programming language other than Python.
Working with mathematical formulas#
The ease of implementing mathematical formulas that work on arrays is one of the things that make NumPy so widely used in the scientific Python community.
For example, this is the mean square error formula (a central formula used in supervised machine learning models that deal with regression):
Implementing this formula is simple and straightforward in NumPy:
What makes this work so well is that predictions and labels can contain
one or a thousand values. They only need to be the same size.
You can visualize it this way:
In this example, both the predictions and labels vectors contain three values,
meaning n has a value of three. After we carry out subtractions the values
in the vector are squared. Then NumPy sums the values, and your result is the
error value for that prediction and a score for the quality of the model.
How to save and load NumPy objects#
This section covers np.save, np.savez, np.savetxt,
np.load, np.loadtxt
You will, at some point, want to save your arrays to disk and load them back
without having to re-run the code. Fortunately, there are several ways to save
and load objects with NumPy. The ndarray objects can be saved to and loaded from
the disk files with loadtxt and savetxt functions that handle normal
text files, load and save functions that handle NumPy binary files with
a .npy file extension, and a savez function that handles NumPy files
with a .npz file extension.
The .npy and .npz files store data, shape, dtype, and other information required to reconstruct the ndarray in a way that allows the array to be correctly retrieved, even when the file is on another machine with different architecture.
If you want to store a single ndarray object, store it as a .npy file using
np.save. If you want to store more than one ndarray object in a single file,
save it as a .npz file using np.savez. You can also save several arrays
into a single file in compressed npz format with savez_compressed.
It’s easy to save and load an array with np.save(). Just make sure to
specify the array you want to save and a file name. For example, if you create
this array:
>>> a = np.array([1, 2, 3, 4, 5, 6])
You can save it as “filename.npy” with:
>>> np.save('filename', a)
You can use np.load() to reconstruct your array.
>>> b = np.load('filename.npy')
If you want to check your array, you can run:
>>> print(b) [1 2 3 4 5 6]
You can save a NumPy array as a plain text file like a .csv or .txt file
with np.savetxt.
For example, if you create this array:
>>> csv_arr = np.array([1, 2, 3, 4, 5, 6, 7, 8])
You can easily save it as a .csv file with the name “new_file.csv” like this:
>>> np.savetxt('new_file.csv', csv_arr)
You can quickly and easily load your saved text file using loadtxt():
>>> np.loadtxt('new_file.csv')
array([1., 2., 3., 4., 5., 6., 7., 8.])
The savetxt() and loadtxt() functions accept additional optional
parameters such as header, footer, and delimiter. While text files can be easier
for sharing, .npy and .npz files are smaller and faster to read. If you need more
sophisticated handling of your text file (for example, if you need to work with
lines that contain missing values), you will want to use the genfromtxt
function.
With savetxt, you can specify headers, footers, comments, and more.
Learn more about input and output routines here.
Importing and exporting a CSV#
It’s simple to read in a CSV that contains existing information. The best and easiest way to do this is to use Pandas.
>>> import pandas as pd
>>> # If all of your columns are the same type:
>>> x = pd.read_csv('music.csv', header=0).values
>>> print(x)
[['Billie Holiday' 'Jazz' 1300000 27000000]
['Jimmie Hendrix' 'Rock' 2700000 70000000]
['Miles Davis' 'Jazz' 1500000 48000000]
['SIA' 'Pop' 2000000 74000000]]
>>> # You can also simply select the columns you need:
>>> x = pd.read_csv('music.csv', usecols=['Artist', 'Plays']).values
>>> print(x)
[['Billie Holiday' 27000000]
['Jimmie Hendrix' 70000000]
['Miles Davis' 48000000]
['SIA' 74000000]]
It’s simple to use Pandas in order to export your array as well. If you are new to NumPy, you may want to create a Pandas dataframe from the values in your array and then write the data frame to a CSV file with Pandas.
If you created this array “a”
>>> a = np.array([[-2.58289208, 0.43014843, -1.24082018, 1.59572603], ... [ 0.99027828, 1.17150989, 0.94125714, -0.14692469], ... [ 0.76989341, 0.81299683, -0.95068423, 0.11769564], ... [ 0.20484034, 0.34784527, 1.96979195, 0.51992837]])
You could create a Pandas dataframe
>>> df = pd.DataFrame(a)
>>> print(df)
0 1 2 3
0 -2.582892 0.430148 -1.240820 1.595726
1 0.990278 1.171510 0.941257 -0.146925
2 0.769893 0.812997 -0.950684 0.117696
3 0.204840 0.347845 1.969792 0.519928
You can easily save your dataframe with:
>>> df.to_csv('pd.csv')
And read your CSV with:
>>> data = pd.read_csv('pd.csv')
You can also save your array with the NumPy savetxt method.
>>> np.savetxt('np.csv', a, fmt='%.2f', delimiter=',', header='1, 2, 3, 4')
If you’re using the command line, you can read your saved CSV any time with a command such as:
$ cat np.csv # 1, 2, 3, 4 -2.58,0.43,-1.24,1.60 0.99,1.17,0.94,-0.15 0.77,0.81,-0.95,0.12 0.20,0.35,1.97,0.52
Or you can open the file any time with a text editor!
If you’re interested in learning more about Pandas, take a look at the official Pandas documentation. Learn how to install Pandas with the official Pandas installation information.
Plotting arrays with Matplotlib#
If you need to generate a plot for your values, it’s very simple with Matplotlib.
For example, you may have an array like this one:
>>> a = np.array([2, 1, 5, 7, 4, 6, 8, 14, 10, 9, 18, 20, 22])
If you already have Matplotlib installed, you can import it with:
>>> import matplotlib.pyplot as plt # If you're using Jupyter Notebook, you may also want to run the following # line of code to display your code in the notebook: %matplotlib inline
All you need to do to plot your values is run:
>>> plt.plot(a) # If you are running from a command line, you may need to do this: # >>> plt.show()
For example, you can plot a 1D array like this:
>>> x = np.linspace(0, 5, 20) >>> y = np.linspace(0, 10, 20) >>> plt.plot(x, y, 'purple') # line >>> plt.plot(x, y, 'o') # dots
With Matplotlib, you have access to an enormous number of visualization options.
>>> fig = plt.figure() >>> ax = fig.add_subplot(projection='3d') >>> X = np.arange(-5, 5, 0.15) >>> Y = np.arange(-5, 5, 0.15) >>> X, Y = np.meshgrid(X, Y) >>> R = np.sqrt(X**2 + Y**2) >>> Z = np.sin(R) >>> ax.plot_surface(X, Y, Z, rstride=1, cstride=1, cmap='viridis')
To read more about Matplotlib and what it can do, take a look at the official documentation. For directions regarding installing Matplotlib, see the official installation section.
Image credits: Jay Alammar https://jalammar.github.io/
On this pageشروع سریع
NumPy quickstart#
Prerequisites#
You’ll need to know a bit of Python. For a refresher, see the Python tutorial.
To work the examples, you’ll need matplotlib installed
in addition to NumPy.
Learner profile
This is a quick overview of arrays in NumPy. It demonstrates how n-dimensional (\(n>=2\)) arrays are represented and can be manipulated. In particular, if you don’t know how to apply common functions to n-dimensional arrays (without using for-loops), or if you want to understand axis and shape properties for n-dimensional arrays, this article might be of help.
Learning Objectives
After reading, you should be able to:
Understand the difference between one-, two- and n-dimensional arrays in NumPy;
Understand how to apply some linear algebra operations to n-dimensional arrays without using for-loops;
Understand axis and shape properties for n-dimensional arrays.
The basics#
NumPy’s main object is the homogeneous multidimensional array. It is a table of elements (usually numbers), all of the same type, indexed by a tuple of non-negative integers. In NumPy dimensions are called axes.
For example, the array for the coordinates of a point in 3D space,
[1, 2, 1], has one axis. That axis has 3 elements in it, so we say
it has a length of 3. In the example pictured below, the array has 2
axes. The first axis has a length of 2, the second axis has a length of
3.
[[1., 0., 0.], [0., 1., 2.]]
NumPy’s array class is called ndarray. It is also known by the alias
array. Note that numpy.array is not the same as the Standard
Python Library class array.array, which only handles one-dimensional
arrays and offers less functionality. The more important attributes of
an ndarray object are:
- ndarray.ndim
the number of axes (dimensions) of the array.
- ndarray.shape
the dimensions of the array. This is a tuple of integers indicating the size of the array in each dimension. For a matrix with n rows and m columns,
shapewill be(n,m). The length of theshapetuple is therefore the number of axes,ndim.- ndarray.size
the total number of elements of the array. This is equal to the product of the elements of
shape.- ndarray.dtype
an object describing the type of the elements in the array. One can create or specify dtype’s using standard Python types. Additionally NumPy provides types of its own. numpy.int32, numpy.int16, and numpy.float64 are some examples.
- ndarray.itemsize
the size in bytes of each element of the array. For example, an array of elements of type
float64hasitemsize8 (=64/8), while one of typecomplex32hasitemsize4 (=32/8). It is equivalent tondarray.dtype.itemsize.- ndarray.data
the buffer containing the actual elements of the array. Normally, we won’t need to use this attribute because we will access the elements in an array using indexing facilities.
An example#
>>> import numpy as np
>>> a = np.arange(15).reshape(3, 5)
>>> a
array([[ 0, 1, 2, 3, 4],
[ 5, 6, 7, 8, 9],
[10, 11, 12, 13, 14]])
>>> a.shape
(3, 5)
>>> a.ndim
2
>>> a.dtype.name
'int64'
>>> a.itemsize
8
>>> a.size
15
>>> type(a)
<class 'numpy.ndarray'>
>>> b = np.array([6, 7, 8])
>>> b
array([6, 7, 8])
>>> type(b)
<class 'numpy.ndarray'>
Array creation#
There are several ways to create arrays.
For example, you can create an array from a regular Python list or tuple
using the array function. The type of the resulting array is deduced
from the type of the elements in the sequences.
>>> import numpy as np
>>> a = np.array([2, 3, 4])
>>> a
array([2, 3, 4])
>>> a.dtype
dtype('int64')
>>> b = np.array([1.2, 3.5, 5.1])
>>> b.dtype
dtype('float64')
A frequent error consists in calling array with multiple arguments,
rather than providing a single sequence as an argument.
>>> a = np.array(1, 2, 3, 4) # WRONG Traceback (most recent call last): ... TypeError: array() takes from 1 to 2 positional arguments but 4 were given >>> a = np.array([1, 2, 3, 4]) # RIGHT
array transforms sequences of sequences into two-dimensional arrays,
sequences of sequences of sequences into three-dimensional arrays, and
so on.
>>> b = np.array([(1.5, 2, 3), (4, 5, 6)])
>>> b
array([[1.5, 2. , 3. ],
[4. , 5. , 6. ]])
The type of the array can also be explicitly specified at creation time:
>>> c = np.array([[1, 2], [3, 4]], dtype=np.complex128)
>>> c
array([[1.+0.j, 2.+0.j],
[3.+0.j, 4.+0.j]])
Often, the elements of an array are originally unknown, but its size is known. Hence, NumPy offers several functions to create arrays with initial placeholder content. These minimize the necessity of growing arrays, an expensive operation.
The function zeros creates an array full of zeros, the function
ones creates an array full of ones, and the function empty
creates an array whose initial content is random and depends on the
state of the memory. By default, the dtype of the created array is
float64, but it can be specified via the key word argument dtype.
>>> np.zeros((3, 4))
array([[0., 0., 0., 0.],
[0., 0., 0., 0.],
[0., 0., 0., 0.]])
>>> np.ones((2, 3, 4), dtype=np.int16)
array([[[1, 1, 1, 1],
[1, 1, 1, 1],
[1, 1, 1, 1]],
[[1, 1, 1, 1],
[1, 1, 1, 1],
[1, 1, 1, 1]]], dtype=int16)
>>> np.empty((2, 3))
array([[3.73603959e-262, 6.02658058e-154, 6.55490914e-260], # may vary
[5.30498948e-313, 3.14673309e-307, 1.00000000e+000]])
To create sequences of numbers, NumPy provides the arange function
which is analogous to the Python built-in range, but returns an
array.
>>> np.arange(10, 30, 5) array([10, 15, 20, 25]) >>> np.arange(0, 2, 0.3) # it accepts float arguments array([0. , 0.3, 0.6, 0.9, 1.2, 1.5, 1.8])
When arange is used with floating point arguments, it is generally
not possible to predict the number of elements obtained, due to the
finite floating point precision. For this reason, it is usually better
to use the function linspace that receives as an argument the number
of elements that we want, instead of the step:
>>> from numpy import pi >>> np.linspace(0, 2, 9) # 9 numbers from 0 to 2 array([0. , 0.25, 0.5 , 0.75, 1. , 1.25, 1.5 , 1.75, 2. ]) >>> x = np.linspace(0, 2 * pi, 100) # useful to evaluate function at lots of points >>> f = np.sin(x)
See also
array,
zeros,
zeros_like,
ones,
ones_like,
empty,
empty_like,
arange,
linspace,
random.Generator.random,
random.Generator.normal,
fromfunction,
fromfile
Printing arrays#
When you print an array, NumPy displays it in a similar way to nested lists, but with the following layout:
the last axis is printed from left to right,
the second-to-last is printed from top to bottom,
the rest are also printed from top to bottom, with each slice separated from the next by an empty line.
One-dimensional arrays are then printed as rows, bidimensionals as matrices and tridimensionals as lists of matrices.
>>> a = np.arange(6) # 1d array >>> print(a) [0 1 2 3 4 5] >>> >>> b = np.arange(12).reshape(4, 3) # 2d array >>> print(b) [[ 0 1 2] [ 3 4 5] [ 6 7 8] [ 9 10 11]] >>> >>> c = np.arange(24).reshape(2, 3, 4) # 3d array >>> print(c) [[[ 0 1 2 3] [ 4 5 6 7] [ 8 9 10 11]] [[12 13 14 15] [16 17 18 19] [20 21 22 23]]]
See below to get
more details on reshape.
If an array is too large to be printed, NumPy automatically skips the central part of the array and only prints the corners:
>>> print(np.arange(10000)) [ 0 1 2 ... 9997 9998 9999] >>> >>> print(np.arange(10000).reshape(100, 100)) [[ 0 1 2 ... 97 98 99] [ 100 101 102 ... 197 198 199] [ 200 201 202 ... 297 298 299] ... [9700 9701 9702 ... 9797 9798 9799] [9800 9801 9802 ... 9897 9898 9899] [9900 9901 9902 ... 9997 9998 9999]]
To disable this behaviour and force NumPy to print the entire array, you
can change the printing options using set_printoptions.
>>> np.set_printoptions(threshold=sys.maxsize) # sys module should be imported
Basic operations#
Arithmetic operators on arrays apply elementwise. A new array is created and filled with the result.
>>> a = np.array([20, 30, 40, 50]) >>> b = np.arange(4) >>> b array([0, 1, 2, 3]) >>> c = a - b >>> c array([20, 29, 38, 47]) >>> b**2 array([0, 1, 4, 9]) >>> 10 * np.sin(a) array([ 9.12945251, -9.88031624, 7.4511316 , -2.62374854]) >>> a < 35 array([ True, True, False, False])
Unlike in many matrix languages, the product operator * operates
elementwise in NumPy arrays. The matrix product can be performed using
the @ operator (in python >=3.5) or the dot function or method:
>>> A = np.array([[1, 1],
... [0, 1]])
>>> B = np.array([[2, 0],
... [3, 4]])
>>> A * B # elementwise product
array([[2, 0],
[0, 4]])
>>> A @ B # matrix product
array([[5, 4],
[3, 4]])
>>> A.dot(B) # another matrix product
array([[5, 4],
[3, 4]])
Some operations, such as += and *=, act in place to modify an
existing array rather than create a new one.
>>> rg = np.random.default_rng(1) # create instance of default random number generator
>>> a = np.ones((2, 3), dtype=np.int_)
>>> b = rg.random((2, 3))
>>> a *= 3
>>> a
array([[3, 3, 3],
[3, 3, 3]])
>>> b += a
>>> b
array([[3.51182162, 3.9504637 , 3.14415961],
[3.94864945, 3.31183145, 3.42332645]])
>>> a += b # b is not automatically converted to integer type
Traceback (most recent call last):
...
numpy._core._exceptions._UFuncOutputCastingError: Cannot cast ufunc 'add' output from dtype('float64') to dtype('int64') with casting rule 'same_kind'
When operating with arrays of different types, the type of the resulting array corresponds to the more general or precise one (a behavior known as upcasting).
>>> a = np.ones(3, dtype=np.int32)
>>> b = np.linspace(0, pi, 3)
>>> b.dtype.name
'float64'
>>> c = a + b
>>> c
array([1. , 2.57079633, 4.14159265])
>>> c.dtype.name
'float64'
>>> d = np.exp(c * 1j)
>>> d
array([ 0.54030231+0.84147098j, -0.84147098+0.54030231j,
-0.54030231-0.84147098j])
>>> d.dtype.name
'complex128'
Many unary operations, such as computing the sum of all the elements in
the array, are implemented as methods of the ndarray class.
>>> a = rg.random((2, 3))
>>> a
array([[0.82770259, 0.40919914, 0.54959369],
[0.02755911, 0.75351311, 0.53814331]])
>>> a.sum()
3.1057109529998157
>>> a.min()
0.027559113243068367
>>> a.max()
0.8277025938204418
By default, these operations apply to the array as though it were a list
of numbers, regardless of its shape. However, by specifying the axis
parameter you can apply an operation along the specified axis of an
array:
>>> b = np.arange(12).reshape(3, 4)
>>> b
array([[ 0, 1, 2, 3],
[ 4, 5, 6, 7],
[ 8, 9, 10, 11]])
>>>
>>> b.sum(axis=0) # sum of each column
array([12, 15, 18, 21])
>>>
>>> b.min(axis=1) # min of each row
array([0, 4, 8])
>>>
>>> b.cumsum(axis=1) # cumulative sum along each row
array([[ 0, 1, 3, 6],
[ 4, 9, 15, 22],
[ 8, 17, 27, 38]])
Universal functions#
NumPy provides familiar mathematical functions such as sin, cos, and
exp. In NumPy, these are called “universal
functions” (ufunc). Within NumPy, these functions
operate elementwise on an array, producing an array as output.
>>> B = np.arange(3) >>> B array([0, 1, 2]) >>> np.exp(B) array([1. , 2.71828183, 7.3890561 ]) >>> np.sqrt(B) array([0. , 1. , 1.41421356]) >>> C = np.array([2., -1., 4.]) >>> np.add(B, C) array([2., 0., 6.])
See also
all,
any,
apply_along_axis,
argmax,
argmin,
argsort,
average,
bincount,
ceil,
clip,
conj,
corrcoef,
cov,
cross,
cumprod,
cumsum,
diff,
dot,
floor,
inner,
invert,
lexsort,
max,
maximum,
mean,
median,
min,
minimum,
nonzero,
outer,
prod,
re,
round,
sort,
std,
sum,
trace,
transpose,
var,
vdot,
vectorize,
where
Indexing, slicing and iterating#
One-dimensional arrays can be indexed, sliced and iterated over, much like lists and other Python sequences.
>>> a = np.arange(10)**3 >>> a array([ 0, 1, 8, 27, 64, 125, 216, 343, 512, 729]) >>> a[2] 8 >>> a[2:5] array([ 8, 27, 64]) >>> # equivalent to a[0:6:2] = 1000; >>> # from start to position 6, exclusive, set every 2nd element to 1000 >>> a[:6:2] = 1000 >>> a array([1000, 1, 1000, 27, 1000, 125, 216, 343, 512, 729]) >>> a[::-1] # reversed a array([ 729, 512, 343, 216, 125, 1000, 27, 1000, 1, 1000]) >>> for i in a: ... print(i**(1 / 3.)) ... 9.999999999999998 # may vary 1.0 9.999999999999998 3.0 9.999999999999998 4.999999999999999 5.999999999999999 6.999999999999999 7.999999999999999 8.999999999999998
Multidimensional arrays can have one index per axis. These indices are given in a tuple separated by commas:
>>> def f(x, y):
... return 10 * x + y
...
>>> b = np.fromfunction(f, (5, 4), dtype=np.int_)
>>> b
array([[ 0, 1, 2, 3],
[10, 11, 12, 13],
[20, 21, 22, 23],
[30, 31, 32, 33],
[40, 41, 42, 43]])
>>> b[2, 3]
23
>>> b[0:5, 1] # each row in the second column of b
array([ 1, 11, 21, 31, 41])
>>> b[:, 1] # equivalent to the previous example
array([ 1, 11, 21, 31, 41])
>>> b[1:3, :] # each column in the second and third row of b
array([[10, 11, 12, 13],
[20, 21, 22, 23]])
When fewer indices are provided than the number of axes, the missing
indices are considered complete slices:
>>> b[-1] # the last row. Equivalent to b[-1, :] array([40, 41, 42, 43])
The expression within brackets in b[i] is treated as an i
followed by as many instances of : as needed to represent the
remaining axes. NumPy also allows you to write this using dots as
b[i, ...].
The dots (...) represent as many colons as needed to produce a
complete indexing tuple. For example, if x is an array with 5
axes, then
x[1, 2, ...]is equivalent tox[1, 2, :, :, :],x[..., 3]tox[:, :, :, :, 3]andx[4, ..., 5, :]tox[4, :, :, 5, :].
>>> c = np.array([[[ 0, 1, 2], # a 3D array (two stacked 2D arrays)
... [ 10, 12, 13]],
... [[100, 101, 102],
... [110, 112, 113]]])
>>> c.shape
(2, 2, 3)
>>> c[1, ...] # same as c[1, :, :] or c[1]
array([[100, 101, 102],
[110, 112, 113]])
>>> c[..., 2] # same as c[:, :, 2]
array([[ 2, 13],
[102, 113]])
Iterating over multidimensional arrays is done with respect to the first axis:
>>> for row in b: ... print(row) ... [0 1 2 3] [10 11 12 13] [20 21 22 23] [30 31 32 33] [40 41 42 43]
However, if one wants to perform an operation on each element in the
array, one can use the flat attribute which is an
iterator
over all the elements of the array:
>>> for element in b.flat: ... print(element) ... 0 1 2 3 10 11 12 13 20 21 22 23 30 31 32 33 40 41 42 43
See also
Indexing on ndarrays,
Indexing routines (reference),
newaxis,
ndenumerate,
indices
Shape manipulation#
Changing the shape of an array#
An array has a shape given by the number of elements along each axis:
>>> a = np.floor(10 * rg.random((3, 4)))
>>> a
array([[3., 7., 3., 4.],
[1., 4., 2., 2.],
[7., 2., 4., 9.]])
>>> a.shape
(3, 4)
The shape of an array can be changed with various commands. Note that the following three commands all return a modified array, but do not change the original array:
>>> a.ravel() # returns the array, flattened
array([3., 7., 3., 4., 1., 4., 2., 2., 7., 2., 4., 9.])
>>> a.reshape(6, 2) # returns the array with a modified shape
array([[3., 7.],
[3., 4.],
[1., 4.],
[2., 2.],
[7., 2.],
[4., 9.]])
>>> a.T # returns the array, transposed
array([[3., 1., 7.],
[7., 4., 2.],
[3., 2., 4.],
[4., 2., 9.]])
>>> a.T.shape
(4, 3)
>>> a.shape
(3, 4)
The order of the elements in the array resulting from ravel is
normally “C-style”, that is, the rightmost index “changes the fastest”,
so the element after a[0, 0] is a[0, 1]. If the array is reshaped to some
other shape, again the array is treated as “C-style”. NumPy normally
creates arrays stored in this order, so ravel will usually not need to
copy its argument, but if the array was made by taking slices of another
array or created with unusual options, it may need to be copied. The
functions ravel and reshape can also be instructed, using an
optional argument, to use FORTRAN-style arrays, in which the leftmost
index changes the fastest.
The reshape function returns its
argument with a modified shape, whereas the
ndarray.resize method modifies the array
itself:
>>> a
array([[3., 7., 3., 4.],
[1., 4., 2., 2.],
[7., 2., 4., 9.]])
>>> a.resize((2, 6))
>>> a
array([[3., 7., 3., 4., 1., 4.],
[2., 2., 7., 2., 4., 9.]])
If a dimension is given as -1 in a reshaping operation, the other
dimensions are automatically calculated:
>>> a.reshape(3, -1)
array([[3., 7., 3., 4.],
[1., 4., 2., 2.],
[7., 2., 4., 9.]])
See also
ndarray.shape,
reshape,
resize,
ravel
Stacking together different arrays#
Several arrays can be stacked together along different axes:
>>> a = np.floor(10 * rg.random((2, 2)))
>>> a
array([[9., 7.],
[5., 2.]])
>>> b = np.floor(10 * rg.random((2, 2)))
>>> b
array([[1., 9.],
[5., 1.]])
>>> np.vstack((a, b))
array([[9., 7.],
[5., 2.],
[1., 9.],
[5., 1.]])
>>> np.hstack((a, b))
array([[9., 7., 1., 9.],
[5., 2., 5., 1.]])
The function column_stack stacks 1D arrays as columns into a 2D array.
It is equivalent to hstack only for 2D arrays:
>>> from numpy import newaxis
>>> np.column_stack((a, b)) # with 2D arrays
array([[9., 7., 1., 9.],
[5., 2., 5., 1.]])
>>> a = np.array([4., 2.])
>>> b = np.array([3., 8.])
>>> np.column_stack((a, b)) # returns a 2D array
array([[4., 3.],
[2., 8.]])
>>> np.hstack((a, b)) # the result is different
array([4., 2., 3., 8.])
>>> a[:, newaxis] # view `a` as a 2D column vector
array([[4.],
[2.]])
>>> np.column_stack((a[:, newaxis], b[:, newaxis]))
array([[4., 3.],
[2., 8.]])
>>> np.hstack((a[:, newaxis], b[:, newaxis])) # the result is the same
array([[4., 3.],
[2., 8.]])
In general, for arrays with more than two dimensions,
hstack stacks along their second
axes, vstack stacks along their
first axes, and concatenate
allows for an optional arguments giving the number of the axis along
which the concatenation should happen.
Note
In complex cases, r_ and c_ are useful for creating arrays by stacking
numbers along one axis. They allow the use of range literals :.
>>> np.r_[1:4, 0, 4] array([1, 2, 3, 0, 4])
When used with arrays as arguments,
r_ and
c_ are similar to
vstack and
hstack in their default behavior,
but allow for an optional argument giving the number of the axis along
which to concatenate.
See also
hstack,
vstack,
column_stack,
concatenate,
c_,
r_
Splitting one array into several smaller ones#
Using hsplit, you can split an
array along its horizontal axis, either by specifying the number of
equally shaped arrays to return, or by specifying the columns after
which the division should occur:
>>> a = np.floor(10 * rg.random((2, 12)))
>>> a
array([[6., 7., 6., 9., 0., 5., 4., 0., 6., 8., 5., 2.],
[8., 5., 5., 7., 1., 8., 6., 7., 1., 8., 1., 0.]])
>>> # Split `a` into 3
>>> np.hsplit(a, 3)
[array([[6., 7., 6., 9.],
[8., 5., 5., 7.]]), array([[0., 5., 4., 0.],
[1., 8., 6., 7.]]), array([[6., 8., 5., 2.],
[1., 8., 1., 0.]])]
>>> # Split `a` after the third and the fourth column
>>> np.hsplit(a, (3, 4))
[array([[6., 7., 6.],
[8., 5., 5.]]), array([[9.],
[7.]]), array([[0., 5., 4., 0., 6., 8., 5., 2.],
[1., 8., 6., 7., 1., 8., 1., 0.]])]
vsplit splits along the vertical
axis, and array_split allows
one to specify along which axis to split.
Copies and views#
When operating and manipulating arrays, their data is sometimes copied into a new array and sometimes not. This is often a source of confusion for beginners. There are three cases:
No copy at all#
Simple assignments make no copy of objects or their data.
>>> a = np.array([[ 0, 1, 2, 3], ... [ 4, 5, 6, 7], ... [ 8, 9, 10, 11]]) >>> b = a # no new object is created >>> b is a # a and b are two names for the same ndarray object True
Python passes mutable objects as references, so function calls make no copy.
>>> def f(x): ... print(id(x)) ... >>> id(a) # id is a unique identifier of an object 148293216 # may vary >>> f(a) 148293216 # may vary
View or shallow copy#
Different array objects can share the same data. The view method
creates a new array object that looks at the same data.
>>> c = a.view()
>>> c is a
False
>>> c.base is a # c is a view of the data owned by a
True
>>> c.flags.owndata
False
>>>
>>> c = c.reshape((2, 6)) # a's shape doesn't change, reassigned c is still a view of a
>>> a.shape
(3, 4)
>>> c[0, 4] = 1234 # a's data changes
>>> a
array([[ 0, 1, 2, 3],
[1234, 5, 6, 7],
[ 8, 9, 10, 11]])
Slicing an array returns a view of it:
>>> s = a[:, 1:3]
>>> s[:] = 10 # s[:] is a view of s. Note the difference between s = 10 and s[:] = 10
>>> a
array([[ 0, 10, 10, 3],
[1234, 10, 10, 7],
[ 8, 10, 10, 11]])
Deep copy#
The copy method makes a complete copy of the array and its data.
>>> d = a.copy() # a new array object with new data is created
>>> d is a
False
>>> d.base is a # d doesn't share anything with a
False
>>> d[0, 0] = 9999
>>> a
array([[ 0, 10, 10, 3],
[1234, 10, 10, 7],
[ 8, 10, 10, 11]])
Sometimes copy should be called after slicing if the original array is not required anymore.
For example, suppose a is a huge intermediate result and the final result b only contains
a small fraction of a, a deep copy should be made when constructing b with slicing:
>>> a = np.arange(int(1e8)) >>> b = a[:100].copy() >>> del a # the memory of ``a`` can be released.
If b = a[:100] is used instead, a is referenced by b and will persist in memory
even if del a is executed.
See also Copies and views.
Functions and methods overview#
Here is a list of some useful NumPy functions and methods names ordered in categories. See Routines and objects by topic for the full list.
- Array Creation
arange,array,copy,empty,empty_like,eye,fromfile,fromfunction,identity,linspace,logspace,mgrid,ogrid,ones,ones_like,r_,zeros,zeros_like- Conversions
ndarray.astype,atleast_1d,atleast_2d,atleast_3d, mat- Manipulations
array_split,column_stack,concatenate,diagonal,dsplit,dstack,hsplit,hstack,ndarray.item,newaxis,ravel,repeat,reshape,resize,squeeze,swapaxes,take,transpose,vsplit,vstack- Questions
all,any,nonzero,where- Ordering
argmax,argmin,argsort,max,min,ptp,searchsorted,sort- Operations
choose,compress,cumprod,cumsum,inner,ndarray.fill,imag,prod,put,putmask,real,sum- Basic Statistics
cov,mean,std,var- Basic Linear Algebra
cross,dot,outer,linalg.svd,vdot
Less basic#
Broadcasting rules#
Broadcasting allows universal functions to deal in a meaningful way with inputs that do not have exactly the same shape.
The first rule of broadcasting is that if all input arrays do not have the same number of dimensions, a “1” will be repeatedly prepended to the shapes of the smaller arrays until all the arrays have the same number of dimensions.
The second rule of broadcasting ensures that arrays with a size of 1 along a particular dimension act as if they had the size of the array with the largest shape along that dimension. The value of the array element is assumed to be the same along that dimension for the “broadcast” array.
After application of the broadcasting rules, the sizes of all arrays must match. More details can be found in Broadcasting.
Advanced indexing and index tricks#
NumPy offers more indexing facilities than regular Python sequences. In addition to indexing by integers and slices, as we saw before, arrays can be indexed by arrays of integers and arrays of booleans.
Indexing with arrays of indices#
>>> a = np.arange(12)**2 # the first 12 square numbers
>>> i = np.array([1, 1, 3, 8, 5]) # an array of indices
>>> a[i] # the elements of `a` at the positions `i`
array([ 1, 1, 9, 64, 25])
>>>
>>> j = np.array([[3, 4], [9, 7]]) # a bidimensional array of indices
>>> a[j] # the same shape as `j`
array([[ 9, 16],
[81, 49]])
When the indexed array a is multidimensional, a single array of
indices refers to the first dimension of a. The following example
shows this behavior by converting an image of labels into a color image
using a palette.
>>> palette = np.array([[0, 0, 0], # black
... [255, 0, 0], # red
... [0, 255, 0], # green
... [0, 0, 255], # blue
... [255, 255, 255]]) # white
>>> image = np.array([[0, 1, 2, 0], # each value corresponds to a color in the palette
... [0, 3, 4, 0]])
>>> palette[image] # the (2, 4, 3) color image
array([[[ 0, 0, 0],
[255, 0, 0],
[ 0, 255, 0],
[ 0, 0, 0]],
[[ 0, 0, 0],
[ 0, 0, 255],
[255, 255, 255],
[ 0, 0, 0]]])
We can also give indexes for more than one dimension. The arrays of indices for each dimension must have the same shape.
>>> a = np.arange(12).reshape(3, 4)
>>> a
array([[ 0, 1, 2, 3],
[ 4, 5, 6, 7],
[ 8, 9, 10, 11]])
>>> i = np.array([[0, 1], # indices for the first dim of `a`
... [1, 2]])
>>> j = np.array([[2, 1], # indices for the second dim
... [3, 3]])
>>>
>>> a[i, j] # i and j must have equal shape
array([[ 2, 5],
[ 7, 11]])
>>>
>>> a[i, 2]
array([[ 2, 6],
[ 6, 10]])
>>>
>>> a[:, j]
array([[[ 2, 1],
[ 3, 3]],
[[ 6, 5],
[ 7, 7]],
[[10, 9],
[11, 11]]])
In Python, arr[i, j] is exactly the same as arr[(i, j)]—so we can
put i and j in a tuple and then do the indexing with that.
>>> l = (i, j)
>>> # equivalent to a[i, j]
>>> a[l]
array([[ 2, 5],
[ 7, 11]])
However, we can not do this by putting i and j into an array,
because this array will be interpreted as indexing the first dimension
of a.
>>> s = np.array([i, j])
>>> # not what we want
>>> a[s]
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
IndexError: index 3 is out of bounds for axis 0 with size 3
>>> # same as `a[i, j]`
>>> a[tuple(s)]
array([[ 2, 5],
[ 7, 11]])
Another common use of indexing with arrays is the search of the maximum value of time-dependent series:
>>> time = np.linspace(20, 145, 5) # time scale
>>> data = np.sin(np.arange(20)).reshape(5, 4) # 4 time-dependent series
>>> time
array([ 20. , 51.25, 82.5 , 113.75, 145. ])
>>> data
array([[ 0. , 0.84147098, 0.90929743, 0.14112001],
[-0.7568025 , -0.95892427, -0.2794155 , 0.6569866 ],
[ 0.98935825, 0.41211849, -0.54402111, -0.99999021],
[-0.53657292, 0.42016704, 0.99060736, 0.65028784],
[-0.28790332, -0.96139749, -0.75098725, 0.14987721]])
>>> # index of the maxima for each series
>>> ind = data.argmax(axis=0)
>>> ind
array([2, 0, 3, 1])
>>> # times corresponding to the maxima
>>> time_max = time[ind]
>>>
>>> data_max = data[ind, range(data.shape[1])] # => data[ind[0], 0], data[ind[1], 1]...
>>> time_max
array([ 82.5 , 20. , 113.75, 51.25])
>>> data_max
array([0.98935825, 0.84147098, 0.99060736, 0.6569866 ])
>>> np.all(data_max == data.max(axis=0))
True
You can also use indexing with arrays as a target to assign to:
>>> a = np.arange(5) >>> a array([0, 1, 2, 3, 4]) >>> a[[1, 3, 4]] = 0 >>> a array([0, 0, 2, 0, 0])
However, when the list of indices contains repetitions, the assignment is done several times, leaving behind the last value:
>>> a = np.arange(5) >>> a[[0, 0, 2]] = [1, 2, 3] >>> a array([2, 1, 3, 3, 4])
This is reasonable enough, but watch out if you want to use Python’s
+= construct, as it may not do what you expect:
>>> a = np.arange(5) >>> a[[0, 0, 2]] += 1 >>> a array([1, 1, 3, 3, 4])
Even though 0 occurs twice in the list of indices, the 0th element is
only incremented once. This is because Python requires a += 1 to be
equivalent to a = a + 1.
Indexing with boolean arrays#
When we index arrays with arrays of (integer) indices we are providing the list of indices to pick. With boolean indices the approach is different; we explicitly choose which items in the array we want and which ones we don’t.
The most natural way one can think of for boolean indexing is to use boolean arrays that have the same shape as the original array:
>>> a = np.arange(12).reshape(3, 4)
>>> b = a > 4
>>> b # `b` is a boolean with `a`'s shape
array([[False, False, False, False],
[False, True, True, True],
[ True, True, True, True]])
>>> a[b] # 1d array with the selected elements
array([ 5, 6, 7, 8, 9, 10, 11])
This property can be very useful in assignments:
>>> a[b] = 0 # All elements of `a` higher than 4 become 0
>>> a
array([[0, 1, 2, 3],
[4, 0, 0, 0],
[0, 0, 0, 0]])
You can look at the following example to see how to use boolean indexing to generate an image of the Mandelbrot set:
>>> import numpy as np >>> import matplotlib.pyplot as plt >>> def mandelbrot(h, w, maxit=20, r=2): ... """Returns an image of the Mandelbrot fractal of size (h,w).""" ... x = np.linspace(-2.5, 1.5, 4*h+1) ... y = np.linspace(-1.5, 1.5, 3*w+1) ... A, B = np.meshgrid(x, y) ... C = A + B*1j ... z = np.zeros_like(C) ... divtime = maxit + np.zeros(z.shape, dtype=np.int_) ... ... for i in range(maxit): ... z = z**2 + C ... diverge = abs(z) > r # who is diverging ... div_now = diverge & (divtime == maxit) # who is diverging now ... divtime[div_now] = i # note when ... z[diverge] = r # avoid diverging too much ... ... return divtime >>> plt.clf() >>> plt.imshow(mandelbrot(400, 400))
The second way of indexing with booleans is more similar to integer indexing; for each dimension of the array we give a 1D boolean array selecting the slices we want:
>>> a = np.arange(12).reshape(3, 4)
>>> b1 = np.array([False, True, True]) # first dim selection
>>> b2 = np.array([True, False, True, False]) # second dim selection
>>>
>>> a[b1, :] # selecting rows
array([[ 4, 5, 6, 7],
[ 8, 9, 10, 11]])
>>>
>>> a[b1] # same thing
array([[ 4, 5, 6, 7],
[ 8, 9, 10, 11]])
>>>
>>> a[:, b2] # selecting columns
array([[ 0, 2],
[ 4, 6],
[ 8, 10]])
>>>
>>> a[b1, b2] # a weird thing to do
array([ 4, 10])
Note that the length of the 1D boolean array must coincide with the
length of the dimension (or axis) you want to slice. In the previous
example, b1 has length 3 (the number of rows in a), and
b2 (of length 4) is suitable to index the 2nd axis (columns) of
a.
The ix_() function#
The ix_ function can be used to combine different vectors so as to
obtain the result for each n-uplet. For example, if you want to compute
all the a+b*c for all the triplets taken from each of the vectors a, b
and c:
>>> a = np.array([2, 3, 4, 5])
>>> b = np.array([8, 5, 4])
>>> c = np.array([5, 4, 6, 8, 3])
>>> ax, bx, cx = np.ix_(a, b, c)
>>> ax
array([[[2]],
[[3]],
[[4]],
[[5]]])
>>> bx
array([[[8],
[5],
[4]]])
>>> cx
array([[[5, 4, 6, 8, 3]]])
>>> ax.shape, bx.shape, cx.shape
((4, 1, 1), (1, 3, 1), (1, 1, 5))
>>> result = ax + bx * cx
>>> result
array([[[42, 34, 50, 66, 26],
[27, 22, 32, 42, 17],
[22, 18, 26, 34, 14]],
[[43, 35, 51, 67, 27],
[28, 23, 33, 43, 18],
[23, 19, 27, 35, 15]],
[[44, 36, 52, 68, 28],
[29, 24, 34, 44, 19],
[24, 20, 28, 36, 16]],
[[45, 37, 53, 69, 29],
[30, 25, 35, 45, 20],
[25, 21, 29, 37, 17]]])
>>> result[3, 2, 4]
17
>>> a[3] + b[2] * c[4]
17
You could also implement the reduce as follows:
>>> def ufunc_reduce(ufct, *vectors): ... vs = np.ix_(*vectors) ... r = ufct.identity ... for v in vs: ... r = ufct(r, v) ... return r
and then use it as:
>>> ufunc_reduce(np.add, a, b, c)
array([[[15, 14, 16, 18, 13],
[12, 11, 13, 15, 10],
[11, 10, 12, 14, 9]],
[[16, 15, 17, 19, 14],
[13, 12, 14, 16, 11],
[12, 11, 13, 15, 10]],
[[17, 16, 18, 20, 15],
[14, 13, 15, 17, 12],
[13, 12, 14, 16, 11]],
[[18, 17, 19, 21, 16],
[15, 14, 16, 18, 13],
[14, 13, 15, 17, 12]]])
The advantage of this version of reduce compared to the normal ufunc.reduce is that it makes use of the broadcasting rules in order to avoid creating a temporary array that needs as much memory as the output array multiplied by the number of vectors.
Indexing with strings#
See Structured arrays.
Tricks and tips#
Here we give a list of short and useful tips.
“Automatic” reshaping#
To change the dimensions of an array, you can omit one of the sizes which will then be deduced automatically:
>>> a = np.arange(30)
>>> b = a.reshape((2, -1, 3)) # -1 means "whatever is needed"
>>> b.shape
(2, 5, 3)
>>> b
array([[[ 0, 1, 2],
[ 3, 4, 5],
[ 6, 7, 8],
[ 9, 10, 11],
[12, 13, 14]],
[[15, 16, 17],
[18, 19, 20],
[21, 22, 23],
[24, 25, 26],
[27, 28, 29]]])
Vector stacking#
How do we construct a 2D array from a list of equally-sized row vectors?
In MATLAB this is quite easy: if x and y are two vectors of the
same length you only need do m=[x;y]. In NumPy this works via the
functions column_stack, dstack, hstack and vstack,
depending on the dimension in which the stacking is to be done. For
example:
>>> x = np.arange(0, 10, 2)
>>> y = np.arange(5)
>>> m = np.vstack([x, y])
>>> m
array([[0, 2, 4, 6, 8],
[0, 1, 2, 3, 4]])
>>> xy = np.hstack([x, y])
>>> xy
array([0, 2, 4, 6, 8, 0, 1, 2, 3, 4])
The logic behind those functions in more than two dimensions can be strange.
See also
NumPy for MATLAB users
Histograms#
The NumPy histogram function applied to an array returns a pair of
vectors: the histogram of the array and a vector of the bin edges. Beware:
matplotlib also has a function to build histograms (called hist,
as in Matlab) that differs from the one in NumPy. The main difference is
that pylab.hist plots the histogram automatically, while
numpy.histogram only generates the data.
>>> import numpy as np >>> rg = np.random.default_rng(1) >>> import matplotlib.pyplot as plt >>> # Build a vector of 10000 normal deviates with variance 0.5^2 and mean 2 >>> mu, sigma = 2, 0.5 >>> v = rg.normal(mu, sigma, 10000) >>> # Plot a normalized histogram with 50 bins >>> plt.hist(v, bins=50, density=True) # matplotlib version (plot) (array...) >>> # Compute the histogram with numpy and then plot it >>> (n, bins) = np.histogram(v, bins=50, density=True) # NumPy version (no plot) >>> plt.plot(.5 * (bins[1:] + bins[:-1]), n)
With Matplotlib >=3.4 you can also use plt.stairs(n, bins).
Further reading#
The Python tutorial
NumPy reference
SciPy Tutorial
SciPy Lecture Notes
A matlab, R, IDL, NumPy/SciPy dictionary
tutorial-svd
اندیسگذاری
Indexing on ndarrays#
See also
Indexing routines
ndarrays can be indexed using the standard Python
x[obj] syntax, where x is the array and obj the selection.
There are different kinds of indexing available depending on obj:
basic indexing, advanced indexing and field access.
Most of the following examples show the use of indexing when referencing data in an array. The examples work just as well when assigning to an array. See Assigning values to indexed arrays for specific examples and explanations on how assignments work.
Note that in Python, x[(exp1, exp2, ..., expN)] is equivalent to
x[exp1, exp2, ..., expN]; the latter is just syntactic sugar
for the former.
Basic indexing#
Single element indexing#
Single element indexing works exactly like that for other standard Python sequences. It is 0-based, and accepts negative indices for indexing from the end of the array.
>>> x = np.arange(10) >>> x[2] 2 >>> x[-2] 8
It is not necessary to separate each dimension’s index into its own set of square brackets.
>>> x = x.reshape((2, 5)) # now x is 2-dimensional >>> x[1, 3] 8 >>> x[1, -1] 9
Note that if one indexes a multidimensional array with fewer indices than dimensions, one gets a subdimensional array. For example:
>>> x[0] array([0, 1, 2, 3, 4])
That is, each index specified selects the array corresponding to the rest of the dimensions selected. In the above example, choosing 0 means that the remaining dimension of length 5 is being left unspecified, and that what is returned is an array of that dimensionality and size. It must be noted that the returned array is a view, i.e., it is not a copy of the original, but points to the same values in memory as does the original array. In this case, the 1-D array at the first position (0) is returned. So using a single index on the returned array, results in a single element being returned. That is:
>>> x[0][2] 2
So note that x[0, 2] == x[0][2] though the second case is more
inefficient as a new temporary array is created after the first index
that is subsequently indexed by 2.
Note
NumPy uses C-order indexing. That means that the last index usually represents the most rapidly changing memory location, unlike Fortran or IDL, where the first index represents the most rapidly changing location in memory. This difference represents a great potential for confusion.
Slicing and striding#
Basic slicing extends Python’s basic concept of slicing to N
dimensions. Basic slicing occurs when obj is a slice object
(constructed by start:stop:step notation inside of brackets), an
integer, or a tuple of slice objects and integers. Ellipsis
and newaxis objects can be interspersed with these as
well.
The simplest case of indexing with N integers returns an array scalar representing the corresponding item. As in Python, all indices are zero-based: for the i-th index \(n_i\), the valid range is \(0 \le n_i < d_i\) where \(d_i\) is the i-th element of the shape of the array. Negative indices are interpreted as counting from the end of the array (i.e., if \(n_i < 0\), it means \(n_i + d_i\)).
All arrays generated by basic slicing are always views of the original array.
Note
NumPy slicing creates a view instead of a copy as in the case of
built-in Python sequences such as string, tuple and list.
Care must be taken when extracting
a small portion from a large array which becomes useless after the
extraction, because the small portion extracted contains a reference
to the large original array whose memory will not be released until
all arrays derived from it are garbage-collected. In such cases an
explicit copy() is recommended.
The standard rules of sequence slicing apply to basic slicing on a per-dimension basis (including using a step index). Some useful concepts to remember include:
The basic slice syntax is
i:j:kwhere i is the starting index, j is the stopping index, and k is the step (\(k\neq0\)). This selects the m elements (in the corresponding dimension) with index values i, i + k, …, i + (m - 1) k where \(m = q + (r\neq0)\) and q and r are the quotient and remainder obtained by dividing j - i by k: j - i = q k + r, so that i + (m - 1) k < j. For example:>>> x = np.array([0, 1, 2, 3, 4, 5, 6, 7, 8, 9]) >>> x[1:7:2] array([1, 3, 5])
Negative i and j are interpreted as n + i and n + j where n is the number of elements in the corresponding dimension. Negative k makes stepping go towards smaller indices. From the above example:
>>> x[-2:10] array([8, 9]) >>> x[-3:3:-1] array([7, 6, 5, 4])
Assume n is the number of elements in the dimension being sliced. Then, if i is not given it defaults to 0 for k > 0 and n - 1 for k < 0 . If j is not given it defaults to n for k > 0 and -n-1 for k < 0 . If k is not given it defaults to 1. Note that
::is the same as:and means select all indices along this axis. From the above example:>>> x[5:] array([5, 6, 7, 8, 9])
If the number of objects in the selection tuple is less than N, then
:is assumed for any subsequent dimensions. For example:>>> x = np.array([[[1],[2],[3]], [[4],[5],[6]]]) >>> x.shape (2, 3, 1) >>> x[1:2] array([[[4], [5], [6]]])An integer, i, returns the same values as
i:i+1except the dimensionality of the returned object is reduced by 1. In particular, a selection tuple with the p-th element an integer (and all other entries:) returns the corresponding sub-array with dimension N - 1. If N = 1 then the returned object is an array scalar. These objects are explained in Scalars.If the selection tuple has all entries
:except the p-th entry which is a slice objecti:j:k, then the returned array has dimension N formed by stacking, along the p-th axis, the sub-arrays returned by integer indexing of elements i, i+k, …, i + (m - 1) k < j.Basic slicing with more than one non-
:entry in the slicing tuple, acts like repeated application of slicing using a single non-:entry, where the non-:entries are successively taken (with all other non-:entries replaced by:). Thus,x[ind1, ..., ind2,:]acts likex[ind1][..., ind2, :]under basic slicing.Warning
The above is not true for advanced indexing.
You may use slicing to set values in the array, but (unlike lists) you can never grow the array. The size of the value to be set in
x[obj] = valuemust be (broadcastable to) the same shape asx[obj].A slicing tuple can always be constructed as obj and used in the
x[obj]notation. Slice objects can be used in the construction in place of the[start:stop:step]notation. For example,x[1:10:5, ::-1]can also be implemented asobj = (slice(1, 10, 5), slice(None, None, -1)); x[obj]. This can be useful for constructing generic code that works on arrays of arbitrary dimensions. See Dealing with variable numbers of indices within programs for more information.
Dimensional indexing tools#
There are some tools to facilitate the easy matching of array shapes with expressions and in assignments.
Ellipsis expands to the number of : objects needed for the
selection tuple to index all dimensions. In most cases, this means that the
length of the expanded selection tuple is x.ndim. There may only be a
single ellipsis present.
From the above example:
>>> x[..., 0]
array([[1, 2, 3],
[4, 5, 6]])
This is equivalent to:
>>> x[:, :, 0]
array([[1, 2, 3],
[4, 5, 6]])
Each newaxis object in the selection tuple serves to expand
the dimensions of the resulting selection by one unit-length
dimension. The added dimension is the position of the newaxis
object in the selection tuple. newaxis is an alias for
None, and None can be used in place of this with the same result.
From the above example:
>>> x[:, np.newaxis, :, :].shape (2, 1, 3, 1) >>> x[:, None, :, :].shape (2, 1, 3, 1)
This can be handy to combine two arrays in a way that otherwise would require explicit reshaping operations. For example:
>>> x = np.arange(5)
>>> x[:, np.newaxis] + x[np.newaxis, :]
array([[0, 1, 2, 3, 4],
[1, 2, 3, 4, 5],
[2, 3, 4, 5, 6],
[3, 4, 5, 6, 7],
[4, 5, 6, 7, 8]])
Advanced indexing#
Advanced indexing is triggered when the selection object, obj, is a
non-tuple sequence object, an ndarray (of data type integer or bool),
or a tuple with at least one sequence object or ndarray (of data type
integer or bool). There are two types of advanced indexing: integer
and Boolean.
Advanced indexing always returns a copy of the data (contrast with basic slicing that returns a view).
Warning
The definition of advanced indexing means that x[(1, 2, 3),] is
fundamentally different than x[(1, 2, 3)]. The latter is
equivalent to x[1, 2, 3] which will trigger basic selection while
the former will trigger advanced indexing. Be sure to understand
why this occurs.
Integer array indexing#
Integer array indexing allows selection of arbitrary items in the array based on their N-dimensional index. Each integer array represents a number of indices into that dimension.
Negative values are permitted in the index arrays and work as they do with single indices or slices:
>>> x = np.arange(10, 1, -1) >>> x array([10, 9, 8, 7, 6, 5, 4, 3, 2]) >>> x[np.array([3, 3, 1, 8])] array([7, 7, 9, 2]) >>> x[np.array([3, 3, -3, 8])] array([7, 7, 4, 2])
If the index values are out of bounds then an IndexError is thrown:
>>> x = np.array([[1, 2], [3, 4], [5, 6]])
>>> x[np.array([1, -1])]
array([[3, 4],
[5, 6]])
>>> x[np.array([3, 4])]
Traceback (most recent call last):
...
IndexError: index 3 is out of bounds for axis 0 with size 3
When the index consists of as many integer arrays as dimensions of the array being indexed, the indexing is straightforward, but different from slicing.
Advanced indices always are broadcast and iterated as one:
result[i_1, ..., i_M] == x[ind_1[i_1, ..., i_M], ind_2[i_1, ..., i_M],
..., ind_N[i_1, ..., i_M]]
Note that the resulting shape is identical to the (broadcast) indexing array
shapes ind_1, ..., ind_N. If the indices cannot be broadcast to the
same shape, an exception IndexError: shape mismatch: indexing arrays could
not be broadcast together with shapes... is raised.
Indexing with multidimensional index arrays tend to be more unusual uses, but they are permitted, and they are useful for some problems. We’ll start with the simplest multidimensional case:
>>> y = np.arange(35).reshape(5, 7)
>>> y
array([[ 0, 1, 2, 3, 4, 5, 6],
[ 7, 8, 9, 10, 11, 12, 13],
[14, 15, 16, 17, 18, 19, 20],
[21, 22, 23, 24, 25, 26, 27],
[28, 29, 30, 31, 32, 33, 34]])
>>> y[np.array([0, 2, 4]), np.array([0, 1, 2])]
array([ 0, 15, 30])
In this case, if the index arrays have a matching shape, and there is an
index array for each dimension of the array being indexed, the resultant
array has the same shape as the index arrays, and the values correspond
to the index set for each position in the index arrays. In this example,
the first index value is 0 for both index arrays, and thus the first value
of the resultant array is y[0, 0]. The next value is y[2, 1], and
the last is y[4, 2].
If the index arrays do not have the same shape, there is an attempt to broadcast them to the same shape. If they cannot be broadcast to the same shape, an exception is raised:
>>> y[np.array([0, 2, 4]), np.array([0, 1])] Traceback (most recent call last): ... IndexError: shape mismatch: indexing arrays could not be broadcast together with shapes (3,) (2,)
The broadcasting mechanism permits index arrays to be combined with scalars for other indices. The effect is that the scalar value is used for all the corresponding values of the index arrays:
>>> y[np.array([0, 2, 4]), 1] array([ 1, 15, 29])
Jumping to the next level of complexity, it is possible to only partially index an array with index arrays. It takes a bit of thought to understand what happens in such cases. For example if we just use one index array with y:
>>> y[np.array([0, 2, 4])]
array([[ 0, 1, 2, 3, 4, 5, 6],
[14, 15, 16, 17, 18, 19, 20],
[28, 29, 30, 31, 32, 33, 34]])
It results in the construction of a new array where each value of the index array selects one row from the array being indexed and the resultant array has the resulting shape (number of index elements, size of row).
In general, the shape of the resultant array will be the concatenation of the shape of the index array (or the shape that all the index arrays were broadcast to) with the shape of any unused dimensions (those not indexed) in the array being indexed.
Example
From each row, a specific element should be selected. The row index is just
[0, 1, 2] and the column index specifies the element to choose for the
corresponding row, here [0, 1, 0]. Using both together the task
can be solved using advanced indexing:
>>> x = np.array([[1, 2], [3, 4], [5, 6]]) >>> x[[0, 1, 2], [0, 1, 0]] array([1, 4, 5])
To achieve a behaviour similar to the basic slicing above, broadcasting can be
used. The function ix_ can help with this broadcasting. This is best
understood with an example.
Example
From a 4x3 array the corner elements should be selected using advanced
indexing. Thus all elements for which the column is one of [0, 2] and
the row is one of [0, 3] need to be selected. To use advanced indexing
one needs to select all elements explicitly. Using the method explained
previously one could write:
>>> x = np.array([[ 0, 1, 2],
... [ 3, 4, 5],
... [ 6, 7, 8],
... [ 9, 10, 11]])
>>> rows = np.array([[0, 0],
... [3, 3]], dtype=np.intp)
>>> columns = np.array([[0, 2],
... [0, 2]], dtype=np.intp)
>>> x[rows, columns]
array([[ 0, 2],
[ 9, 11]])
However, since the indexing arrays above just repeat themselves,
broadcasting can be used (compare operations such as
rows[:, np.newaxis] + columns) to simplify this:
>>> rows = np.array([0, 3], dtype=np.intp)
>>> columns = np.array([0, 2], dtype=np.intp)
>>> rows[:, np.newaxis]
array([[0],
[3]])
>>> x[rows[:, np.newaxis], columns]
array([[ 0, 2],
[ 9, 11]])
This broadcasting can also be achieved using the function ix_:
>>> x[np.ix_(rows, columns)]
array([[ 0, 2],
[ 9, 11]])
Note that without the np.ix_ call, only the diagonal elements would
be selected:
>>> x[rows, columns] array([ 0, 11])
This difference is the most important thing to remember about indexing with multiple advanced indices.
Example
A real-life example of where advanced indexing may be useful is for a color lookup table where we want to map the values of an image into RGB triples for display. The lookup table could have a shape (nlookup, 3). Indexing such an array with an image with shape (ny, nx) with dtype=np.uint8 (or any integer type so long as values are with the bounds of the lookup table) will result in an array of shape (ny, nx, 3) where a triple of RGB values is associated with each pixel location.
Boolean array indexing#
This advanced indexing occurs when obj is an array object of Boolean
type, such as may be returned from comparison operators. A single
boolean index array is practically identical to x[obj.nonzero()] where,
as described above, obj.nonzero() returns a
tuple (of length obj.ndim) of integer index
arrays showing the True elements of obj. However, it is
faster when obj.shape == x.shape.
If obj.ndim == x.ndim, x[obj]
returns a 1-dimensional array filled with the elements of x
corresponding to the True values of obj. The search order
will be row-major, C-style. An index error will be raised if
the shape of obj does not match the corresponding dimensions of x,
regardless of whether those values are True or
False.
A common use case for this is filtering for desired element values.
For example, one may wish to select all entries from an array which
are not numpy.nan:
>>> x = np.array([[1., 2.], [np.nan, 3.], [np.nan, np.nan]]) >>> x[~np.isnan(x)] array([1., 2., 3.])
Or wish to add a constant to all negative elements:
>>> x = np.array([1., -1., -2., 3]) >>> x[x < 0] += 20 >>> x array([ 1., 19., 18., 3.])
In general if an index includes a Boolean array, the result will be
identical to inserting obj.nonzero() into the same position
and using the integer array indexing mechanism described above.
x[ind_1, boolean_array, ind_2] is equivalent to
x[(ind_1,) + boolean_array.nonzero() + (ind_2,)].
If there is only one Boolean array and no integer indexing array present, this is straightforward. Care must only be taken to make sure that the boolean index has exactly as many dimensions as it is supposed to work with.
In general, when the boolean array has fewer dimensions than the array being
indexed, this is equivalent to x[b, ...], which means x is indexed by b
followed by as many : as are needed to fill out the rank of x. Thus the
shape of the result is one dimension containing the number of True elements of
the boolean array, followed by the remaining dimensions of the array being
indexed:
>>> x = np.arange(35).reshape(5, 7)
>>> b = x > 20
>>> b[:, 5]
array([False, False, False, True, True])
>>> x[b[:, 5]]
array([[21, 22, 23, 24, 25, 26, 27],
[28, 29, 30, 31, 32, 33, 34]])
Here the 4th and 5th rows are selected from the indexed array and combined to make a 2-D array.
Example
From an array, select all rows which sum up to less or equal two:
>>> x = np.array([[0, 1], [1, 1], [2, 2]])
>>> rowsum = x.sum(-1)
>>> x[rowsum <= 2, :]
array([[0, 1],
[1, 1]])
Combining multiple Boolean indexing arrays or a Boolean with an integer
indexing array can best be understood with the
obj.nonzero() analogy. The function ix_
also supports boolean arrays and will work without any surprises.
Example
Use boolean indexing to select all rows adding up to an even
number. At the same time columns 0 and 2 should be selected with an
advanced integer index. Using the ix_ function this can be done
with:
>>> x = np.array([[ 0, 1, 2],
... [ 3, 4, 5],
... [ 6, 7, 8],
... [ 9, 10, 11]])
>>> rows = (x.sum(-1) % 2) == 0
>>> rows
array([False, True, False, True])
>>> columns = [0, 2]
>>> x[np.ix_(rows, columns)]
array([[ 3, 5],
[ 9, 11]])
Without the np.ix_ call, only the diagonal elements would be
selected.
Or without np.ix_ (compare the integer array examples):
>>> rows = rows.nonzero()[0]
>>> x[rows[:, np.newaxis], columns]
array([[ 3, 5],
[ 9, 11]])
Example
Use a 2-D boolean array of shape (2, 3) with four True elements to select rows from a 3-D array of shape (2, 3, 5) results in a 2-D result of shape (4, 5):
>>> x = np.arange(30).reshape(2, 3, 5)
>>> x
array([[[ 0, 1, 2, 3, 4],
[ 5, 6, 7, 8, 9],
[10, 11, 12, 13, 14]],
[[15, 16, 17, 18, 19],
[20, 21, 22, 23, 24],
[25, 26, 27, 28, 29]]])
>>> b = np.array([[True, True, False], [False, True, True]])
>>> x[b]
array([[ 0, 1, 2, 3, 4],
[ 5, 6, 7, 8, 9],
[20, 21, 22, 23, 24],
[25, 26, 27, 28, 29]])
Combining advanced and basic indexing#
When there is at least one slice (:), ellipsis (...) or newaxis
in the index (or the array has more dimensions than there are advanced indices),
then the behaviour can be more complicated. It is like concatenating the
indexing result for each advanced index element.
In the simplest case, there is only a single advanced index combined with a slice. For example:
>>> y = np.arange(35).reshape(5,7)
>>> y[np.array([0, 2, 4]), 1:3]
array([[ 1, 2],
[15, 16],
[29, 30]])
In effect, the slice and index array operation are independent. The slice operation extracts columns with index 1 and 2, (i.e. the 2nd and 3rd columns), followed by the index array operation which extracts rows with index 0, 2 and 4 (i.e the first, third and fifth rows). This is equivalent to:
>>> y[:, 1:3][np.array([0, 2, 4]), :]
array([[ 1, 2],
[15, 16],
[29, 30]])
A single advanced index can, for example, replace a slice and the result array will be the same. However, it is a copy and may have a different memory layout. A slice is preferable when it is possible. For example:
>>> x = np.array([[ 0, 1, 2], ... [ 3, 4, 5], ... [ 6, 7, 8], ... [ 9, 10, 11]]) >>> x[1:2, 1:3] array([[4, 5]]) >>> x[1:2, [1, 2]] array([[4, 5]])
The easiest way to understand a combination of multiple advanced indices may be to think in terms of the resulting shape. There are two parts to the indexing operation, the subspace defined by the basic indexing (excluding integers) and the subspace from the advanced indexing part. Two cases of index combination need to be distinguished:
The advanced indices are separated by a slice,
Ellipsisornewaxis. For examplex[arr1, :, arr2].The advanced indices are all next to each other. For example
x[..., arr1, arr2, :]but notx[arr1, :, 1]since1is an advanced index in this regard.
In the first case, the dimensions resulting from the advanced indexing operation come first in the result array, and the subspace dimensions after that. In the second case, the dimensions from the advanced indexing operations are inserted into the result array at the same spot as they were in the initial array (the latter logic is what makes simple advanced indexing behave just like slicing).
Example
Suppose x.shape is (10, 20, 30) and ind is a (2, 5, 2)-shaped
indexing intp array, then result = x[..., ind, :] has
shape (10, 2, 5, 2, 30) because the (20,)-shaped subspace has been
replaced with a (2, 5, 2)-shaped broadcasted indexing subspace. If
we let i, j, k loop over the (2, 5, 2)-shaped subspace then
result[..., i, j, k, :] = x[..., ind[i, j, k], :]. This example
produces the same result as x.take(ind, axis=-2).
Example
Let x.shape be (10, 20, 30, 40, 50) and suppose ind_1
and ind_2 can be broadcast to the shape (2, 3, 4). Then
x[:, ind_1, ind_2] has shape (10, 2, 3, 4, 40, 50) because the
(20, 30)-shaped subspace from X has been replaced with the
(2, 3, 4) subspace from the indices. However,
x[:, ind_1, :, ind_2] has shape (2, 3, 4, 10, 30, 50) because there
is no unambiguous place to drop in the indexing subspace, thus
it is tacked-on to the beginning. It is always possible to use
.transpose() to move the subspace
anywhere desired. Note that this example cannot be replicated
using take.
Example
Slicing can be combined with broadcasted boolean indices:
>>> x = np.arange(35).reshape(5, 7)
>>> b = x > 20
>>> b
array([[False, False, False, False, False, False, False],
[False, False, False, False, False, False, False],
[False, False, False, False, False, False, False],
[ True, True, True, True, True, True, True],
[ True, True, True, True, True, True, True]])
>>> x[b[:, 5], 1:3]
array([[22, 23],
[29, 30]])
Field access#
See also
Structured arrays
If the ndarray object is a structured array the fields
of the array can be accessed by indexing the array with strings,
dictionary-like.
Indexing x['field-name'] returns a new view to the array,
which is of the same shape as x (except when the field is a
sub-array) but of data type x.dtype['field-name'] and contains
only the part of the data in the specified field. Also,
record array scalars can be “indexed” this way.
Indexing into a structured array can also be done with a list of field names,
e.g. x[['field-name1', 'field-name2']]. As of NumPy 1.16, this returns a
view containing only those fields. In older versions of NumPy, it returned a
copy. See the user guide section on Structured arrays for more
information on multifield indexing.
If the accessed field is a sub-array, the dimensions of the sub-array are appended to the shape of the result. For example:
>>> x = np.zeros((2, 2), dtype=[('a', np.int32), ('b', np.float64, (3, 3))])
>>> x['a'].shape
(2, 2)
>>> x['a'].dtype
dtype('int32')
>>> x['b'].shape
(2, 2, 3, 3)
>>> x['b'].dtype
dtype('float64')
Flat iterator indexing#
x.flat returns an iterator that will iterate
over the entire array (in C-contiguous style with the last index
varying the fastest). This iterator object can also be indexed using
basic slicing or advanced indexing as long as the selection object is
not a tuple. This should be clear from the fact that x.flat is a 1-dimensional view. It can be used for integer
indexing with 1-dimensional C-style-flat indices. The shape of any
returned array is therefore the shape of the integer indexing object.
Assigning values to indexed arrays#
As mentioned, one can select a subset of an array to assign to using a single index, slices, and index and mask arrays. The value being assigned to the indexed array must be shape consistent (the same shape or broadcastable to the shape the index produces). For example, it is permitted to assign a constant to a slice:
>>> x = np.arange(10) >>> x[2:7] = 1
or an array of the right size:
>>> x[2:7] = np.arange(5)
Note that assignments may result in changes if assigning higher types to lower types (like floats to ints) or even exceptions (assigning complex to floats or ints):
>>> x[1] = 1.2 >>> x[1] 1 >>> x[1] = 1.2j Traceback (most recent call last): ... TypeError: can't convert complex to int
Unlike some of the references (such as array and mask indices) assignments are always made to the original data in the array (indeed, nothing else would make sense!). Note though, that some actions may not work as one may naively expect. This particular example is often surprising to people:
>>> x = np.arange(0, 50, 10) >>> x array([ 0, 10, 20, 30, 40]) >>> x[np.array([1, 1, 3, 1])] += 1 >>> x array([ 0, 11, 20, 31, 40])
Where people expect that the 1st location will be incremented by 3.
In fact, it will only be incremented by 1. The reason is that
a new array is extracted from the original (as a temporary) containing
the values at 1, 1, 3, 1, then the value 1 is added to the temporary,
and then the temporary is assigned back to the original array. Thus
the value of the array at x[1] + 1 is assigned to x[1] three times,
rather than being incremented 3 times.
Dealing with variable numbers of indices within programs#
The indexing syntax is very powerful but limiting when dealing with a variable number of indices. For example, if you want to write a function that can handle arguments with various numbers of dimensions without having to write special case code for each number of possible dimensions, how can that be done? If one supplies to the index a tuple, the tuple will be interpreted as a list of indices. For example:
>>> z = np.arange(81).reshape(3, 3, 3, 3) >>> indices = (1, 1, 1, 1) >>> z[indices] 40
So one can use code to construct tuples of any number of indices and then use these within an index.
Slices can be specified within programs by using the slice() function in Python. For example:
>>> indices = (1, 1, 1, slice(0, 2)) # same as [1, 1, 1, 0:2] >>> z[indices] array([39, 40])
Likewise, ellipsis can be specified by code by using the Ellipsis object:
>>> indices = (1, Ellipsis, 1) # same as [1, ..., 1]
>>> z[indices]
array([[28, 31, 34],
[37, 40, 43],
[46, 49, 52]])
For this reason, it is possible to use the output from the
np.nonzero() function directly as an index since
it always returns a tuple of index arrays.
Because of the special treatment of tuples, they are not automatically converted to an array as a list would be. As an example:
>>> z[[1, 1, 1, 1]] # produces a large array
array([[[[27, 28, 29],
[30, 31, 32], ...
>>> z[(1, 1, 1, 1)] # returns a single value
40
Detailed notes#
These are some detailed notes, which are not of importance for day to day indexing (in no particular order):
The native NumPy indexing type is
intpand may differ from the default integer array type.intpis the smallest data type sufficient to safely index any array; for advanced indexing it may be faster than other types.For advanced assignments, there is in general no guarantee for the iteration order. This means that if an element is set more than once, it is not possible to predict the final result.
An empty (tuple) index is a full scalar index into a zero-dimensional array.
x[()]returns a scalar ifxis zero-dimensional and a view otherwise. On the other hand,x[...]always returns a view.If a zero-dimensional array is present in the index and it is a full integer index the result will be a scalar and not a zero-dimensional array. (Advanced indexing is not triggered.)
When an ellipsis (
...) is present but has no size (i.e. replaces zero:) the result will still always be an array. A view if no advanced index is present, otherwise a copy.The
nonzeroequivalence for Boolean arrays does not hold for zero dimensional boolean arrays.When the result of an advanced indexing operation has no elements but an individual index is out of bounds, whether or not an
IndexErroris raised is undefined (e.g.x[[], [123]]with123being out of bounds).When a casting error occurs during assignment (for example updating a numerical array using a sequence of strings), the array being assigned to may end up in an unpredictable partially updated state. However, if any other error (such as an out of bounds index) occurs, the array will remain unchanged.
The memory layout of an advanced indexing result is optimized for each indexing operation and no particular memory order can be assumed.
When using a subclass (especially one which manipulates its shape), the default
ndarray.__setitem__behaviour will call__getitem__for basic indexing but not for advanced indexing. For such a subclass it may be preferable to callndarray.__setitem__with a base class ndarray view on the data. This must be done if the subclasses__getitem__does not return views.