Dask¶
Dask is a flexible parallel computing library for analytic computing.
Dask is composed of two components:
 Dynamic task scheduling optimized for computation. This is similar to Airflow, Luigi, Celery, or Make, but optimized for interactive computational workloads.
 “Big Data” collections like parallel arrays, dataframes, and lists that extend common interfaces like NumPy, Pandas, or Python iterators to largerthanmemory or distributed environments. These parallel collections run on top of the dynamic task schedulers.
Dask emphasizes the following virtues:
 Familiar: Provides parallelized NumPy array and Pandas DataFrame objects
 Flexible: Provides a task scheduling interface for more custom workloads and integration with other projects.
 Native: Enables distributed computing in Pure Python with access to the PyData stack.
 Fast: Operates with low overhead, low latency, and minimal serialization necessary for fast numerical algorithms
 Scales up: Runs resiliently on clusters with 1000s of cores
 Scales down: Trivial to set up and run on a laptop in a single process
 Responsive: Designed with interactive computing in mind it provides rapid feedback and diagnostics to aid humans
See the dask.distributed documentation (separate website) for more technical information on Dask’s distributed scheduler,
Familiar user interface¶
Dask DataFrame mimics Pandas
import pandas as pd import dask.dataframe as dd
df = pd.read_csv('20150101.csv') df = dd.read_csv('2015**.csv')
df.groupby(df.user_id).value.mean() df.groupby(df.user_id).value.mean().compute()
Dask Array mimics NumPy
import numpy as np import dask.array as da
f = h5py.File('myfile.hdf5') f = h5py.File('myfile.hdf5')
x = np.array(f['/smalldata']) x = da.from_array(f['/bigdata'],
chunks=(1000, 1000))
x  x.mean(axis=1) x  x.mean(axis=1).compute()
Dask Bag mimics iterators, Toolz, and PySpark
import dask.bag as db
b = db.read_text('2015**.json.gz').map(json.loads)
b.pluck('name').frequencies().topk(10, lambda pair: pair[1]).compute()
Dask Delayed mimics for loops and wraps custom code
from dask import delayed
L = []
for fn in filenames: # Use for loops to build up computation
data = delayed(load)(fn) # Delay execution of function
L.append(delayed(process)(data)) # Build connections between variables
result = delayed(summarize)(L)
result.compute()
The concurrent.futures interface provides general submission of custom tasks:
from dask.distributed import Client
client = Client('scheduler:port')
futures = []
for fn in filenames:
future = client.submit(load, fn)
futures.append(future)
summary = client.submit(summarize, futures)
summary.result()
Scales from laptops to clusters¶
Dask is convenient on a laptop. It installs trivially with
conda
or pip
and extends the size of convenient datasets from “fits in
memory” to “fits on disk”.
Dask can scale to a cluster of 100s of machines. It is resilient, elastic, data local, and low latency. For more information see documentation on the distributed scheduler.
This ease of transition between singlemachine to moderate cluster enables users both to start simple and to grow when necessary.
Complex Algorithms¶
Dask represents parallel computations with task graphs. These
directed acyclic graphs may have arbitrary structure, which enables both
developers and users the freedom to build sophisticated algorithms and to
handle messy situations not easily managed by the map/filter/groupby
paradigm common in most data engineering frameworks.
We originally needed this complexity to build complex algorithms for ndimensional arrays but have found it to be equally valuable when dealing with messy situations in everyday problems.
Index¶
Getting Started
Install Dask¶
You can install dask with conda
, with pip
, or by installing from source.
Conda¶
To install the latest version of Dask from the condaforge repository using conda:
conda install dask c condaforge
This installs dask and all common dependencies, including Pandas and NumPy.
Pip¶
To install Dask with pip
there are a few options, depending on which
dependencies you would like to keep up to date:
pip install dask[complete]
: Install everythingpip install dask[array]
: Install dask and numpypip install dask[bag]
: Install dask and cloudpicklepip install dask[dataframe]
: Install dask, numpy, and pandaspip install dask
: Install only dask, which depends only on the standard library. This is appropriate if you only want the task schedulers.
We do this so that users of the lightweight core dask scheduler aren’t required to download the more exotic dependencies of the collections (numpy, pandas, etc..)
Install from Source¶
To install dask from source, clone the repository from github:
git clone https://github.com/dask/dask.git
cd dask
python setup.py install
or use pip
locally if you want to install all dependencies as well:
pip install e .[complete]
You can view the list of all dependencies within the extras_require
field
of setup.py
.
Test¶
Test dask with py.test
:
cd dask
py.test dask
Although please aware that installing dask naively may not install all
requirements by default. Please read the pip
section above that discusses
requirements. You may choose to install the dask[complete]
which includes
all dependencies for all collections. Alternatively you may choose to test
only certain submodules depending on the libraries within your environment.
For example to test only dask core and dask array we would run tests as
follows:
py.test dask/tests dask/array/tests
Use Cases¶
Dask is a versatile tool that supports a variety of workloads. This page contains brief and illustrative examples for how people use Dask in practice. This page emphasizes breadth and hopefully inspires readers to find new ways that Dask can serve them beyond their original intent.
Overview¶
Dask use cases can be roughly divided in the following two categories:
 Large NumPy/Pandas/Lists with dask.array, dask.dataframe, dask.bag to analyze large datasets with familiar techniques. This is similar to Databases, Spark, or big array libraries.
 Custom task scheduling. You submit a graph of functions that depend on each other for custom workloads. This is similar to Luigi, Airflow, Celery, or Makefiles.
Most people today approach Dask assuming it is a framework like Spark, designed for the first use case around large collections of uniformly shaped data. However, many of the more productive and novel use cases fall into the second category, using Dask to parallelize custom workflows.
Dask compute environments can be divided into the following two categories:
 Single machine parallelism with threads or processes: The Dask singlemachine scheduler leverages the full CPU power of a laptop or a large workstation and changes the space limitation from “fits in memory” to “fits on disk”. This scheduler is simple to use and doesn’t have the computational or conceptual overhead of most “big data” systems.
 Distributed cluster parallelism on multiple nodes: The Dask distributed scheduler coordinates the actions of multiple machines on a cluster. It scales anywhere from a single machine to a thousand machines, but not significantly beyond.
The single machine scheduler is useful to more individuals (more people have personal laptops than have access to clusters) and probably accounts for 80+% of the use of Dask today. The distributed machine scheduler is useful to larger organizations like universities, research labs, or private companies.
Below we give specific examples of how people use Dask. We start with large NumPy/Pandas/List examples because they’re somewhat more familiar to people looking at “big data” frameworks. We then follow with custom scheduling examples, which tend to be applicable more often, and are arguably a bit more interesting.
Collection Examples¶
Dask contains large parallel collections for ndimensional arrays (similar to NumPy), dataframes (similar to Pandas), and lists (similar to PyToolz or PySpark).
On disk arrays¶
Scientists studying the earth have 10GB to 100GB of regularly gridded weather data on their laptop’s hard drive stored as many individual HDF5 or NetCDF files. They use dask.array to treat this stack of HDF5 or NetCDF files as a single NumPy array (or a collection of NumPy arrays with the XArray project). They slice, perform reductions, perform seasonal averaging etc. all with straight Numpy syntax. These computations take a few minutes (reading 100GB from disk is somewhat slow) but previously infeasible computations become convenient from the comfort of a personal laptop.
It’s not so much parallel computing that is valuable here but rather the ability to comfortably compute on largerthanmemory data without special hardware.
import h5py
dataset = h5py.File('myfile.hdf5')['/x']
import dask.array as da
x = da.from_array(dataset, chunks=dataset.chunks)
y = x[::10]  x.mean(axis=0)
y.compute()
Directory of CSV or tabular HDF files¶
Analysts studying time series data have a large directory of CSV, HDF, or otherwise formatted tabular files. They usually use Pandas for this kind of data but either the volume is too large or dealing with a large number of files is confusing. They use dask.dataframe to logically wrap all of these different files into one logical dataframe that is built on demand to save space. Most of their Pandas workflow is the same (Dask.dataframe is a subset of Pandas) so they switch from Pandas to Dask.dataframe and back easily without significantly changing their code.
import dask.dataframe as dd
df = dd.read_csv('data/2016*.*.csv', parse_dates=['timestamp'])
df.groupby(df.timestamp.dt.hour).value.mean().compute()
Directory of CSV files on HDFS¶
The same analyst as above uses dask.dataframe with the dask.distributed scheduler to analyze terabytes of data on their institution’s Hadoop cluster straight from Python. This uses the HDFS3 Python library for HDFS management
This solution is particularly attractive because it stays within the Python ecosystem, and uses the speed and algorithm set of Pandas, a tool with which the analyst is already very comfortable.
from dask.distributed import Client
client = Client('clusteraddress:8786')
import dask.dataframe as dd
df = dd.read_csv('hdfs://data/2016*.*.csv', parse_dates=['timestamp'])
df.groupby(df.timestamp.dt.hour).value.mean().compute()
Directories of custom format files¶
The same analyst has a bunch of files of a custom format not supported by Dask.dataframe, or perhaps these files are in a directory structure that encodes important information about his data (such as the date or other metadata.) They use dask.delayed to teach Dask.dataframe how to load the data and then pass into dask.dataframe for tabular algorithms.
 Example Notebook: https://gist.github.com/mrocklin/e7b7b3a65f2835cda813096332ec73ca
JSON data¶
Data Engineers with click stream data from a website or mechanical engineers with telemetry data from mechanical instruments have large volumes of data in JSON or some other semistructured format. They use dask.bag to manipulate many Python objects in parallel either on their personal machine, where they stream the data through memory or across a cluster.
import dask.bag as db
import json
records = db.read_text('data/2015**.json').map(json.loads)
records.filter(lambda d: d['name'] == 'Alice').pluck('id').frequencies()
Custom Examples¶
The large collections (array, dataframe, bag) are wonderful when they fit the application, for example if you want to perform a groupby on a directory of CSV data. However several parallel computing applications don’t fit neatly into one of these higher level abstractions. Fortunately, Dask provides a wide variety of ways to parallelize more custom applications. These use the same machinery as the arrays and dataframes, but allow the user to develop custom algorithms specific to their problem.
Embarrassingly parallel computation¶
A programmer has a function that they want to run many times on different inputs. Their function and inputs might use arrays or dataframes internally, but conceptually their problem isn’t a single large array or dataframe.
They want to run these functions in parallel on their laptop while they prototype but they also intend to eventually use an inhouse cluster. They wrap their function in dask.delayed and let the appropriate dask scheduler parallelize and load balance the work.
def process(data):
...
return ...
Normal Sequential Processing:
results = [process(x) for x in inputs]
Build Dask Computation:
from dask import compute, delayed
values = [delayed(process)(x) for x in inputs]
Multiple Threads:
import dask.threaded
results = compute(*values, get=dask.threaded.get)
Multiple Processes:
import dask.multiprocessing
results = compute(*values, get=dask.multiprocessing.get)
Distributed Cluster:
from dask.distributed import Client
client = Client("clusteraddress:8786")
results = compute(*values, get=client.get)
Complex dependencies¶
A financial analyst has many models that depend on each other in a complex web of computations.
data = [load(fn) for fn in filenames]
reference = load_from_database(query)
A = [model_a(x, reference) for x in data]
B = [model_b(x, reference) for x in data]
roll_A = [roll(A[i], A[i + 1]) for i in range(len(A)  1)]
roll_B = [roll(B[i], B[i + 1]) for i in range(len(B)  1)]
compare = [compare_ab(a, b) for a, b in zip(A, B)]
results = summarize(compare, roll_A, roll_B)
These models are time consuming and need to be run on a variety of inputs and situations. The analyst has his code now as a collection of Python functions and is trying to figure out how to parallelize such a codebase. They use dask.delayed to wrap their function calls and capture the implicit parallelism.
from dask import compute, delayed
data = [delayed(load)(fn) for fn in filenames]
reference = delayed(load_from_database)(query)
A = [delayed(model_a)(x, reference) for x in data]
B = [delayed(model_b)(x, reference) for x in data]
roll_A = [delayed(roll)(A[i], A[i + 1]) for i in range(len(A)  1)]
roll_B = [delayed(roll)(B[i], B[i + 1]) for i in range(len(B)  1)]
compare = [delayed(compare_ab)(a, b) for a, b in zip(A, B)]
lazy_results = delayed(summarize)(compare, roll_A, roll_B)
They then depend on the dask schedulers to run this complex web of computations in parallel.
results = compute(lazy_results)
They appreciate how easy it was to transition from the experimental code to a scalable parallel version. This code is also easy enough for their teammates to understand easily and extend in the future.
Algorithm developer¶
A graduate student in machine learning is prototyping novel parallel algorithms. They are in a situation much like the financial analyst above except that they need to benchmark and profile their computation heavily under a variety of situations and scales. The dask profiling tools (single machine diagnostics and distributed diagnostics) provide the feedback they need to understand their parallel performance, including how long each task takes, how intense communication is, and their scheduling overhead. They scale their algorithm between 1 and 50 cores on single workstations and then scale out to a cluster running their computation at thousands of cores. They don’t have access to an institutional cluster, so instead they use daskec2 to easily provision clusters of varying sizes.
Their algorithm is written the same in all cases, drastically reducing the cognitive load, and letting the readers of their work experiment with their system on their own machines, aiding reproducibility.
ScikitLearn or Joblib User¶
A data scientist wants to scale their machine learning pipeline to run on their
cluster to accelerate parameter searches. They already use the sklearn
njobs=
parameter to accelerate their computation on their local computer
with Joblib. Now they wrap their sklearn
code with a context manager to
parallelize the exact same code across a cluster (also available with
IPyParallel)
import distributed.joblib
with joblib.parallel_backend('distributed',
scheduler_host=('192.168.1.100', 8786)):
result = GridSearchCV( ... ) # normal sklearn code
Academic Cluster Administrator¶
A system administrator for a university compute cluster wants to enable many researchers to use the available cluster resources, which are currently lying idle. The research faculty and graduate students lack experience with job schedulers and MPI, but are comfortable interacting with Python code through a Jupyter notebook.
Teaching the faculty and graduate students to parallelize software has proven time consuming. Instead the administrator sets up dask.distributed on a sandbox allocation of the cluster and broadly publishes the address of the scheduler, pointing researchers to the dask.distributed quickstart. Utilization of the cluster climbs steadily over the next week as researchers are more easily able to parallelize their computations without having to learn foreign interfaces. The administrator is happy because resources are being used without significant handholding.
As utilization increases the administrator has a new problem; the shared dask.distributed cluster is being overused. The administrator tracks use through Dask diagnostics to identify which users are taking most of the resources. They contact these users and teach them how to launch their own dask.distributed clusters using the traditional job scheduler on their cluster, making space for more new users in the sandbox allocation.
Financial Modeling Team¶
Similar to the case above, a team of modelers working at a financial institution run a complex network of computational models on top of each other. They started using dask.delayed individually, as suggested above, but realized that they often perform highly overlapping computations, such as always reading the same data.
Now they decide to use the same Dask cluster collaboratively to save on these costs. Because Dask intelligently hashes computations in a way similar to how Git works, they find that when two people submit similar computations the overlapping part of the computation runs only once.
Ever since working collaboratively on the same cluster they find that their frequently running jobs run much faster, because most of the work is already done by previous users. When they share scripts with colleagues they find that those repeated scripts complete immediately rather than taking several hours.
They are now able to iterate and share data as a team more effectively, decreasing their time to result and increasing their competitive edge.
As this becomes more heavily used on the company cluster they decide to set up
an autoscaling system. They use their dynamic job scheduler (perhaps SGE,
LSF, Mesos, or Marathon) to run a single daskscheduler
24/7 and then scale
up and down the number of daskworkers
running on the cluster based on
computational load. This solution ends up being more responsive (and thus more
heavily used) than their previous attempts to provide institutionwide access
to parallel computing but because it responds to load it still acts as a good
citizen in the cluster.
Streaming data engineering¶
A data engineer responsible for watching a data feed needs to scale out a continuous process. They combine dask.distributed with normal Python Queues to produce a rudimentary but effective stream processing system.
Because dask.distributed is elastic, they can scale up or scale down their cluster resources in response to demand.
Examples¶
Array¶
Creating Dask arrays from NumPy arrays¶
We can create Dask arrays from any object that implements NumPy slicing, like a
numpy.ndarray
or ondisk formats like h5py or netCDF Dataset objects. This
is particularly useful with on disk arrays that don’t fit in memory but, for
simplicity’s sake, we show how this works on a NumPy array.
The following example uses da.from_array
to create a Dask array from a NumPy
array, which isn’t particularly valuable (the NumPy array already works in
memory just fine) but is easy to play with.
>>> import numpy as np
>>> import dask.array as da
>>> x = np.arange(1000)
>>> y = da.from_array(x, chunks=(100))
>>> y.mean().compute()
499.5
Creating Dask arrays from HDF5 Datasets¶
We can construct dask array objects from other array objects that support
numpystyle slicing. In this example, we wrap a dask array around an HDF5 dataset,
chunking that dataset into blocks of size (1000, 1000)
:
>>> import h5py
>>> f = h5py.File('myfile.hdf5')
>>> dset = f['/data/path']
>>> import dask.array as da
>>> x = da.from_array(dset, chunks=(1000, 1000))
Often we have many such datasets. We can use the stack
or concatenate
functions to bind many dask arrays into one:
>>> dsets = [h5py.File(fn)['/data'] for fn in sorted(glob('myfiles.*.hdf5')]
>>> arrays = [da.from_array(dset, chunks=(1000, 1000)) for dset in dsets]
>>> x = da.stack(arrays, axis=0) # Stack along a new first axis
Note that none of the data is loaded into memory yet, the dask array just
contains a graph of tasks showing how to load the data. This allows
dask.array
to do work on datasets that don’t fit into RAM.
Creating random arrays¶
In a simple case, we can create arrays with random data using the da.random
module.
>>> import dask.array as da
>>> x = da.random.normal(0, 1, size=(100000,100000), chunks=(1000, 1000))
>>> x.mean().compute()
0.0002280808453825202
Build Custom Dask.Array Function¶
As discussed in the array design document
to create a
dask Array
object we need the following:
 A dask graph
 A name specifying a set of keys within that graph
 A
chunks
tuple giving chunk shape information  A NumPy dtype
Often dask.array
functions take other Array
objects as inputs along
with parameters, add tasks to a new dask dictionary, create a new chunks
tuple, and then construct and return a new Array
object. The hard parts
are invariably creating the right tasks and creating a new chunks
tuple.
Careful review of the array design document
is suggested.
Example eye¶
Consider this simple example with the eye
function.
from dask.base import tokenize
def eye(n, blocksize):
chunks = ((blocksize,) * n // blocksize,
(blocksize,) * n // blocksize)
name = 'eye' + tokenize(n, blocksize) # unique identifier
dsk = {(name, i, j): (np.eye, blocksize)
if i == j else
(np.zeros, (blocksize, blocksize))
for i in range(n // blocksize)
for j in range(n // blocksize)}
dtype = np.eye(0).dtype # take dtype default from numpy
return Array(dsk, name, chunks, dtype)
This example is particularly simple because it doesn’t take any Array
objects as input.
Example diag¶
Consider the function diag
that takes a 1d vector and produces a 2d matrix
with the values of the vector along the diagonal. Consider the case where the
input is a 1d array with chunk sizes (2, 3, 4)
in the first dimension like
this:
[x_0, x_1], [x_2, x_3, x_4], [x_5, x_6, x_7, x_8]
We need to create a 2d matrix with chunks equal to ((2, 3, 4), (2, 3, 4))
where the ith block along the diagonal of the output is the result of calling
np.diag
on the ith
block of the input and all other blocks are zero.
from dask.base import tokenize
def diag(v):
"""Construct a diagonal array, with ``v`` on the diagonal."""
assert v.ndim == 1
chunks = (v.chunks[0], v.chunks[0]) # repeat chunks twice
name = 'diag' + tokenize(v) # unique identifier
dsk = {(name, i, j): (np.diag, (v.name, i))
if i == j else
(np.zeros, (v.chunks[0][i], v.chunks[0][j]))
for i in range(len(v.chunks[0]))
for j in range(len(v.chunks[0]))}
dsk.update(v.dask) # include dask graph of the input
dtype = v.dtype # output has the same dtype as the input
return Array(dsk, name, chunks, dtype)
>>> x = da.arange(9, chunks=((2, 3, 4),))
>>> x
dask.array<arange1, shape=(9,), chunks=((2, 3, 4)), dtype=int64>
>>> M = diag(x)
>>> M
dask.array<diag2, shape=(9, 9), chunks=((2, 3, 4), (2, 3, 4)), dtype=int64>
>>> M.compute()
array([[0, 0, 0, 0, 0, 0, 0, 0, 0],
[0, 1, 0, 0, 0, 0, 0, 0, 0],
[0, 0, 2, 0, 0, 0, 0, 0, 0],
[0, 0, 0, 3, 0, 0, 0, 0, 0],
[0, 0, 0, 0, 4, 0, 0, 0, 0],
[0, 0, 0, 0, 0, 5, 0, 0, 0],
[0, 0, 0, 0, 0, 0, 6, 0, 0],
[0, 0, 0, 0, 0, 0, 0, 7, 0],
[0, 0, 0, 0, 0, 0, 0, 0, 8]])
Bag¶
Read JSON records from disk¶
We commonly use dask.bag
to process unstructured or semistructured data:
>>> import dask.bag as db
>>> import json
>>> js = db.read_text('logs/2015*.json.gz').map(json.loads)
>>> js.take(2)
({'name': 'Alice', 'location': {'city': 'LA', 'state': 'CA'}},
{'name': 'Bob', 'location': {'city': 'NYC', 'state': 'NY'})
>>> result = js.pluck('name').frequencies() # just another Bag
>>> dict(result) # Evaluate Result
{'Alice': 10000, 'Bob': 5555, 'Charlie': ...}
Word count¶
In this example, we’ll use dask
to count the number of words in text files
(Enron email dataset, 6.4 GB) both locally and on a cluster (along with the
distributed and hdfs3 libraries).
Local computation¶
Download the first text file (76 MB) in the dataset to your local machine:
$ wget https://s3.amazonaws.com/blazedata/enronemail/edrmenronv2_allenp_xml.zip/merged.txt
Import dask.bag
and create a bag
from the single text file:
>>> import dask.bag as db
>>> b = db.read_text('merged.txt', blocksize=10000000)
View the first ten lines of the text file with .take()
:
>>> b.take(10)
('Date: Tue, 26 Sep 2000 09:26:00 0700 (PDT)\r\n',
'From: Phillip K Allen\r\n',
'To: pallen70@hotmail.com\r\n',
'Subject: Investment Structure\r\n',
'XSDOC: 948896\r\n',
'XZLID: zledrmenronv2allenp1713.eml\r\n',
'\r\n',
' Forwarded by Phillip K Allen/HOU/ECT on 09/26/2000 \r\n',
'04:26 PM \r\n',
'\r\n')
We can write a word count expression using the bag
methods to split the
lines into words, concatenate the nested lists of words into a single list,
count the frequencies of each word, then list the top 10 words by their count:
>>> wordcount = b.str.split().concat().frequencies().topk(10, lambda x: x[1])
Note that the combined operations in the previous expression are lazy. We can
trigger the word count computation using .compute()
:
>>> wordcount.compute()
[('P', 288093),
('1999', 280917),
('2000', 277093),
('FO', 255844),
('AC', 254962),
('1', 240458),
('0', 233198),
('2', 224739),
('O', 223927),
('3', 221407)]
This computation required about 7 seconds to run on a laptop with 8 cores and 16 GB RAM.
Cluster computation with HDFS¶
Next, we’ll use dask
along with the distributed and hdfs3 libraries
to count the number of words in all of the text files stored in a Hadoop
Distributed File System (HDFS).
Copy the text data from Amazon S3 into HDFS on the cluster:
$ hadoop distcp s3n://AWS_SECRET_ID:AWS_SECRET_KEY@blazedata/enronemail hdfs:///tmp/enron
where AWS_SECRET_ID
and AWS_SECRET_KEY
are valid AWS credentials.
We can now start a distributed
scheduler and workers on the cluster,
replacing SCHEDULER_IP
and SCHEDULER_PORT
with the IP address and port of
the distributed
scheduler:
$ daskscheduler # On the head node
$ daskworker SCHEDULER_IP:SCHEDULER_PORT nprocs 4 nthreads 1 # On the compute nodes
Because our computations use pure Python rather than numeric libraries (e.g., NumPy, pandas), we started the workers with multiple processes rather than with multiple threads. This helps us avoid issues with the Python Global Interpreter Lock (GIL) and increases efficiency.
In Python, import the hdfs3
and the distributed
methods used in this
example:
>>> from dask.distributed import Client, progress
Initialize a connection to the distributed
executor:
>>> client = Client('SCHEDULER_IP:SCHEDULER_PORT')
Create a bag
from the text files stored in HDFS. This expression will not
read data from HDFS until the computation is triggered:
>>> import dask.bag as db
>>> b = db.read_text('hdfs:///tmp/enron/*/*')
We can write a word count expression using the same bag
methods as the
local dask
example:
>>> wordcount = b.str.split().concat().frequencies().topk(10, lambda x: x[1])
We are ready to count the number of words in all of the text files using
distributed
workers. We can map the wordcount
expression to a future
that triggers the computation on the cluster.
>>> future = clinet.compute(wordcount)
Note that the compute
operation is nonblocking, and you can continue to
work in the Python shell/notebook while the computations are running.
We can check the status of the future
while all of the text files are being
processed:
>>> print(future)
<Future: status: pending, key: finalize0f2f51e2350a886223f11e5a1a7bc948>
>>> progress(future)
[########################################]  100% Completed  8min 15.2s
This computation required about 8 minutes to run on a cluster with three worker
machines, each with 4 cores and 16 GB RAM. For comparison, running the same
computation locally with dask
required about 20 minutes on a single machine
with the same specs.
When the future
finishes reading in all of the text files and counting
words, the results will exist on each worker. To sum the word counts for all of
the text files, we need to gather the results from the dask.distributed
workers:
>>> results = client.gather(future)
Finally, we print the top 10 words from all of the text files:
>>> print(results)
[('0', 67218227),
('the', 19588747),
('', 14126955),
('to', 11893912),
('N/A', 11814994),
('of', 11725144),
('and', 10254267),
('in', 6685245),
('a', 5470711),
('or', 5227787)]
The complete Python script for this example is shown below:
# wordcount.py
# Local computation
import dask.bag as db
b = db.read_text('merged.txt')
b.take(10)
wordcount = b.str.split().concat().frequencies().topk(10, lambda x: x[1])
wordcount.compute()
# Cluster computation with HDFS
from dask.distributed import Client, progress
client = Client('SCHEDULER_IP:SCHEDULER_PORT')
b = db.read_text('hdfs:///tmp/enron/*/*')
wordcount = b.str.split().concat().frequencies().topk(10, lambda x: x[1])
future = client.compute(wordcount)
print(future)
progress(future)
results = client.gather(future)
print(results)
DataFrame¶
Dataframes from CSV files¶
Suppose we have a collection of CSV files with data:
data1.csv:
time,temperature,humidity
0,22,58
1,21,57
2,25,57
3,26,55
4,22,53
5,23,59
data2.csv:
time,temperature,humidity
0,24,85
1,26,83
2,27,85
3,25,92
4,25,83
5,23,81
data3.csv:
time,temperature,humidity
0,18,51
1,15,57
2,18,55
3,19,51
4,19,52
5,19,57
and so on.
We can create Dask dataframes from CSV files using dd.read_csv
.
>>> import dask.dataframe as dd
>>> df = dd.read_csv('data*.csv')
We can work with the Dask dataframe as usual, which is composed of Pandas dataframes. We can list the first few rows.
>>> df.head()
time temperature humidity
0 0 22 58
1 1 21 57
2 2 25 57
3 3 26 55
4 4 22 53
Or we can compute values over the entire dataframe.
>>> df.temperature.mean().compute()
22.055555555555557
>>> df.humidity.std().compute()
14.710829233324224
Dataframes from HDF5 files¶
This section provides working examples of dask.dataframe
methods to read HDF5 files. HDF5 is a unique technology suite that makes possible the management of large and complex data collections. To learn more about HDF5, visit the HDF Group Tutorial page. For an overview of dask.dataframe
, its limitations, scope, and use, see the DataFrame overview section.
Important Note – dask.dataframe.read_hdf
uses pandas.read_hdf
, thereby inheriting its abilities and limitations. See pandas HDF5 documentation for more information.
Examples Covered¶
 Use
dask.dataframe
to:
 Create dask DataFrame by loading a specific dataset (key) from a single HDF5 file
 Create dask DataFrame from a single HDF5 file with multiple datasets (keys)
 Create dask DataFrame by loading multiple HDF5 files with different datasets (keys)
Generate Example Data¶
Here is some code to generate sample HDF5 files.
import string, json, random
import pandas as pd
import numpy as np
# dict to keep track of hdf5 filename and each key
fileKeys = {}
for i in range(10):
# randomly pick letter as dataset key
groupkey = random.choice(list(string.ascii_lowercase))
# randomly pick a number as hdf5 filename
filename = 'my' + str(np.random.randint(100)) + '.h5'
# Make a dataframe; 26 rows, 2 columns
df = pd.DataFrame({'x': np.random.randint(1, 1000, 26),
'y': np.random.randint(1, 1000, 26)},
index=list(string.ascii_lowercase))
# Write hdf5 to current directory
df.to_hdf(filename, key='/' + groupkey, format='table')
fileKeys[filename] = groupkey
print(fileKeys) # prints hdf5 filenames and keys for each
Read single dataset from HDF5¶
The first order of dask.dataframe
business is creating a dask DataFrame using a single HDF5 file’s dataset. The code to accomplish this task is:
import dask.dataframe as dd
df = dd.read_hdf('my86.h5', key='/c')
Load multiple datasets from single HDF5 file¶
Loading multiple datasets from a single file requires a small tweak and use of the wildcard character:
import dask.dataframe as dd
df = dd.read_hdf('my86.h5', key='/*')
Learn more about dask.dataframe
methods by visiting the API documentation.
Create dask DataFrame from multiple HDF5 files¶
The next example is a natural progression from the previous example (e.g. using a wildcard). Add a wildcard for the key and path parameters to read multiple files and multiple keys:
import dask.dataframe as dd
df = dd.read_hdf('./*.h5', key='/*')
These exercises cover the basics of using dask.dataframe
to work with HDF5 data. For more information on the user functions to manipulate and explore dataframes (visualize, describe, compute, etc.) see API documentation. To explore the other data formats supported by dask.dataframe
, visit the section on creating dataframes .
Delayed¶
Build Custom Arrays¶
Here we have a serial blocked computation for computing the mean of all positive elements in a large, on disk array:
x = h5py.File('myfile.hdf5')['/x'] # Trillion element array on disk
sums = []
counts = []
for i in range(1000000): # One million times
chunk = x[1000000*i:1000000*(i + 1)] # Pull out chunk
positive = chunk[chunk > 0] # Filter out negative elements
sums.append(positive.sum()) # Sum chunk
counts.append(positive.size) # Count chunk
result = sum(sums) / sum(counts) # Aggregate results
Below is the same code, parallelized using dask.delayed
:
x = delayed(h5py.File('myfile.hdf5')['/x']) # Trillion element array on disk
sums = []
counts = []
for i in range(1000000): # One million times
chunk = x[1000000*i:1000000*(i + 1)] # Pull out chunk
positive = chunk[chunk > 0] # Filter out negative elements
sums.append(positive.sum()) # Sum chunk
counts.append(positive.size) # Count chunk
result = delayed(sum)(sums) / delayed(sum)(counts) # Aggregate results
result.compute() # Perform the computation
Only 3 lines had to change to make this computation parallel instead of serial.
 Wrap the original array in
delayed
. This makes all the slices on it returnDelayed
objects.  Wrap both calls to
sum
withdelayed
.  Call the
compute
method on the result.
While the for loop above still iterates fully, it’s just building up a graph of the computation that needs to happen, without actually doing any computing.
Data Processing Pipelines¶
Now, rebuilding the example from custom graphs:
from dask import delayed, value
@delayed
def load(filename):
...
@delayed
def clean(data):
...
@delayed
def analyze(sequence_of_data):
...
@delayed
def store(result):
with open(..., 'w') as f:
f.write(result)
files = ['myfile.a.data', 'myfile.b.data', 'myfile.c.data']
loaded = [load(i) for i in files]
cleaned = [clean(i) for i in loaded]
analyzed = analyze(cleaned)
stored = store(analyzed)
stored.compute()
This builds the same graph as seen before, but using normal Python syntax. In
fact, the only difference between Python code that would do this in serial, and
the parallel version with dask is the delayed
decorators on the functions, and
the call to compute
at the end.
Distributed Concurrent.futures¶
Tutorial¶
A Dask tutorial from July 2015 (fairly old) is available here: https://github.com/dask/dasktutorial
Dask Cheat Sheet¶
The 300KB pdf dask cheat sheet
is a single page summary
of all the most important information about using dask.
Collections
Dask collections are the main interaction point for users. They look like NumPy and pandas but generate dask graphs internally. If you are a dask user then you should start here.
Array¶
Dask arrays implement a subset of the NumPy interface on large arrays using blocked algorithms and task scheduling.
Overview¶
Dask Array implements a subset of the NumPy ndarray interface using blocked algorithms, cutting up the large array into many small arrays. This lets us compute on arrays larger than memory using all of our cores. We coordinate these blocked algorithms using dask graphs.
Design¶
Dask arrays coordinate many NumPy arrays arranged into a grid. These NumPy arrays may live on disk or on other machines.
Common Uses¶
Today Dask array is commonly used in the sort of gridded data analysis that arises in weather, climate modeling, or oceanography, especially when data sizes become inconveniently large. Dask array complements large ondisk array stores like HDF5, NetCDF, and BColz. Additionally Dask array is commonly used to speed up expensive inmemory computations using multiple cores, such as you might find in image analysis or statistical and machine learning applications.
Scope¶
The dask.array
library supports the following interface from numpy
:
 Arithmetic and scalar mathematics,
+, *, exp, log, ...
 Reductions along axes,
sum(), mean(), std(), sum(axis=0), ...
 Tensor contractions / dot products / matrix multiply,
tensordot
 Axis reordering / transpose,
transpose
 Slicing,
x[:100, 500:100:2]
 Fancy indexing along single axes with lists or numpy arrays,
x[:, [10, 1, 5]]
 The array protocol
__array__
 Some linear algebra
svd, qr, solve, solve_triangular, lstsq
See the dask.array API for a more extensive list of functionality.
Execution¶
By default Dask array uses the threaded scheduler in order to avoid data transfer costs and because NumPy releases the GIL well. It is also quite effective on a cluster using the dask.distributed scheduler.
Limitations¶
Dask array does not implement the entire numpy interface. Users expecting this will be disappointed. Notably, Dask array has the following limitations:
 Dask array does not implement all of
np.linalg
. This has been done by a number of excellent BLAS/LAPACK implementations, and is the focus of numerous ongoing academic research projects.  Dask array with unknown shapes do not support all operations
 Dask array does not attempt operations like
sort
which are notoriously difficult to do in parallel, and are of somewhat diminished value on very large data (you rarely actually need a full sort). Often we include parallelfriendly alternatives liketopk
.  It is very inefficient to iterate over a Dask array with for loops.
 Dask development is driven by immediate need, and so many lesser used functions have not been implemented. Community contributions are encouraged.
Create Dask Arrays¶
We store and manipulate large arrays in a wide variety of ways. There are some standards like HDF5 and NetCDF but just as often people use custom storage solutions. This page talks about how to build dask graphs to interact with your array.
In principle we need functions that return NumPy arrays. These functions and their arrangement can be as simple or as complex as the situation dictates.
Simple case  Format Supports NumPy Slicing¶
Many storage formats have Python projects that expose storage using NumPy
slicing syntax. These include HDF5, NetCDF, BColz, Zarr, GRIB, etc.. For
example the HDF5
file format has the h5py
Python project, which
provides a Dataset
object into which we can slice in NumPy fashion.
>>> import h5py
>>> f = h5py.File('myfile.hdf5') # HDF5 file
>>> d = f['/data/path'] # Pointer on ondisk array
>>> d.shape # d can be very large
(1000000, 1000000)
>>> x = d[:5, :5] # We slice to get numpy arrays
It is common for Python wrappers of ondisk array formats to present a NumPy
slicing syntax. The full dataset looks like a NumPy array with .shape
and
.dtype
attributes even though the data hasn’t yet been loaded in and still
lives on disk. Slicing in to this arraylike object fetches the appropriate
data from disk and returns that region as an inmemory NumPy array.
For this common case dask.array
presents the convenience function
da.from_array
>>> import dask.array as da
>>> x = da.from_array(d, chunks=(1000, 1000))
Concatenation and Stacking¶
Often we store data in several different locations and want to stitch them together.
>>> filenames = sorted(glob('2015**.hdf5')
>>> dsets = [h5py.File(fn)['/data'] for fn in filenames]
>>> arrays = [da.from_array(dset, chunks=(1000, 1000)) for dset in dsets]
>>> x = da.concatenate(arrays, axis=0) # Concatenate arrays along first axis
For more information see concatenation and stacking docs.
Using dask.delayed
¶
You can create a plan to arrange many numpy arrays into a grid with normal for
loops using dask.delayed and then convert each of these
Dask.delayed objects into a singlechunk Dask array with da.from_delayed
.
You can then arrange these singlechunk Dask arrays into a larger
multiplechunk Dask array using concatenation and stacking,
as described above.
From Dask.dataframe¶
You can create dask arrays from dask dataframes using the .values
attribute
or the .to_records()
method.
>>> x = df.values
>>> x = df.to_records()
However these arrays do not have known chunk sizes (dask.dataframe does not track the number of rows in each partition) and so some operations like slicing will not operate correctly.
Chunks¶
We always specify a chunks
argument to tell dask.array how to break up the
underlying array into chunks. This strongly impacts performance. We can
specify chunks
in one of three ways
 a blocksize like
1000
 a blockshape like
(1000, 1000)
 explicit sizes of all blocks along all dimensions,
like
((1000, 1000, 500), (400, 400))
Your chunks input will be normalized and stored in the third and most explicit form.
For performance, a good choice of chunks
follows the following rules:
 A chunk should be small enough to fit comfortably in memory. We’ll have many chunks in memory at once.
 A chunk must be large enough so that computations on that chunk take significantly longer than the 1ms overhead per task that dask scheduling incurs. A task should take longer than 100ms.
 Chunks should align with the computation that you want to do. For example if you plan to frequently slice along a particular dimension then it’s more efficient if your chunks are aligned so that you have to touch fewer chunks. If you want to add two arrays then its convenient if those arrays have matching chunks patterns.
Unknown Chunks¶
Some arrays have unknown chunk sizes. These are designated using np.nan
rather than an integer. These arrays support many but not all operations. In
particular, opeations like slicing are not possible and will result in an
error.
>>> x.shape
(np.nan, np.nan)
>>> x[0]
ValueError: Array chunk sizes unknown
Chunks Examples¶
We show of how different inputs for chunks=
cut up the following array:
1 2 3 4 5 6
7 8 9 0 1 2
3 4 5 6 7 8
9 0 1 2 3 4
5 6 7 8 9 0
1 2 3 4 5 6
We show how different chunks=
arguments split the array into different blocks
chunks=3: Symmetric blocks of size 3:
1 2 3 4 5 6
7 8 9 0 1 2
3 4 5 6 7 8
9 0 1 2 3 4
5 6 7 8 9 0
1 2 3 4 5 6
chunks=2: Symmetric blocks of size 2:
1 2 3 4 5 6
7 8 9 0 1 2
3 4 5 6 7 8
9 0 1 2 3 4
5 6 7 8 9 0
1 2 3 4 5 6
chunks=(3, 2): Asymmetric but repeated blocks of size (3, 2)
:
1 2 3 4 5 6
7 8 9 0 1 2
3 4 5 6 7 8
9 0 1 2 3 4
5 6 7 8 9 0
1 2 3 4 5 6
chunks=(1, 6): Asymmetric but repeated blocks of size (1, 6)
:
1 2 3 4 5 6
7 8 9 0 1 2
3 4 5 6 7 8
9 0 1 2 3 4
5 6 7 8 9 0
1 2 3 4 5 6
chunks=((2, 4), (3, 3)): Asymmetric and nonrepeated blocks:
1 2 3 4 5 6
7 8 9 0 1 2
3 4 5 6 7 8
9 0 1 2 3 4
5 6 7 8 9 0
1 2 3 4 5 6
chunks=((2, 2, 1, 1), (3, 2, 1)): Asymmetric and nonrepeated blocks:
1 2 3 4 5 6
7 8 9 0 1 2
3 4 5 6 7 8
9 0 1 2 3 4
5 6 7 8 9 0
1 2 3 4 5 6
Discussion
The latter examples are rarely provided by users on original data but arise from complex slicing and broadcasting operations. Generally people use the simplest form until they need more complex forms. The choice of chunks should align with the computations you want to do.
For example, if you plan to take out thin slices along the first dimension then you might want to make that dimension skinnier than the others. If you plan to do linear algebra then you might want more symmetric blocks.
Store Dask Arrays¶
In Memory¶
If you have a small amount of data, you can call np.array
or .compute()
on your Dask array to turn in to a normal NumPy array:
>>> x = da.arange(6, chunks=3)
>>> y = x**2
>>> np.array(y)
array([0, 1, 4, 9, 16, 25])
>>> y.compute()
array([0, 1, 4, 9, 16, 25])
HDF5¶
Use the to_hdf5
function to store data into HDF5 using h5py
:
>>> da.to_hdf5('myfile.hdf5', '/y', y) # doctest: +SKIP
Store several arrays in one computation with the function
da.to_hdf5
by passing in a dict:
>>> da.to_hdf5('myfile.hdf5', {'/x': x, '/y': y}) # doctest: +SKIP
Other OnDisk Storage¶
Alternatively, you can store dask arrays in any object that supports numpystyle
slice assignment like h5py.Dataset
, or bcolz.carray
:
>>> import bcolz # doctest: +SKIP
>>> out = bcolz.zeros(shape=y.shape, rootdir='myfile.bcolz') # doctest: +SKIP
>>> da.store(y, out) # doctest: +SKIP
You can store several arrays in one computation by passing lists of sources and destinations:
>>> da.store([array1, array2], [output1, output2]) # doctest: +SKIP
API¶
Top level user functions:
all (a[, axis, keepdims, split_every]) 
Test whether all array elements along a given axis evaluate to True. 
angle (*args, **kwargs) 
Return the angle of the complex argument. 
any (a[, axis, keepdims, split_every]) 
Test whether any array element along a given axis evaluates to True. 
arange (*args, **kwargs) 
Return evenly spaced values from start to stop with step size step. 
arccos (x[, out]) 
Trigonometric inverse cosine, elementwise. 
arccosh (x[, out]) 
Inverse hyperbolic cosine, elementwise. 
arcsin (x[, out]) 
Inverse sine, elementwise. 
arcsinh (x[, out]) 
Inverse hyperbolic sine elementwise. 
arctan (x[, out]) 
Trigonometric inverse tangent, elementwise. 
arctan2 (x1, x2[, out]) 
Elementwise arc tangent of x1/x2 choosing the quadrant correctly. 
arctanh (x[, out]) 
Inverse hyperbolic tangent elementwise. 
argmax (x[, axis, split_every]) 
Indices of the maximum values along an axis. 
argmin (x[, axis, split_every]) 
Return the indices of the minimum values along an axis. 
around (x[, decimals]) 
Evenly round to the given number of decimals. 
array (object[, dtype, copy, order, subok, ndmin]) 
Create an array. 
bincount (x[, weights, minlength]) 
Count number of occurrences of each value in array of nonnegative ints. 
broadcast_to (x, shape) 
Broadcast an array to a new shape. 
coarsen (reduction, x, axes[, trim_excess]) 
Coarsen array by applying reduction to fixed size neighborhoods 
ceil (x[, out]) 
Return the ceiling of the input, elementwise. 
choose (a, choices) 
Construct an array from an index array and a set of arrays to choose from. 
clip (*args, **kwargs) 
Clip (limit) the values in an array. 
compress (condition, a[, axis]) 
Return selected slices of an array along given axis. 
concatenate (seq[, axis]) 
Concatenate arrays along an existing axis 
conj (x[, out]) 
Return the complex conjugate, elementwise. 
copysign (x1, x2[, out]) 
Change the sign of x1 to that of x2, elementwise. 
corrcoef (x[, y, rowvar]) 
Return correlation coefficients. 
cos (x[, out]) 
Cosine elementwise. 
cosh (x[, out]) 
Hyperbolic cosine, elementwise. 
cov (m[, y, rowvar, bias, ddof]) 
Estimate a covariance matrix, given data. 
cumprod (x, axis[, dtype]) 
Return the cumulative product of elements along a given axis. 
cumsum (x, axis[, dtype]) 
Return the cumulative sum of the elements along a given axis. 
deg2rad (x[, out]) 
Convert angles from degrees to radians. 
degrees (x[, out]) 
Convert angles from radians to degrees. 
diag (v) 
Extract a diagonal or construct a diagonal array. 
digitize (x, bins[, right]) 
Return the indices of the bins to which each value in input array belongs. 
dot (a, b[, out]) 
Dot product of two arrays. 
dstack (tup) 
Stack arrays in sequence depth wise (along third axis). 
empty 
Blocked variant of empty 
exp (x[, out]) 
Calculate the exponential of all elements in the input array. 
expm1 (x[, out]) 
Calculate exp(x)  1 for all elements in the array. 
eye (N, chunks[, M, k, dtype]) 
Return a 2D Array with ones on the diagonal and zeros elsewhere. 
fabs (x[, out]) 
Compute the absolute values elementwise. 
fft.fft (a[, n, axis]) 
Wrapping of numpy.fft.fft 
fft.ifft (a[, n, axis]) 
Wrapping of numpy.fft.ifft 
fft.hfft (a[, n, axis]) 
Wrapping of numpy.fft.hfft 
fft.ihfft (a[, n, axis]) 
Wrapping of numpy.fft.ihfft 
fft.rfft (a[, n, axis]) 
Wrapping of numpy.fft.rfft 
fft.irfft (a[, n, axis]) 
Wrapping of numpy.fft.irfft 
fix (*args, **kwargs) 
Round to nearest integer towards zero. 
floor (x[, out]) 
Return the floor of the input, elementwise. 
fmax (x1, x2[, out]) 
Elementwise maximum of array elements. 
fmin (x1, x2[, out]) 
Elementwise minimum of array elements. 
fmod (x1, x2[, out]) 
Return the elementwise remainder of division. 
frexp (x[, out1, out2]) 
Decompose the elements of x into mantissa and twos exponent. 
fromfunction (func[, chunks, shape, dtype]) 
Construct an array by executing a function over each coordinate. 
full 
Blocked variant of full 
histogram (a[, bins, range, normed, weights, ...]) 
Blocked variant of numpy.histogram. 
hstack (tup) 
Stack arrays in sequence horizontally (column wise). 
hypot (x1, x2[, out]) 
Given the “legs” of a right triangle, return its hypotenuse. 
imag (*args, **kwargs) 
Return the imaginary part of the elements of the array. 
insert (arr, obj, values, axis) 
Insert values along the given axis before the given indices. 
isclose (arr1, arr2[, rtol, atol, equal_nan]) 
Returns a boolean array where two arrays are elementwise equal within a tolerance. 
iscomplex (*args, **kwargs) 
Returns a bool array, where True if input element is complex. 
isfinite (x[, out]) 
Test elementwise for finiteness (not infinity or not Not a Number). 
isinf (x[, out]) 
Test elementwise for positive or negative infinity. 
isnan (x[, out]) 
Test elementwise for NaN and return result as a boolean array. 
isnull (values) 
pandas.isnull for dask arrays 
isreal (*args, **kwargs) 
Returns a bool array, where True if input element is real. 
ldexp (x1, x2[, out]) 
Returns x1 * 2**x2, elementwise. 
linspace (start, stop[, num, chunks, dtype]) 
Return num evenly spaced values over the closed interval [start, stop]. 
log (x[, out]) 
Natural logarithm, elementwise. 
log10 (x[, out]) 
Return the base 10 logarithm of the input array, elementwise. 
log1p (x[, out]) 
Return the natural logarithm of one plus the input array, elementwise. 
log2 (x[, out]) 
Base2 logarithm of x. 
logaddexp (x1, x2[, out]) 
Logarithm of the sum of exponentiations of the inputs. 
logaddexp2 (x1, x2[, out]) 
Logarithm of the sum of exponentiations of the inputs in base2. 
logical_and (x1, x2[, out]) 
Compute the truth value of x1 AND x2 elementwise. 
logical_not (x[, out]) 
Compute the truth value of NOT x elementwise. 
logical_or (x1, x2[, out]) 
Compute the truth value of x1 OR x2 elementwise. 
logical_xor (x1, x2[, out]) 
Compute the truth value of x1 XOR x2, elementwise. 
max (a[, axis, keepdims, split_every]) 
Return the maximum of an array or maximum along an axis. 
maximum (x1, x2[, out]) 
Elementwise maximum of array elements. 
mean (a[, axis, dtype, keepdims, split_every]) 
Compute the arithmetic mean along the specified axis. 
min (a[, axis, keepdims, split_every]) 
Return the minimum of an array or minimum along an axis. 
minimum (x1, x2[, out]) 
Elementwise minimum of array elements. 
modf (x[, out1, out2]) 
Return the fractional and integral parts of an array, elementwise. 
moment (a, order[, axis, dtype, keepdims, ...]) 

nanargmax (x[, axis, split_every]) 

nanargmin (x[, axis, split_every]) 

nancumprod (x, axis[, dtype]) 
Return the cumulative product of array elements over a given axis treating Not a Numbers (NaNs) as one. 
nancumsum (x, axis[, dtype]) 
Return the cumulative sum of array elements over a given axis treating Not a Numbers (NaNs) as zero. 
nanmax (a[, axis, keepdims, split_every]) 
Return the maximum of an array or maximum along an axis, ignoring any NaNs. 
nanmean (a[, axis, dtype, keepdims, split_every]) 
Compute the arithmetic mean along the specified axis, ignoring NaNs. 
nanmin (a[, axis, keepdims, split_every]) 
Return minimum of an array or minimum along an axis, ignoring any NaNs. 
nanprod (a[, axis, dtype, keepdims, split_every]) 
Return the product of array elements over a given axis treating Not a Numbers (NaNs) as zero. 
nanstd (a[, axis, dtype, keepdims, ddof, ...]) 
Compute the standard deviation along the specified axis, while ignoring NaNs. 
nansum (a[, axis, dtype, keepdims, split_every]) 
Return the sum of array elements over a given axis treating Not a Numbers (NaNs) as zero. 
nanvar (a[, axis, dtype, keepdims, ddof, ...]) 
Compute the variance along the specified axis, while ignoring NaNs. 
nextafter (x1, x2[, out]) 
Return the next floatingpoint value after x1 towards x2, elementwise. 
notnull (values) 
pandas.notnull for dask arrays 
ones 
Blocked variant of ones 
percentile (a, q[, interpolation]) 
Approximate percentile of 1D array 
prod (a[, axis, dtype, keepdims, split_every]) 
Return the product of array elements over a given axis. 
rad2deg (x[, out]) 
Convert angles from radians to degrees. 
radians (x[, out]) 
Convert angles from degrees to radians. 
random 

ravel (array) 
Return a flattened array. 
real (*args, **kwargs) 
Return the real part of the elements of the array. 
rechunk (x, chunks[, threshold, block_size_limit]) 
Convert blocks in dask array x for new chunks. 
repeat (a, b[, out]) 
Dot product of two arrays. 
reshape (array, shape) 
Gives a new shape to an array without changing its data. 
rint (x[, out]) 
Round elements of the array to the nearest integer. 
round (a[, decimals]) 
Round an array to the given number of decimals. 
sign (x[, out]) 
Returns an elementwise indication of the sign of a number. 
signbit (x[, out]) 
Returns elementwise True where signbit is set (less than zero). 
sin (x[, out]) 
Trigonometric sine, elementwise. 
sinh (x[, out]) 
Hyperbolic sine, elementwise. 
sqrt (x[, out]) 
Return the positive squareroot of an array, elementwise. 
square (x[, out]) 
Return the elementwise square of the input. 
squeeze (a[, axis]) 
Remove singledimensional entries from the shape of an array. 
stack (seq[, axis]) 
Stack arrays along a new axis 
std (a[, axis, dtype, keepdims, ddof, ...]) 
Compute the standard deviation along the specified axis. 
sum (a[, axis, dtype, keepdims, split_every]) 
Sum of array elements over a given axis. 
take (a, indices[, axis]) 
Take elements from an array along an axis. 
tan (x[, out]) 
Compute tangent elementwise. 
tanh (x[, out]) 
Compute hyperbolic tangent elementwise. 
tensordot (lhs, rhs[, axes]) 
Compute tensor dot product along specified axes for arrays >= 1D. 
topk (k, x) 
The top k elements of an array 
transpose (a[, axes]) 
Permute the dimensions of an array. 
tril (m[, k]) 
Lower triangle of an array with elements above the kth diagonal zeroed. 
triu (m[, k]) 
Upper triangle of an array with elements above the kth diagonal zeroed. 
trunc (x[, out]) 
Return the truncated value of the input, elementwise. 
unique (x) 
Find the unique elements of an array. 
var (a[, axis, dtype, keepdims, ddof, ...]) 
Compute the variance along the specified axis. 
vnorm (a[, ord, axis, dtype, keepdims, ...]) 
Vector norm 
vstack (tup) 
Stack arrays in sequence vertically (row wise). 
where (condition, [x, y]) 
Return elements, either from x or y, depending on condition. 
zeros 
Blocked variant of zeros 
Linear Algebra¶
linalg.cholesky (a[, lower]) 
Returns the Cholesky decomposition, \(A = L L^*\) or \(A = U^* U\) of a Hermitian positivedefinite matrix A. 
linalg.inv (a) 
Compute the inverse of a matrix with LU decomposition and forward / backward substitutions. 
linalg.lstsq (a, b) 
Return the leastsquares solution to a linear matrix equation using QR decomposition. 
linalg.lu (a) 
Compute the lu decomposition of a matrix. 
linalg.qr (a[, name]) 
Compute the qr factorization of a matrix. 
linalg.solve (a, b[, sym_pos]) 
Solve the equation a x = b for x . 
linalg.solve_triangular (a, b[, lower]) 
Solve the equation a x = b for x, assuming a is a triangular matrix. 
linalg.svd (a[, name]) 
Compute the singular value decomposition of a matrix. 
linalg.svd_compressed (a, k[, n_power_iter, ...]) 
Randomly compressed rankk thin Singular Value Decomposition. 
linalg.tsqr (data[, name, compute_svd]) 
Direct TallandSkinny QR algorithm 
Random¶
random.beta (a, b[, size]) 
The Beta distribution over [0, 1] . 
random.binomial (n, p[, size]) 
Draw samples from a binomial distribution. 
random.chisquare (df[, size]) 
Draw samples from a chisquare distribution. 
random.different_seeds 

random.exponential ([scale, size]) 
Exponential distribution. 
random.f (dfnum, dfden[, size]) 
Draw samples from a F distribution. 
random.gamma (shape[, scale, size]) 
Draw samples from a Gamma distribution. 
random.geometric (p[, size]) 
Draw samples from the geometric distribution. 
random.gumbel ([loc, scale, size]) 
Gumbel distribution. 
random.hypergeometric (ngood, nbad, nsample) 
Draw samples from a Hypergeometric distribution. 
random.laplace ([loc, scale, size]) 
Draw samples from the Laplace or double exponential distribution with specified location (or mean) and scale (decay). 
random.logistic ([loc, scale, size]) 
Draw samples from a Logistic distribution. 
random.lognormal ([mean, sigma, size]) 
Return samples drawn from a lognormal distribution. 
random.logseries (p[, size]) 
Draw samples from a Logarithmic Series distribution. 
random.negative_binomial (n, p[, size]) 
Draw samples from a negative_binomial distribution. 
random.noncentral_chisquare (df, nonc[, size]) 
Draw samples from a noncentral chisquare distribution. 
random.noncentral_f (dfnum, dfden, nonc[, size]) 
Draw samples from the noncentral F distribution. 
random.normal ([loc, scale, size]) 
Draw random samples from a normal (Gaussian) distribution. 
random.pareto (a[, size]) 
Draw samples from a Pareto II or Lomax distribution with specified shape. 
random.poisson ([lam, size]) 
Draw samples from a Poisson distribution. 
random.power (a[, size]) 
Draws samples in [0, 1] from a power distribution with positive exponent a  1. 
random.random ([size]) 
Return random floats in the halfopen interval [0.0, 1.0). 
random.random_sample ([size]) 
Return random floats in the halfopen interval [0.0, 1.0). 
random.rayleigh ([scale, size]) 
Draw samples from a Rayleigh distribution. 
random.standard_cauchy ([size]) 
Standard Cauchy distribution with mode = 0. 
random.standard_exponential ([size]) 
Draw samples from the standard exponential distribution. 
random.standard_gamma (shape[, size]) 
Draw samples from a Standard Gamma distribution. 
random.standard_normal ([size]) 
Returns samples from a Standard Normal distribution (mean=0, stdev=1). 
random.standard_t (df[, size]) 
Standard Student’s t distribution with df degrees of freedom. 
random.triangular (left, mode, right[, size]) 
Draw samples from the triangular distribution. 
random.uniform ([low, high, size]) 
Draw samples from a uniform distribution. 
random.vonmises (mu, kappa[, size]) 
Draw samples from a von Mises distribution. 
random.wald (mean, scale[, size]) 
Draw samples from a Wald, or Inverse Gaussian, distribution. 
random.weibull (a[, size]) 
Weibull distribution. 
random.zipf (a[, size]) 
Standard distributions 
Slightly Overlapping Ghost Computations¶
ghost.ghost (x, depth, boundary) 
Share boundaries between neighboring blocks 
ghost.map_overlap (x, func, depth[, ...]) 
Create and Store Arrays¶
from_array (x, chunks[, name, lock, fancy]) 
Create dask array from something that looks like an array 
from_delayed (value, shape, dtype[, name]) 
Create a dask array from a dask delayed value 
from_npy_stack (dirname[, mmap_mode]) 
Load dask array from stack of npy files 
store (sources, targets[, lock, regions, compute]) 
Store dask arrays in arraylike objects, overwrite data in target 
to_hdf5 (filename, *args, **kwargs) 
Store arrays in HDF5 file 
to_npy_stack (dirname, x[, axis]) 
Write dask array to a stack of .npy files 
Internal functions¶
map_blocks (func, *args, **kwargs) 
Map a function across all blocks of a dask array. 
atop (func, out_ind, *args, **kwargs) 
Tensor operation: Generalized inner and outer products 
top (func, output, out_indices, ...) 
Tensor operation 
Other functions¶

dask.array.
from_array
(x, chunks, name=None, lock=False, fancy=True)¶ Create dask array from something that looks like an array
Input must have a
.shape
and support numpystyle slicing.Parameters: x : array_like
chunks : int, tuple
How to chunk the array. Must be one of the following forms:  A blocksize like 1000.  A blockshape like (1000, 1000).  Explicit sizes of all blocks along all dimensions
like ((1000, 1000, 500), (400, 400)).
name : str, optional
The key name to use for the array. Defaults to a hash of
x
.lock : bool or Lock, optional
If
x
doesn’t support concurrent reads then provide a lock here, or pass in True to have dask.array create one for you.fancy : bool, optional
If
x
doesn’t support fancy indexing (e.g. indexing with lists or arrays) then set to False. Default is True.Examples
>>> x = h5py.File('...')['/data/path'] >>> a = da.from_array(x, chunks=(1000, 1000))
If your underlying datastore does not support concurrent reads then include the
lock=True
keyword argument orlock=mylock
if you want multiple arrays to coordinate around the same lock.>>> a = da.from_array(x, chunks=(1000, 1000), lock=True)

dask.array.
from_delayed
(value, shape, dtype, name=None)¶ Create a dask array from a dask delayed value
This routine is useful for constructing dask arrays in an adhoc fashion using dask delayed, particularly when combined with stack and concatenate.
The dask array will consist of a single chunk.
Examples
>>> from dask import delayed >>> value = delayed(np.ones)(5) >>> array = from_delayed(value, (5,), float) >>> array dask.array<fromva..., shape=(5,), dtype=float64, chunksize=(5,)> >>> array.compute() array([ 1., 1., 1., 1., 1.])

dask.array.
store
(sources, targets, lock=True, regions=None, compute=True, **kwargs)¶ Store dask arrays in arraylike objects, overwrite data in target
This stores dask arrays into object that supports numpystyle setitem indexing. It stores values chunk by chunk so that it does not have to fill up memory. For best performance you can align the block size of the storage target with the block size of your array.
If your data fits in memory then you may prefer calling
np.array(myarray)
instead.Parameters: sources: Array or iterable of Arrays
targets: arraylike or iterable of arraylikes
These should support setitem syntax
target[10:20] = ...
lock: boolean or threading.Lock, optional
Whether or not to lock the data stores while storing. Pass True (lock each file individually), False (don’t lock) or a particular
threading.Lock
object to be shared among all writes.regions: tuple of slices or iterable of tuple of slices
Each
region
tuple inregions
should be such thattarget[region].shape = source.shape
for the corresponding source and target in sources and targets, respectively.compute: boolean, optional
If true compute immediately, return
dask.delayed.Delayed
otherwiseExamples
>>> x = ...
>>> import h5py >>> f = h5py.File('myfile.hdf5') >>> dset = f.create_dataset('/data', shape=x.shape, ... chunks=x.chunks, ... dtype='f8')
>>> store(x, dset)
Alternatively store many arrays at the same time
>>> store([x, y, z], [dset1, dset2, dset3])

dask.array.
topk
(k, x)¶ The top k elements of an array
Returns the k greatest elements of the array in sorted order. Only works on arrays of a single dimension.
This assumes that
k
is small. All results will be returned in a single chunk.Examples
>>> x = np.array([5, 1, 3, 6]) >>> d = from_array(x, chunks=2) >>> d.topk(2).compute() array([6, 5])

dask.array.
coarsen
(reduction, x, axes, trim_excess=False)¶ Coarsen array by applying reduction to fixed size neighborhoods
Parameters: reduction: function
Function like np.sum, np.mean, etc...
x: np.ndarray
Array to be coarsened
axes: dict
Mapping of axis to coarsening factor
Examples
>>> x = np.array([1, 2, 3, 4, 5, 6]) >>> coarsen(np.sum, x, {0: 2}) array([ 3, 7, 11]) >>> coarsen(np.max, x, {0: 3}) array([3, 6])
Provide dictionary of scale per dimension
>>> x = np.arange(24).reshape((4, 6)) >>> x array([[ 0, 1, 2, 3, 4, 5], [ 6, 7, 8, 9, 10, 11], [12, 13, 14, 15, 16, 17], [18, 19, 20, 21, 22, 23]])
>>> coarsen(np.min, x, {0: 2, 1: 3}) array([[ 0, 3], [12, 15]])
You must avoid excess elements explicitly
>>> x = np.array([1, 2, 3, 4, 5, 6, 7, 8]) >>> coarsen(np.min, x, {0: 3}, trim_excess=True) array([1, 4])

dask.array.
stack
(seq, axis=0)¶ Stack arrays along a new axis
Given a sequence of dask Arrays form a new dask Array by stacking them along a new dimension (axis=0 by default)
See also
Examples
Create slices
>>> import dask.array as da >>> import numpy as np
>>> data = [from_array(np.ones((4, 4)), chunks=(2, 2)) ... for i in range(3)]
>>> x = da.stack(data, axis=0) >>> x.shape (3, 4, 4)
>>> da.stack(data, axis=1).shape (4, 3, 4)
>>> da.stack(data, axis=1).shape (4, 4, 3)
Result is a new dask Array

dask.array.
concatenate
(seq, axis=0)¶ Concatenate arrays along an existing axis
Given a sequence of dask Arrays form a new dask Array by stacking them along an existing dimension (axis=0 by default)
See also
Examples
Create slices
>>> import dask.array as da >>> import numpy as np
>>> data = [from_array(np.ones((4, 4)), chunks=(2, 2)) ... for i in range(3)]
>>> x = da.concatenate(data, axis=0) >>> x.shape (12, 4)
>>> da.concatenate(data, axis=1).shape (4, 12)
Result is a new dask Array

dask.array.
all
(a, axis=None, keepdims=False, split_every=None)¶ Test whether all array elements along a given axis evaluate to True.
Parameters: a : array_like
Input array or object that can be converted to an array.
axis : None or int or tuple of ints, optional
Axis or axes along which a logical AND reduction is performed. The default (axis = None) is perform a logical OR over all the dimensions of the input array. axis may be negative, in which case it counts from the last to the first axis.
New in version 1.7.0.
If this is a tuple of ints, a reduction is performed on multiple axes, instead of a single axis or all the axes as before.
out : ndarray, optional
Alternate output array in which to place the result. It must have the same shape as the expected output and its type is preserved (e.g., if
dtype(out)
is float, the result will consist of 0.0’s and 1.0’s). See doc.ufuncs (Section “Output arguments”) for more details.keepdims : bool, optional
If this is set to True, the axes which are reduced are left in the result as dimensions with size one. With this option, the result will broadcast correctly against the original arr.
Returns: all : ndarray, bool
A new boolean or array is returned unless out is specified, in which case a reference to out is returned.
See also
ndarray.all
 equivalent method
any
 Test whether any element along a given axis evaluates to True.
Notes
Not a Number (NaN), positive infinity and negative infinity evaluate to True because these are not equal to zero.
Examples
>>> np.all([[True,False],[True,True]]) False
>>> np.all([[True,False],[True,True]], axis=0) array([ True, False], dtype=bool)
>>> np.all([1, 4, 5]) True
>>> np.all([1.0, np.nan]) True
>>> o=np.array([False]) >>> z=np.all([1, 4, 5], out=o) >>> id(z), id(o), z (28293632, 28293632, array([ True], dtype=bool))

dask.array.
angle
(*args, **kwargs)¶ Return the angle of the complex argument.
Parameters: z : array_like
A complex number or sequence of complex numbers.
deg : bool, optional
Return angle in degrees if True, radians if False (default).
Returns: angle : {ndarray, scalar}
The counterclockwise angle from the positive real axis on the complex plane, with dtype as numpy.float64.
See also
arctan2
,absolute
Examples
>>> np.angle([1.0, 1.0j, 1+1j]) # in radians array([ 0. , 1.57079633, 0.78539816]) >>> np.angle(1+1j, deg=True) # in degrees 45.0

dask.array.
any
(a, axis=None, keepdims=False, split_every=None)¶ Test whether any array element along a given axis evaluates to True.
Returns single boolean unless axis is not
None
Parameters: a : array_like
Input array or object that can be converted to an array.
axis : None or int or tuple of ints, optional
Axis or axes along which a logical OR reduction is performed. The default (axis = None) is perform a logical OR over all the dimensions of the input array. axis may be negative, in which case it counts from the last to the first axis.
New in version 1.7.0.
If this is a tuple of ints, a reduction is performed on multiple axes, instead of a single axis or all the axes as before.
out : ndarray, optional
Alternate output array in which to place the result. It must have the same shape as the expected output and its type is preserved (e.g., if it is of type float, then it will remain so, returning 1.0 for True and 0.0 for False, regardless of the type of a). See doc.ufuncs (Section “Output arguments”) for details.
keepdims : bool, optional
If this is set to True, the axes which are reduced are left in the result as dimensions with size one. With this option, the result will broadcast correctly against the original arr.
Returns: any : bool or ndarray
A new boolean or ndarray is returned unless out is specified, in which case a reference to out is returned.
See also
ndarray.any
 equivalent method
all
 Test whether all elements along a given axis evaluate to True.
Notes
Not a Number (NaN), positive infinity and negative infinity evaluate to True because these are not equal to zero.
Examples
>>> np.any([[True, False], [True, True]]) True
>>> np.any([[True, False], [False, False]], axis=0) array([ True, False], dtype=bool)
>>> np.any([1, 0, 5]) True
>>> np.any(np.nan) True
>>> o=np.array([False]) >>> z=np.any([1, 4, 5], out=o) >>> z, o (array([ True], dtype=bool), array([ True], dtype=bool)) >>> # Check now that z is a reference to o >>> z is o True >>> id(z), id(o) # identity of z and o (191614240, 191614240)

dask.array.
arange
(*args, **kwargs)¶ Return evenly spaced values from start to stop with step size step.
The values are halfopen [start, stop), so including start and excluding stop. This is basically the same as python’s range function but for dask arrays.
When using a noninteger step, such as 0.1, the results will often not be consistent. It is better to use linspace for these cases.
Parameters: start : int, optional
The starting value of the sequence. The default is 0.
stop : int
The end of the interval, this value is excluded from the interval.
step : int, optional
The spacing between the values. The default is 1 when not specified. The last value of the sequence.
chunks : int
The number of samples on each block. Note that the last block will have fewer samples if
len(array) % chunks != 0
.Returns: samples : dask array
See also

dask.array.
arccos
(x[, out])¶ Trigonometric inverse cosine, elementwise.
The inverse of cos so that, if
y = cos(x)
, thenx = arccos(y)
.Parameters: x : array_like
xcoordinate on the unit circle. For real arguments, the domain is [1, 1].
out : ndarray, optional
Array of the same shape as a, to store results in. See doc.ufuncs (Section “Output arguments”) for more details.
Returns: angle : ndarray
The angle of the ray intersecting the unit circle at the given xcoordinate in radians [0, pi]. If x is a scalar then a scalar is returned, otherwise an array of the same shape as x is returned.
Notes
arccos is a multivalued function: for each x there are infinitely many numbers z such that cos(z) = x. The convention is to return the angle z whose real part lies in [0, pi].
For realvalued input data types, arccos always returns real output. For each value that cannot be expressed as a real number or infinity, it yields
nan
and sets the invalid floating point error flag.For complexvalued input, arccos is a complex analytic function that has branch cuts [inf, 1] and [1, inf] and is continuous from above on the former and from below on the latter.
The inverse cos is also known as acos or cos^1.
References
M. Abramowitz and I.A. Stegun, “Handbook of Mathematical Functions”, 10th printing, 1964, pp. 79. http://www.math.sfu.ca/~cbm/aands/
Examples
We expect the arccos of 1 to be 0, and of 1 to be pi:
>>> np.arccos([1, 1]) array([ 0. , 3.14159265])
Plot arccos:
>>> import matplotlib.pyplot as plt >>> x = np.linspace(1, 1, num=100) >>> plt.plot(x, np.arccos(x)) >>> plt.axis('tight') >>> plt.show()

dask.array.
arccosh
(x[, out])¶ Inverse hyperbolic cosine, elementwise.
Parameters: x : array_like
Input array.
out : ndarray, optional
Array of the same shape as x, to store results in. See doc.ufuncs (Section “Output arguments”) for details.
Returns: arccosh : ndarray
Array of the same shape as x.
Notes
arccosh is a multivalued function: for each x there are infinitely many numbers z such that cosh(z) = x. The convention is to return the z whose imaginary part lies in [pi, pi] and the real part in
[0, inf]
.For realvalued input data types, arccosh always returns real output. For each value that cannot be expressed as a real number or infinity, it yields
nan
and sets the invalid floating point error flag.For complexvalued input, arccosh is a complex analytical function that has a branch cut [inf, 1] and is continuous from above on it.
References
[R76] M. Abramowitz and I.A. Stegun, “Handbook of Mathematical Functions”, 10th printing, 1964, pp. 86. http://www.math.sfu.ca/~cbm/aands/ [R77] Wikipedia, “Inverse hyperbolic function”, http://en.wikipedia.org/wiki/Arccosh Examples
>>> np.arccosh([np.e, 10.0]) array([ 1.65745445, 2.99322285]) >>> np.arccosh(1) 0.0

dask.array.
arcsin
(x[, out])¶ Inverse sine, elementwise.
Parameters: x : array_like
ycoordinate on the unit circle.
out : ndarray, optional
Array of the same shape as x, in which to store the results. See doc.ufuncs (Section “Output arguments”) for more details.
Returns: angle : ndarray
The inverse sine of each element in x, in radians and in the closed interval
[pi/2, pi/2]
. If x is a scalar, a scalar is returned, otherwise an array.Notes
arcsin is a multivalued function: for each x there are infinitely many numbers z such that \(sin(z) = x\). The convention is to return the angle z whose real part lies in [pi/2, pi/2].
For realvalued input data types, arcsin always returns real output. For each value that cannot be expressed as a real number or infinity, it yields
nan
and sets the invalid floating point error flag.For complexvalued input, arcsin is a complex analytic function that has, by convention, the branch cuts [inf, 1] and [1, inf] and is continuous from above on the former and from below on the latter.
The inverse sine is also known as asin or sin^{1}.
References
Abramowitz, M. and Stegun, I. A., Handbook of Mathematical Functions, 10th printing, New York: Dover, 1964, pp. 79ff. http://www.math.sfu.ca/~cbm/aands/
Examples
>>> np.arcsin(1) # pi/2 1.5707963267948966 >>> np.arcsin(1) # pi/2 1.5707963267948966 >>> np.arcsin(0) 0.0

dask.array.
arcsinh
(x[, out])¶ Inverse hyperbolic sine elementwise.
Parameters: x : array_like
Input array.
out : ndarray, optional
Array into which the output is placed. Its type is preserved and it must be of the right shape to hold the output. See doc.ufuncs.
Returns: out : ndarray
Array of of the same shape as x.
Notes
arcsinh is a multivalued function: for each x there are infinitely many numbers z such that sinh(z) = x. The convention is to return the z whose imaginary part lies in [pi/2, pi/2].
For realvalued input data types, arcsinh always returns real output. For each value that cannot be expressed as a real number or infinity, it returns
nan
and sets the invalid floating point error flag.For complexvalued input, arccos is a complex analytical function that has branch cuts [1j, infj] and [1j, infj] and is continuous from the right on the former and from the left on the latter.
The inverse hyperbolic sine is also known as asinh or
sinh^1
.References
[R78] M. Abramowitz and I.A. Stegun, “Handbook of Mathematical Functions”, 10th printing, 1964, pp. 86. http://www.math.sfu.ca/~cbm/aands/ [R79] Wikipedia, “Inverse hyperbolic function”, http://en.wikipedia.org/wiki/Arcsinh Examples
>>> np.arcsinh(np.array([np.e, 10.0])) array([ 1.72538256, 2.99822295])

dask.array.
arctan
(x[, out])¶ Trigonometric inverse tangent, elementwise.
The inverse of tan, so that if
y = tan(x)
thenx = arctan(y)
.Parameters: x : array_like
Input values. arctan is applied to each element of x.
Returns: out : ndarray
Out has the same shape as x. Its real part is in
[pi/2, pi/2]
(arctan(+/inf)
returns+/pi/2
). It is a scalar if x is a scalar.See also
Notes
arctan is a multivalued function: for each x there are infinitely many numbers z such that tan(z) = x. The convention is to return the angle z whose real part lies in [pi/2, pi/2].
For realvalued input data types, arctan always returns real output. For each value that cannot be expressed as a real number or infinity, it yields
nan
and sets the invalid floating point error flag.For complexvalued input, arctan is a complex analytic function that has [1j, infj] and [1j, infj] as branch cuts, and is continuous from the left on the former and from the right on the latter.
The inverse tangent is also known as atan or tan^{1}.
References
Abramowitz, M. and Stegun, I. A., Handbook of Mathematical Functions, 10th printing, New York: Dover, 1964, pp. 79. http://www.math.sfu.ca/~cbm/aands/
Examples
We expect the arctan of 0 to be 0, and of 1 to be pi/4:
>>> np.arctan([0, 1]) array([ 0. , 0.78539816])
>>> np.pi/4 0.78539816339744828
Plot arctan:
>>> import matplotlib.pyplot as plt >>> x = np.linspace(10, 10) >>> plt.plot(x, np.arctan(x)) >>> plt.axis('tight') >>> plt.show()

dask.array.
arctan2
(x1, x2[, out])¶ Elementwise arc tangent of
x1/x2
choosing the quadrant correctly.The quadrant (i.e., branch) is chosen so that
arctan2(x1, x2)
is the signed angle in radians between the ray ending at the origin and passing through the point (1,0), and the ray ending at the origin and passing through the point (x2, x1). (Note the role reversal: the “ycoordinate” is the first function parameter, the “xcoordinate” is the second.) By IEEE convention, this function is defined for x2 = +/0 and for either or both of x1 and x2 = +/inf (see Notes for specific values).This function is not defined for complexvalued arguments; for the socalled argument of complex values, use angle.
Parameters: x1 : array_like, realvalued
ycoordinates.
x2 : array_like, realvalued
xcoordinates. x2 must be broadcastable to match the shape of x1 or vice versa.
Returns: angle : ndarray
Array of angles in radians, in the range
[pi, pi]
.Notes
arctan2 is identical to the atan2 function of the underlying C library. The following special values are defined in the C standard: [R80]
x1 x2 arctan2(x1,x2) +/ 0 +0 +/ 0 +/ 0 0 +/ pi > 0 +/inf +0 / +pi < 0 +/inf 0 / pi +/inf +inf +/ (pi/4) +/inf inf +/ (3*pi/4) Note that +0 and 0 are distinct floating point numbers, as are +inf and inf.
References
[R80] (1, 2) ISO/IEC standard 9899:1999, “Programming language C.” Examples
Consider four points in different quadrants:
>>> x = np.array([1, +1, +1, 1]) >>> y = np.array([1, 1, +1, +1]) >>> np.arctan2(y, x) * 180 / np.pi array([135., 45., 45., 135.])
Note the order of the parameters. arctan2 is defined also when x2 = 0 and at several other special points, obtaining values in the range
[pi, pi]
:>>> np.arctan2([1., 1.], [0., 0.]) array([ 1.57079633, 1.57079633]) >>> np.arctan2([0., 0., np.inf], [+0., 0., np.inf]) array([ 0. , 3.14159265, 0.78539816])

dask.array.
arctanh
(x[, out])¶ Inverse hyperbolic tangent elementwise.
Parameters: x : array_like
Input array.
Returns: out : ndarray
Array of the same shape as x.
See also
emath.arctanh
Notes
arctanh is a multivalued function: for each x there are infinitely many numbers z such that tanh(z) = x. The convention is to return the z whose imaginary part lies in [pi/2, pi/2].
For realvalued input data types, arctanh always returns real output. For each value that cannot be expressed as a real number or infinity, it yields
nan
and sets the invalid floating point error flag.For complexvalued input, arctanh is a complex analytical function that has branch cuts [1, inf] and [1, inf] and is continuous from above on the former and from below on the latter.
The inverse hyperbolic tangent is also known as atanh or
tanh^1
.References
[R81] M. Abramowitz and I.A. Stegun, “Handbook of Mathematical Functions”, 10th printing, 1964, pp. 86. http://www.math.sfu.ca/~cbm/aands/ [R82] Wikipedia, “Inverse hyperbolic function”, http://en.wikipedia.org/wiki/Arctanh Examples
>>> np.arctanh([0, 0.5]) array([ 0. , 0.54930614])

dask.array.
argmax
(x, axis=None, split_every=None)¶ Indices of the maximum values along an axis.
Parameters: a : array_like
Input array.
axis : int, optional
By default, the index is into the flattened array, otherwise along the specified axis.
Returns: index_array : ndarray of ints
Array of indices into the array. It has the same shape as a.shape with the dimension along axis removed.
See also
ndarray.argmax
,argmin
amax
 The maximum value along a given axis.
unravel_index
 Convert a flat index into an index tuple.
Notes
In case of multiple occurrences of the maximum values, the indices corresponding to the first occurrence are returned.
Examples
>>> a = np.arange(6).reshape(2,3) >>> a array([[0, 1, 2], [3, 4, 5]]) >>> np.argmax(a) 5 >>> np.argmax(a, axis=0) array([1, 1, 1]) >>> np.argmax(a, axis=1) array([2, 2])
>>> b = np.arange(6) >>> b[1] = 5 >>> b array([0, 5, 2, 3, 4, 5]) >>> np.argmax(b) # Only the first occurrence is returned. 1

dask.array.
argmin
(x, axis=None, split_every=None)¶ Return the indices of the minimum values along an axis.
See also
argmax
 Similar function. Please refer to numpy.argmax for detailed documentation.

dask.array.
around
(x, decimals=0)¶ Evenly round to the given number of decimals.
Parameters: a : array_like
Input data.
decimals : int, optional
Number of decimal places to round to (default: 0). If decimals is negative, it specifies the number of positions to the left of the decimal point.
out : ndarray, optional
Alternative output array in which to place the result. It must have the same shape as the expected output, but the type of the output values will be cast if necessary. See doc.ufuncs (Section “Output arguments”) for details.
Returns: rounded_array : ndarray
An array of the same type as a, containing the rounded values. Unless out was specified, a new array is created. A reference to the result is returned.
The real and imaginary parts of complex numbers are rounded separately. The result of rounding a float is a float.
Notes
For values exactly halfway between rounded decimal values, Numpy rounds to the nearest even value. Thus 1.5 and 2.5 round to 2.0, 0.5 and 0.5 round to 0.0, etc. Results may also be surprising due to the inexact representation of decimal fractions in the IEEE floating point standard [R83] and errors introduced when scaling by powers of ten.
References
[R83] (1, 2) “Lecture Notes on the Status of IEEE 754”, William Kahan, http://www.cs.berkeley.edu/~wkahan/ieee754status/IEEE754.PDF [R84] “How Futile are Mindless Assessments of Roundoff in FloatingPoint Computation?”, William Kahan, http://www.cs.berkeley.edu/~wkahan/Mindless.pdf Examples
>>> np.around([0.37, 1.64]) array([ 0., 2.]) >>> np.around([0.37, 1.64], decimals=1) array([ 0.4, 1.6]) >>> np.around([.5, 1.5, 2.5, 3.5, 4.5]) # rounds to nearest even value array([ 0., 2., 2., 4., 4.]) >>> np.around([1,2,3,11], decimals=1) # ndarray of ints is returned array([ 1, 2, 3, 11]) >>> np.around([1,2,3,11], decimals=1) array([ 0, 0, 0, 10])

dask.array.
array
(object, dtype=None, copy=True, order=None, subok=False, ndmin=0)¶ Create an array.
Parameters: object : array_like
An array, any object exposing the array interface, an object whose __array__ method returns an array, or any (nested) sequence.
dtype : datatype, optional
The desired datatype for the array. If not given, then the type will be determined as the minimum type required to hold the objects in the sequence. This argument can only be used to ‘upcast’ the array. For downcasting, use the .astype(t) method.
copy : bool, optional
If true (default), then the object is copied. Otherwise, a copy will only be made if __array__ returns a copy, if obj is a nested sequence, or if a copy is needed to satisfy any of the other requirements (dtype, order, etc.).
order : {‘C’, ‘F’, ‘A’}, optional
Specify the order of the array. If order is ‘C’ (default), then the array will be in Ccontiguous order (lastindex varies the fastest). If order is ‘F’, then the returned array will be in Fortrancontiguous order (firstindex varies the fastest). If order is ‘A’, then the returned array may be in any order (either C, Fortrancontiguous, or even discontiguous).
subok : bool, optional
If True, then subclasses will be passedthrough, otherwise the returned array will be forced to be a baseclass array (default).
ndmin : int, optional
Specifies the minimum number of dimensions that the resulting array should have. Ones will be prepended to the shape as needed to meet this requirement.
Returns: out : ndarray
An array object satisfying the specified requirements.
Examples
>>> np.array([1, 2, 3]) array([1, 2, 3])
Upcasting:
>>> np.array([1, 2, 3.0]) array([ 1., 2., 3.])
More than one dimension:
>>> np.array([[1, 2], [3, 4]]) array([[1, 2], [3, 4]])
Minimum dimensions 2:
>>> np.array([1, 2, 3], ndmin=2) array([[1, 2, 3]])
Type provided:
>>> np.array([1, 2, 3], dtype=complex) array([ 1.+0.j, 2.+0.j, 3.+0.j])
Datatype consisting of more than one element:
>>> x = np.array([(1,2),(3,4)],dtype=[('a','<i4'),('b','<i4')]) >>> x['a'] array([1, 3])
Creating an array from subclasses:
>>> np.array(np.mat('1 2; 3 4')) array([[1, 2], [3, 4]])
>>> np.array(np.mat('1 2; 3 4'), subok=True) matrix([[1, 2], [3, 4]])

dask.array.
bincount
(x, weights=None, minlength=None)¶ Count number of occurrences of each value in array of nonnegative ints.
The number of bins (of size 1) is one larger than the largest value in x. If minlength is specified, there will be at least this number of bins in the output array (though it will be longer if necessary, depending on the contents of x). Each bin gives the number of occurrences of its index value in x. If weights is specified the input array is weighted by it, i.e. if a value
n
is found at positioni
,out[n] += weight[i]
instead ofout[n] += 1
.Parameters: x : array_like, 1 dimension, nonnegative ints
Input array.
weights : array_like, optional
Weights, array of the same shape as x.
minlength : int, optional
New in version 1.6.0.
A minimum number of bins for the output array.
Returns: out : ndarray of ints
The result of binning the input array. The length of out is equal to
np.amax(x)+1
.Raises: ValueError
If the input is not 1dimensional, or contains elements with negative values, or if minlength is nonpositive.
TypeError
If the type of the input is float or complex.
Examples
>>> np.bincount(np.arange(5)) array([1, 1, 1, 1, 1]) >>> np.bincount(np.array([0, 1, 1, 3, 2, 1, 7])) array([1, 3, 1, 1, 0, 0, 0, 1])
>>> x = np.array([0, 1, 1, 3, 2, 1, 7, 23]) >>> np.bincount(x).size == np.amax(x)+1 True
The input array needs to be of integer dtype, otherwise a TypeError is raised:
>>> np.bincount(np.arange(5, dtype=np.float)) Traceback (most recent call last): File "<stdin>", line 1, in <module> TypeError: array cannot be safely cast to required type
A possible use of
bincount
is to perform sums over variablesize chunks of an array, using theweights
keyword.>>> w = np.array([0.3, 0.5, 0.2, 0.7, 1., 0.6]) # weights >>> x = np.array([0, 1, 1, 2, 2, 2]) >>> np.bincount(x, weights=w) array([ 0.3, 0.7, 1.1])

dask.array.
broadcast_to
(x, shape)¶ Broadcast an array to a new shape.
Parameters: array : array_like
The array to broadcast.
shape : tuple
The shape of the desired array.
subok : bool, optional
If True, then subclasses will be passedthrough, otherwise the returned array will be forced to be a baseclass array (default).
Returns: broadcast : array
A readonly view on the original array with the given shape. It is typically not contiguous. Furthermore, more than one element of a broadcasted array may refer to a single memory location.
Raises: ValueError
If the array is not compatible with the new shape according to NumPy’s broadcasting rules.
Examples
>>> x = np.array([1, 2, 3]) >>> np.broadcast_to(x, (3, 3)) array([[1, 2, 3], [1, 2, 3], [1, 2, 3]])

dask.array.
coarsen
(reduction, x, axes, trim_excess=False) Coarsen array by applying reduction to fixed size neighborhoods
Parameters: reduction: function
Function like np.sum, np.mean, etc...
x: np.ndarray
Array to be coarsened
axes: dict
Mapping of axis to coarsening factor
Examples
>>> x = np.array([1, 2, 3, 4, 5, 6]) >>> coarsen(np.sum, x, {0: 2}) array([ 3, 7, 11]) >>> coarsen(np.max, x, {0: 3}) array([3, 6])
Provide dictionary of scale per dimension
>>> x = np.arange(24).reshape((4, 6)) >>> x array([[ 0, 1, 2, 3, 4, 5], [ 6, 7, 8, 9, 10, 11], [12, 13, 14, 15, 16, 17], [18, 19, 20, 21, 22, 23]])
>>> coarsen(np.min, x, {0: 2, 1: 3}) array([[ 0, 3], [12, 15]])
You must avoid excess elements explicitly
>>> x = np.array([1, 2, 3, 4, 5, 6, 7, 8]) >>> coarsen(np.min, x, {0: 3}, trim_excess=True) array([1, 4])

dask.array.
ceil
(x[, out])¶ Return the ceiling of the input, elementwise.
The ceil of the scalar x is the smallest integer i, such that i >= x. It is often denoted as \(\lceil x \rceil\).
Parameters: x : array_like
Input data.
Returns: y : {ndarray, scalar}
The ceiling of each element in x, with float dtype.
Examples
>>> a = np.array([1.7, 1.5, 0.2, 0.2, 1.5, 1.7, 2.0]) >>> np.ceil(a) array([1., 1., 0., 1., 2., 2., 2.])

dask.array.
choose
(a, choices)¶ Construct an array from an index array and a set of arrays to choose from.
First of all, if confused or uncertain, definitely look at the Examples  in its full generality, this function is less simple than it might seem from the following code description (below ndi = numpy.lib.index_tricks):
np.choose(a,c) == np.array([c[a[I]][I] for I in ndi.ndindex(a.shape)])
.But this omits some subtleties. Here is a fully general summary:
Given an “index” array (a) of integers and a sequence of n arrays (choices), a and each choice array are first broadcast, as necessary, to arrays of a common shape; calling these Ba and Bchoices[i], i = 0,...,n1 we have that, necessarily,
Ba.shape == Bchoices[i].shape
for each i. Then, a new array with shapeBa.shape
is created as follows: if
mode=raise
(the default), then, first of all, each element of a (and thus Ba) must be in the range [0, n1]; now, suppose that i (in that range) is the value at the (j0, j1, ..., jm) position in Ba  then the value at the same position in the new array is the value in Bchoices[i] at that same position;  if
mode=wrap
, values in a (and thus Ba) may be any (signed) integer; modular arithmetic is used to map integers outside the range [0, n1] back into that range; and then the new array is constructed as above;  if
mode=clip
, values in a (and thus Ba) may be any (signed) integer; negative integers are mapped to 0; values greater than n1 are mapped to n1; and then the new array is constructed as above.
Parameters: a : int array
This array must contain integers in [0, n1], where n is the number of choices, unless
mode=wrap
ormode=clip
, in which cases any integers are permissible.choices : sequence of arrays
Choice arrays. a and all of the choices must be broadcastable to the same shape. If choices is itself an array (not recommended), then its outermost dimension (i.e., the one corresponding to
choices.shape[0]
) is taken as defining the “sequence”.out : array, optional
If provided, the result will be inserted into this array. It should be of the appropriate shape and dtype.
mode : {‘raise’ (default), ‘wrap’, ‘clip’}, optional
Specifies how indices outside [0, n1] will be treated:
 ‘raise’ : an exception is raised
 ‘wrap’ : value becomes value mod n
 ‘clip’ : values < 0 are mapped to 0, values > n1 are mapped to n1
Returns: merged_array : array
The merged result.
Raises: ValueError: shape mismatch
If a and each choice array are not all broadcastable to the same shape.
See also
ndarray.choose
 equivalent method
Notes
To reduce the chance of misinterpretation, even though the following “abuse” is nominally supported, choices should neither be, nor be thought of as, a single array, i.e., the outermost sequencelike container should be either a list or a tuple.
Examples
>>> choices = [[0, 1, 2, 3], [10, 11, 12, 13], ... [20, 21, 22, 23], [30, 31, 32, 33]] >>> np.choose([2, 3, 1, 0], choices ... # the first element of the result will be the first element of the ... # third (2+1) "array" in choices, namely, 20; the second element ... # will be the second element of the fourth (3+1) choice array, i.e., ... # 31, etc. ... ) array([20, 31, 12, 3]) >>> np.choose([2, 4, 1, 0], choices, mode='clip') # 4 goes to 3 (41) array([20, 31, 12, 3]) >>> # because there are 4 choice arrays >>> np.choose([2, 4, 1, 0], choices, mode='wrap') # 4 goes to (4 mod 4) array([20, 1, 12, 3]) >>> # i.e., 0
A couple examples illustrating how choose broadcasts:
>>> a = [[1, 0, 1], [0, 1, 0], [1, 0, 1]] >>> choices = [10, 10] >>> np.choose(a, choices) array([[ 10, 10, 10], [10, 10, 10], [ 10, 10, 10]])
>>> # With thanks to Anne Archibald >>> a = np.array([0, 1]).reshape((2,1,1)) >>> c1 = np.array([1, 2, 3]).reshape((1,3,1)) >>> c2 = np.array([1, 2, 3, 4, 5]).reshape((1,1,5)) >>> np.choose(a, (c1, c2)) # result is 2x3x5, res[0,:,:]=c1, res[1,:,:]=c2 array([[[ 1, 1, 1, 1, 1], [ 2, 2, 2, 2, 2], [ 3, 3, 3, 3, 3]], [[1, 2, 3, 4, 5], [1, 2, 3, 4, 5], [1, 2, 3, 4, 5]]])
 if

dask.array.
clip
(*args, **kwargs)¶ Clip (limit) the values in an array.
Given an interval, values outside the interval are clipped to the interval edges. For example, if an interval of
[0, 1]
is specified, values smaller than 0 become 0, and values larger than 1 become 1.Parameters: a : array_like
Array containing elements to clip.
a_min : scalar or array_like
Minimum value.
a_max : scalar or array_like
Maximum value. If a_min or a_max are array_like, then they will be broadcasted to the shape of a.
out : ndarray, optional
The results will be placed in this array. It may be the input array for inplace clipping. out must be of the right shape to hold the output. Its type is preserved.
Returns: clipped_array : ndarray
An array with the elements of a, but where values < a_min are replaced with a_min, and those > a_max with a_max.
See also
numpy.doc.ufuncs
 Section “Output arguments”
Examples
>>> a = np.arange(10) >>> np.clip(a, 1, 8) array([1, 1, 2, 3, 4, 5, 6, 7, 8, 8]) >>> a array([0, 1, 2, 3, 4, 5, 6, 7, 8, 9]) >>> np.clip(a, 3, 6, out=a) array([3, 3, 3, 3, 4, 5, 6, 6, 6, 6]) >>> a = np.arange(10) >>> a array([0, 1, 2, 3, 4, 5, 6, 7, 8, 9]) >>> np.clip(a, [3,4,1,1,1,4,4,4,4,4], 8) array([3, 4, 2, 3, 4, 5, 6, 7, 8, 8])

dask.array.
compress
(condition, a, axis=None)¶ Return selected slices of an array along given axis.
When working along a given axis, a slice along that axis is returned in output for each index where condition evaluates to True. When working on a 1D array, compress is equivalent to extract.
Parameters: condition : 1D array of bools
Array that selects which entries to return. If len(condition) is less than the size of a along the given axis, then output is truncated to the length of the condition array.
a : array_like
Array from which to extract a part.
axis : int, optional
Axis along which to take slices. If None (default), work on the flattened array.
out : ndarray, optional
Output array. Its type is preserved and it must be of the right shape to hold the output.
Returns: compressed_array : ndarray
A copy of a without the slices along axis for which condition is false.
See also
take
,choose
,diag
,diagonal
,select
ndarray.compress
 Equivalent method in ndarray
np.extract
 Equivalent method when working on 1D arrays
numpy.doc.ufuncs
 Section “Output arguments”
Examples
>>> a = np.array([[1, 2], [3, 4], [5, 6]]) >>> a array([[1, 2], [3, 4], [5, 6]]) >>> np.compress([0, 1], a, axis=0) array([[3, 4]]) >>> np.compress([False, True, True], a, axis=0) array([[3, 4], [5, 6]]) >>> np.compress([False, True], a, axis=1) array([[2], [4], [6]])
Working on the flattened array does not return slices along an axis but selects elements.
>>> np.compress([False, True], a) array([2])

dask.array.
concatenate
(seq, axis=0) Concatenate arrays along an existing axis
Given a sequence of dask Arrays form a new dask Array by stacking them along an existing dimension (axis=0 by default)
See also
Examples
Create slices
>>> import dask.array as da >>> import numpy as np
>>> data = [from_array(np.ones((4, 4)), chunks=(2, 2)) ... for i in range(3)]
>>> x = da.concatenate(data, axis=0) >>> x.shape (12, 4)
>>> da.concatenate(data, axis=1).shape (4, 12)
Result is a new dask Array

dask.array.
conj
(x[, out])¶ Return the complex conjugate, elementwise.
The complex conjugate of a complex number is obtained by changing the sign of its imaginary part.
Parameters: x : array_like
Input value.
Returns: y : ndarray
The complex conjugate of x, with same dtype as y.
Examples
>>> np.conjugate(1+2j) (12j)
>>> x = np.eye(2) + 1j * np.eye(2) >>> np.conjugate(x) array([[ 1.1.j, 0.0.j], [ 0.0.j, 1.1.j]])

dask.array.
copysign
(x1, x2[, out])¶ Change the sign of x1 to that of x2, elementwise.
If both arguments are arrays or sequences, they have to be of the same length. If x2 is a scalar, its sign will be copied to all elements of x1.
Parameters: x1 : array_like
Values to change the sign of.
x2 : array_like
The sign of x2 is copied to x1.
out : ndarray, optional
Array into which the output is placed. Its type is preserved and it must be of the right shape to hold the output. See doc.ufuncs.
Returns: out : array_like
The values of x1 with the sign of x2.
Examples
>>> np.copysign(1.3, 1) 1.3 >>> 1/np.copysign(0, 1) inf >>> 1/np.copysign(0, 1) inf
>>> np.copysign([1, 0, 1], 1.1) array([1., 0., 1.]) >>> np.copysign([1, 0, 1], np.arange(3)1) array([1., 0., 1.])

dask.array.
corrcoef
(x, y=None, rowvar=1)¶ Return correlation coefficients.
Please refer to the documentation for cov for more detail. The relationship between the correlation coefficient matrix, P, and the covariance matrix, C, is
\[P_{ij} = \frac{ C_{ij} } { \sqrt{ C_{ii} * C_{jj} } }\]The values of P are between 1 and 1, inclusive.
Parameters: x : array_like
A 1D or 2D array containing multiple variables and observations. Each row of m represents a variable, and each column a single observation of all those variables. Also see rowvar below.
y : array_like, optional
An additional set of variables and observations. y has the same shape as m.
rowvar : int, optional
If rowvar is nonzero (default), then each row represents a variable, with observations in the columns. Otherwise, the relationship is transposed: each column represents a variable, while the rows contain observations.
bias : int, optional
Default normalization is by
(N  1)
, whereN
is the number of observations (unbiased estimate). If bias is 1, then normalization is byN
. These values can be overridden by using the keywordddof
in numpy versions >= 1.5.ddof : {None, int}, optional
New in version 1.5.
If not
None
normalization is by(N  ddof)
, whereN
is the number of observations; this overrides the value implied bybias
. The default value isNone
.Returns: out : ndarray
The correlation coefficient matrix of the variables.
See also
cov
 Covariance matrix

dask.array.
cos
(x[, out])¶ Cosine elementwise.
Parameters: x : array_like
Input array in radians.
out : ndarray, optional
Output array of same shape as x.
Returns: y : ndarray
The corresponding cosine values.
Raises: ValueError: invalid return array shape
if out is provided and out.shape != x.shape (See Examples)
Notes
If out is provided, the function writes the result into it, and returns a reference to out. (See Examples)
References
M. Abramowitz and I. A. Stegun, Handbook of Mathematical Functions. New York, NY: Dover, 1972.
Examples
>>> np.cos(np.array([0, np.pi/2, np.pi])) array([ 1.00000000e+00, 6.12303177e17, 1.00000000e+00]) >>> >>> # Example of providing the optional output parameter >>> out2 = np.cos([0.1], out1) >>> out2 is out1 True >>> >>> # Example of ValueError due to provision of shape mismatched `out` >>> np.cos(np.zeros((3,3)),np.zeros((2,2))) Traceback (most recent call last): File "<stdin>", line 1, in <module> ValueError: invalid return array shape

dask.array.
cosh
(x[, out])¶ Hyperbolic cosine, elementwise.
Equivalent to
1/2 * (np.exp(x) + np.exp(x))
andnp.cos(1j*x)
.Parameters: x : array_like
Input array.
Returns: out : ndarray
Output array of same shape as x.
Examples
>>> np.cosh(0) 1.0
The hyperbolic cosine describes the shape of a hanging cable:
>>> import matplotlib.pyplot as plt >>> x = np.linspace(4, 4, 1000) >>> plt.plot(x, np.cosh(x)) >>> plt.show()

dask.array.
cov
(m, y=None, rowvar=1, bias=0, ddof=None)¶ Estimate a covariance matrix, given data.
Covariance indicates the level to which two variables vary together. If we examine Ndimensional samples, \(X = [x_1, x_2, ... x_N]^T\), then the covariance matrix element \(C_{ij}\) is the covariance of \(x_i\) and \(x_j\). The element \(C_{ii}\) is the variance of \(x_i\).
Parameters: m : array_like
A 1D or 2D array containing multiple variables and observations. Each row of m represents a variable, and each column a single observation of all those variables. Also see rowvar below.
y : array_like, optional
An additional set of variables and observations. y has the same form as that of m.
rowvar : int, optional
If rowvar is nonzero (default), then each row represents a variable, with observations in the columns. Otherwise, the relationship is transposed: each column represents a variable, while the rows contain observations.
bias : int, optional
Default normalization is by
(N  1)
, whereN
is the number of observations given (unbiased estimate). If bias is 1, then normalization is byN
. These values can be overridden by using the keywordddof
in numpy versions >= 1.5.ddof : int, optional
New in version 1.5.
If not
None
normalization is by(N  ddof)
, whereN
is the number of observations; this overrides the value implied bybias
. The default value isNone
.Returns: out : ndarray
The covariance matrix of the variables.
See also
corrcoef
 Normalized covariance matrix
Examples
Consider two variables, \(x_0\) and \(x_1\), which correlate perfectly, but in opposite directions:
>>> x = np.array([[0, 2], [1, 1], [2, 0]]).T >>> x array([[0, 1, 2], [2, 1, 0]])
Note how \(x_0\) increases while \(x_1\) decreases. The covariance matrix shows this clearly:
>>> np.cov(x) array([[ 1., 1.], [1., 1.]])
Note that element \(C_{0,1}\), which shows the correlation between \(x_0\) and \(x_1\), is negative.
Further, note how x and y are combined:
>>> x = [2.1, 1, 4.3] >>> y = [3, 1.1, 0.12] >>> X = np.vstack((x,y)) >>> print np.cov(X) [[ 11.71 4.286 ] [ 4.286 2.14413333]] >>> print np.cov(x, y) [[ 11.71 4.286 ] [ 4.286 2.14413333]] >>> print np.cov(x) 11.71

dask.array.
cumprod
(x, axis, dtype=None)¶ Return the cumulative product of elements along a given axis.
Parameters: a : array_like
Input array.
axis : int, optional
Axis along which the cumulative product is computed. By default the input is flattened.
dtype : dtype, optional
Type of the returned array, as well as of the accumulator in which the elements are multiplied. If dtype is not specified, it defaults to the dtype of a, unless a has an integer dtype with a precision less than that of the default platform integer. In that case, the default platform integer is used instead.
out : ndarray, optional
Alternative output array in which to place the result. It must have the same shape and buffer length as the expected output but the type of the resulting values will be cast if necessary.
Returns: cumprod : ndarray
A new array holding the result is returned unless out is specified, in which case a reference to out is returned.
See also
numpy.doc.ufuncs
 Section “Output arguments”
Notes
Arithmetic is modular when using integer types, and no error is raised on overflow.
Examples
>>> a = np.array([1,2,3]) >>> np.cumprod(a) # intermediate results 1, 1*2 ... # total product 1*2*3 = 6 array([1, 2, 6]) >>> a = np.array([[1, 2, 3], [4, 5, 6]]) >>> np.cumprod(a, dtype=float) # specify type of output array([ 1., 2., 6., 24., 120., 720.])
The cumulative product for each column (i.e., over the rows) of a:
>>> np.cumprod(a, axis=0) array([[ 1, 2, 3], [ 4, 10, 18]])
The cumulative product for each row (i.e. over the columns) of a:
>>> np.cumprod(a,axis=1) array([[ 1, 2, 6], [ 4, 20, 120]])

dask.array.
cumsum
(x, axis, dtype=None)¶ Return the cumulative sum of the elements along a given axis.
Parameters: a : array_like
Input array.
axis : int, optional
Axis along which the cumulative sum is computed. The default (None) is to compute the cumsum over the flattened array.
dtype : dtype, optional
Type of the returned array and of the accumulator in which the elements are summed. If dtype is not specified, it defaults to the dtype of a, unless a has an integer dtype with a precision less than that of the default platform integer. In that case, the default platform integer is used.
out : ndarray, optional
Alternative output array in which to place the result. It must have the same shape and buffer length as the expected output but the type will be cast if necessary. See doc.ufuncs (Section “Output arguments”) for more details.
Returns: cumsum_along_axis : ndarray.
A new array holding the result is returned unless out is specified, in which case a reference to out is returned. The result has the same size as a, and the same shape as a if axis is not None or a is a 1d array.
See also
sum
 Sum array elements.
trapz
 Integration of array values using the composite trapezoidal rule.
diff
 Calculate the nth order discrete difference along given axis.
Notes
Arithmetic is modular when using integer types, and no error is raised on overflow.
Examples
>>> a = np.array([[1,2,3], [4,5,6]]) >>> a array([[1, 2, 3], [4, 5, 6]]) >>> np.cumsum(a) array([ 1, 3, 6, 10, 15, 21]) >>> np.cumsum(a, dtype=float) # specifies type of output value(s) array([ 1., 3., 6., 10., 15., 21.])
>>> np.cumsum(a,axis=0) # sum over rows for each of the 3 columns array([[1, 2, 3], [5, 7, 9]]) >>> np.cumsum(a,axis=1) # sum over columns for each of the 2 rows array([[ 1, 3, 6], [ 4, 9, 15]])

dask.array.
deg2rad
(x[, out])¶ Convert angles from degrees to radians.
Parameters: x : array_like
Angles in degrees.
Returns: y : ndarray
The corresponding angle in radians.
See also
rad2deg
 Convert angles from radians to degrees.
unwrap
 Remove large jumps in angle by wrapping.
Notes
New in version 1.3.0.
deg2rad(x)
isx * pi / 180
.Examples
>>> np.deg2rad(180) 3.1415926535897931

dask.array.
degrees
(x[, out])¶ Convert angles from radians to degrees.
Parameters: x : array_like
Input array in radians.
out : ndarray, optional
Output array of same shape as x.
Returns: y : ndarray of floats
The corresponding degree values; if out was supplied this is a reference to it.
See also
rad2deg
 equivalent function
Examples
Convert a radian array to degrees
>>> rad = np.arange(12.)*np.pi/6 >>> np.degrees(rad) array([ 0., 30., 60., 90., 120., 150., 180., 210., 240., 270., 300., 330.])
>>> out = np.zeros((rad.shape)) >>> r = degrees(rad, out) >>> np.all(r == out) True

dask.array.
diag
(v)¶ Extract a diagonal or construct a diagonal array.
See the more detailed documentation for
numpy.diagonal
if you use this function to extract a diagonal and wish to write to the resulting array; whether it returns a copy or a view depends on what version of numpy you are using.Parameters: v : array_like
If v is a 2D array, return a copy of its kth diagonal. If v is a 1D array, return a 2D array with v on the kth diagonal.
k : int, optional
Diagonal in question. The default is 0. Use k>0 for diagonals above the main diagonal, and k<0 for diagonals below the main diagonal.
Returns: out : ndarray
The extracted diagonal or constructed diagonal array.
See also
Examples
>>> x = np.arange(9).reshape((3,3)) >>> x array([[0, 1, 2], [3, 4, 5], [6, 7, 8]])
>>> np.diag(x) array([0, 4, 8]) >>> np.diag(x, k=1) array([1, 5]) >>> np.diag(x, k=1) array([3, 7])
>>> np.diag(np.diag(x)) array([[0, 0, 0], [0, 4, 0], [0, 0, 8]])

dask.array.
digitize
(x, bins, right=False)¶ Return the indices of the bins to which each value in input array belongs.
Each index
i
returned is such thatbins[i1] <= x < bins[i]
if bins is monotonically increasing, orbins[i1] > x >= bins[i]
if bins is monotonically decreasing. If values in x are beyond the bounds of bins, 0 orlen(bins)
is returned as appropriate. If right is True, then the right bin is closed so that the indexi
is such thatbins[i1] < x <= bins[i]
or bins[i1] >= x > bins[i]`` if bins is monotonically increasing or decreasing, respectively.Parameters: x : array_like
Input array to be binned. It has to be 1dimensional.
bins : array_like
Array of bins. It has to be 1dimensional and monotonic.
right : bool, optional
Indicating whether the intervals include the right or the left bin edge. Default behavior is (right==False) indicating that the interval does not include the right edge. The left bin and is open in this case. Ie., bins[i1] <= x < bins[i] is the default behavior for monotonically increasing bins.
Returns: out : ndarray of ints
Output array of indices, of same shape as x.
Raises: ValueError
If the input is not 1dimensional, or if bins is not monotonic.
TypeError
If the type of the input is complex.
Notes
If values in x are such that they fall outside the bin range, attempting to index bins with the indices that digitize returns will result in an IndexError.
Examples
>>> x = np.array([0.2, 6.4, 3.0, 1.6]) >>> bins = np.array([0.0, 1.0, 2.5, 4.0, 10.0]) >>> inds = np.digitize(x, bins) >>> inds array([1, 4, 3, 2]) >>> for n in range(x.size): ... print bins[inds[n]1], "<=", x[n], "<", bins[inds[n]] ... 0.0 <= 0.2 < 1.0 4.0 <= 6.4 < 10.0 2.5 <= 3.0 < 4.0 1.0 <= 1.6 < 2.5
>>> x = np.array([1.2, 10.0, 12.4, 15.5, 20.]) >>> bins = np.array([0,5,10,15,20]) >>> np.digitize(x,bins,right=True) array([1, 2, 3, 4, 4]) >>> np.digitize(x,bins,right=False) array([1, 3, 3, 4, 5])

dask.array.
dot
(a, b, out=None)¶ Dot product of two arrays.
For 2D arrays it is equivalent to matrix multiplication, and for 1D arrays to inner product of vectors (without complex conjugation). For N dimensions it is a sum product over the last axis of a and the secondtolast of b:
dot(a, b)[i,j,k,m] = sum(a[i,j,:] * b[k,:,m])
Parameters: a : array_like
First argument.
b : array_like
Second argument.
out : ndarray, optional
Output argument. This must have the exact kind that would be returned if it was not used. In particular, it must have the right type, must be Ccontiguous, and its dtype must be the dtype that would be returned for dot(a,b). This is a performance feature. Therefore, if these conditions are not met, an exception is raised, instead of attempting to be flexible.
Returns: output : ndarray
Returns the dot product of a and b. If a and b are both scalars or both 1D arrays then a scalar is returned; otherwise an array is returned. If out is given, then it is returned.
Raises: ValueError
If the last dimension of a is not the same size as the secondtolast dimension of b.
See also
vdot
 Complexconjugating dot product.
tensordot
 Sum products over arbitrary axes.
einsum
 Einstein summation convention.
Examples
>>> np.dot(3, 4) 12
Neither argument is complexconjugated:
>>> np.dot([2j, 3j], [2j, 3j]) (13+0j)
For 2D arrays it’s the matrix product:
>>> a = [[1, 0], [0, 1]] >>> b = [[4, 1], [2, 2]] >>> np.dot(a, b) array([[4, 1], [2, 2]])
>>> a = np.arange(3*4*5*6).reshape((3,4,5,6)) >>> b = np.arange(3*4*5*6)[::1].reshape((5,4,6,3)) >>> np.dot(a, b)[2,3,2,1,2,2] 499128 >>> sum(a[2,3,2,:] * b[1,2,:,2]) 499128

dask.array.
dstack
(tup)¶ Stack arrays in sequence depth wise (along third axis).
Takes a sequence of arrays and stack them along the third axis to make a single array. Rebuilds arrays divided by dsplit. This is a simple way to stack 2D arrays (images) into a single 3D array for processing.
Parameters: tup : sequence of arrays
Arrays to stack. All of them must have the same shape along all but the third axis.
Returns: stacked : ndarray
The array formed by stacking the given arrays.
See also
vstack
 Stack along first axis.
hstack
 Stack along second axis.
concatenate
 Join arrays.
dsplit
 Split array along third axis.
Notes
Equivalent to
np.concatenate(tup, axis=2)
.Examples
>>> a = np.array((1,2,3)) >>> b = np.array((2,3,4)) >>> np.dstack((a,b)) array([[[1, 2], [2, 3], [3, 4]]])
>>> a = np.array([[1],[2],[3]]) >>> b = np.array([[2],[3],[4]]) >>> np.dstack((a,b)) array([[[1, 2]], [[2, 3]], [[3, 4]]])

dask.array.
empty
()¶ Blocked variant of empty
Follows the signature of empty exactly except that it also requires a keyword argument chunks=(...)
Original signature follows below. empty(shape, dtype=float, order=’C’)
Return a new array of given shape and type, without initializing entries.
Parameters: shape : int or tuple of int
Shape of the empty array
dtype : datatype, optional
Desired output datatype.
order : {‘C’, ‘F’}, optional
Whether to store multidimensional data in C (rowmajor) or Fortran (columnmajor) order in memory.
Notes
empty, unlike zeros, does not set the array values to zero, and may therefore be marginally faster. On the other hand, it requires the user to manually set all the values in the array, and should be used with caution.
Examples
>>> np.empty([2, 2]) array([[ 9.74499359e+001, 6.69583040e309], [ 2.13182611e314, 3.06959433e309]]) #random
>>> np.empty([2, 2], dtype=int) array([[1073741821, 1067949133], [ 496041986, 19249760]]) #random

dask.array.
exp
(x[, out])¶ Calculate the exponential of all elements in the input array.
Parameters: x : array_like
Input values.
Returns: out : ndarray
Output array, elementwise exponential of x.
See also
expm1
 Calculate
exp(x)  1
for all elements in the array. exp2
 Calculate
2**x
for all elements in the array.
Notes
The irrational number
e
is also known as Euler’s number. It is approximately 2.718281, and is the base of the natural logarithm,ln
(this means that, if \(x = \ln y = \log_e y\), then \(e^x = y\). For real input,exp(x)
is always positive.For complex arguments,
x = a + ib
, we can write \(e^x = e^a e^{ib}\). The first term, \(e^a\), is already known (it is the real argument, described above). The second term, \(e^{ib}\), is \(\cos b + i \sin b\), a function with magnitude 1 and a periodic phase.References
[R85] Wikipedia, “Exponential function”, http://en.wikipedia.org/wiki/Exponential_function [R86] M. Abramovitz and I. A. Stegun, “Handbook of Mathematical Functions with Formulas, Graphs, and Mathematical Tables,” Dover, 1964, p. 69, http://www.math.sfu.ca/~cbm/aands/page_69.htm Examples
Plot the magnitude and phase of
exp(x)
in the complex plane:>>> import matplotlib.pyplot as plt
>>> x = np.linspace(2*np.pi, 2*np.pi, 100) >>> xx = x + 1j * x[:, np.newaxis] # a + ib over complex plane >>> out = np.exp(xx)
>>> plt.subplot(121) >>> plt.imshow(np.abs(out), ... extent=[2*np.pi, 2*np.pi, 2*np.pi, 2*np.pi]) >>> plt.title('Magnitude of exp(x)')
>>> plt.subplot(122) >>> plt.imshow(np.angle(out), ... extent=[2*np.pi, 2*np.pi, 2*np.pi, 2*np.pi]) >>> plt.title('Phase (angle) of exp(x)') >>> plt.show()

dask.array.
expm1
(x[, out])¶ Calculate
exp(x)  1
for all elements in the array.Parameters: x : array_like
Input values.
Returns: out : ndarray
Elementwise exponential minus one:
out = exp(x)  1
.See also
log1p
log(1 + x)
, the inverse of expm1.
Notes
This function provides greater precision than
exp(x)  1
for small values ofx
.Examples
The true value of
exp(1e10)  1
is1.00000000005e10
to about 32 significant digits. This example shows the superiority of expm1 in this case.>>> np.expm1(1e10) 1.00000000005e10 >>> np.exp(1e10)  1 1.000000082740371e10

dask.array.
eye
(N, chunks, M=None, k=0, dtype=<type 'float'>)¶ Return a 2D Array with ones on the diagonal and zeros elsewhere.
Parameters: N : int
Number of rows in the output.
chunks: int
chunk size of resulting blocks
M : int, optional
Number of columns in the output. If None, defaults to N.
k : int, optional
Index of the diagonal: 0 (the default) refers to the main diagonal, a positive value refers to an upper diagonal, and a negative value to a lower diagonal.
dtype : datatype, optional
Datatype of the returned array.
Returns: I : Array of shape (N,M)
An array where all elements are equal to zero, except for the kth diagonal, whose values are equal to one.

dask.array.
fabs
(x[, out])¶ Compute the absolute values elementwise.
This function returns the absolute values (positive magnitude) of the data in x. Complex values are not handled, use absolute to find the absolute values of complex data.
Parameters: x : array_like
The array of numbers for which the absolute values are required. If x is a scalar, the result y will also be a scalar.
out : ndarray, optional
Array into which the output is placed. Its type is preserved and it must be of the right shape to hold the output. See doc.ufuncs.
Returns: y : {ndarray, scalar}
The absolute values of x, the returned values are always floats.
See also
absolute
 Absolute values including complex types.
Examples
>>> np.fabs(1) 1.0 >>> np.fabs([1.2, 1.2]) array([ 1.2, 1.2])

dask.array.
fix
(*args, **kwargs)¶ Round to nearest integer towards zero.
Round an array of floats elementwise to nearest integer towards zero. The rounded values are returned as floats.
Parameters: x : array_like
An array of floats to be rounded
y : ndarray, optional
Output array
Returns: out : ndarray of floats
The array of rounded numbers
Examples
>>> np.fix(3.14) 3.0 >>> np.fix(3) 3.0 >>> np.fix([2.1, 2.9, 2.1, 2.9]) array([ 2., 2., 2., 2.])

dask.array.
floor
(x[, out])¶ Return the floor of the input, elementwise.
The floor of the scalar x is the largest integer i, such that i <= x. It is often denoted as \(\lfloor x \rfloor\).
Parameters: x : array_like
Input data.
Returns: y : {ndarray, scalar}
The floor of each element in x.
Notes
Some spreadsheet programs calculate the “floortowardszero”, in other words
floor(2.5) == 2
. NumPy instead uses the definition of floor where floor(2.5) == 3.Examples
>>> a = np.array([1.7, 1.5, 0.2, 0.2, 1.5, 1.7, 2.0]) >>> np.floor(a) array([2., 2., 1., 0., 1., 1., 2.])

dask.array.
fmax
(x1, x2[, out])¶ Elementwise maximum of array elements.
Compare two arrays and returns a new array containing the elementwise maxima. If one of the elements being compared is a NaN, then the nonnan element is returned. If both elements are NaNs then the first is returned. The latter distinction is important for complex NaNs, which are defined as at least one of the real or imaginary parts being a NaN. The net effect is that NaNs are ignored when possible.
Parameters: x1, x2 : array_like
The arrays holding the elements to be compared. They must have the same shape.
Returns: y : {ndarray, scalar}
The minimum of x1 and x2, elementwise. Returns scalar if both x1 and x2 are scalars.
See also
Notes
New in version 1.3.0.
The fmax is equivalent to
np.where(x1 >= x2, x1, x2)
when neither x1 nor x2 are NaNs, but it is faster and does proper broadcasting.Examples
>>> np.fmax([2, 3, 4], [1, 5, 2]) array([ 2., 5., 4.])
>>> np.fmax(np.eye(2), [0.5, 2]) array([[ 1. , 2. ], [ 0.5, 2. ]])
>>> np.fmax([np.nan, 0, np.nan],[0, np.nan, np.nan]) array([ 0., 0., NaN])

dask.array.
fmin
(x1, x2[, out])¶ Elementwise minimum of array elements.
Compare two arrays and returns a new array containing the elementwise minima. If one of the elements being compared is a NaN, then the nonnan element is returned. If both elements are NaNs then the first is returned. The latter distinction is important for complex NaNs, which are defined as at least one of the real or imaginary parts being a NaN. The net effect is that NaNs are ignored when possible.
Parameters: x1, x2 : array_like
The arrays holding the elements to be compared. They must have the same shape.
Returns: y : {ndarray, scalar}
The minimum of x1 and x2, elementwise. Returns scalar if both x1 and x2 are scalars.
See also
Notes
New in version 1.3.0.
The fmin is equivalent to
np.where(x1 <= x2, x1, x2)
when neither x1 nor x2 are NaNs, but it is faster and does proper broadcasting.Examples
>>> np.fmin([2, 3, 4], [1, 5, 2]) array([2, 5, 4])
>>> np.fmin(np.eye(2), [0.5, 2]) array([[ 1. , 2. ], [ 0.5, 2. ]])
>>> np.fmin([np.nan, 0, np.nan],[0, np.nan, np.nan]) array([ 0., 0., NaN])

dask.array.
fmod
(x1, x2[, out])¶ Return the elementwise remainder of division.
This is the NumPy implementation of the C library function fmod, the remainder has the same sign as the dividend x1. It is equivalent to the Matlab(TM)
rem
function and should not be confused with the Python modulus operatorx1 % x2
.Parameters: x1 : array_like
Dividend.
x2 : array_like
Divisor.
Returns: y : array_like
The remainder of the division of x1 by x2.
See also
remainder
 Equivalent to the Python
%
operator.
divide
Notes
The result of the modulo operation for negative dividend and divisors is bound by conventions. For fmod, the sign of result is the sign of the dividend, while for remainder the sign of the result is the sign of the divisor. The fmod function is equivalent to the Matlab(TM)
rem
function.Examples
>>> np.fmod([3, 2, 1, 1, 2, 3], 2) array([1, 0, 1, 1, 0, 1]) >>> np.remainder([3, 2, 1, 1, 2, 3], 2) array([1, 0, 1, 1, 0, 1])
>>> np.fmod([5, 3], [2, 2.]) array([ 1., 1.]) >>> a = np.arange(3, 3).reshape(3, 2) >>> a array([[3, 2], [1, 0], [ 1, 2]]) >>> np.fmod(a, [2,2]) array([[1, 0], [1, 0], [ 1, 0]])

dask.array.
frexp
(x[, out1, out2])¶ Decompose the elements of x into mantissa and twos exponent.
Returns (mantissa, exponent), where x = mantissa * 2**exponent`. The mantissa is lies in the open interval(1, 1), while the twos exponent is a signed integer.
Parameters: x : array_like
Array of numbers to be decomposed.
out1: ndarray, optional
Output array for the mantissa. Must have the same shape as x.
out2: ndarray, optional
Output array for the exponent. Must have the same shape as x.
Returns: (mantissa, exponent) : tuple of ndarrays, (float, int)
mantissa is a float array with values between 1 and 1. exponent is an int array which represents the exponent of 2.
See also
ldexp
 Compute
y = x1 * 2**x2
, the inverse of frexp.
Notes
Complex dtypes are not supported, they will raise a TypeError.
Examples
>>> x = np.arange(9) >>> y1, y2 = np.frexp(x) >>> y1 array([ 0. , 0.5 , 0.5 , 0.75 , 0.5 , 0.625, 0.75 , 0.875, 0.5 ]) >>> y2 array([0, 1, 2, 2, 3, 3, 3, 3, 4]) >>> y1 * 2**y2 array([ 0., 1., 2., 3., 4., 5., 6., 7., 8.])

dask.array.
fromfunction
(func, chunks=None, shape=None, dtype=None)¶ Construct an array by executing a function over each coordinate.
The resulting array therefore has a value
fn(x, y, z)
at coordinate(x, y, z)
.Parameters: function : callable
The function is called with N parameters, where N is the rank of shape. Each parameter represents the coordinates of the array varying along a specific axis. For example, if shape were
(2, 2)
, then the parameters in turn be (0, 0), (0, 1), (1, 0), (1, 1).shape : (N,) tuple of ints
Shape of the output array, which also determines the shape of the coordinate arrays passed to function.
dtype : datatype, optional
Datatype of the coordinate arrays passed to function. By default, dtype is float.
Returns: fromfunction : any
The result of the call to function is passed back directly. Therefore the shape of fromfunction is completely determined by function. If function returns a scalar value, the shape of fromfunction would match the shape parameter.
See also
indices
,meshgrid
Notes
Keywords other than dtype are passed to function.
Examples
>>> np.fromfunction(lambda i, j: i == j, (3, 3), dtype=int) array([[ True, False, False], [False, True, False], [False, False, True]], dtype=bool)
>>> np.fromfunction(lambda i, j: i + j, (3, 3), dtype=int) array([[0, 1, 2], [1, 2, 3], [2, 3, 4]])

dask.array.
full
(*args, **kwargs)¶ Blocked variant of full
Follows the signature of full exactly except that it also requires a keyword argument chunks=(...)
Original signature follows below.
Return a new array of given shape and type, filled with fill_value.
Parameters: shape : int or sequence of ints
Shape of the new array, e.g.,
(2, 3)
or2
.fill_value : scalar
Fill value.
dtype : datatype, optional
The desired datatype for the array, e.g., numpy.int8. Default is is chosen as np.array(fill_value).dtype.
order : {‘C’, ‘F’}, optional
Whether to store multidimensional data in C or Fortrancontiguous (row or columnwise) order in memory.
Returns: out : ndarray
Array of fill_value with the given shape, dtype, and order.
See also
zeros_like
 Return an array of zeros with shape and type of input.
ones_like
 Return an array of ones with shape and type of input.
empty_like
 Return an empty array with shape and type of input.
full_like
 Fill an array with shape and type of input.
zeros
 Return a new array setting values to zero.
ones
 Return a new array setting values to one.
empty
 Return a new uninitialized array.
Examples
>>> np.full((2, 2), np.inf) array([[ inf, inf], [ inf, inf]]) >>> np.full((2, 2), 10, dtype=np.int) array([[10, 10], [10, 10]])

dask.array.
histogram
(a, bins=None, range=None, normed=False, weights=None, density=None)¶ Blocked variant of numpy.histogram.
Follows the signature of numpy.histogram exactly with the following exceptions:
 Either an iterable specifying the
bins
or the number ofbins
and arange
argument is required as computingmin
andmax
over blocked arrays is an expensive operation that must be performed explicitly. weights
must be a dask.array.Array with the same block structure asa
.
Examples
Using number of bins and range:
>>> import dask.array as da >>> import numpy as np >>> x = da.from_array(np.arange(10000), chunks=10) >>> h, bins = da.histogram(x, bins=10, range=[0, 10000]) >>> bins array([ 0., 1000., 2000., 3000., 4000., 5000., 6000., 7000., 8000., 9000., 10000.]) >>> h.compute() array([1000, 1000, 1000, 1000, 1000, 1000, 1000, 1000, 1000, 1000])
Explicitly specifying the bins:
>>> h, bins = da.histogram(x, bins=np.array([0, 5000, 10000])) >>> bins array([ 0, 5000, 10000]) >>> h.compute() array([5000, 5000])
 Either an iterable specifying the

dask.array.
hstack
(tup)¶ Stack arrays in sequence horizontally (column wise).
Take a sequence of arrays and stack them horizontally to make a single array. Rebuild arrays divided by hsplit.
Parameters: tup : sequence of ndarrays
All arrays must have the same shape along all but the second axis.
Returns: stacked : ndarray
The array formed by stacking the given arrays.
See also
vstack
 Stack arrays in sequence vertically (row wise).
dstack
 Stack arrays in sequence depth wise (along third axis).
concatenate
 Join a sequence of arrays together.
hsplit
 Split array along second axis.
Notes
Equivalent to
np.concatenate(tup, axis=1)
Examples
>>> a = np.array((1,2,3)) >>> b = np.array((2,3,4)) >>> np.hstack((a,b)) array([1, 2, 3, 2, 3, 4]) >>> a = np.array([[1],[2],[3]]) >>> b = np.array([[2],[3],[4]]) >>> np.hstack((a,b)) array([[1, 2], [2, 3], [3, 4]])

dask.array.
hypot
(x1, x2[, out])¶ Given the “legs” of a right triangle, return its hypotenuse.
Equivalent to
sqrt(x1**2 + x2**2)
, elementwise. If x1 or x2 is scalar_like (i.e., unambiguously castable to a scalar type), it is broadcast for use with each element of the other argument. (See Examples)Parameters: x1, x2 : array_like
Leg of the triangle(s).
out : ndarray, optional
Array into which the output is placed. Its type is preserved and it must be of the right shape to hold the output. See doc.ufuncs.
Returns: z : ndarray
The hypotenuse of the triangle(s).
Examples
>>> np.hypot(3*np.ones((3, 3)), 4*np.ones((3, 3))) array([[ 5., 5., 5.], [ 5., 5., 5.], [ 5., 5., 5.]])
Example showing broadcast of scalar_like argument:
>>> np.hypot(3*np.ones((3, 3)), [4]) array([[ 5., 5., 5.], [ 5., 5., 5.], [ 5., 5., 5.]])

dask.array.
imag
(*args, **kwargs)¶ Return the imaginary part of the elements of the array.
Parameters: val : array_like
Input array.
Returns: out : ndarray
Output array. If val is real, the type of val is used for the output. If val has complex elements, the returned type is float.
Examples
>>> a = np.array([1+2j, 3+4j, 5+6j]) >>> a.imag array([ 2., 4., 6.]) >>> a.imag = np.array([8, 10, 12]) >>> a array([ 1. +8.j, 3.+10.j, 5.+12.j])

dask.array.
insert
(arr, obj, values, axis)¶ Insert values along the given axis before the given indices.
Parameters: arr : array_like
Input array.
obj : int, slice or sequence of ints
Object that defines the index or indices before which values is inserted.
New in version 1.8.0.
Support for multiple insertions when obj is a single scalar or a sequence with one element (similar to calling insert multiple times).
values : array_like
Values to insert into arr. If the type of values is different from that of arr, values is converted to the type of arr. values should be shaped so that
arr[...,obj,...] = values
is legal.axis : int, optional
Axis along which to insert values. If axis is None then arr is flattened first.
Returns: out : ndarray
A copy of arr with values inserted. Note that insert does not occur inplace: a new array is returned. If axis is None, out is a flattened array.
See also
append
 Append elements at the end of an array.
concatenate
 Join a sequence of arrays together.
delete
 Delete elements from an array.
Notes
Note that for higher dimensional inserts obj=0 behaves very different from obj=[0] just like arr[:,0,:] = values is different from arr[:,[0],:] = values.
Examples
>>> a = np.array([[1, 1], [2, 2], [3, 3]]) >>> a array([[1, 1], [2, 2], [3, 3]]) >>> np.insert(a, 1, 5) array([1, 5, 1, 2, 2, 3, 3]) >>> np.insert(a, 1, 5, axis=1) array([[1, 5, 1], [2, 5, 2], [3, 5, 3]])
Difference between sequence and scalars: >>> np.insert(a, [1], [[1],[2],[3]], axis=1) array([[1, 1, 1],
[2, 2, 2], [3, 3, 3]])>>> np.array_equal(np.insert(a, 1, [1, 2, 3], axis=1), ... np.insert(a, [1], [[1],[2],[3]], axis=1)) True
>>> b = a.flatten() >>> b array([1, 1, 2, 2, 3, 3]) >>> np.insert(b, [2, 2], [5, 6]) array([1, 1, 5, 6, 2, 2, 3, 3])
>>> np.insert(b, slice(2, 4), [5, 6]) array([1, 1, 5, 2, 6, 2, 3, 3])
>>> np.insert(b, [2, 2], [7.13, False]) # type casting array([1, 1, 7, 0, 2, 2, 3, 3])
>>> x = np.arange(8).reshape(2, 4) >>> idx = (1, 3) >>> np.insert(x, idx, 999, axis=1) array([[ 0, 999, 1, 2, 999, 3], [ 4, 999, 5, 6, 999, 7]])

dask.array.
isclose
(arr1, arr2, rtol=1e05, atol=1e08, equal_nan=False)¶ Returns a boolean array where two arrays are elementwise equal within a tolerance.
The tolerance values are positive, typically very small numbers. The relative difference (rtol * abs(b)) and the absolute difference atol are added together to compare against the absolute difference between a and b.
Parameters: a, b : array_like
Input arrays to compare.
rtol : float
The relative tolerance parameter (see Notes).
atol : float
The absolute tolerance parameter (see Notes).
equal_nan : bool
Whether to compare NaN’s as equal. If True, NaN’s in a will be considered equal to NaN’s in b in the output array.
Returns: y : array_like
Returns a boolean array of where a and b are equal within the given tolerance. If both a and b are scalars, returns a single boolean value.
See also
allclose
Notes
New in version 1.7.0.
For finite values, isclose uses the following equation to test whether two floating point values are equivalent.
absolute(a  b) <= (atol + rtol * absolute(b))The above equation is not symmetric in a and b, so that isclose(a, b) might be different from isclose(b, a) in some rare cases.
Examples
>>> np.isclose([1e10,1e7], [1.00001e10,1e8]) array([True, False]) >>> np.isclose([1e10,1e8], [1.00001e10,1e9]) array([True, True]) >>> np.isclose([1e10,1e8], [1.0001e10,1e9]) array([False, True]) >>> np.isclose([1.0, np.nan], [1.0, np.nan]) array([True, False]) >>> np.isclose([1.0, np.nan], [1.0, np.nan], equal_nan=True) array([True, True])

dask.array.
iscomplex
(*args, **kwargs)¶ Returns a bool array, where True if input element is complex.
What is tested is whether the input has a nonzero imaginary part, not if the input type is complex.
Parameters: x : array_like
Input array.
Returns: out : ndarray of bools
Output array.
Examples
>>> np.iscomplex([1+1j, 1+0j, 4.5, 3, 2, 2j]) array([ True, False, False, False, False, True], dtype=bool)

dask.array.
isfinite
(x[, out])¶ Test elementwise for finiteness (not infinity or not Not a Number).
The result is returned as a boolean array.
Parameters: x : array_like
Input values.
out : ndarray, optional
Array into which the output is placed. Its type is preserved and it must be of the right shape to hold the output. See doc.ufuncs.
Returns: y : ndarray, bool
For scalar input, the result is a new boolean with value True if the input is finite; otherwise the value is False (input is either positive infinity, negative infinity or Not a Number).
For array input, the result is a boolean array with the same dimensions as the input and the values are True if the corresponding element of the input is finite; otherwise the values are False (element is either positive infinity, negative infinity or Not a Number).
Notes
Not a Number, positive infinity and negative infinity are considered to be nonfinite.
Numpy uses the IEEE Standard for Binary FloatingPoint for Arithmetic (IEEE 754). This means that Not a Number is not equivalent to infinity. Also that positive infinity is not equivalent to negative infinity. But infinity is equivalent to positive infinity. Errors result if the second argument is also supplied when x is a scalar input, or if first and second arguments have different shapes.
Examples
>>> np.isfinite(1) True >>> np.isfinite(0) True >>> np.isfinite(np.nan) False >>> np.isfinite(np.inf) False >>> np.isfinite(np.NINF) False >>> np.isfinite([np.log(1.),1.,np.log(0)]) array([False, True, False], dtype=bool)
>>> x = np.array([np.inf, 0., np.inf]) >>> y = np.array([2, 2, 2]) >>> np.isfinite(x, y) array([0, 1, 0]) >>> y array([0, 1, 0])

dask.array.
isinf
(x[, out])¶ Test elementwise for positive or negative infinity.
Returns a boolean array of the same shape as x, True where
x == +/inf
, otherwise False.Parameters: x : array_like
Input values
out : array_like, optional
An array with the same shape as x to store the result.
Returns: y : bool (scalar) or boolean ndarray
For scalar input, the result is a new boolean with value True if the input is positive or negative infinity; otherwise the value is False.
For array input, the result is a boolean array with the same shape as the input and the values are True where the corresponding element of the input is positive or negative infinity; elsewhere the values are False. If a second argument was supplied the result is stored there. If the type of that array is a numeric type the result is represented as zeros and ones, if the type is boolean then as False and True, respectively. The return value y is then a reference to that array.
Notes
Numpy uses the IEEE Standard for Binary FloatingPoint for Arithmetic (IEEE 754).
Errors result if the second argument is supplied when the first argument is a scalar, or if the first and second arguments have different shapes.
Examples
>>> np.isinf(np.inf) True >>> np.isinf(np.nan) False >>> np.isinf(np.NINF) True >>> np.isinf([np.inf, np.inf, 1.0, np.nan]) array([ True, True, False, False], dtype=bool)
>>> x = np.array([np.inf, 0., np.inf]) >>> y = np.array([2, 2, 2]) >>> np.isinf(x, y) array([1, 0, 1]) >>> y array([1, 0, 1])

dask.array.
isnan
(x[, out])¶ Test elementwise for NaN and return result as a boolean array.
Parameters: x : array_like
Input array.
Returns: y : {ndarray, bool}
For scalar input, the result is a new boolean with value True if the input is NaN; otherwise the value is False.
For array input, the result is a boolean array of the same dimensions as the input and the values are True if the corresponding element of the input is NaN; otherwise the values are False.
Notes
Numpy uses the IEEE Standard for Binary FloatingPoint for Arithmetic (IEEE 754). This means that Not a Number is not equivalent to infinity.
Examples
>>> np.isnan(np.nan) True >>> np.isnan(np.inf) False >>> np.isnan([np.log(1.),1.,np.log(0)]) array([ True, False, False], dtype=bool)

dask.array.
isnull
(values)¶ pandas.isnull for dask arrays

dask.array.
isreal
(*args, **kwargs)¶ Returns a bool array, where True if input element is real.
If element has complex type with zero complex part, the return value for that element is True.
Parameters: x : array_like
Input array.
Returns: out : ndarray, bool
Boolean array of same shape as x.
Examples
>>> np.isreal([1+1j, 1+0j, 4.5, 3, 2, 2j]) array([False, True, True, True, True, False], dtype=bool)

dask.array.
ldexp
(x1, x2[, out])¶ Returns x1 * 2**x2, elementwise.
The mantissas x1 and twos exponents x2 are used to construct floating point numbers
x1 * 2**x2
.Parameters: x1 : array_like
Array of multipliers.
x2 : array_like, int
Array of twos exponents.
out : ndarray, optional
Output array for the result.
Returns: y : ndarray or scalar
The result of
x1 * 2**x2
.See also
frexp
 Return (y1, y2) from
x = y1 * 2**y2
, inverse to ldexp.
Notes
Complex dtypes are not supported, they will raise a TypeError.
ldexp is useful as the inverse of frexp, if used by itself it is more clear to simply use the expression
x1 * 2**x2
.Examples
>>> np.ldexp(5, np.arange(4)) array([ 5., 10., 20., 40.], dtype=float32)
>>> x = np.arange(6) >>> np.ldexp(*np.frexp(x)) array([ 0., 1., 2., 3., 4., 5.])

dask.array.
linspace
(start, stop, num=50, chunks=None, dtype=None)¶ Return num evenly spaced values over the closed interval [start, stop].
TODO: implement the endpoint, restep, and dtype keyword args
Parameters: start : scalar
The starting value of the sequence.
stop : scalar
The last value of the sequence.
num : int, optional
Number of samples to include in the returned dask array, including the endpoints.
chunks : int
The number of samples on each block. Note that the last block will have fewer samples if num % blocksize != 0
Returns: samples : dask array
See also

dask.array.
log
(x[, out])¶ Natural logarithm, elementwise.
The natural logarithm log is the inverse of the exponential function, so that log(exp(x)) = x. The natural logarithm is logarithm in base e.
Parameters: x : array_like
Input value.
Returns: y : ndarray
The natural logarithm of x, elementwise.
Notes
Logarithm is a multivalued function: for each x there is an infinite number of z such that exp(z) = x. The convention is to return the z whose imaginary part lies in [pi, pi].
For realvalued input data types, log always returns real output. For each value that cannot be expressed as a real number or infinity, it yields
nan
and sets the invalid floating point error flag.For complexvalued input, log is a complex analytical function that has a branch cut [inf, 0] and is continuous from above on it. log handles the floatingpoint negative zero as an infinitesimal negative number, conforming to the C99 standard.
References
[R87] M. Abramowitz and I.A. Stegun, “Handbook of Mathematical Functions”, 10th printing, 1964, pp. 67. http://www.math.sfu.ca/~cbm/aands/ [R88] Wikipedia, “Logarithm”. http://en.wikipedia.org/wiki/Logarithm Examples
>>> np.log([1, np.e, np.e**2, 0]) array([ 0., 1., 2., Inf])

dask.array.
log10
(x[, out])¶ Return the base 10 logarithm of the input array, elementwise.
Parameters: x : array_like
Input values.
Returns: y : ndarray
The logarithm to the base 10 of x, elementwise. NaNs are returned where x is negative.
See also
emath.log10
Notes
Logarithm is a multivalued function: for each x there is an infinite number of z such that 10**z = x. The convention is to return the z whose imaginary part lies in [pi, pi].
For realvalued input data types, log10 always returns real output. For each value that cannot be expressed as a real number or infinity, it yields
nan
and sets the invalid floating point error flag.For complexvalued input, log10 is a complex analytical function that has a branch cut [inf, 0] and is continuous from above on it. log10 handles the floatingpoint negative zero as an infinitesimal negative number, conforming to the C99 standard.
References
[R89] M. Abramowitz and I.A. Stegun, “Handbook of Mathematical Functions”, 10th printing, 1964, pp. 67. http://www.math.sfu.ca/~cbm/aands/ [R90] Wikipedia, “Logarithm”. http://en.wikipedia.org/wiki/Logarithm Examples
>>> np.log10([1e15, 3.]) array([15., NaN])

dask.array.
log1p
(x[, out])¶ Return the natural logarithm of one plus the input array, elementwise.
Calculates
log(1 + x)
.Parameters: x : array_like
Input values.
Returns: y : ndarray
Natural logarithm of 1 + x, elementwise.
See also
expm1
exp(x)  1
, the inverse of log1p.
Notes
For realvalued input, log1p is accurate also for x so small that 1 + x == 1 in floatingpoint accuracy.
Logarithm is a multivalued function: for each x there is an infinite number of z such that exp(z) = 1 + x. The convention is to return the z whose imaginary part lies in [pi, pi].
For realvalued input data types, log1p always returns real output. For each value that cannot be expressed as a real number or infinity, it yields
nan
and sets the invalid floating point error flag.For complexvalued input, log1p is a complex analytical function that has a branch cut [inf, 1] and is continuous from above on it. log1p handles the floatingpoint negative zero as an infinitesimal negative number, conforming to the C99 standard.
References
[R91] M. Abramowitz and I.A. Stegun, “Handbook of Mathematical Functions”, 10th printing, 1964, pp. 67. http://www.math.sfu.ca/~cbm/aands/ [R92] Wikipedia, “Logarithm”. http://en.wikipedia.org/wiki/Logarithm Examples
>>> np.log1p(1e99) 1e99 >>> np.log(1 + 1e99) 0.0

dask.array.
log2
(x[, out])¶ Base2 logarithm of x.
Parameters: x : array_like
Input values.
Returns: y : ndarray
Base2 logarithm of x.
Notes
New in version 1.3.0.
Logarithm is a multivalued function: for each x there is an infinite number of z such that 2**z = x. The convention is to return the z whose imaginary part lies in [pi, pi].
For realvalued input data types, log2 always returns real output. For each value that cannot be expressed as a real number or infinity, it yields
nan
and sets the invalid floating point error flag.For complexvalued input, log2 is a complex analytical function that has a branch cut [inf, 0] and is continuous from above on it. log2 handles the floatingpoint negative zero as an infinitesimal negative number, conforming to the C99 standard.
Examples
>>> x = np.array([0, 1, 2, 2**4]) >>> np.log2(x) array([Inf, 0., 1., 4.])
>>> xi = np.array([0+1.j, 1, 2+0.j, 4.j]) >>> np.log2(xi) array([ 0.+2.26618007j, 0.+0.j , 1.+0.j , 2.+2.26618007j])

dask.array.
logaddexp
(x1, x2[, out])¶ Logarithm of the sum of exponentiations of the inputs.
Calculates
log(exp(x1) + exp(x2))
. This function is useful in statistics where the calculated probabilities of events may be so small as to exceed the range of normal floating point numbers. In such cases the logarithm of the calculated probability is stored. This function allows adding probabilities stored in such a fashion.Parameters: x1, x2 : array_like
Input values.
Returns: result : ndarray
Logarithm of
exp(x1) + exp(x2)
.See also
logaddexp2
 Logarithm of the sum of exponentiations of inputs in base 2.
Notes
New in version 1.3.0.
Examples
>>> prob1 = np.log(1e50) >>> prob2 = np.log(2.5e50) >>> prob12 = np.logaddexp(prob1, prob2) >>> prob12 113.87649168120691 >>> np.exp(prob12) 3.5000000000000057e50

dask.array.
logaddexp2
(x1, x2[, out])¶ Logarithm of the sum of exponentiations of the inputs in base2.
Calculates
log2(2**x1 + 2**x2)
. This function is useful in machine learning when the calculated probabilities of events may be so small as to exceed the range of normal floating point numbers. In such cases the base2 logarithm of the calculated probability can be used instead. This function allows adding probabilities stored in such a fashion.Parameters: x1, x2 : array_like
Input values.
out : ndarray, optional
Array to store results in.
Returns: result : ndarray
Base2 logarithm of
2**x1 + 2**x2
.See also
logaddexp
 Logarithm of the sum of exponentiations of the inputs.
Notes
New in version 1.3.0.
Examples
>>> prob1 = np.log2(1e50) >>> prob2 = np.log2(2.5e50) >>> prob12 = np.logaddexp2(prob1, prob2) >>> prob1, prob2, prob12 (166.09640474436813, 164.77447664948076, 164.28904982231052) >>> 2**prob12 3.4999999999999914e50

dask.array.
logical_and
(x1, x2[, out])¶ Compute the truth value of x1 AND x2 elementwise.
Parameters: x1, x2 : array_like
Input arrays. x1 and x2 must be of the same shape.
Returns: y : {ndarray, bool}
Boolean result with the same shape as x1 and x2 of the logical AND operation on corresponding elements of x1 and x2.
See also
logical_or
,logical_not
,logical_xor
,bitwise_and
Examples
>>> np.logical_and(True, False) False >>> np.logical_and([True, False], [False, False]) array([False, False], dtype=bool)
>>> x = np.arange(5) >>> np.logical_and(x>1, x<4) array([False, False, True, True, False], dtype=bool)

dask.array.
logical_not
(x[, out])¶ Compute the truth value of NOT x elementwise.
Parameters: x : array_like
Logical NOT is applied to the elements of x.
Returns: y : bool or ndarray of bool
Boolean result with the same shape as x of the NOT operation on elements of x.
See also
Examples
>>> np.logical_not(3) False >>> np.logical_not([True, False, 0, 1]) array([False, True, True, False], dtype=bool)
>>> x = np.arange(5) >>> np.logical_not(x<3) array([False, False, False, True, True], dtype=bool)

dask.array.
logical_or
(x1, x2[, out])¶ Compute the truth value of x1 OR x2 elementwise.
Parameters: x1, x2 : array_like
Logical OR is applied to the elements of x1 and x2. They have to be of the same shape.
Returns: y : {ndarray, bool}
Boolean result with the same shape as x1 and x2 of the logical OR operation on elements of x1 and x2.
See also
logical_and
,logical_not
,logical_xor
,bitwise_or
Examples
>>> np.logical_or(True, False) True >>> np.logical_or([True, False], [False, False]) array([ True, False], dtype=bool)
>>> x = np.arange(5) >>> np.logical_or(x < 1, x > 3) array([ True, False, False, False, True], dtype=bool)

dask.array.
logical_xor
(x1, x2[, out])¶ Compute the truth value of x1 XOR x2, elementwise.
Parameters: x1, x2 : array_like
Logical XOR is applied to the elements of x1 and x2. They must be broadcastable to the same shape.
Returns: y : bool or ndarray of bool
Boolean result of the logical XOR operation applied to the elements of x1 and x2; the shape is determined by whether or not broadcasting of one or both arrays was required.
See also
logical_and
,logical_or
,logical_not
,bitwise_xor
Examples
>>> np.logical_xor(True, False) True >>> np.logical_xor([True, True, False, False], [True, False, True, False]) array([False, True, True, False], dtype=bool)
>>> x = np.arange(5) >>> np.logical_xor(x < 1, x > 3) array([ True, False, False, False, True], dtype=bool)
Simple example showing support of broadcasting
>>> np.logical_xor(0, np.eye(2)) array([[ True, False], [False, True]], dtype=bool)

dask.array.
max
(a, axis=None, keepdims=False, split_every=None)¶ Return the maximum of an array or maximum along an axis.
Parameters: a : array_like
Input data.
axis : int, optional
Axis along which to operate. By default, flattened input is used.
out : ndarray, optional
Alternative output array in which to place the result. Must be of the same shape and buffer length as the expected output. See doc.ufuncs (Section “Output arguments”) for more details.
keepdims : bool, optional
If this is set to True, the axes which are reduced are left in the result as dimensions with size one. With this option, the result will broadcast correctly against the original arr.
Returns: amax : ndarray or scalar
Maximum of a. If axis is None, the result is a scalar value. If axis is given, the result is an array of dimension
a.ndim  1
.See also
amin
 The minimum value of an array along a given axis, propagating any NaNs.
nanmax
 The maximum value of an array along a given axis, ignoring any NaNs.
maximum
 Elementwise maximum of two arrays, propagating any NaNs.
fmax
 Elementwise maximum of two arrays, ignoring any NaNs.
argmax
 Return the indices of the maximum values.
Notes
NaN values are propagated, that is if at least one item is NaN, the corresponding max value will be NaN as well. To ignore NaN values (MATLAB behavior), please use nanmax.
Don’t use amax for elementwise comparison of 2 arrays; when
a.shape[0]
is 2,maximum(a[0], a[1])
is faster thanamax(a, axis=0)
.Examples
>>> a = np.arange(4).reshape((2,2)) >>> a array([[0, 1], [2, 3]]) >>> np.amax(a) # Maximum of the flattened array 3 >>> np.amax(a, axis=0) # Maxima along the first axis array([2, 3]) >>> np.amax(a, axis=1) # Maxima along the second axis array([1, 3])
>>> b = np.arange(5, dtype=np.float) >>> b[2] = np.NaN >>> np.amax(b) nan >>> np.nanmax(b) 4.0

dask.array.
maximum
(x1, x2[, out])¶ Elementwise maximum of array elements.
Compare two arrays and returns a new array containing the elementwise maxima. If one of the elements being compared is a NaN, then that element is returned. If both elements are NaNs then the first is returned. The latter distinction is important for complex NaNs, which are defined as at least one of the real or imaginary parts being a NaN. The net effect is that NaNs are propagated.
Parameters: x1, x2 : array_like
The arrays holding the elements to be compared. They must have the same shape, or shapes that can be broadcast to a single shape.
Returns: y : {ndarray, scalar}
The maximum of x1 and x2, elementwise. Returns scalar if both x1 and x2 are scalars.
See also
Notes
The maximum is equivalent to
np.where(x1 >= x2, x1, x2)
when neither x1 nor x2 are nans, but it is faster and does proper broadcasting.Examples
>>> np.maximum([2, 3, 4], [1, 5, 2]) array([2, 5, 4])
>>> np.maximum(np.eye(2), [0.5, 2]) # broadcasting array([[ 1. , 2. ], [ 0.5, 2. ]])
>>> np.maximum([np.nan, 0, np.nan], [0, np.nan, np.nan]) array([ NaN, NaN, NaN]) >>> np.maximum(np.Inf, 1) inf

dask.array.
mean
(a, axis=None, dtype=None, keepdims=False, split_every=None)¶ Compute the arithmetic mean along the specified axis.
Returns the average of the array elements. The average is taken over the flattened array by default, otherwise over the specified axis. float64 intermediate and return values are used for integer inputs.
Parameters: a : array_like
Array containing numbers whose mean is desired. If a is not an array, a conversion is attempted.
axis : int, optional
Axis along which the means are computed. The default is to compute the mean of the flattened array.
dtype : datatype, optional
Type to use in computing the mean. For integer inputs, the default is float64; for floating point inputs, it is the same as the input dtype.
out : ndarray, optional
Alternate output array in which to place the result. The default is
None
; if provided, it must have the same shape as the expected output, but the type will be cast if necessary. See doc.ufuncs for details.keepdims : bool, optional
If this is set to True, the axes which are reduced are left in the result as dimensions with size one. With this option, the result will broadcast correctly against the original arr.
Returns: m : ndarray, see dtype parameter above
If out=None, returns a new array containing the mean values, otherwise a reference to the output array is returned.
Notes
The arithmetic mean is the sum of the elements along the axis divided by the number of elements.
Note that for floatingpoint input, the mean is computed using the same precision the input has. Depending on the input data, this can cause the results to be inaccurate, especially for float32 (see example below). Specifying a higherprecision accumulator using the dtype keyword can alleviate this issue.
Examples
>>> a = np.array([[1, 2], [3, 4]]) >>> np.mean(a) 2.5 >>> np.mean(a, axis=0) array([ 2., 3.]) >>> np.mean(a, axis=1) array([ 1.5, 3.5])
In single precision, mean can be inaccurate:
>>> a = np.zeros((2, 512*512), dtype=np.float32) >>> a[0, :] = 1.0 >>> a[1, :] = 0.1 >>> np.mean(a) 0.546875
Computing the mean in float64 is more accurate:
>>> np.mean(a, dtype=np.float64) 0.55000000074505806

dask.array.
min
(a, axis=None, keepdims=False, split_every=None)¶ Return the minimum of an array or minimum along an axis.
Parameters: a : array_like
Input data.
axis : int, optional
Axis along which to operate. By default, flattened input is used.
out : ndarray, optional
Alternative output array in which to place the result. Must be of the same shape and buffer length as the expected output. See doc.ufuncs (Section “Output arguments”) for more details.
keepdims : bool, optional
If this is set to True, the axes which are reduced are left in the result as dimensions with size one. With this option, the result will broadcast correctly against the original arr.
Returns: amin : ndarray or scalar
Minimum of a. If axis is None, the result is a scalar value. If axis is given, the result is an array of dimension
a.ndim  1
.See also
amax
 The maximum value of an array along a given axis, propagating any NaNs.
nanmin
 The minimum value of an array along a given axis, ignoring any NaNs.
minimum
 Elementwise minimum of two arrays, propagating any NaNs.
fmin
 Elementwise minimum of two arrays, ignoring any NaNs.
argmin
 Return the indices of the minimum values.
Notes
NaN values are propagated, that is if at least one item is NaN, the corresponding min value will be NaN as well. To ignore NaN values (MATLAB behavior), please use nanmin.
Don’t use amin for elementwise comparison of 2 arrays; when
a.shape[0]
is 2,minimum(a[0], a[1])
is faster thanamin(a, axis=0)
.Examples
>>> a = np.arange(4).reshape((2,2)) >>> a array([[0, 1], [2, 3]]) >>> np.amin(a) # Minimum of the flattened array 0 >>> np.amin(a, axis=0) # Minima along the first axis array([0, 1]) >>> np.amin(a, axis=1) # Minima along the second axis array([0, 2])
>>> b = np.arange(5, dtype=np.float) >>> b[2] = np.NaN >>> np.amin(b) nan >>> np.nanmin(b) 0.0

dask.array.
minimum
(x1, x2[, out])¶ Elementwise minimum of array elements.
Compare two arrays and returns a new array containing the elementwise minima. If one of the elements being compared is a NaN, then that element is returned. If both elements are NaNs then the first is returned. The latter distinction is important for complex NaNs, which are defined as at least one of the real or imaginary parts being a NaN. The net effect is that NaNs are propagated.
Parameters: x1, x2 : array_like
The arrays holding the elements to be compared. They must have the same shape, or shapes that can be broadcast to a single shape.
Returns: y : {ndarray, scalar}
The minimum of x1 and x2, elementwise. Returns scalar if both x1 and x2 are scalars.
See also
Notes
The minimum is equivalent to
np.where(x1 <= x2, x1, x2)
when neither x1 nor x2 are NaNs, but it is faster and does proper broadcasting.Examples
>>> np.minimum([2, 3, 4], [1, 5, 2]) array([1, 3, 2])
>>> np.minimum(np.eye(2), [0.5, 2]) # broadcasting array([[ 0.5, 0. ], [ 0. , 1. ]])
>>> np.minimum([np.nan, 0, np.nan],[0, np.nan, np.nan]) array([ NaN, NaN, NaN]) >>> np.minimum(np.Inf, 1) inf

dask.array.
modf
(x[, out1, out2])¶ Return the fractional and integral parts of an array, elementwise.
The fractional and integral parts are negative if the given number is negative.
Parameters: x : array_like
Input array.
Returns: y1 : ndarray
Fractional part of x.
y2 : ndarray
Integral part of x.
Notes
For integer input the return values are floats.
Examples
>>> np.modf([0, 3.5]) (array([ 0. , 0.5]), array([ 0., 3.])) >>> np.modf(0.5) (0.5, 0)

dask.array.
moment
(a, order, axis=None, dtype=None, keepdims=False, ddof=0, split_every=None)¶

dask.array.
nanargmax
(x, axis=None, split_every=None)¶

dask.array.
nanargmin
(x, axis=None, split_every=None)¶

dask.array.
nancumprod
(x, axis, dtype=None)¶ Return the cumulative product of array elements over a given axis treating Not a Numbers (NaNs) as one. The cumulative product does not change when NaNs are encountered and leading NaNs are replaced by ones.
Ones are returned for slices that are allNaN or empty.
New in version 1.12.0.
Parameters: a : array_like
Input array.
axis : int, optional
Axis along which the cumulative product is computed. By default the input is flattened.
dtype : dtype, optional
Type of the returned array, as well as of the accumulator in which the elements are multiplied. If dtype is not specified, it defaults to the dtype of a, unless a has an integer dtype with a precision less than that of the default platform integer. In that case, the default platform integer is used instead.
out : ndarray, optional
Alternative output array in which to place the result. It must have the same shape and buffer length as the expected output but the type of the resulting values will be cast if necessary.
Returns: nancumprod : ndarray
A new array holding the result is returned unless out is specified, in which case it is returned.
See also
numpy.cumprod
 Cumulative product across array propagating NaNs.
isnan
 Show which elements are NaN.
Examples
>>> np.nancumprod(1) array([1]) >>> np.nancumprod([1]) array([1]) >>> np.nancumprod([1, np.nan]) array([ 1., 1.]) >>> a = np.array([[1, 2], [3, np.nan]]) >>> np.nancumprod(a) array([ 1., 2., 6., 6.]) >>> np.nancumprod(a, axis=0) array([[ 1., 2.], [ 3., 2.]]) >>> np.nancumprod(a, axis=1) array([[ 1., 2.], [ 3., 3.]])

dask.array.
nancumsum
(x, axis, dtype=None)¶ Return the cumulative sum of array elements over a given axis treating Not a Numbers (NaNs) as zero. The cumulative sum does not change when NaNs are encountered and leading NaNs are replaced by zeros.
Zeros are returned for slices that are allNaN or empty.
New in version 1.12.0.
Parameters: a : array_like
Input array.
axis : int, optional
Axis along which the cumulative sum is computed. The default (None) is to compute the cumsum over the flattened array.
dtype : dtype, optional
Type of the returned array and of the accumulator in which the elements are summed. If dtype is not specified, it defaults to the dtype of a, unless a has an integer dtype with a precision less than that of the default platform integer. In that case, the default platform integer is used.
out : ndarray, optional
Alternative output array in which to place the result. It must have the same shape and buffer length as the expected output but the type will be cast if necessary. See doc.ufuncs (Section “Output arguments”) for more details.
Returns: nancumsum : ndarray.
A new array holding the result is returned unless out is specified, in which it is returned. The result has the same size as a, and the same shape as a if axis is not None or a is a 1d array.
See also
numpy.cumsum
 Cumulative sum across array propagating NaNs.
isnan
 Show which elements are NaN.
Examples
>>> np.nancumsum(1) array([1]) >>> np.nancumsum([1]) array([1]) >>> np.nancumsum([1, np.nan]) array([ 1., 1.]) >>> a = np.array([[1, 2], [3, np.nan]]) >>> np.nancumsum(a) array([ 1., 3., 6., 6.]) >>> np.nancumsum(a, axis=0) array([[ 1., 2.], [ 4., 2.]]) >>> np.nancumsum(a, axis=1) array([[ 1., 3.], [ 3., 3.]])

dask.array.
nanmax
(a, axis=None, keepdims=False, split_every=None)¶ Return the maximum of an array or maximum along an axis, ignoring any NaNs. When allNaN slices are encountered a
RuntimeWarning
is raised and NaN is returned for that slice.Parameters: a : array_like
Array containing numbers whose maximum is desired. If a is not an array, a conversion is attempted.
axis : int, optional
Axis along which the maximum is computed. The default is to compute the maximum of the flattened array.
out : ndarray, optional
Alternate output array in which to place the result. The default is
None
; if provided, it must have the same shape as the expected output, but the type will be cast if necessary. See doc.ufuncs for details.New in version 1.8.0.
keepdims : bool, optional
If this is set to True, the axes which are reduced are left in the result as dimensions with size one. With this option, the result will broadcast correctly against the original a.
New in version 1.8.0.
Returns: nanmax : ndarray
An array with the same shape as a, with the specified axis removed. If a is a 0d array, or if axis is None, an ndarray scalar is returned. The same dtype as a is returned.
See also
nanmin
 The minimum value of an array along a given axis, ignoring any NaNs.
amax
 The maximum value of an array along a given axis, propagating any NaNs.
fmax
 Elementwise maximum of two arrays, ignoring any NaNs.
maximum
 Elementwise maximum of two arrays, propagating any NaNs.
isnan
 Shows which elements are Not a Number (NaN).
isfinite
 Shows which elements are neither NaN nor infinity.
Notes
Numpy uses the IEEE Standard for Binary FloatingPoint for Arithmetic (IEEE 754). This means that Not a Number is not equivalent to infinity. Positive infinity is treated as a very large number and negative infinity is treated as a very small (i.e. negative) number.
If the input has a integer type the function is equivalent to np.max.
Examples
>>> a = np.array([[1, 2], [3, np.nan]]) >>> np.nanmax(a) 3.0 >>> np.nanmax(a, axis=0) array([ 3., 2.]) >>> np.nanmax(a, axis=1) array([ 2., 3.])
When positive infinity and negative infinity are present:
>>> np.nanmax([1, 2, np.nan, np.NINF]) 2.0 >>> np.nanmax([1, 2, np.nan, np.inf]) inf

dask.array.
nanmean
(a, axis=None, dtype=None, keepdims=False, split_every=None)¶ Compute the arithmetic mean along the specified axis, ignoring NaNs.
Returns the average of the array elements. The average is taken over the flattened array by default, otherwise over the specified axis. float64 intermediate and return values are used for integer inputs.
For allNaN slices, NaN is returned and a RuntimeWarning is raised.
New in version 1.8.0.
Parameters: a : array_like
Array containing numbers whose mean is desired. If a is not an array, a conversion is attempted.
axis : int, optional
Axis along which the means are computed. The default is to compute the mean of the flattened array.
dtype : datatype, optional
Type to use in computing the mean. For integer inputs, the default is float64; for inexact inputs, it is the same as the input dtype.
out : ndarray, optional
Alternate output array in which to place the result. The default is
None
; if provided, it must have the same shape as the expected output, but the type will be cast if necessary. See doc.ufuncs for details.keepdims : bool, optional
If this is set to True, the axes which are reduced are left in the result as dimensions with size one. With this option, the result will broadcast correctly against the original arr.
Returns: m : ndarray, see dtype parameter above
If out=None, returns a new array containing the mean values, otherwise a reference to the output array is returned. Nan is returned for slices that contain only NaNs.
Notes
The arithmetic mean is the sum of the nonNaN elements along the axis divided by the number of nonNaN elements.
Note that for floatingpoint input, the mean is computed using the same precision the input has. Depending on the input data, this can cause the results to be inaccurate, especially for float32. Specifying a higherprecision accumulator using the dtype keyword can alleviate this issue.
Examples
>>> a = np.array([[1, np.nan], [3, 4]]) >>> np.nanmean(a) 2.6666666666666665 >>> np.nanmean(a, axis=0) array([ 2., 4.]) >>> np.nanmean(a, axis=1) array([ 1., 3.5])

dask.array.
nanmin
(a, axis=None, keepdims=False, split_every=None)¶ Return minimum of an array or minimum along an axis, ignoring any NaNs. When allNaN slices are encountered a
RuntimeWarning
is raised and Nan is returned for that slice.Parameters: a : array_like
Array containing numbers whose minimum is desired. If a is not an array, a conversion is attempted.
axis : int, optional
Axis along which the minimum is computed. The default is to compute the minimum of the flattened array.
out : ndarray, optional
Alternate output array in which to place the result. The default is
None
; if provided, it must have the same shape as the expected output, but the type will be cast if necessary. See doc.ufuncs for details.New in version 1.8.0.
keepdims : bool, optional
If this is set to True, the axes which are reduced are left in the result as dimensions with size one. With this option, the result will broadcast correctly against the original a.
New in version 1.8.0.
Returns: nanmin : ndarray
An array with the same shape as a, with the specified axis removed. If a is a 0d array, or if axis is None, an ndarray scalar is returned. The same dtype as a is returned.
See also
nanmax
 The maximum value of an array along a given axis, ignoring any NaNs.
amin
 The minimum value of an array along a given axis, propagating any NaNs.
fmin
 Elementwise minimum of two arrays, ignoring any NaNs.
minimum
 Elementwise minimum of two arrays, propagating any NaNs.
isnan
 Shows which elements are Not a Number (NaN).
isfinite
 Shows which elements are neither NaN nor infinity.
Notes
Numpy uses the IEEE Standard for Binary FloatingPoint for Arithmetic (IEEE 754). This means that Not a Number is not equivalent to infinity. Positive infinity is treated as a very large number and negative infinity is treated as a very small (i.e. negative) number.
If the input has a integer type the function is equivalent to np.min.
Examples
>>> a = np.array([[1, 2], [3, np.nan]]) >>> np.nanmin(a) 1.0 >>> np.nanmin(a, axis=0) array([ 1., 2.]) >>> np.nanmin(a, axis=1) array([ 1., 3.])
When positive infinity and negative infinity are present:
>>> np.nanmin([1, 2, np.nan, np.inf]) 1.0 >>> np.nanmin([1, 2, np.nan, np.NINF]) inf

dask.array.
nanprod
(a, axis=None, dtype=None, keepdims=False, split_every=None)¶ Return the product of array elements over a given axis treating Not a Numbers (NaNs) as zero.
One is returned for slices that are allNaN or empty.
New in version 1.10.0.
Parameters: a : array_like
Array containing numbers whose sum is desired. If a is not an array, a conversion is attempted.
axis : int, optional
Axis along which the product is computed. The default is to compute the product of the flattened array.
dtype : datatype, optional
The type of the returned array and of the accumulator in which the elements are summed. By default, the dtype of a is used. An exception is when a has an integer type with less precision than the platform (u)intp. In that case, the default will be either (u)int32 or (u)int64 depending on whether the platform is 32 or 64 bits. For inexact inputs, dtype must be inexact.
out : ndarray, optional
Alternate output array in which to place the result. The default is
None
. If provided, it must have the same shape as the expected output, but the type will be cast if necessary. See doc.ufuncs for details. The casting of NaN to integer can yield unexpected results.keepdims : bool, optional
If True, the axes which are reduced are left in the result as dimensions with size one. With this option, the result will broadcast correctly against the original arr.
Returns: y : ndarray or numpy scalar
See also
numpy.prod
 Product across array propagating NaNs.
isnan
 Show which elements are NaN.
Notes
Numpy integer arithmetic is modular. If the size of a product exceeds the size of an integer accumulator, its value will wrap around and the result will be incorrect. Specifying
dtype=double
can alleviate that problem.Examples
>>> np.nanprod(1) 1 >>> np.nanprod([1]) 1 >>> np.nanprod([1, np.nan]) 1.0 >>> a = np.array([[1, 2], [3, np.nan]]) >>> np.nanprod(a) 6.0 >>> np.nanprod(a, axis=0) array([ 3., 2.])

dask.array.
nanstd
(a, axis=None, dtype=None, keepdims=False, ddof=0, split_every=None)¶ Compute the standard deviation along the specified axis, while ignoring NaNs.
Returns the standard deviation, a measure of the spread of a distribution, of the nonNaN array elements. The standard deviation is computed for the flattened array by default, otherwise over the specified axis.
For allNaN slices or slices with zero degrees of freedom, NaN is returned and a RuntimeWarning is raised.
New in version 1.8.0.
Parameters: a : array_like
Calculate the standard deviation of the nonNaN values.
axis : int, optional
Axis along which the standard deviation is computed. The default is to compute the standard deviation of the flattened array.
dtype : dtype, optional
Type to use in computing the standard deviation. For arrays of integer type the default is float64, for arrays of float types it is the same as the array type.
out : ndarray, optional
Alternative output array in which to place the result. It must have the same shape as the expected output but the type (of the calculated values) will be cast if necessary.
ddof : int, optional
Means Delta Degrees of Freedom. The divisor used in calculations is
N  ddof
, whereN
represents the number of nonNaN elements. By default ddof is zero.keepdims : bool, optional
If this is set to True, the axes which are reduced are left in the result as dimensions with size one. With this option, the result will broadcast correctly against the original arr.
Returns: standard_deviation : ndarray, see dtype parameter above.
If out is None, return a new array containing the standard deviation, otherwise return a reference to the output array. If ddof is >= the number of nonNaN elements in a slice or the slice contains only NaNs, then the result for that slice is NaN.
Notes
The standard deviation is the square root of the average of the squared deviations from the mean:
std = sqrt(mean(abs(x  x.mean())**2))
.The average squared deviation is normally calculated as
x.sum() / N
, whereN = len(x)
. If, however, ddof is specified, the divisorN  ddof
is used instead. In standard statistical practice,ddof=1
provides an unbiased estimator of the variance of the infinite population.ddof=0
provides a maximum likelihood estimate of the variance for normally distributed variables. The standard deviation computed in this function is the square root of the estimated variance, so even withddof=1
, it will not be an unbiased estimate of the standard deviation per se.Note that, for complex numbers, std takes the absolute value before squaring, so that the result is always real and nonnegative.
For floatingpoint input, the std is computed using the same precision the input has. Depending on the input data, this can cause the results to be inaccurate, especially for float32 (see example below). Specifying a higheraccuracy accumulator using the dtype keyword can alleviate this issue.
Examples
>>> a = np.array([[1, np.nan], [3, 4]]) >>> np.nanstd(a) 1.247219128924647 >>> np.nanstd(a, axis=0) array([ 1., 0.]) >>> np.nanstd(a, axis=1) array([ 0., 0.5])

dask.array.
nansum
(a, axis=None, dtype=None, keepdims=False, split_every=None)¶ Return the sum of array elements over a given axis treating Not a Numbers (NaNs) as zero.
FutureWarning: In Numpy versions <= 1.8 Nan is returned for slices that are allNaN or empty. In later versions zero will be returned.
Parameters: a : array_like
Array containing numbers whose sum is desired. If a is not an array, a conversion is attempted.
axis : int, optional
Axis along which the sum is computed. The default is to compute the sum of the flattened array.
dtype : datatype, optional
The type of the returned array and of the accumulator in which the elements are summed. By default, the dtype of a is used. An exception is when a has an integer type with less precision than the platform (u)intp. In that case, the default will be either (u)int32 or (u)int64 depending on whether the platform is 32 or 64 bits. For inexact inputs, dtype must be inexact.
New in version 1.8.0.
out : ndarray, optional
Alternate output array in which to place the result. The default is
None
. If provided, it must have the same shape as the expected output, but the type will be cast if necessary. See doc.ufuncs for details. The casting of NaN to integer can yield unexpected results.New in version 1.8.0.
keepdims : bool, optional
If True, the axes which are reduced are left in the result as dimensions with size one. With this option, the result will broadcast correctly against the original arr.
New in version 1.8.0.
Returns: y : ndarray or numpy scalar
See also
Notes
If both positive and negative infinity are present, the sum will be Not A Number (NaN).
Numpy integer arithmetic is modular. If the size of a sum exceeds the size of an integer accumulator, its value will wrap around and the result will be incorrect. Specifying
dtype=double
can alleviate that problem.Examples
>>> np.nansum(1) 1 >>> np.nansum([1]) 1 >>> np.nansum([1, np.nan]) 1.0 >>> a = np.array([[1, 1], [1, np.nan]]) >>> np.nansum(a) 3.0 >>> np.nansum(a, axis=0) array([ 2., 1.]) >>> np.nansum([1, np.nan, np.inf]) inf >>> np.nansum([1, np.nan, np.NINF]) inf >>> np.nansum([1, np.nan, np.inf, np.inf]) # both +/ infinity present nan

dask.array.
nanvar
(a, axis=None, dtype=None, keepdims=False, ddof=0, split_every=None)¶ Compute the variance along the specified axis, while ignoring NaNs.
Returns the variance of the array elements, a measure of the spread of a distribution. The variance is computed for the flattened array by default, otherwise over the specified axis.
For allNaN slices or slices with zero degrees of freedom, NaN is returned and a RuntimeWarning is raised.
New in version 1.8.0.
Parameters: a : array_like
Array containing numbers whose variance is desired. If a is not an array, a conversion is attempted.
axis : int, optional
Axis along which the variance is computed. The default is to compute the variance of the flattened array.
dtype : datatype, optional
Type to use in computing the variance. For arrays of integer type the default is float32; for arrays of float types it is the same as the array type.
out : ndarray, optional
Alternate output array in which to place the result. It must have the same shape as the expected output, but the type is cast if necessary.
ddof : int, optional
“Delta Degrees of Freedom”: the divisor used in the calculation is
N  ddof
, whereN
represents the number of nonNaN elements. By default ddof is zero.keepdims : bool, optional
If this is set to True, the axes which are reduced are left in the result as dimensions with size one. With this option, the result will broadcast correctly against the original arr.
Returns: variance : ndarray, see dtype parameter above
If out is None, return a new array containing the variance, otherwise return a reference to the output array. If ddof is >= the number of nonNaN elements in a slice or the slice contains only NaNs, then the result for that slice is NaN.
See also
numpy.doc.ufuncs
 Section “Output arguments”
Notes
The variance is the average of the squared deviations from the mean, i.e.,
var = mean(abs(x  x.mean())**2)
.The mean is normally calculated as
x.sum() / N
, whereN = len(x)
. If, however, ddof is specified, the divisorN  ddof
is used instead. In standard statistical practice,ddof=1
provides an unbiased estimator of the variance of a hypothetical infinite population.ddof=0
provides a maximum likelihood estimate of the variance for normally distributed variables.Note that for complex numbers, the absolute value is taken before squaring, so that the result is always real and nonnegative.
For floatingpoint input, the variance is computed using the same precision the input has. Depending on the input data, this can cause the results to be inaccurate, especially for float32 (see example below). Specifying a higheraccuracy accumulator using the
dtype
keyword can alleviate this issue.Examples
>>> a = np.array([[1, np.nan], [3, 4]]) >>> np.var(a) 1.5555555555555554 >>> np.nanvar(a, axis=0) array([ 1., 0.]) >>> np.nanvar(a, axis=1) array([ 0., 0.25])

dask.array.
nextafter
(x1, x2[, out])¶ Return the next floatingpoint value after x1 towards x2, elementwise.
Parameters: x1 : array_like
Values to find the next representable value of.
x2 : array_like
The direction where to look for the next representable value of x1.
out : ndarray, optional
Array into which the output is placed. Its type is preserved and it must be of the right shape to hold the output. See doc.ufuncs.
Returns: out : array_like
The next representable values of x1 in the direction of x2.
Examples
>>> eps = np.finfo(np.float64).eps >>> np.nextafter(1, 2) == eps + 1 True >>> np.nextafter([1, 2], [2, 1]) == [eps + 1, 2  eps] array([ True, True], dtype=bool)

dask.array.
notnull
(values)¶ pandas.notnull for dask arrays

dask.array.
ones
()¶ Blocked variant of ones
Follows the signature of ones exactly except that it also requires a keyword argument chunks=(...)
Original signature follows below.
Return a new array of given shape and type, filled with ones.
Parameters: shape : int or sequence of ints
Shape of the new array, e.g.,
(2, 3)
or2
.dtype : datatype, optional
The desired datatype for the array, e.g., numpy.int8. Default is numpy.float64.
order : {‘C’, ‘F’}, optional
Whether to store multidimensional data in C or Fortrancontiguous (row or columnwise) order in memory.
Returns: out : ndarray
Array of ones with the given shape, dtype, and order.
See also
zeros
,ones_like
Examples
>>> np.ones(5) array([ 1., 1., 1., 1., 1.])
>>> np.ones((5,), dtype=np.int) array([1, 1, 1, 1, 1])
>>> np.ones((2, 1)) array([[ 1.], [ 1.]])
>>> s = (2,2) >>> np.ones(s) array([[ 1., 1.], [ 1., 1.]])

dask.array.
percentile
(a, q, interpolation='linear')¶ Approximate percentile of 1D array
See numpy.percentile for more information

dask.array.
prod
(a, axis=None, dtype=None, keepdims=False, split_every=None)¶ Return the product of array elements over a given axis.
Parameters: a : array_like
Input data.
axis : None or int or tuple of ints, optional
Axis or axes along which a product is performed. The default (axis = None) is perform a product over all the dimensions of the input array. axis may be negative, in which case it counts from the last to the first axis.
New in version 1.7.0.
If this is a tuple of ints, a product is performed on multiple axes, instead of a single axis or all the axes as before.
dtype : datatype, optional
The datatype of the returned array, as well as of the accumulator in which the elements are multiplied. By default, if a is of integer type, dtype is the default platform integer. (Note: if the type of a is unsigned, then so is dtype.) Otherwise, the dtype is the same as that of a.
out : ndarray, optional
Alternative output array in which to place the result. It must have the same shape as the expected output, but the type of the output values will be cast if necessary.
keepdims : bool, optional
If this is set to True, the axes which are reduced are left in the result as dimensions with size one. With this option, the result will broadcast correctly against the original arr.
Returns: product_along_axis : ndarray, see dtype parameter above.
An array shaped as a but with the specified axis removed. Returns a reference to out if specified.
See also
ndarray.prod
 equivalent method
numpy.doc.ufuncs
 Section “Output arguments”
Notes
Arithmetic is modular when using integer types, and no error is raised on overflow. That means that, on a 32bit platform:
>>> x = np.array([536870910, 536870910, 536870910, 536870910]) >>> np.prod(x) #random 16
Examples
By default, calculate the product of all elements:
>>> np.prod([1.,2.]) 2.0
Even when the input array is twodimensional:
>>> np.prod([[1.,2.],[3.,4.]]) 24.0
But we can also specify the axis over which to multiply:
>>> np.prod([[1.,2.],[3.,4.]], axis=1) array([ 2., 12.])
If the type of x is unsigned, then the output type is the unsigned platform integer:
>>> x = np.array([1, 2, 3], dtype=np.uint8) >>> np.prod(x).dtype == np.uint True
If x is of a signed integer type, then the output type is the default platform integer:
>>> x = np.array([1, 2, 3], dtype=np.int8) >>> np.prod(x).dtype == np.int True

dask.array.
rad2deg
(x[, out])¶ Convert angles from radians to degrees.
Parameters: x : array_like
Angle in radians.
out : ndarray, optional
Array into which the output is placed. Its type is preserved and it must be of the right shape to hold the output. See doc.ufuncs.
Returns: y : ndarray
The corresponding angle in degrees.
See also
deg2rad
 Convert angles from degrees to radians.
unwrap
 Remove large jumps in angle by wrapping.
Notes
New in version 1.3.0.
rad2deg(x) is
180 * x / pi
.Examples
>>> np.rad2deg(np.pi/2) 90.0

dask.array.
radians
(x[, out])¶ Convert angles from degrees to radians.
Parameters: x : array_like
Input array in degrees.
out : ndarray, optional
Output array of same shape as x.
Returns: y : ndarray
The corresponding radian values.
See also
deg2rad
 equivalent function
Examples
Convert a degree array to radians
>>> deg = np.arange(12.) * 30. >>> np.radians(deg) array([ 0. , 0.52359878, 1.04719755, 1.57079633, 2.0943951 , 2.61799388, 3.14159265, 3.66519143, 4.1887902 , 4.71238898, 5.23598776, 5.75958653])
>>> out = np.zeros((deg.shape)) >>> ret = np.radians(deg, out) >>> ret is out True

dask.array.
ravel
(array)¶ Return a flattened array.
A 1D array, containing the elements of the input, is returned. A copy is made only if needed.
Parameters: a : array_like
Input array. The elements in a are read in the order specified by order, and packed as a 1D array.
order : {‘C’,’F’, ‘A’, ‘K’}, optional
The elements of a are read using this index order. ‘C’ means to index the elements in Clike order, with the last axis index changing fastest, back to the first axis index changing slowest. ‘F’ means to index the elements in Fortranlike index order, with the first index changing fastest, and the last index changing slowest. Note that the ‘C’ and ‘F’ options take no account of the memory layout of the underlying array, and only refer to the order of axis indexing. ‘A’ means to read the elements in Fortranlike index order if a is Fortran contiguous in memory, Clike order otherwise. ‘K’ means to read the elements in the order they occur in memory, except for reversing the data when strides are negative. By default, ‘C’ index order is used.
Returns: 1d_array : ndarray
Output of the same dtype as a, and of shape
(a.size,)
.See also
ndarray.flat
 1D iterator over an array.
ndarray.flatten
 1D array copy of the elements of an array in rowmajor order.
Notes
In Clike (rowmajor) order, in two dimensions, the row index varies the slowest, and the column index the quickest. This can be generalized to multiple dimensions, where rowmajor order implies that the index along the first axis varies slowest, and the index along the last quickest. The opposite holds for Fortranlike, or columnmajor, index ordering.
Examples
It is equivalent to
reshape(1, order=order)
.>>> x = np.array([[1, 2, 3], [4, 5, 6]]) >>> print np.ravel(x) [1 2 3 4 5 6]
>>> print x.reshape(1) [1 2 3 4 5 6]
>>> print np.ravel(x, order='F') [1 4 2 5 3 6]
When
order
is ‘A’, it will preserve the array’s ‘C’ or ‘F’ ordering:>>> print np.ravel(x.T) [1 4 2 5 3 6] >>> print np.ravel(x.T, order='A') [1 2 3 4 5 6]
When
order
is ‘K’, it will preserve orderings that are neither ‘C’ nor ‘F’, but won’t reverse axes:>>> a = np.arange(3)[::1]; a array([2, 1, 0]) >>> a.ravel(order='C') array([2, 1, 0]) >>> a.ravel(order='K') array([2, 1, 0])
>>> a = np.arange(12).reshape(2,3,2).swapaxes(1,2); a array([[[ 0, 2, 4], [ 1, 3, 5]], [[ 6, 8, 10], [ 7, 9, 11]]]) >>> a.ravel(order='C') array([ 0, 2, 4, 1, 3, 5, 6, 8, 10, 7, 9, 11]) >>> a.ravel(order='K') array([ 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11])

dask.array.
real
(*args, **kwargs)¶ Return the real part of the elements of the array.
Parameters: val : array_like
Input array.
Returns: out : ndarray
Output array. If val is real, the type of val is used for the output. If val has complex elements, the returned type is float.
Examples
>>> a = np.array([1+2j, 3+4j, 5+6j]) >>> a.real array([ 1., 3., 5.]) >>> a.real = 9 >>> a array([ 9.+2.j, 9.+4.j, 9.+6.j]) >>> a.real = np.array([9, 8, 7]) >>> a array([ 9.+2.j, 8.+4.j, 7.+6.j])

dask.array.
rechunk
(x, chunks, threshold=4, block_size_limit=100000000.0)¶ Convert blocks in dask array x for new chunks.
>>> import dask.array as da >>> a = np.random.uniform(0, 1, 7**4).reshape((7,) * 4) >>> x = da.from_array(a, chunks=((2, 3, 2),)*4) >>> x.chunks ((2, 3, 2), (2, 3, 2), (2, 3, 2), (2, 3, 2))
>>> y = rechunk(x, chunks=((2, 4, 1), (4, 2, 1), (4, 3), (7,))) >>> y.chunks ((2, 4, 1), (4, 2, 1), (4, 3), (7,))
chunks also accept dict arguments mapping axis to blockshape
>>> y = rechunk(x, chunks={1: 2}) # rechunk axis 1 with blockshape 2
Parameters: x: dask array
chunks: tuple
The new block dimensions to create
threshold: int
The graph growth factor under which we don’t bother introducing an intermediate step
block_size_limit: int
The maximum block size (in bytes) we want to produce during an intermediate step

dask.array.
reshape
(array, shape)¶ Gives a new shape to an array without changing its data.
Parameters: a : array_like
Array to be reshaped.
newshape : int or tuple of ints
The new shape should be compatible with the original shape. If an integer, then the result will be a 1D array of that length. One shape dimension can be 1. In this case, the value is inferred from the length of the array and remaining dimensions.
order : {‘C’, ‘F’, ‘A’}, optional
Read the elements of a using this index order, and place the elements into the reshaped array using this index order. ‘C’ means to read / write the elements using Clike index order, with the last axis index changing fastest, back to the first axis index changing slowest. ‘F’ means to read / write the elements using Fortranlike index order, with the first index changing fastest, and the last index changing slowest. Note that the ‘C’ and ‘F’ options take no account of the memory layout of the underlying array, and only refer to the order of indexing. ‘A’ means to read / write the elements in Fortranlike index order if a is Fortran contiguous in memory, Clike order otherwise.
Returns: reshaped_array : ndarray
This will be a new view object if possible; otherwise, it will be a copy. Note there is no guarantee of the memory layout (C or Fortran contiguous) of the returned array.
See also
ndarray.reshape
 Equivalent method.
Notes
It is not always possible to change the shape of an array without copying the data. If you want an error to be raise if the data is copied, you should assign the new shape to the shape attribute of the array:
>>> a = np.zeros((10, 2)) # A transpose make the array noncontiguous >>> b = a.T # Taking a view makes it possible to modify the shape without modifying the # initial object. >>> c = b.view() >>> c.shape = (20) AttributeError: incompatible shape for a noncontiguous array
The order keyword gives the index ordering both for fetching the values from a, and then placing the values into the output array. For example, let’s say you have an array:
>>> a = np.arange(6).reshape((3, 2)) >>> a array([[0, 1], [2, 3], [4, 5]])
You can think of reshaping as first raveling the array (using the given index order), then inserting the elements from the raveled array into the new array using the same kind of index ordering as was used for the raveling.
>>> np.reshape(a, (2, 3)) # Clike index ordering array([[0, 1, 2], [3, 4, 5]]) >>> np.reshape(np.ravel(a), (2, 3)) # equivalent to C ravel then C reshape array([[0, 1, 2], [3, 4, 5]]) >>> np.reshape(a, (2, 3), order='F') # Fortranlike index ordering array([[0, 4, 3], [2, 1, 5]]) >>> np.reshape(np.ravel(a, order='F'), (2, 3), order='F') array([[0, 4, 3], [2, 1, 5]])
Examples
>>> a = np.array([[1,2,3], [4,5,6]]) >>> np.reshape(a, 6) array([1, 2, 3, 4, 5, 6]) >>> np.reshape(a, 6, order='F') array([1, 4, 2, 5, 3, 6])
>>> np.reshape(a, (3,1)) # the unspecified value is inferred to be 2 array([[1, 2], [3, 4], [5, 6]])

dask.array.
rint
(x[, out])¶ Round elements of the array to the nearest integer.
Parameters: x : array_like
Input array.
Returns: out : {ndarray, scalar}
Output array is same shape and type as x.
Examples
>>> a = np.array([1.7, 1.5, 0.2, 0.2, 1.5, 1.7, 2.0]) >>> np.rint(a) array([2., 2., 0., 0., 2., 2., 2.])

dask.array.
sign
(x[, out])¶ Returns an elementwise indication of the sign of a number.
The sign function returns
1 if x < 0, 0 if x==0, 1 if x > 0
.Parameters: x : array_like
Input values.
Returns: y : ndarray
The sign of x.
Examples
>>> np.sign([5., 4.5]) array([1., 1.]) >>> np.sign(0) 0

dask.array.
signbit
(x[, out])¶ Returns elementwise True where signbit is set (less than zero).
Parameters: x : array_like
The input value(s).
out : ndarray, optional
Array into which the output is placed. Its type is preserved and it must be of the right shape to hold the output. See doc.ufuncs.
Returns: result : ndarray of bool
Output array, or reference to out if that was supplied.
Examples
>>> np.signbit(1.2) True >>> np.signbit(np.array([1, 2.3, 2.1])) array([False, True, False], dtype=bool)

dask.array.
sin
(x[, out])¶ Trigonometric sine, elementwise.
Parameters: x : array_like
Angle, in radians (\(2 \pi\) rad equals 360 degrees).
Returns: y : array_like
The sine of each element of x.
Notes
The sine is one of the fundamental functions of trigonometry (the mathematical study of triangles). Consider a circle of radius 1 centered on the origin. A ray comes in from the \(+x\) axis, makes an angle at the origin (measured counterclockwise from that axis), and departs from the origin. The \(y\) coordinate of the outgoing ray’s intersection with the unit circle is the sine of that angle. It ranges from 1 for \(x=3\pi / 2\) to +1 for \(\pi / 2.\) The function has zeroes where the angle is a multiple of \(\pi\). Sines of angles between \(\pi\) and \(2\pi\) are negative. The numerous properties of the sine and related functions are included in any standard trigonometry text.
Examples
Print sine of one angle:
>>> np.sin(np.pi/2.) 1.0
Print sines of an array of angles given in degrees:
>>> np.sin(np.array((0., 30., 45., 60., 90.)) * np.pi / 180. ) array([ 0. , 0.5 , 0.70710678, 0.8660254 , 1. ])
Plot the sine function:
>>> import matplotlib.pylab as plt >>> x = np.linspace(np.pi, np.pi, 201) >>> plt.plot(x, np.sin(x)) >>> plt.xlabel('Angle [rad]') >>> plt.ylabel('sin(x)') >>> plt.axis('tight') >>> plt.show()

dask.array.
sinh
(x[, out])¶ Hyperbolic sine, elementwise.
Equivalent to
1/2 * (np.exp(x)  np.exp(x))
or1j * np.sin(1j*x)
.Parameters: x : array_like
Input array.
out : ndarray, optional
Output array of same shape as x.
Returns: y : ndarray
The corresponding hyperbolic sine values.
Raises: ValueError: invalid return array shape
if out is provided and out.shape != x.shape (See Examples)
Notes
If out is provided, the function writes the result into it, and returns a reference to out. (See Examples)
References
M. Abramowitz and I. A. Stegun, Handbook of Mathematical Functions. New York, NY: Dover, 1972, pg. 83.
Examples
>>> np.sinh(0) 0.0 >>> np.sinh(np.pi*1j/2) 1j >>> np.sinh(np.pi*1j) # (exact value is 0) 1.2246063538223773e016j >>> # Discrepancy due to vagaries of floating point arithmetic.
>>> # Example of providing the optional output parameter >>> out2 = np.sinh([0.1], out1) >>> out2 is out1 True
>>> # Example of ValueError due to provision of shape mismatched `out` >>> np.sinh(np.zeros((3,3)),np.zeros((2,2))) Traceback (most recent call last): File "<stdin>", line 1, in <module> ValueError: invalid return array shape

dask.array.
sqrt
(x[, out])¶ Return the positive squareroot of an array, elementwise.
Parameters: x : array_like
The values whose squareroots are required.
out : ndarray, optional
Alternate array object in which to put the result; if provided, it must have the same shape as x
Returns: y : ndarray
An array of the same shape as x, containing the positive squareroot of each element in x. If any element in x is complex, a complex array is returned (and the squareroots of negative reals are calculated). If all of the elements in x are real, so is y, with negative elements returning
nan
. If out was provided, y is a reference to it.See also
lib.scimath.sqrt
 A version which returns complex numbers when given negative reals.
Notes
sqrt has–consistent with common convention–as its branch cut the real “interval” [inf, 0), and is continuous from above on it. A branch cut is a curve in the complex plane across which a given complex function fails to be continuous.
Examples
>>> np.sqrt([1,4,9]) array([ 1., 2., 3.])
>>> np.sqrt([4, 1, 3+4J]) array([ 2.+0.j, 0.+1.j, 1.+2.j])
>>> np.sqrt([4, 1, numpy.inf]) array([ 2., NaN, Inf])

dask.array.
square
(x[, out])¶ Return the elementwise square of the input.
Parameters: x : array_like
Input data.
Returns: out : ndarray
Elementwise x*x, of the same shape and dtype as x. Returns scalar if x is a scalar.
See also
numpy.linalg.matrix_power
,sqrt
,power
Examples
>>> np.square([1j, 1]) array([1.0.j, 1.+0.j])

dask.array.
squeeze
(a, axis=None)¶ Remove singledimensional entries from the shape of an array.
Parameters: a : array_like
Input data.
axis : None or int or tuple of ints, optional
New in version 1.7.0.
Selects a subset of the singledimensional entries in the shape. If an axis is selected with shape entry greater than one, an error is raised.
Returns: squeezed : ndarray
The input array, but with with all or a subset of the dimensions of length 1 removed. This is always a itself or a view into a.
Examples
>>> x = np.array([[[0], [1], [2]]]) >>> x.shape (1, 3, 1) >>> np.squeeze(x).shape (3,) >>> np.squeeze(x, axis=(2,)).shape (1, 3)

dask.array.
stack
(seq, axis=0) Stack arrays along a new axis
Given a sequence of dask Arrays form a new dask Array by stacking them along a new dimension (axis=0 by default)
See also
Examples
Create slices
>>> import dask.array as da >>> import numpy as np
>>> data = [from_array(np.ones((4, 4)), chunks=(2, 2)) ... for i in range(3)]
>>> x = da.stack(data, axis=0) >>> x.shape (3, 4, 4)
>>> da.stack(data, axis=1).shape (4, 3, 4)
>>> da.stack(data, axis=1).shape (4, 4, 3)
Result is a new dask Array

dask.array.
std
(a, axis=None, dtype=None, keepdims=False, ddof=0, split_every=None)¶ Compute the standard deviation along the specified axis.
Returns the standard deviation, a measure of the spread of a distribution, of the array elements. The standard deviation is computed for the flattened array by default, otherwise over the specified axis.
Parameters: a : array_like
Calculate the standard deviation of these values.
axis : int, optional
Axis along which the standard deviation is computed. The default is to compute the standard deviation of the flattened array.
dtype : dtype, optional
Type to use in computing the standard deviation. For arrays of integer type the default is float64, for arrays of float types it is the same as the array type.
out : ndarray, optional
Alternative output array in which to place the result. It must have the same shape as the expected output but the type (of the calculated values) will be cast if necessary.
ddof : int, optional
Means Delta Degrees of Freedom. The divisor used in calculations is
N  ddof
, whereN
represents the number of elements. By default ddof is zero.keepdims : bool, optional
If this is set to True, the axes which are reduced are left in the result as dimensions with size one. With this option, the result will broadcast correctly against the original arr.
Returns: standard_deviation : ndarray, see dtype parameter above.
If out is None, return a new array containing the standard deviation, otherwise return a reference to the output array.
Notes
The standard deviation is the square root of the average of the squared deviations from the mean, i.e.,
std = sqrt(mean(abs(x  x.mean())**2))
.The average squared deviation is normally calculated as
x.sum() / N
, whereN = len(x)
. If, however, ddof is specified, the divisorN  ddof
is used instead. In standard statistical practice,ddof=1
provides an unbiased estimator of the variance of the infinite population.ddof=0
provides a maximum likelihood estimate of the variance for normally distributed variables. The standard deviation computed in this function is the square root of the estimated variance, so even withddof=1
, it will not be an unbiased estimate of the standard deviation per se.Note that, for complex numbers, std takes the absolute value before squaring, so that the result is always real and nonnegative.
For floatingpoint input, the std is computed using the same precision the input has. Depending on the input data, this can cause the results to be inaccurate, especially for float32 (see example below). Specifying a higheraccuracy accumulator using the dtype keyword can alleviate this issue.
Examples
>>> a = np.array([[1, 2], [3, 4]]) >>> np.std(a) 1.1180339887498949 >>> np.std(a, axis=0) array([ 1., 1.]) >>> np.std(a, axis=1) array([ 0.5, 0.5])
In single precision, std() can be inaccurate:
>>> a = np.zeros((2,512*512), dtype=np.float32) >>> a[0,:] = 1.0 >>> a[1,:] = 0.1 >>> np.std(a) 0.45172946707416706
Computing the standard deviation in float64 is more accurate:
>>> np.std(a, dtype=np.float64) 0.44999999925552653

dask.array.
sum
(a, axis=None, dtype=None, keepdims=False, split_every=None)¶ Sum of array elements over a given axis.
Parameters: a : array_like
Elements to sum.
axis : None or int or tuple of ints, optional
Axis or axes along which a sum is performed. The default (axis = None) is perform a sum over all the dimensions of the input array. axis may be negative, in which case it counts from the last to the first axis.
New in version 1.7.0.
If this is a tuple of ints, a sum is performed on multiple axes, instead of a single axis or all the axes as before.
dtype : dtype, optional
The type of the returned array and of the accumulator in which the elements are summed. By default, the dtype of a is used. An exception is when a has an integer type with less precision than the default platform integer. In that case, the default platform integer is used instead.
out : ndarray, optional
Array into which the output is placed. By default, a new array is created. If out is given, it must be of the appropriate shape (the shape of a with axis removed, i.e.,
numpy.delete(a.shape, axis)
). Its type is preserved. See doc.ufuncs (Section “Output arguments”) for more details.keepdims : bool, optional
If this is set to True, the axes which are reduced are left in the result as dimensions with size one. With this option, the result will broadcast correctly against the original arr.
Returns: sum_along_axis : ndarray
An array with the same shape as a, with the specified axis removed. If a is a 0d array, or if axis is None, a scalar is returned. If an output array is specified, a reference to out is returned.
See also
ndarray.sum
 Equivalent method.
cumsum
 Cumulative sum of array elements.
trapz
 Integration of array values using the composite trapezoidal rule.
mean
,average
Notes
Arithmetic is modular when using integer types, and no error is raised on overflow.
Examples
>>> np.sum([0.5, 1.5]) 2.0 >>> np.sum([0.5, 0.7, 0.2, 1.5], dtype=np.int32) 1 >>> np.sum([[0, 1], [0, 5]]) 6 >>> np.sum([[0, 1], [0, 5]], axis=0) array([0, 6]) >>> np.sum([[0, 1], [0, 5]], axis=1) array([1, 5])
If the accumulator is too small, overflow occurs:
>>> np.ones(128, dtype=np.int8).sum(dtype=np.int8) 128

dask.array.
take
(a, indices, axis=0)¶ Take elements from an array along an axis.
This function does the same thing as “fancy” indexing (indexing arrays using arrays); however, it can be easier to use if you need elements along a given axis.
Parameters: a : array_like
The source array.
indices : array_like
The indices of the values to extract.
New in version 1.8.0.
Also allow scalars for indices.
axis : int, optional
The axis over which to select values. By default, the flattened input array is used.
out : ndarray, optional
If provided, the result will be placed in this array. It should be of the appropriate shape and dtype.
mode : {‘raise’, ‘wrap’, ‘clip’}, optional
Specifies how outofbounds indices will behave.
 ‘raise’ – raise an error (default)
 ‘wrap’ – wrap around
 ‘clip’ – clip to the range
‘clip’ mode means that all indices that are too large are replaced by the index that addresses the last element along that axis. Note that this disables indexing with negative numbers.
Returns: subarray : ndarray
The returned array has the same type as a.
See also
ndarray.take
 equivalent method
Examples
>>> a = [4, 3, 5, 7, 6, 8] >>> indices = [0, 1, 4] >>> np.take(a, indices) array([4, 3, 6])
In this example if a is an ndarray, “fancy” indexing can be used.
>>> a = np.array(a) >>> a[indices] array([4, 3, 6])
If indices is not one dimensional, the output also has these dimensions.
>>> np.take(a, [[0, 1], [2, 3]]) array([[4, 3], [5, 7]])

dask.array.
tan
(x[, out])¶ Compute tangent elementwise.
Equivalent to
np.sin(x)/np.cos(x)
elementwise.Parameters: x : array_like
Input array.
out : ndarray, optional
Output array of same shape as x.
Returns: y : ndarray
The corresponding tangent values.
Raises: ValueError: invalid return array shape
if out is provided and out.shape != x.shape (See Examples)
Notes
If out is provided, the function writes the result into it, and returns a reference to out. (See Examples)
References
M. Abramowitz and I. A. Stegun, Handbook of Mathematical Functions. New York, NY: Dover, 1972.
Examples
>>> from math import pi >>> np.tan(np.array([pi,pi/2,pi])) array([ 1.22460635e16, 1.63317787e+16, 1.22460635e16]) >>> >>> # Example of providing the optional output parameter illustrating >>> # that what is returned is a reference to said parameter >>> out2 = np.cos([0.1], out1) >>> out2 is out1 True >>> >>> # Example of ValueError due to provision of shape mismatched `out` >>> np.cos(np.zeros((3,3)),np.zeros((2,2))) Traceback (most recent call last): File "<stdin>", line 1, in <module> ValueError: invalid return array shape

dask.array.
tanh
(x[, out])¶ Compute hyperbolic tangent elementwise.
Equivalent to
np.sinh(x)/np.cosh(x)
or1j * np.tan(1j*x)
.Parameters: x : array_like
Input array.
out : ndarray, optional
Output array of same shape as x.
Returns: y : ndarray
The corresponding hyperbolic tangent values.
Raises: ValueError: invalid return array shape
if out is provided and out.shape != x.shape (See Examples)
Notes
If out is provided, the function writes the result into it, and returns a reference to out. (See Examples)
References
[R93] M. Abramowitz and I. A. Stegun, Handbook of Mathematical Functions. New York, NY: Dover, 1972, pg. 83. http://www.math.sfu.ca/~cbm/aands/ [R94] Wikipedia, “Hyperbolic function”, http://en.wikipedia.org/wiki/Hyperbolic_function Examples
>>> np.tanh((0, np.pi*1j, np.pi*1j/2)) array([ 0. +0.00000000e+00j, 0. 1.22460635e16j, 0. +1.63317787e+16j])
>>> # Example of providing the optional output parameter illustrating >>> # that what is returned is a reference to said parameter >>> out2 = np.tanh([0.1], out1) >>> out2 is out1 True
>>> # Example of ValueError due to provision of shape mismatched `out` >>> np.tanh(np.zeros((3,3)),np.zeros((2,2))) Traceback (most recent call last): File "<stdin>", line 1, in <module> ValueError: invalid return array shape

dask.array.
tensordot
(lhs, rhs, axes=2)¶ Compute tensor dot product along specified axes for arrays >= 1D.
Given two tensors (arrays of dimension greater than or equal to one), a and b, and an array_like object containing two array_like objects,
(a_axes, b_axes)
, sum the products of a‘s and b‘s elements (components) over the axes specified bya_axes
andb_axes
. The third argument can be a single nonnegative integer_like scalar,N
; if it is such, then the lastN
dimensions of a and the firstN
dimensions of b are summed over.Parameters: a, b : array_like, len(shape) >= 1
Tensors to “dot”.
axes : variable type
 integer_like scalar Number of axes to sum over (applies to both arrays); or
 (2,) array_like, both elements array_like of the same length List of axes to be summed over, first sequence applying to a, second to b.
See also
dot
,einsum
Notes
When there is more than one axis to sum over  and they are not the last (first) axes of a (b)  the argument axes should consist of two sequences of the same length, with the first axis to sum over given first in both sequences, the second axis second, and so forth.
Examples
A “traditional” example:
>>> a = np.arange(60.).reshape(3,4,5) >>> b = np.arange(24.).reshape(4,3,2) >>> c = np.tensordot(a,b, axes=([1,0],[0,1])) >>> c.shape (5, 2) >>> c array([[ 4400., 4730.], [ 4532., 4874.], [ 4664., 5018.], [ 4796., 5162.], [ 4928., 5306.]]) >>> # A slower but equivalent way of computing the same... >>> d = np.zeros((5,2)) >>> for i in range(5): ... for j in range(2): ... for k in range(3): ... for n in range(4): ... d[i,j] += a[k,n,i] * b[n,k,j] >>> c == d array([[ True, True], [ True, True], [ True, True], [ True, True], [ True, True]], dtype=bool)
An extended example taking advantage of the overloading of + and *:
>>> a = np.array(range(1, 9)) >>> a.shape = (2, 2, 2) >>> A = np.array(('a', 'b', 'c', 'd'), dtype=object) >>> A.shape = (2, 2) >>> a; A array([[[1, 2], [3, 4]], [[5, 6], [7, 8]]]) array([[a, b], [c, d]], dtype=object)
>>> np.tensordot(a, A) # third argument default is 2 array([abbcccdddd, aaaaabbbbbbcccccccdddddddd], dtype=object)
>>> np.tensordot(a, A, 1) array([[[acc, bdd], [aaacccc, bbbdddd]], [[aaaaacccccc, bbbbbdddddd], [aaaaaaacccccccc, bbbbbbbdddddddd]]], dtype=object)
>>> np.tensordot(a, A, 0) # "Left for reader" (result too long to incl.) array([[[[[a, b], [c, d]], ...
>>> np.tensordot(a, A, (0, 1)) array([[[abbbbb, cddddd], [aabbbbbb, ccdddddd]], [[aaabbbbbbb, cccddddddd], [aaaabbbbbbbb, ccccdddddddd]]], dtype=object)
>>> np.tensordot(a, A, (2, 1)) array([[[abb, cdd], [aaabbbb, cccdddd]], [[aaaaabbbbbb, cccccdddddd], [aaaaaaabbbbbbbb, cccccccdddddddd]]], dtype=object)
>>> np.tensordot(a, A, ((0, 1), (0, 1))) array([abbbcccccddddddd, aabbbbccccccdddddddd], dtype=object)
>>> np.tensordot(a, A, ((2, 1), (1, 0))) array([acccbbdddd, aaaaacccccccbbbbbbdddddddd], dtype=object)

dask.array.
topk
(k, x) The top k elements of an array
Returns the k greatest elements of the array in sorted order. Only works on arrays of a single dimension.
This assumes that
k
is small. All results will be returned in a single chunk.Examples
>>> x = np.array([5, 1, 3, 6]) >>> d = from_array(x, chunks=2) >>> d.topk(2).compute() array([6, 5])

dask.array.
transpose
(a, axes=None)¶ Permute the dimensions of an array.
Parameters: a : array_like
Input array.
axes : list of ints, optional
By default, reverse the dimensions, otherwise permute the axes according to the values given.
Returns: p : ndarray
a with its axes permuted. A view is returned whenever possible.
See also
rollaxis
Examples
>>> x = np.arange(4).reshape((2,2)) >>> x array([[0, 1], [2, 3]])
>>> np.transpose(x) array([[0, 2], [1, 3]])
>>> x = np.ones((1, 2, 3)) >>> np.transpose(x, (1, 0, 2)).shape (2, 1, 3)

dask.array.
tril
(m, k=0)¶ Lower triangle of an array with elements above the kth diagonal zeroed.
Parameters: m : array_like, shape (M, M)
Input array.
k : int, optional
Diagonal above which to zero elements. k = 0 (the default) is the main diagonal, k < 0 is below it and k > 0 is above.
Returns: tril : ndarray, shape (M, M)
Lower triangle of m, of same shape and datatype as m.
See also
triu
 upper triangle of an array

dask.array.
triu
(m, k=0)¶ Upper triangle of an array with elements above the kth diagonal zeroed.
Parameters: m : array_like, shape (M, N)
Input array.
k : int, optional
Diagonal above which to zero elements. k = 0 (the default) is the main diagonal, k < 0 is below it and k > 0 is above.
Returns: triu : ndarray, shape (M, N)
Upper triangle of m, of same shape and datatype as m.
See also
tril
 lower triangle of an array

dask.array.
trunc
(x[, out])¶ Return the truncated value of the input, elementwise.
The truncated value of the scalar x is the nearest integer i which is closer to zero than x is. In short, the fractional part of the signed number x is discarded.
Parameters: x : array_like
Input data.
Returns: y : {ndarray, scalar}
The truncated value of each element in x.
Notes
New in version 1.3.0.
Examples
>>> a = np.array([1.7, 1.5, 0.2, 0.2, 1.5, 1.7, 2.0]) >>> np.trunc(a) array([1., 1., 0., 0., 1., 1., 2.])

dask.array.
unique
(x)¶ Find the unique elements of an array.
Returns the sorted unique elements of an array. There are two optional outputs in addition to the unique elements: the indices of the input array that give the unique values, and the indices of the unique array that reconstruct the input array.
Parameters: ar : array_like
Input array. This will be flattened if it is not already 1D.
return_index : bool, optional
If True, also return the indices of ar that result in the unique array.
return_inverse : bool, optional
If True, also return the indices of the unique array that can be used to reconstruct ar.
Returns: unique : ndarray
The sorted unique values.
unique_indices : ndarray, optional
The indices of the first occurrences of the unique values in the (flattened) original array. Only provided if return_index is True.
unique_inverse : ndarray, optional
The indices to reconstruct the (flattened) original array from the unique array. Only provided if return_inverse is True.
See also
numpy.lib.arraysetops
 Module with a number of other functions for performing set operations on arrays.
Examples
>>> np.unique([1, 1, 2, 2, 3, 3]) array([1, 2, 3]) >>> a = np.array([[1, 1], [2, 3]]) >>> np.unique(a) array([1, 2, 3])
Return the indices of the original array that give the unique values:
>>> a = np.array(['a', 'b', 'b', 'c', 'a']) >>> u, indices = np.unique(a, return_index=True) >>> u array(['a', 'b', 'c'], dtype='S1') >>> indices array([0, 1, 3]) >>> a[indices] array(['a', 'b', 'c'], dtype='S1')
Reconstruct the input array from the unique values:
>>> a = np.array([1, 2, 6, 4, 2, 3, 2]) >>> u, indices = np.unique(a, return_inverse=True) >>> u array([1, 2, 3, 4, 6]) >>> indices array([0, 1, 4, 3, 1, 2, 1]) >>> u[indices] array([1, 2, 6, 4, 2, 3, 2])

dask.array.
var
(a, axis=None, dtype=None, keepdims=False, ddof=0, split_every=None)¶ Compute the variance along the specified axis.
Returns the variance of the array elements, a measure of the spread of a distribution. The variance is computed for the flattened array by default, otherwise over the specified axis.
Parameters: a : array_like
Array containing numbers whose variance is desired. If a is not an array, a conversion is attempted.
axis : int, optional
Axis along which the variance is computed. The default is to compute the variance of the flattened array.
dtype : datatype, optional
Type to use in computing the variance. For arrays of integer type the default is float32; for arrays of float types it is the same as the array type.
out : ndarray, optional
Alternate output array in which to place the result. It must have the same shape as the expected output, but the type is cast if necessary.
ddof : int, optional
“Delta Degrees of Freedom”: the divisor used in the calculation is
N  ddof
, whereN
represents the number of elements. By default ddof is zero.keepdims : bool, optional
If this is set to True, the axes which are reduced are left in the result as dimensions with size one. With this option, the result will broadcast correctly against the original arr.
Returns: variance : ndarray, see dtype parameter above
If
out=None
, returns a new array containing the variance; otherwise, a reference to the output array is returned.Notes
The variance is the average of the squared deviations from the mean, i.e.,
var = mean(abs(x  x.mean())**2)
.The mean is normally calculated as
x.sum() / N
, whereN = len(x)
. If, however, ddof is specified, the divisorN  ddof
is used instead. In standard statistical practice,ddof=1
provides an unbiased estimator of the variance of a hypothetical infinite population.ddof=0
provides a maximum likelihood estimate of the variance for normally distributed variables.Note that for complex numbers, the absolute value is taken before squaring, so that the result is always real and nonnegative.
For floatingpoint input, the variance is computed using the same precision the input has. Depending on the input data, this can cause the results to be inaccurate, especially for float32 (see example below). Specifying a higheraccuracy accumulator using the
dtype
keyword can alleviate this issue.Examples
>>> a = np.array([[1,2],[3,4]]) >>> np.var(a) 1.25 >>> np.var(a, axis=0) array([ 1., 1.]) >>> np.var(a, axis=1) array([ 0.25, 0.25])
In single precision, var() can be inaccurate:
>>> a = np.zeros((2,512*512), dtype=np.float32) >>> a[0,:] = 1.0 >>> a[1,:] = 0.1 >>> np.var(a) 0.20405951142311096
Computing the variance in float64 is more accurate:
>>> np.var(a, dtype=np.float64) 0.20249999932997387 >>> ((10.55)**2 + (0.10.55)**2)/2 0.20250000000000001

dask.array.
vnorm
(a, ord=None, axis=None, dtype=None, keepdims=False, split_every=None)¶ Vector norm
See np.linalg.norm

dask.array.
vstack
(tup)¶ Stack arrays in sequence vertically (row wise).
Take a sequence of arrays and stack them vertically to make a single array. Rebuild arrays divided by vsplit.
Parameters: tup : sequence of ndarrays
Tuple containing arrays to be stacked. The arrays must have the same shape along all but the first axis.
Returns: stacked : ndarray
The array formed by stacking the given arrays.
See also
hstack
 Stack arrays in sequence horizontally (column wise).
dstack
 Stack arrays in sequence depth wise (along third dimension).
concatenate
 Join a sequence of arrays together.
vsplit
 Split array into a list of multiple subarrays vertically.
Notes
Equivalent to
np.concatenate(tup, axis=0)
if tup contains arrays that are at least 2dimensional.Examples
>>> a = np.array([1, 2, 3]) >>> b = np.array([2, 3, 4]) >>> np.vstack((a,b)) array([[1, 2, 3], [2, 3, 4]])
>>> a = np.array([[1], [2], [3]]) >>> b = np.array([[2], [3], [4]]) >>> np.vstack((a,b)) array([[1], [2], [3], [2], [3], [4]])

dask.array.
where
(condition[, x, y])¶ Return elements, either from x or y, depending on condition.
If only condition is given, return
condition.nonzero()
.Parameters: condition : array_like, bool
When True, yield x, otherwise yield y.
x, y : array_like, optional
Values from which to choose. x and y need to have the same shape as condition.
Returns: out : ndarray or tuple of ndarrays
If both x and y are specified, the output array contains elements of x where condition is True, and elements from y elsewhere.
If only condition is given, return the tuple
condition.nonzero()
, the indices where condition is True.See also
nonzero
,choose
Notes
If x and y are given and input arrays are 1D, where is equivalent to:
[xv if c else yv for (c,xv,yv) in zip(condition,x,y)]
Examples
>>> np.where([[True, False], [True, True]], ... [[1, 2], [3, 4]], ... [[9, 8], [7, 6]]) array([[1, 8], [3, 4]])
>>> np.where([[0, 1], [1, 0]]) (array([0, 1]), array([1, 0]))
>>> x = np.arange(9.).reshape(3, 3) >>> np.where( x > 5 ) (array([2, 2, 2]), array([0, 1, 2])) >>> x[np.where( x > 3.0 )] # Note: result is 1D. array([ 4., 5., 6., 7., 8.]) >>> np.where(x < 5, x, 1) # Note: broadcasting. array([[ 0., 1., 2.], [ 3., 4., 1.], [1., 1., 1.]])
Find the indices of elements of x that are in goodvalues.
>>> goodvalues = [3, 4, 7] >>> ix = np.in1d(x.ravel(), goodvalues).reshape(x.shape) >>> ix array([[False, False, False], [ True, True, False], [False, True, False]], dtype=bool) >>> np.where(ix) (array([1, 1, 2]), array([0, 1, 1]))

dask.array.
zeros
()¶ Blocked variant of zeros
Follows the signature of zeros exactly except that it also requires a keyword argument chunks=(...)
Original signature follows below. zeros(shape, dtype=float, order=’C’)
Return a new array of given shape and type, filled with zeros.
Parameters: shape : int or sequence of ints
Shape of the new array, e.g.,
(2, 3)
or2
.dtype : datatype, optional
The desired datatype for the array, e.g., numpy.int8. Default is numpy.float64.
order : {‘C’, ‘F’}, optional
Whether to store multidimensional data in C or Fortrancontiguous (row or columnwise) order in memory.
Returns: out : ndarray
Array of zeros with the given shape, dtype, and order.
See also
Examples
>>> np.zeros(5) array([ 0., 0., 0., 0., 0.])
>>> np.zeros((5,), dtype=numpy.int) array([0, 0, 0, 0, 0])
>>> np.zeros((2, 1)) array([[ 0.], [ 0.]])
>>> s = (2,2) >>> np.zeros(s) array([[ 0., 0.], [ 0., 0.]])
>>> np.zeros((2,), dtype=[('x', 'i4'), ('y', 'i4')]) # custom dtype array([(0, 0), (0, 0)], dtype=[('x', '<i4'), ('y', '<i4')])

dask.array.linalg.
cholesky
(a, lower=False)¶ Returns the Cholesky decomposition, \(A = L L^*\) or \(A = U^* U\) of a Hermitian positivedefinite matrix A.
Parameters: a : (M, M) array_like
Matrix to be decomposed
lower : bool, optional
Whether to compute the upper or lower triangular Cholesky factorization. Default is uppertriangular.
Returns: c : (M, M) Array
Upper or lowertriangular Cholesky factor of a.

dask.array.linalg.
inv
(a)¶ Compute the inverse of a matrix with LU decomposition and forward / backward substitutions.
Parameters: a : array_like
Square matrix to be inverted.
Returns: ainv : Array
Inverse of the matrix a.

dask.array.linalg.
lstsq
(a, b)¶ Return the leastsquares solution to a linear matrix equation using QR decomposition.
Solves the equation a x = b by computing a vector x that minimizes the Euclidean 2norm  b  a x ^2. The equation may be under, well, or over determined (i.e., the number of linearly independent rows of a can be less than, equal to, or greater than its number of linearly independent columns). If a is square and of full rank, then x (but for roundoff error) is the “exact” solution of the equation.
Parameters: a : (M, N) array_like
“Coefficient” matrix.
b : (M,) array_like
Ordinate or “dependent variable” values.
Returns: x : (N,) Array
Leastsquares solution. If b is twodimensional, the solutions are in the K columns of x.
residuals : (1,) Array
Sums of residuals; squared Euclidean 2norm for each column in
b  a*x
.rank : Array
Rank of matrix a.
s : (min(M, N),) Array
Singular values of a.

dask.array.linalg.
lu
(a)¶ Compute the lu decomposition of a matrix.
Returns: p: Array, permutation matrix
l: Array, lower triangular matrix with unit diagonal.
u: Array, upper triangular matrix
Examples
>>> p, l, u = da.linalg.lu(x)

dask.array.linalg.
qr
(a, name=None)¶ Compute the qr factorization of a matrix.
Returns: q: Array, orthonormal
r: Array, uppertriangular
See also
np.linalg.qr
 Equivalent NumPy Operation
dask.array.linalg.tsqr
 Actual implementation with citation
Examples
>>> q, r = da.linalg.qr(x)

dask.array.linalg.
solve
(a, b, sym_pos=False)¶ Solve the equation
a x = b
forx
. By default, use LU decomposition and forward / backward substitutions. Whensym_pos
isTrue
, use Cholesky decomposition.Parameters: a : (M, M) array_like
A square matrix.
b : (M,) or (M, N) array_like
Righthand side matrix in
a x = b
.sym_pos : bool
Assume a is symmetric and positive definite. If
True
, use Cholesky decomposition.Returns: x : (M,) or (M, N) Array
Solution to the system
a x = b
. Shape of the return matches the shape of b.

dask.array.linalg.
solve_triangular
(a, b, lower=False)¶ Solve the equation a x = b for x, assuming a is a triangular matrix.
Parameters: a : (M, M) array_like
A triangular matrix
b : (M,) or (M, N) array_like
Righthand side matrix in a x = b
lower : bool, optional
Use only data contained in the lower triangle of a. Default is to use upper triangle.
Returns: x : (M,) or (M, N) array
Solution to the system a x = b. Shape of return matches b.

dask.array.linalg.
svd
(a, name=None)¶ Compute the singular value decomposition of a matrix.
Returns: u: Array, unitary / orthogonal
s: Array, singular values in decreasing order (largest first)
v: Array, unitary / orthogonal
See also
np.linalg.svd
 Equivalent NumPy Operation
dask.array.linalg.tsqr
 Actual implementation with citation
Examples
>>> u, s, v = da.linalg.svd(x)

dask.array.linalg.
svd_compressed
(a, k, n_power_iter=0, seed=None, name=None)¶ Randomly compressed rankk thin Singular Value Decomposition.
This computes the approximate singular value decomposition of a large array. This algorithm is generally faster than the normal algorithm but does not provide exact results. One can balance between performance and accuracy with input parameters (see below).
Parameters: a: Array
Input array
k: int
Rank of the desired thin SVD decomposition.
n_power_iter: int
Number of power iterations, useful when the singular values decay slowly. Error decreases exponentially as n_power_iter increases. In practice, set n_power_iter <= 4.
Returns: u: Array, unitary / orthogonal
s: Array, singular values in decreasing order (largest first)
v: Array, unitary / orthogonal
References
N. Halko, P. G. Martinsson, and J. A. Tropp. Finding structure with randomness: Probabilistic algorithms for constructing approximate matrix decompositions. SIAM Rev., Survey and Review section, Vol. 53, num. 2, pp. 217288, June 2011 http://arxiv.org/abs/0909.4061
Examples
>>> u, s, vt = svd_compressed(x, 20)

dask.array.linalg.
tsqr
(data, name=None, compute_svd=False)¶ Direct TallandSkinny QR algorithm
As presented in:
A. Benson, D. Gleich, and J. Demmel. Direct QR factorizations for tallandskinny matrices in MapReduce architectures. IEEE International Conference on Big Data, 2013. http://arxiv.org/abs/1301.1071This algorithm is used to compute both the QR decomposition and the Singular Value Decomposition. It requires that the input array have a single column of blocks, each of which fit in memory.
If blocks are of size
(n, k)
then this algorithm has memory use that scales asn**2 * k * nthreads
.Parameters: data: Array
compute_svd: bool
Whether to compute the SVD rather than the QR decomposition
See also

dask.array.ghost.
ghost
(x, depth, boundary)¶ Share boundaries between neighboring blocks
Parameters: x: da.Array
A dask array
depth: dict
The size of the shared boundary per axis
boundary: dict
The boundary condition on each axis. Options are ‘reflect’, ‘periodic’, ‘nearest’, ‘none’, or an array value. Such a value will fill the boundary with that value.
The depth input informs how many cells to overlap between neighboring
blocks ``{0: 2, 2: 5}`` means share two cells in 0 axis, 5 cells in 2 axis.
Axes missing from this input will not be overlapped.
Examples
>>> import numpy as np >>> import dask.array as da
>>> x = np.arange(64).reshape((8, 8)) >>> d = da.from_array(x, chunks=(4, 4)) >>> d.chunks ((4, 4), (4, 4))
>>> g = da.ghost.ghost(d, depth={0: 2, 1: 1}, ... boundary={0: 100, 1: 'reflect'}) >>> g.chunks ((8, 8), (6, 6))
>>> np.array(g) array([[100, 100, 100, 100, 100, 100, 100, 100, 100, 100, 100, 100], [100, 100, 100, 100, 100, 100, 100, 100, 100, 100, 100, 100], [ 0, 0, 1, 2, 3, 4, 3, 4, 5, 6, 7, 7], [ 8, 8, 9, 10, 11, 12, 11, 12, 13, 14, 15, 15], [ 16, 16, 17, 18, 19, 20, 19, 20, 21, 22, 23, 23], [ 24, 24, 25, 26, 27, 28, 27, 28, 29, 30, 31, 31], [ 32, 32, 33, 34, 35, 36, 35, 36, 37, 38, 39, 39], [ 40, 40, 41, 42, 43, 44, 43, 44, 45, 46, 47, 47], [ 16, 16, 17, 18, 19, 20, 19, 20, 21, 22, 23, 23], [ 24, 24, 25, 26, 27, 28, 27, 28, 29, 30, 31, 31], [ 32, 32, 33, 34, 35, 36, 35, 36, 37, 38, 39, 39], [ 40, 40, 41, 42, 43, 44, 43, 44, 45, 46, 47, 47], [ 48, 48, 49, 50, 51, 52, 51, 52, 53, 54, 55, 55], [ 56, 56, 57, 58, 59, 60, 59, 60, 61, 62, 63, 63], [100, 100, 100, 100, 100, 100, 100, 100, 100, 100, 100, 100], [100, 100, 100, 100, 100, 100, 100, 100, 100, 100, 100, 100]])

dask.array.ghost.
map_overlap
(x, func, depth, boundary=None, trim=True, **kwargs)¶

dask.array.
from_array
(x, chunks, name=None, lock=False, fancy=True) Create dask array from something that looks like an array
Input must have a
.shape
and support numpystyle slicing.Parameters: x : array_like
chunks : int, tuple
How to chunk the array. Must be one of the following forms:  A blocksize like 1000.  A blockshape like (1000, 1000).  Explicit sizes of all blocks along all dimensions
like ((1000, 1000, 500), (400, 400)).
name : str, optional
The key name to use for the array. Defaults to a hash of
x
.lock : bool or Lock, optional
If
x
doesn’t support concurrent reads then provide a lock here, or pass in True to have dask.array create one for you.fancy : bool, optional
If
x
doesn’t support fancy indexing (e.g. indexing with lists or arrays) then set to False. Default is True.Examples
>>> x = h5py.File('...')['/data/path'] >>> a = da.from_array(x, chunks=(1000, 1000))
If your underlying datastore does not support concurrent reads then include the
lock=True
keyword argument orlock=mylock
if you want multiple arrays to coordinate around the same lock.>>> a = da.from_array(x, chunks=(1000, 1000), lock=True)

dask.array.
from_delayed
(value, shape, dtype, name=None) Create a dask array from a dask delayed value
This routine is useful for constructing dask arrays in an adhoc fashion using dask delayed, particularly when combined with stack and concatenate.
The dask array will consist of a single chunk.
Examples
>>> from dask import delayed >>> value = delayed(np.ones)(5) >>> array = from_delayed(value, (5,), float) >>> array dask.array<fromva..., shape=(5,), dtype=float64, chunksize=(5,)> >>> array.compute() array([ 1., 1., 1., 1., 1.])

dask.array.
from_npy_stack
(dirname, mmap_mode='r')¶ Load dask array from stack of npy files
See
da.to_npy_stack
for docstringParameters: dirname: string
Directory of .npy files
mmap_mode: (None or ‘r’)
Read data in memory map mode

dask.array.
store
(sources, targets, lock=True, regions=None, compute=True, **kwargs) Store dask arrays in arraylike objects, overwrite data in target
This stores dask arrays into object that supports numpystyle setitem indexing. It stores values chunk by chunk so that it does not have to fill up memory. For best performance you can align the block size of the storage target with the block size of your array.
If your data fits in memory then you may prefer calling
np.array(myarray)
instead.Parameters: sources: Array or iterable of Arrays
targets: arraylike or iterable of arraylikes
These should support setitem syntax
target[10:20] = ...
lock: boolean or threading.Lock, optional
Whether or not to lock the data stores while storing. Pass True (lock each file individually), False (don’t lock) or a particular
threading.Lock
object to be shared among all writes.regions: tuple of slices or iterable of tuple of slices
Each
region
tuple inregions
should be such thattarget[region].shape = source.shape
for the corresponding source and target in sources and targets, respectively.compute: boolean, optional
If true compute immediately, return
dask.delayed.Delayed
otherwiseExamples
>>> x = ...
>>> import h5py >>> f = h5py.File('myfile.hdf5') >>> dset = f.create_dataset('/data', shape=x.shape, ... chunks=x.chunks, ... dtype='f8')
>>> store(x, dset)
Alternatively store many arrays at the same time
>>> store([x, y, z], [dset1, dset2, dset3])

dask.array.
to_hdf5
(filename, *args, **kwargs)¶ Store arrays in HDF5 file
This saves several dask arrays into several datapaths in an HDF5 file. It creates the necessary datasets and handles clean file opening/closing.
>>> da.to_hdf5('myfile.hdf5', '/x', x)
or
>>> da.to_hdf5('myfile.hdf5', {'/x': x, '/y': y})
Optionally provide arguments as though to
h5py.File.create_dataset
>>> da.to_hdf5('myfile.hdf5', '/x', x, compression='lzf', shuffle=True)
This can also be used as a method on a single Array
>>> x.to_hdf5('myfile.hdf5', '/x')
See also
da.store
,h5py.File.create_dataset

dask.array.
to_npy_stack
(dirname, x, axis=0)¶ Write dask array to a stack of .npy files
This partitions the dask.array along one axis and stores each block along that axis as a single .npy file in the specified directory
See also
Examples
>>> x = da.ones((5, 10, 10), chunks=(2, 4, 4)) >>> da.to_npy_stack('data/', x, axis=0)
The
.npy
files store numpy arrays forx[0:2], x[2:4], and x[4:5]
respectively, as is specified by the chunk size along the zeroth axis. The info file stores the dtype, chunks, and axis information of the array.You can load these stacks with the
da.from_npy_stack
function.>>> y = da.from_npy_stack('data/')

dask.array.fft.
fft
(a, n=None, axis=1)¶ Wrapping of numpy.fft.fft
The axis along which the FFT is applied must have a one chunk. To change the array’s chunking use dask.Array.rechunk.
The numpy.fft.fft docstring follows below:
Compute the onedimensional discrete Fourier Transform.
This function computes the onedimensional npoint discrete Fourier Transform (DFT) with the efficient Fast Fourier Transform (FFT) algorithm [CT].
Parameters: a : array_like
Input array, can be complex.
n : int, optional
Length of the transformed axis of the output. If n is smaller than the length of the input, the input is cropped. If it is larger, the input is padded with zeros. If n is not given, the length of the input (along the axis specified by axis) is used.
axis : int, optional
Axis over which to compute the FFT. If not given, the last axis is used.
Returns: out : complex ndarray
The truncated or zeropadded input, transformed along the axis indicated by axis, or the last one if axis is not specified.
Raises: IndexError
if axes is larger than the last axis of a.
See also
numpy.fft
 for definition of the DFT and conventions used.
ifft
 The inverse of fft.
fft2
 The twodimensional FFT.
fftn
 The ndimensional FFT.
rfftn
 The ndimensional FFT of real input.
fftfreq
 Frequency bins for given FFT parameters.
Notes
FFT (Fast Fourier Transform) refers to a way the discrete Fourier Transform (DFT) can be calculated efficiently, by using symmetries in the calculated terms. The symmetry is highest when n is a power of 2, and the transform is therefore most efficient for these sizes.
The DFT is defined, with the conventions used in this implementation, in the documentation for the numpy.fft module.
References
[CT] Cooley, James W., and John W. Tukey, 1965, “An algorithm for the machine calculation of complex Fourier series,” Math. Comput. 19: 297301. Examples
>>> np.fft.fft(np.exp(2j * np.pi * np.arange(8) / 8)) array([ 3.44505240e16 +1.14383329e17j, 8.00000000e+00 5.71092652e15j, 2.33482938e16 +1.22460635e16j, 1.64863782e15 +1.77635684e15j, 9.95839695e17 +2.33482938e16j, 0.00000000e+00 +1.66837030e15j, 1.14383329e17 +1.22460635e16j, 1.64863782e15 +1.77635684e15j])
>>> import matplotlib.pyplot as plt >>> t = np.arange(256) >>> sp = np.fft.fft(np.sin(t)) >>> freq = np.fft.fftfreq(t.shape[1]) >>> plt.plot(freq, sp.real, freq, sp.imag) [<matplotlib.lines.Line2D object at 0x...>, <matplotlib.lines.Line2D object at 0x...>] >>> plt.show()
In this example, real input has an FFT which is Hermitian, i.e., symmetric in the real part and antisymmetric in the imaginary part, as described in the numpy.fft documentation.

dask.array.fft.
ifft
(a, n=None, axis=1)¶ Wrapping of numpy.fft.ifft
The axis along which the FFT is applied must have a one chunk. To change the array’s chunking use dask.Array.rechunk.
The numpy.fft.ifft docstring follows below:
Compute the onedimensional inverse discrete Fourier Transform.
This function computes the inverse of the onedimensional npoint discrete Fourier transform computed by fft. In other words,
ifft(fft(a)) == a
to within numerical accuracy. For a general description of the algorithm and definitions, see numpy.fft.The input should be ordered in the same way as is returned by fft, i.e.,
a[0]
should contain the zero frequency term,a[1:n/2+1]
should contain the positivefrequency terms, anda[n/2+1:]
should contain the negativefrequency terms, in order of decreasingly negative frequency. See numpy.fft for details.Parameters: a : array_like
Input array, can be complex.
n : int, optional
Length of the transformed axis of the output. If n is smaller than the length of the input, the input is cropped. If it is larger, the input is padded with zeros. If n is not given, the length of the input (along the axis specified by axis) is used. See notes about padding issues.
axis : int, optional
Axis over which to compute the inverse DFT. If not given, the last axis is used.
Returns: out : complex ndarray
The truncated or zeropadded input, transformed along the axis indicated by axis, or the last one if axis is not specified.
Raises: IndexError
If axes is larger than the last axis of a.
See also
numpy.fft
 An introduction, with definitions and general explanations.
fft
 The onedimensional (forward) FFT, of which ifft is the inverse
ifft2
 The twodimensional inverse FFT.
ifftn
 The ndimensional inverse FFT.
Notes
If the input parameter n is larger than the size of the input, the input is padded by appending zeros at the end. Even though this is the common approach, it might lead to surprising results. If a different padding is desired, it must be performed before calling ifft.
Examples
>>> np.fft.ifft([0, 4, 0, 0]) array([ 1.+0.j, 0.+1.j, 1.+0.j, 0.1.j])
Create and plot a bandlimited signal with random phases:
>>> import matplotlib.pyplot as plt >>> t = np.arange(400) >>> n = np.zeros((400,), dtype=complex) >>> n[40:60] = np.exp(1j*np.random.uniform(0, 2*np.pi, (20,))) >>> s = np.fft.ifft(n) >>> plt.plot(t, s.real, 'b', t, s.imag, 'r') [<matplotlib.lines.Line2D object at 0x...>, <matplotlib.lines.Line2D object at 0x...>] >>> plt.legend(('real', 'imaginary')) <matplotlib.legend.Legend object at 0x...> >>> plt.show()

dask.array.fft.
hfft
(a, n=None, axis=1)¶ Wrapping of numpy.fft.hfft
The axis along which the FFT is applied must have a one chunk. To change the array’s chunking use dask.Array.rechunk.
The numpy.fft.hfft docstring follows below:
Compute the FFT of a signal whose spectrum has Hermitian symmetry.
Parameters: a : array_like
The input array.
n : int, optional
The length of the FFT.
axis : int, optional
The axis over which to compute the FFT, assuming Hermitian symmetry of the spectrum. Default is the last axis.
Returns: out : ndarray
The transformed input.
Notes
hfft/ihfft are a pair analogous to rfft/irfft, but for the opposite case: here the signal is real in the frequency domain and has Hermite symmetry in the time domain. So here it’s hfft for which you must supply the length of the result if it is to be odd:
ihfft(hfft(a), len(a)) == a
, within numerical accuracy.Examples
>>> signal = np.array([[1, 1.j], [1.j, 2]]) >>> np.conj(signal.T)  signal # check Hermitian symmetry array([[ 0.0.j, 0.+0.j], [ 0.+0.j, 0.0.j]]) >>> freq_spectrum = np.fft.hfft(signal) >>> freq_spectrum array([[ 1., 1.], [ 2., 2.]])

dask.array.fft.
ihfft
(a, n=None, axis=1)¶ Wrapping of numpy.fft.ihfft
The axis along which the FFT is applied must have a one chunk. To change the array’s chunking use dask.Array.rechunk.
The numpy.fft.ihfft docstring follows below:
Compute the inverse FFT of a signal whose spectrum has Hermitian symmetry.
Parameters: a : array_like
Input array.
n : int, optional
Length of the inverse FFT.
axis : int, optional
Axis over which to compute the inverse FFT, assuming Hermitian symmetry of the spectrum. Default is the last axis.
Returns: out : ndarray
The transformed input.
Notes
hfft/ihfft are a pair analogous to rfft/irfft, but for the opposite case: here the signal is real in the frequency domain and has Hermite symmetry in the time domain. So here it’s hfft for which you must supply the length of the result if it is to be odd:
ihfft(hfft(a), len(a)) == a
, within numerical accuracy.

dask.array.fft.
rfft
(a, n=None, axis=1)¶ Wrapping of numpy.fft.rfft
The axis along which the FFT is applied must have a one chunk. To change the array’s chunking use dask.Array.rechunk.
The numpy.fft.rfft docstring follows below:
Compute the onedimensional discrete Fourier Transform for real input.
This function computes the onedimensional npoint discrete Fourier Transform (DFT) of a realvalued array by means of an efficient algorithm called the Fast Fourier Transform (FFT).
Parameters: a : array_like
Input array
n : int, optional
Number of points along transformation axis in the input to use. If n is smaller than the length of the input, the input is cropped. If it is larger, the input is padded with zeros. If n is not given, the length of the input (along the axis specified by axis) is used.
axis : int, optional
Axis over which to compute the FFT. If not given, the last axis is used.
Returns: out : complex ndarray
The truncated or zeropadded input, transformed along the axis indicated by axis, or the last one if axis is not specified. If n is even, the length of the transformed axis is
(n/2)+1
. If n is odd, the length is(n+1)/2
.Raises: IndexError
If axis is larger than the last axis of a.
See also
Notes
When the DFT is computed for purely real input, the output is Hermitesymmetric, i.e. the negative frequency terms are just the complex conjugates of the corresponding positivefrequency terms, and the negativefrequency terms are therefore redundant. This function does not compute the negative frequency terms, and the length of the transformed axis of the output is therefore
n//2+1
.When
A = rfft(a)
and fs is the sampling frequency,A[0]
contains the zerofrequency term 0*fs, which is real due to Hermitian symmetry.If n is even,
A[1]
contains the term representing both positive and negative Nyquist frequency (+fs/2 and fs/2), and must also be purely real. If n is odd, there is no term at fs/2;A[1]
contains the largest positive frequency (fs/2*(n1)/n), and is complex in the general case.If the input a contains an imaginary part, it is silently discarded.
Examples
>>> np.fft.fft([0, 1, 0, 0]) array([ 1.+0.j, 0.1.j, 1.+0.j, 0.+1.j]) >>> np.fft.rfft([0, 1, 0, 0]) array([ 1.+0.j, 0.1.j, 1.+0.j])
Notice how the final element of the fft output is the complex conjugate of the second element, for real input. For rfft, this symmetry is exploited to compute only the nonnegative frequency terms.

dask.array.fft.
irfft
(a, n=None, axis=1)¶ Wrapping of numpy.fft.irfft
The axis along which the FFT is applied must have a one chunk. To change the array’s chunking use dask.Array.rechunk.
The numpy.fft.irfft docstring follows below:
Compute the inverse of the npoint DFT for real input.
This function computes the inverse of the onedimensional npoint discrete Fourier Transform of real input computed by rfft. In other words,
irfft(rfft(a), len(a)) == a
to within numerical accuracy. (See Notes below for whylen(a)
is necessary here.)The input is expected to be in the form returned by rfft, i.e. the real zerofrequency term followed by the complex positive frequency terms in order of increasing frequency. Since the discrete Fourier Transform of real input is Hermitesymmetric, the negative frequency terms are taken to be the complex conjugates of the corresponding positive frequency terms.
Parameters: a : array_like
The input array.
n : int, optional
Length of the transformed axis of the output. For n output points,
n//2+1
input points are necessary. If the input is longer than this, it is cropped. If it is shorter than this, it is padded with zeros. If n is not given, it is determined from the length of the input (along the axis specified by axis).axis : int, optional
Axis over which to compute the inverse FFT.
Returns: out : ndarray
The truncated or zeropadded input, transformed along the axis indicated by axis, or the last one if axis is not specified. The length of the transformed axis is n, or, if n is not given,
2*(m1)
where m is the length of the transformed axis of the input. To get an odd number of output points, n must be specified.Raises: IndexError
If axis is larger than the last axis of a.
See also
Notes
Returns the real valued npoint inverse discrete Fourier transform of a, where a contains the nonnegative frequency terms of a Hermitesymmetric sequence. n is the length of the result, not the input.
If you specify an n such that a must be zeropadded or truncated, the extra/removed values will be added/removed at high frequencies. One can thus resample a series to m points via Fourier interpolation by:
a_resamp = irfft(rfft(a), m)
.Examples
>>> np.fft.ifft([1, 1j, 1, 1j]) array([ 0.+0.j, 1.+0.j, 0.+0.j, 0.+0.j]) >>> np.fft.irfft([1, 1j, 1]) array([ 0., 1., 0., 0.])
Notice how the last term in the input to the ordinary ifft is the complex conjugate of the second term, and the output has zero imaginary part everywhere. When calling irfft, the negative frequencies are not specified, and the output array is purely real.

dask.array.random.
beta
(a, b, size=None)¶ The Beta distribution over
[0, 1]
.The Beta distribution is a special case of the Dirichlet distribution, and is related to the Gamma distribution. It has the probability distribution function
\[f(x; a,b) = \frac{1}{B(\alpha, \beta)} x^{\alpha  1} (1  x)^{\beta  1},\]where the normalisation, B, is the beta function,
\[B(\alpha, \beta) = \int_0^1 t^{\alpha  1} (1  t)^{\beta  1} dt.\]It is often seen in Bayesian inference and order statistics.
Parameters: a : float
Alpha, nonnegative.
b : float
Beta, nonnegative.
size : tuple of ints, optional
The number of samples to draw. The output is packed according to the size given.
Returns: out : ndarray
Array of the given shape, containing values drawn from a Beta distribution.

dask.array.random.
binomial
(n, p, size=None)¶ Draw samples from a binomial distribution.
Samples are drawn from a Binomial distribution with specified parameters, n trials and p probability of success where n an integer >= 0 and p is in the interval [0,1]. (n may be input as a float, but it is truncated to an integer in use)
Parameters: n : float (but truncated to an integer)
parameter, >= 0.
p : float
parameter, >= 0 and <=1.
size : {tuple, int}
Output shape. If the given shape is, e.g.,
(m, n, k)
, thenm * n * k
samples are drawn.Returns: samples : {ndarray, scalar}
where the values are all integers in [0, n].
See also
scipy.stats.distributions.binom
 probability density function, distribution or cumulative density function, etc.
Notes
The probability density for the Binomial distribution is
\[P(N) = \binom{n}{N}p^N(1p)^{nN},\]where \(n\) is the number of trials, \(p\) is the probability of success, and \(N\) is the number of successes.
When estimating the standard error of a proportion in a population by using a random sample, the normal distribution works well unless the product p*n <=5, where p = population proportion estimate, and n = number of samples, in which case the binomial distribution is used instead. For example, a sample of 15 people shows 4 who are left handed, and 11 who are right handed. Then p = 4/15 = 27%. 0.27*15 = 4, so the binomial distribution should be used in this case.
References
[R95] Dalgaard, Peter, “Introductory Statistics with R”, SpringerVerlag, 2002. [R96] Glantz, Stanton A. “Primer of Biostatistics.”, McGrawHill, Fifth Edition, 2002. [R97] Lentner, Marvin, “Elementary Applied Statistics”, Bogden and Quigley, 1972. [R98] Weisstein, Eric W. “Binomial Distribution.” From MathWorld–A Wolfram Web Resource. http://mathworld.wolfram.com/BinomialDistribution.html [R99] Wikipedia, “Binomialdistribution”, http://en.wikipedia.org/wiki/Binomial_distribution Examples
Draw samples from the distribution:
>> n, p = 10, .5 # number of trials, probability of each trial >> s = np.random.binomial(n, p, 1000) # result of flipping a coin 10 times, tested 1000 times.
A real world example. A company drills 9 wildcat oil exploration wells, each with an estimated probability of success of 0.1. All nine wells fail. What is the probability of that happening?
Let’s do 20,000 trials of the model, and count the number that generate zero positive results.
>> sum(np.random.binomial(9,0.1,20000)==0)/20000. answer = 0.38885, or 38%.

dask.array.random.
chisquare
(df, size=None)¶ Draw samples from a chisquare distribution.
When df independent random variables, each with standard normal distributions (mean 0, variance 1), are squared and summed, the resulting distribution is chisquare (see Notes). This distribution is often used in hypothesis testing.
Parameters: df : int
Number of degrees of freedom.
size : tuple of ints, int, optional
Size of the returned array. By default, a scalar is returned.
Returns: output : ndarray
Samples drawn from the distribution, packed in a sizeshaped array.
Raises: ValueError
When df <= 0 or when an inappropriate size (e.g.
size=1
) is given.Notes
The variable obtained by summing the squares of df independent, standard normally distributed random variables:
\[Q = \sum_{i=0}^{\mathtt{df}} X^2_i\]is chisquare distributed, denoted
\[Q \sim \chi^2_k.\]The probability density function of the chisquared distribution is
\[p(x) = \frac{(1/2)^{k/2}}{\Gamma(k/2)} x^{k/2  1} e^{x/2},\]where \(\Gamma\) is the gamma function,
\[\Gamma(x) = \int_0^{\infty} t^{x  1} e^{t} dt.\]References
NIST/SEMATECH eHandbook of Statistical Methods
Examples
>> np.random.chisquare(2,4) array([ 1.89920014, 9.00867716, 3.13710533, 5.62318272])

dask.array.random.
exponential
(scale=1.0, size=None)¶ Exponential distribution.
Its probability density function is
\[f(x; \frac{1}{\beta}) = \frac{1}{\beta} \exp(\frac{x}{\beta}),\]for
x > 0
and 0 elsewhere. \(\beta\) is the scale parameter, which is the inverse of the rate parameter \(\lambda = 1/\beta\). The rate parameter is an alternative, widely used parameterization of the exponential distribution [R102].The exponential distribution is a continuous analogue of the geometric distribution. It describes many common situations, such as the size of raindrops measured over many rainstorms [R100], or the time between page requests to Wikipedia [R101].
Parameters: scale : float
The scale parameter, \(\beta = 1/\lambda\).
size : tuple of ints
Number of samples to draw. The output is shaped according to size.
References
[R100] (1, 2) Peyton Z. Peebles Jr., “Probability, Random Variables and Random Signal Principles”, 4th ed, 2001, p. 57. [R101] (1, 2) “Poisson Process”, Wikipedia, http://en.wikipedia.org/wiki/Poisson_process [R102] (1, 2) “Exponential Distribution, Wikipedia, http://en.wikipedia.org/wiki/Exponential_distribution

dask.array.random.
f
(dfnum, dfden, size=None)¶ Draw samples from a F distribution.
Samples are drawn from an F distribution with specified parameters, dfnum (degrees of freedom in numerator) and dfden (degrees of freedom in denominator), where both parameters should be greater than zero.
The random variate of the F distribution (also known as the Fisher distribution) is a continuous probability distribution that arises in ANOVA tests, and is the ratio of two chisquare variates.
Parameters: dfnum : float
Degrees of freedom in numerator. Should be greater than zero.
dfden : float
Degrees of freedom in denominator. Should be greater than zero.
size : {tuple, int}, optional
Output shape. If the given shape is, e.g.,
(m, n, k)
, thenm * n * k
samples are drawn. By default only one sample is returned.Returns: samples : {ndarray, scalar}
Samples from the Fisher distribution.
See also
scipy.stats.distributions.f
 probability density function, distribution or cumulative density function, etc.
Notes
The F statistic is used to compare ingroup variances to betweengroup variances. Calculating the distribution depends on the sampling, and so it is a function of the respective degrees of freedom in the problem. The variable dfnum is the number of samples minus one, the betweengroups degrees of freedom, while dfden is the withingroups degrees of freedom, the sum of the number of samples in each group minus the number of groups.
References
[R103] Glantz, Stanton A. “Primer of Biostatistics.”, McGrawHill, Fifth Edition, 2002. [R104] Wikipedia, “Fdistribution”, http://en.wikipedia.org/wiki/Fdistribution Examples
An example from Glantz[1], pp 4740. Two groups, children of diabetics (25 people) and children from people without diabetes (25 controls). Fasting blood glucose was measured, case group had a mean value of 86.1, controls had a mean value of 82.2. Standard deviations were 2.09 and 2.49 respectively. Are these data consistent with the null hypothesis that the parents diabetic status does not affect their children’s blood glucose levels? Calculating the F statistic from the data gives a value of 36.01.
Draw samples from the distribution:
>> dfnum = 1. # between group degrees of freedom >> dfden = 48. # within groups degrees of freedom >> s = np.random.f(dfnum, dfden, 1000)
The lower bound for the top 1% of the samples is :
>> sort(s)[10] 7.61988120985
So there is about a 1% chance that the F statistic will exceed 7.62, the measured value is 36, so the null hypothesis is rejected at the 1% level.

dask.array.random.
gamma
(shape, scale=1.0, size=None)¶ Draw samples from a Gamma distribution.
Samples are drawn from a Gamma distribution with specified parameters, shape (sometimes designated “k”) and scale (sometimes designated “theta”), where both parameters are > 0.
Parameters: shape : scalar > 0
The shape of the gamma distribution.
scale : scalar > 0, optional
The scale of the gamma distribution. Default is equal to 1.
size : shape_tuple, optional
Output shape. If the given shape is, e.g.,
(m, n, k)
, thenm * n * k
samples are drawn.Returns: out : ndarray, float
Returns one sample unless size parameter is specified.
See also
scipy.stats.distributions.gamma
 probability density function, distribution or cumulative density function, etc.
Notes
The probability density for the Gamma distribution is
\[p(x) = x^{k1}\frac{e^{x/\theta}}{\theta^k\Gamma(k)},\]where \(k\) is the shape and \(\theta\) the scale, and \(\Gamma\) is the Gamma function.
The Gamma distribution is often used to model the times to failure of electronic components, and arises naturally in processes for which the waiting times between Poisson distributed events are relevant.
References
[R105] Weisstein, Eric W. “Gamma Distribution.” From MathWorld–A Wolfram Web Resource. http://mathworld.wolfram.com/GammaDistribution.html [R106] Wikipedia, “Gammadistribution”, http://en.wikipedia.org/wiki/Gammadistribution Examples
Draw samples from the distribution:
>> shape, scale = 2., 2. # mean and dispersion >> s = np.random.gamma(shape, scale, 1000)
Display the histogram of the samples, along with the probability density function:
>> import matplotlib.pyplot as plt >> import scipy.special as sps >> count, bins, ignored = plt.hist(s, 50, normed=True) >> y = bins**(shape1)*(np.exp(bins/scale) / .. (sps.gamma(shape)*scale**shape)) >> plt.plot(bins, y, linewidth=2, color=’r’) >> plt.show()

dask.array.random.
geometric
(p, size=None)¶ Draw samples from the geometric distribution.
Bernoulli trials are experiments with one of two outcomes: success or failure (an example of such an experiment is flipping a coin). The geometric distribution models the number of trials that must be run in order to achieve success. It is therefore supported on the positive integers,
k = 1, 2, ..
.The probability mass function of the geometric distribution is
\[f(k) = (1  p)^{k  1} p\]where p is the probability of success of an individual trial.
Parameters: p : float
The probability of success of an individual trial.
size : tuple of ints
Number of values to draw from the distribution. The output is shaped according to size.
Returns: out : ndarray
Samples from the geometric distribution, shaped according to size.
Examples
Draw ten thousand values from the geometric distribution, with the probability of an individual success equal to 0.35:
>> z = np.random.geometric(p=0.35, size=10000)
How many trials succeeded after a single run?
>> (z == 1).sum() / 10000. 0.34889999999999999 #random

dask.array.random.
gumbel
(loc=0.0, scale=1.0, size=None)¶ Gumbel distribution.
Draw samples from a Gumbel distribution with specified location and scale. For more information on the Gumbel distribution, see Notes and References below.
Parameters: loc : float
The location of the mode of the distribution.
scale : float
The scale parameter of the distribution.
size : tuple of ints
Output shape. If the given shape is, e.g.,
(m, n, k)
, thenm * n * k
samples are drawn.Returns: out : ndarray
The samples
See also
scipy.stats.gumbel_l
,scipy.stats.gumbel_r
scipy.stats.genextreme
 probability density function, distribution, or cumulative density function, etc. for each of the above
Notes
The Gumbel (or Smallest Extreme Value (SEV) or the Smallest Extreme Value Type I) distribution is one of a class of Generalized Extreme Value (GEV) distributions used in modeling extreme value problems. The Gumbel is a special case of the Extreme Value Type I distribution for maximums from distributions with “exponentiallike” tails.
The probability density for the Gumbel distribution is
\[p(x) = \frac{e^{(x  \mu)/ \beta}}{\beta} e^{ e^{(x  \mu)/ \beta}},\]where \(\mu\) is the mode, a location parameter, and \(\beta\) is the scale parameter.
The Gumbel (named for German mathematician Emil Julius Gumbel) was used very early in the hydrology literature, for modeling the occurrence of flood events. It is also used for modeling maximum wind speed and rainfall rates. It is a “fattailed” distribution  the probability of an event in the tail of the distribution is larger than if one used a Gaussian, hence the surprisingly frequent occurrence of 100year floods. Floods were initially modeled as a Gaussian process, which underestimated the frequency of extreme events.
It is one of a class of extreme value distributions, the Generalized Extreme Value (GEV) distributions, which also includes the Weibull and Frechet.
The function has a mean of \(\mu + 0.57721\beta\) and a variance of \(\frac{\pi^2}{6}\beta^2\).
References
Gumbel, E. J., Statistics of Extremes, New York: Columbia University Press, 1958.
Reiss, R.D. and Thomas, M., Statistical Analysis of Extreme Values from Insurance, Finance, Hydrology and Other Fields, Basel: Birkhauser Verlag, 2001.
Examples
Draw samples from the distribution:
>> mu, beta = 0, 0.1 # location and scale >> s = np.random.gumbel(mu, beta, 1000)
Display the histogram of the samples, along with the probability density function:
>> import matplotlib.pyplot as plt >> count, bins, ignored = plt.hist(s, 30, normed=True) >> plt.plot(bins, (1/beta)*np.exp((bins  mu)/beta) .. * np.exp( np.exp( (bins  mu) /beta) ), .. linewidth=2, color=’r’) >> plt.show()
Show how an extreme value distribution can arise from a Gaussian process and compare to a Gaussian:
>> means = [] >> maxima = [] >> for i in range(0,1000) : .. a = np.random.normal(mu, beta, 1000) .. means.append(a.mean()) .. maxima.append(a.max()) >> count, bins, ignored = plt.hist(maxima, 30, normed=True) >> beta = np.std(maxima)*np.pi/np.sqrt(6) >> mu = np.mean(maxima)  0.57721*beta >> plt.plot(bins, (1/beta)*np.exp((bins  mu)/beta) .. * np.exp(np.exp((bins  mu)/beta)), .. linewidth=2, color=’r’) >> plt.plot(bins, 1/(beta * np.sqrt(2 * np.pi)) .. * np.exp((bins  mu)**2 / (2 * beta**2)), .. linewidth=2, color=’g’) >> plt.show()

dask.array.random.
hypergeometric
(ngood, nbad, nsample, size=None)¶ Draw samples from a Hypergeometric distribution.
Samples are drawn from a Hypergeometric distribution with specified parameters, ngood (ways to make a good selection), nbad (ways to make a bad selection), and nsample = number of items sampled, which is less than or equal to the sum ngood + nbad.
Parameters: ngood : int or array_like
Number of ways to make a good selection. Must be nonnegative.
nbad : int or array_like
Number of ways to make a bad selection. Must be nonnegative.
nsample : int or array_like
Number of items sampled. Must be at least 1 and at most
ngood + nbad
.size : int or tuple of int
Output shape. If the given shape is, e.g.,
(m, n, k)
, thenm * n * k
samples are drawn.Returns: samples : ndarray or scalar
The values are all integers in [0, n].
See also
scipy.stats.distributions.hypergeom
 probability density function, distribution or cumulative density function, etc.
Notes
The probability density for the Hypergeometric distribution is
\[P(x) = \frac{\binom{m}{n}\binom{Nm}{nx}}{\binom{N}{n}},\]where \(0 \le x \le m\) and \(n+mN \le x \le n\)
for P(x) the probability of x successes, n = ngood, m = nbad, and N = number of samples.
Consider an urn with black and white marbles in it, ngood of them black and nbad are white. If you draw nsample balls without replacement, then the Hypergeometric distribution describes the distribution of black balls in the drawn sample.
Note that this distribution is very similar to the Binomial distribution, except that in this case, samples are drawn without replacement, whereas in the Binomial case samples are drawn with replacement (or the sample space is infinite). As the sample space becomes large, this distribution approaches the Binomial.
References
[R107] Lentner, Marvin, “Elementary Applied Statistics”, Bogden and Quigley, 1972. [R108] Weisstein, Eric W. “Hypergeometric Distribution.” From MathWorld–A Wolfram Web Resource. http://mathworld.wolfram.com/HypergeometricDistribution.html [R109] Wikipedia, “Hypergeometricdistribution”, http://en.wikipedia.org/wiki/Hypergeometricdistribution Examples
Draw samples from the distribution:
>> ngood, nbad, nsamp = 100, 2, 10 # number of good, number of bad, and number of samples >> s = np.random.hypergeometric(ngood, nbad, nsamp, 1000) >> hist(s) # note that it is very unlikely to grab both bad items
Suppose you have an urn with 15 white and 15 black marbles. If you pull 15 marbles at random, how likely is it that 12 or more of them are one color?
>> s = np.random.hypergeometric(15, 15, 15, 100000) >> sum(s>=12)/100000. + sum(s<=3)/100000. # answer = 0.003 .. pretty unlikely!

dask.array.random.
laplace
(loc=0.0, scale=1.0, size=None)¶ Draw samples from the Laplace or double exponential distribution with specified location (or mean) and scale (decay).
The Laplace distribution is similar to the Gaussian/normal distribution, but is sharper at the peak and has fatter tails. It represents the difference between two independent, identically distributed exponential random variables.
Parameters: loc : float
The position, \(\mu\), of the distribution peak.
scale : float
\(\lambda\), the exponential decay.
Notes
It has the probability density function
\[f(x; \mu, \lambda) = \frac{1}{2\lambda} \exp\left(\frac{x  \mu}{\lambda}\right).\]The first law of Laplace, from 1774, states that the frequency of an error can be expressed as an exponential function of the absolute magnitude of the error, which leads to the Laplace distribution. For many problems in Economics and Health sciences, this distribution seems to model the data better than the standard Gaussian distribution
References
[R110] Abramowitz, M. and Stegun, I. A. (Eds.). Handbook of Mathematical Functions with Formulas, Graphs, and Mathematical Tables, 9th printing. New York: Dover, 1972. [R111] The Laplace distribution and generalizations By Samuel Kotz, Tomasz J. Kozubowski, Krzysztof Podgorski, Birkhauser, 2001. [R112] Weisstein, Eric W. “Laplace Distribution.” From MathWorld–A Wolfram Web Resource. http://mathworld.wolfram.com/LaplaceDistribution.html [R113] Wikipedia, “Laplace distribution”, http://en.wikipedia.org/wiki/Laplace_distribution Examples
Draw samples from the distribution
>> loc, scale = 0., 1. >> s = np.random.laplace(loc, scale, 1000)
Display the histogram of the samples, along with the probability density function:
>> import matplotlib.pyplot as plt >> count, bins, ignored = plt.hist(s, 30, normed=True) >> x = np.arange(8., 8., .01) >> pdf = np.exp(abs(xloc/scale))/(2.*scale) >> plt.plot(x, pdf)
Plot Gaussian for comparison:
>> g = (1/(scale * np.sqrt(2 * np.pi)) * .. np.exp(  (x  loc)**2 / (2 * scale**2) )) >> plt.plot(x,g)

dask.array.random.
logistic
(loc=0.0, scale=1.0, size=None)¶ Draw samples from a Logistic distribution.
Samples are drawn from a Logistic distribution with specified parameters, loc (location or mean, also median), and scale (>0).
Parameters: loc : float
scale : float > 0.
size : {tuple, int}
Output shape. If the given shape is, e.g.,
(m, n, k)
, thenm * n * k
samples are drawn.Returns: samples : {ndarray, scalar}
where the values are all integers in [0, n].
See also
scipy.stats.distributions.logistic
 probability density function, distribution or cumulative density function, etc.
Notes
The probability density for the Logistic distribution is
\[P(x) = P(x) = \frac{e^{(x\mu)/s}}{s(1+e^{(x\mu)/s})^2},\]where \(\mu\) = location and \(s\) = scale.
The Logistic distribution is used in Extreme Value problems where it can act as a mixture of Gumbel distributions, in Epidemiology, and by the World Chess Federation (FIDE) where it is used in the Elo ranking system, assuming the performance of each player is a logistically distributed random variable.
References
[R114] Reiss, R.D. and Thomas M. (2001), Statistical Analysis of Extreme Values, from Insurance, Finance, Hydrology and Other Fields, Birkhauser Verlag, Basel, pp 132133. [R115] Weisstein, Eric W. “Logistic Distribution.” From MathWorld–A Wolfram Web Resource. http://mathworld.wolfram.com/LogisticDistribution.html [R116] Wikipedia, “Logisticdistribution”, http://en.wikipedia.org/wiki/Logisticdistribution Examples
Draw samples from the distribution:
>> loc, scale = 10, 1 >> s = np.random.logistic(loc, scale, 10000) >> count, bins, ignored = plt.hist(s, bins=50)
# plot against distribution
>> def logist(x, loc, scale): .. return exp((locx)/scale)/(scale*(1+exp((locx)/scale))**2) >> plt.plot(bins, logist(bins, loc, scale)*count.max()/.. logist(bins, loc, scale).max()) >> plt.show()

dask.array.random.
lognormal
(mean=0.0, sigma=1.0, size=None)¶ Return samples drawn from a lognormal distribution.
Draw samples from a lognormal distribution with specified mean, standard deviation, and array shape. Note that the mean and standard deviation are not the values for the distribution itself, but of the underlying normal distribution it is derived from.
Parameters: mean : float
Mean value of the underlying normal distribution
sigma : float, > 0.
Standard deviation of the underlying normal distribution
size : tuple of ints
Output shape. If the given shape is, e.g.,
(m, n, k)
, thenm * n * k
samples are drawn.Returns: samples : ndarray or float
The desired samples. An array of the same shape as size if given, if size is None a float is returned.
See also
scipy.stats.lognorm
 probability density function, distribution, cumulative density function, etc.
Notes
A variable x has a lognormal distribution if log(x) is normally distributed. The probability density function for the lognormal distribution is:
\[p(x) = \frac{1}{\sigma x \sqrt{2\pi}} e^{(\frac{(ln(x)\mu)^2}{2\sigma^2})}\]where \(\mu\) is the mean and \(\sigma\) is the standard deviation of the normally distributed logarithm of the variable. A lognormal distribution results if a random variable is the product of a large number of independent, identicallydistributed variables in the same way that a normal distribution results if the variable is the sum of a large number of independent, identicallydistributed variables.
References
Limpert, E., Stahel, W. A., and Abbt, M., “Lognormal Distributions across the Sciences: Keys and Clues,” BioScience, Vol. 51, No. 5, May, 2001. http://stat.ethz.ch/~stahel/lognormal/bioscience.pdf
Reiss, R.D. and Thomas, M., Statistical Analysis of Extreme Values, Basel: Birkhauser Verlag, 2001, pp. 3132.
Examples
Draw samples from the distribution:
>> mu, sigma = 3., 1. # mean and standard deviation >> s = np.random.lognormal(mu, sigma, 1000)
Display the histogram of the samples, along with the probability density function:
>> import matplotlib.pyplot as plt >> count, bins, ignored = plt.hist(s, 100, normed=True, align=’mid’)
>> x = np.linspace(min(bins), max(bins), 10000) >> pdf = (np.exp((np.log(x)  mu)**2 / (2 * sigma**2)) .. / (x * sigma * np.sqrt(2 * np.pi)))
>> plt.plot(x, pdf, linewidth=2, color=’r’) >> plt.axis(‘tight’) >> plt.show()
Demonstrate that taking the products of random samples from a uniform distribution can be fit well by a lognormal probability density function.
>> # Generate a thousand samples: each is the product of 100 random >> # values, drawn from a normal distribution. >> b = [] >> for i in range(1000): .. a = 10. + np.random.random(100) .. b.append(np.product(a))
>> b = np.array(b) / np.min(b) # scale values to be positive >> count, bins, ignored = plt.hist(b, 100, normed=True, align=’center’) >> sigma = np.std(np.log(b)) >> mu = np.mean(np.log(b))
>> x = np.linspace(min(bins), max(bins), 10000) >> pdf = (np.exp((np.log(x)  mu)**2 / (2 * sigma**2)) .. / (x * sigma * np.sqrt(2 * np.pi)))
>> plt.plot(x, pdf, color=’r’, linewidth=2) >> plt.show()

dask.array.random.
logseries
(p, size=None)¶ Draw samples from a Logarithmic Series distribution.
Samples are drawn from a Log Series distribution with specified parameter, p (probability, 0 < p < 1).
Parameters: loc : float
scale : float > 0.
size : {tuple, int}
Output shape. If the given shape is, e.g.,
(m, n, k)
, thenm * n * k
samples are drawn.Returns: samples : {ndarray, scalar}
where the values are all integers in [0, n].
See also
scipy.stats.distributions.logser
 probability density function, distribution or cumulative density function, etc.
Notes
The probability density for the Log Series distribution is
\[P(k) = \frac{p^k}{k \ln(1p)},\]where p = probability.
The Log Series distribution is frequently used to represent species richness and occurrence, first proposed by Fisher, Corbet, and Williams in 1943 [2]. It may also be used to model the numbers of occupants seen in cars [3].
References
[R117] Buzas, Martin A.; Culver, Stephen J., Understanding regional species diversity through the log series distribution of occurrences: BIODIVERSITY RESEARCH Diversity & Distributions, Volume 5, Number 5, September 1999 , pp. 187195(9). [R118] Fisher, R.A,, A.S. Corbet, and C.B. Williams. 1943. The relation between the number of species and the number of individuals in a random sample of an animal population. Journal of Animal Ecology, 12:4258. [R119] D. J. Hand, F. Daly, D. Lunn, E. Ostrowski, A Handbook of Small Data Sets, CRC Press, 1994. [R120] Wikipedia, “Logarithmicdistribution”, http://en.wikipedia.org/wiki/Logarithmicdistribution Examples
Draw samples from the distribution:
>> a = .6 >> s = np.random.logseries(a, 10000) >> count, bins, ignored = plt.hist(s)
# plot against distribution
>> def logseries(k, p): .. return p**k/(k*log(1p)) >> plt.plot(bins, logseries(bins, a)*count.max()/
logseries(bins, a).max(), ‘r’)>> plt.show()

dask.array.random.
negative_binomial
(n, p, size=None)¶ Draw samples from a negative_binomial distribution.
Samples are drawn from a negative_Binomial distribution with specified parameters, n trials and p probability of success where n is an integer > 0 and p is in the interval [0, 1].
Parameters: n : int
Parameter, > 0.
p : float
Parameter, >= 0 and <=1.
size : int or tuple of ints
Output shape. If the given shape is, e.g.,
(m, n, k)
, thenm * n * k
samples are drawn.Returns: samples : int or ndarray of ints
Drawn samples.
Notes
The probability density for the Negative Binomial distribution is
\[P(N;n,p) = \binom{N+n1}{n1}p^{n}(1p)^{N},\]where \(n1\) is the number of successes, \(p\) is the probability of success, and \(N+n1\) is the number of trials.
The negative binomial distribution gives the probability of n1 successes and N failures in N+n1 trials, and success on the (N+n)th trial.
If one throws a die repeatedly until the third time a “1” appears, then the probability distribution of the number of non“1”s that appear before the third “1” is a negative binomial distribution.
References
[R121] Weisstein, Eric W. “Negative Binomial Distribution.” From MathWorld–A Wolfram Web Resource. http://mathworld.wolfram.com/NegativeBinomialDistribution.html [R122] Wikipedia, “Negative binomial distribution”, http://en.wikipedia.org/wiki/Negative_binomial_distribution Examples
Draw samples from the distribution:
A real world example. A company drills wildcat oil exploration wells, each with an estimated probability of success of 0.1. What is the probability of having one success for each successive well, that is what is the probability of a single success after drilling 5 wells, after 6 wells, etc.?
>> s = np.random.negative_binomial(1, 0.1, 100000) >> for i in range(1, 11): .. probability = sum(s<i) / 100000. .. print i, “wells drilled, probability of one success =”, probability

dask.array.random.
noncentral_chisquare
(df, nonc, size=None)¶ Draw samples from a noncentral chisquare distribution.
The noncentral \(\chi^2\) distribution is a generalisation of the \(\chi^2\) distribution.
Parameters: df : int
Degrees of freedom, should be >= 1.
nonc : float
Noncentrality, should be > 0.
size : int or tuple of ints
Shape of the output.
Notes
The probability density function for the noncentral Chisquare distribution is
\[P(x;df,nonc) = \sum^{\infty}_{i=0} \frac{e^{nonc/2}(nonc/2)^{i}}{i!}P_{Y_{df+2i}}(x),\]where \(Y_{q}\) is the Chisquare with q degrees of freedom.
In Delhi (2007), it is noted that the noncentral chisquare is useful in bombing and coverage problems, the probability of killing the point target given by the noncentral chisquared distribution.
References
[R123] Delhi, M.S. Holla, “On a noncentral chisquare distribution in the analysis of weapon systems effectiveness”, Metrika, Volume 15, Number 1 / December, 1970. [R124] Wikipedia, “Noncentral chisquare distribution” http://en.wikipedia.org/wiki/Noncentral_chisquare_distribution Examples
Draw values from the distribution and plot the histogram
>> import matplotlib.pyplot as plt >> values = plt.hist(np.random.noncentral_chisquare(3, 20, 100000), .. bins=200, normed=True) >> plt.show()
Draw values from a noncentral chisquare with very small noncentrality, and compare to a chisquare.
>> plt.figure() >> values = plt.hist(np.random.noncentral_chisquare(3, .0000001, 100000), .. bins=np.arange(0., 25, .1), normed=True) >> values2 = plt.hist(np.random.chisquare(3, 100000), .. bins=np.arange(0., 25, .1), normed=True) >> plt.plot(values[1][0:1], values[0]values2[0], ‘ob’) >> plt.show()
Demonstrate how large values of noncentrality lead to a more symmetric distribution.
>> plt.figure() >> values = plt.hist(np.random.noncentral_chisquare(3, 20, 100000), .. bins=200, normed=True) >> plt.show()

dask.array.random.
noncentral_f
(dfnum, dfden, nonc, size=None)¶ Draw samples from the noncentral F distribution.
Samples are drawn from an F distribution with specified parameters, dfnum (degrees of freedom in numerator) and dfden (degrees of freedom in denominator), where both parameters > 1. nonc is the noncentrality parameter.
Parameters: dfnum : int
Parameter, should be > 1.
dfden : int
Parameter, should be > 1.
nonc : float
Parameter, should be >= 0.
size : int or tuple of ints
Output shape. If the given shape is, e.g.,
(m, n, k)
, thenm * n * k
samples are drawn.Returns: samples : scalar or ndarray
Drawn samples.
Notes
When calculating the power of an experiment (power = probability of rejecting the null hypothesis when a specific alternative is true) the noncentral F statistic becomes important. When the null hypothesis is true, the F statistic follows a central F distribution. When the null hypothesis is not true, then it follows a noncentral F statistic.
References
Weisstein, Eric W. “Noncentral FDistribution.” From MathWorld–A Wolfram Web Resource. http://mathworld.wolfram.com/NoncentralFDistribution.html
Wikipedia, “Noncentral F distribution”, http://en.wikipedia.org/wiki/Noncentral_Fdistribution
Examples
In a study, testing for a specific alternative to the null hypothesis requires use of the Noncentral F distribution. We need to calculate the area in the tail of the distribution that exceeds the value of the F distribution for the null hypothesis. We’ll plot the two probability distributions for comparison.
>> dfnum = 3 # between group deg of freedom >> dfden = 20 # within groups degrees of freedom >> nonc = 3.0 >> nc_vals = np.random.noncentral_f(dfnum, dfden, nonc, 1000000) >> NF = np.histogram(nc_vals, bins=50, normed=True) >> c_vals = np.random.f(dfnum, dfden, 1000000) >> F = np.histogram(c_vals, bins=50, normed=True) >> plt.plot(F[1][1:], F[0]) >> plt.plot(NF[1][1:], NF[0]) >> plt.show()

dask.array.random.
normal
(loc=0.0, scale=1.0, size=None)¶ Draw random samples from a normal (Gaussian) distribution.
The probability density function of the normal distribution, first derived by De Moivre and 200 years later by both Gauss and Laplace independently [R126], is often called the bell curve because of its characteristic shape (see the example below).
The normal distributions occurs often in nature. For example, it describes the commonly occurring distribution of samples influenced by a large number of tiny, random disturbances, each with its own unique distribution [R126].
Parameters: loc : float
Mean (“centre”) of the distribution.
scale : float
Standard deviation (spread or “width”) of the distribution.
size : tuple of ints
Output shape. If the given shape is, e.g.,
(m, n, k)
, thenm * n * k
samples are drawn.See also
scipy.stats.distributions.norm
 probability density function, distribution or cumulative density function, etc.
Notes
The probability density for the Gaussian distribution is
\[p(x) = \frac{1}{\sqrt{ 2 \pi \sigma^2 }} e^{  \frac{ (x  \mu)^2 } {2 \sigma^2} },\]where \(\mu\) is the mean and \(\sigma\) the standard deviation. The square of the standard deviation, \(\sigma^2\), is called the variance.
The function has its peak at the mean, and its “spread” increases with the standard deviation (the function reaches 0.607 times its maximum at \(x + \sigma\) and \(x  \sigma\) [R126]). This implies that numpy.random.normal is more likely to return samples lying close to the mean, rather than those far away.
References
[R125] Wikipedia, “Normal distribution”, http://en.wikipedia.org/wiki/Normal_distribution [R126] (1, 2, 3, 4) P. R. Peebles Jr., “Central Limit Theorem” in “Probability, Random Variables and Random Signal Principles”, 4th ed., 2001, pp. 51, 51, 125. Examples
Draw samples from the distribution:
>> mu, sigma = 0, 0.1 # mean and standard deviation >> s = np.random.normal(mu, sigma, 1000)
Verify the mean and the variance:
>> abs(mu  np.mean(s)) < 0.01 True
>> abs(sigma  np.std(s, ddof=1)) < 0.01 True
Display the histogram of the samples, along with the probability density function:
>> import matplotlib.pyplot as plt >> count, bins, ignored = plt.hist(s, 30, normed=True) >> plt.plot(bins, 1/(sigma * np.sqrt(2 * np.pi)) * .. np.exp(  (bins  mu)**2 / (2 * sigma**2) ), .. linewidth=2, color=’r’) >> plt.show()

dask.array.random.
pareto
(a, size=None)¶ Draw samples from a Pareto II or Lomax distribution with specified shape.
The Lomax or Pareto II distribution is a shifted Pareto distribution. The classical Pareto distribution can be obtained from the Lomax distribution by adding the location parameter m, see below. The smallest value of the Lomax distribution is zero while for the classical Pareto distribution it is m, where the standard Pareto distribution has location m=1. Lomax can also be considered as a simplified version of the Generalized Pareto distribution (available in SciPy), with the scale set to one and the location set to zero.
The Pareto distribution must be greater than zero, and is unbounded above. It is also known as the “8020 rule”. In this distribution, 80 percent of the weights are in the lowest 20 percent of the range, while the other 20 percent fill the remaining 80 percent of the range.
Parameters: shape : float, > 0.
Shape of the distribution.
size : tuple of ints
Output shape. If the given shape is, e.g.,
(m, n, k)
, thenm * n * k
samples are drawn.See also
scipy.stats.distributions.lomax.pdf
 probability density function, distribution or cumulative density function, etc.
scipy.stats.distributions.genpareto.pdf
 probability density function, distribution or cumulative density function, etc.
Notes
The probability density for the Pareto distribution is
\[p(x) = \frac{am^a}{x^{a+1}}\]where \(a\) is the shape and \(m\) the location
The Pareto distribution, named after the Italian economist Vilfredo Pareto, is a power law probability distribution useful in many real world problems. Outside the field of economics it is generally referred to as the Bradford distribution. Pareto developed the distribution to describe the distribution of wealth in an economy. It has also found use in insurance, web page access statistics, oil field sizes, and many other problems, including the download frequency for projects in Sourceforge [1]. It is one of the socalled “fattailed” distributions.
References
[R127] Francis Hunt and Paul Johnson, On the Pareto Distribution of Sourceforge projects. [R128] Pareto, V. (1896). Course of Political Economy. Lausanne. [R129] Reiss, R.D., Thomas, M.(2001), Statistical Analysis of Extreme Values, Birkhauser Verlag, Basel, pp 2330. [R130] Wikipedia, “Pareto distribution”, http://en.wikipedia.org/wiki/Pareto_distribution Examples
Draw samples from the distribution:
>> a, m = 3., 1. # shape and mode >> s = np.random.pareto(a, 1000) + m
Display the histogram of the samples, along with the probability density function:
>> import matplotlib.pyplot as plt >> count, bins, ignored = plt.hist(s, 100, normed=True, align=’center’) >> fit = a*m**a/bins**(a+1) >> plt.plot(bins, max(count)*fit/max(fit),linewidth=2, color=’r’) >> plt.show()

dask.array.random.
poisson
(lam=1.0, size=None)¶ Draw samples from a Poisson distribution.
The Poisson distribution is the limit of the Binomial distribution for large N.
Parameters: lam : float
Expectation of interval, should be >= 0.
size : int or tuple of ints, optional
Output shape. If the given shape is, e.g.,
(m, n, k)
, thenm * n * k
samples are drawn.Notes
The Poisson distribution
\[f(k; \lambda)=\frac{\lambda^k e^{\lambda}}{k!}\]For events with an expected separation \(\lambda\) the Poisson distribution \(f(k; \lambda)\) describes the probability of \(k\) events occurring within the observed interval \(\lambda\).
Because the output is limited to the range of the C long type, a ValueError is raised when lam is within 10 sigma of the maximum representable value.
References
[R131] Weisstein, Eric W. “Poisson Distribution.” From MathWorld–A Wolfram Web Resource. http://mathworld.wolfram.com/PoissonDistribution.html [R132] Wikipedia, “Poisson distribution”, http://en.wikipedia.org/wiki/Poisson_distribution Examples
Draw samples from the distribution:
>> import numpy as np >> s = np.random.poisson(5, 10000)
Display histogram of the sample:
>> import matplotlib.pyplot as plt >> count, bins, ignored = plt.hist(s, 14, normed=True) >> plt.show()

dask.array.random.
power
(a, size=None)¶ Draws samples in [0, 1] from a power distribution with positive exponent a  1.
Also known as the power function distribution.
Parameters: a : float
parameter, > 0
size : tuple of ints
 Output shape. If the given shape is, e.g.,
(m, n, k)
, then m * n * k
samples are drawn.
Returns: samples : {ndarray, scalar}
The returned samples lie in [0, 1].
Raises: ValueError
If a<1.
Notes
The probability density function is
\[\begin{split}P(x; a) = ax^{a1}, 0 \le x \le 1, a>0.\end{split}\]The power function distribution is just the inverse of the Pareto distribution. It may also be seen as a special case of the Beta distribution.
It is used, for example, in modeling the overreporting of insurance claims.
References
[R133] Christian Kleiber, Samuel Kotz, “Statistical size distributions in economics and actuarial sciences”, Wiley, 2003. [R134] Heckert, N. A. and Filliben, James J. (2003). NIST Handbook 148: Dataplot Reference Manual, Volume 2: Let Subcommands and Library Functions”, National Institute of Standards and Technology Handbook Series, June 2003. http://www.itl.nist.gov/div898/software/dataplot/refman2/auxillar/powpdf.pdf Examples
Draw samples from the distribution:
>> a = 5. # shape >> samples = 1000 >> s = np.random.power(a, samples)
Display the histogram of the samples, along with the probability density function:
>> import matplotlib.pyplot as plt >> count, bins, ignored = plt.hist(s, bins=30) >> x = np.linspace(0, 1, 100) >> y = a*x**(a1.) >> normed_y = samples*np.diff(bins)[0]*y >> plt.plot(x, normed_y) >> plt.show()
Compare the power function distribution to the inverse of the Pareto.
>> from scipy import stats >> rvs = np.random.power(5, 1000000) >> rvsp = np.random.pareto(5, 1000000) >> xx = np.linspace(0,1,100) >> powpdf = stats.powerlaw.pdf(xx,5)
>> plt.figure() >> plt.hist(rvs, bins=50, normed=True) >> plt.plot(xx,powpdf,’r‘) >> plt.title(‘np.random.power(5)’)
>> plt.figure() >> plt.hist(1./(1.+rvsp), bins=50, normed=True) >> plt.plot(xx,powpdf,’r‘) >> plt.title(‘inverse of 1 + np.random.pareto(5)’)
>> plt.figure() >> plt.hist(1./(1.+rvsp), bins=50, normed=True) >> plt.plot(xx,powpdf,’r‘) >> plt.title(‘inverse of stats.pareto(5)’)
 Output shape. If the given shape is, e.g.,

dask.array.random.
random
(size=None)¶ Return random floats in the halfopen interval [0.0, 1.0).
Results are from the “continuous uniform” distribution over the stated interval. To sample \(Unif[a, b), b > a\) multiply the output of random_sample by (ba) and add a:
(b  a) * random_sample() + a
Parameters: size : int or tuple of ints, optional
Defines the shape of the returned array of random floats. If None (the default), returns a single float.
Returns: out : float or ndarray of floats
Array of random floats of shape size (unless
size=None
, in which case a single float is returned).Examples
>> np.random.random_sample() 0.47108547995356098 >> type(np.random.random_sample()) <type ‘float’> >> np.random.random_sample((5,)) array([ 0.30220482, 0.86820401, 0.1654503 , 0.11659149, 0.54323428])
Threebytwo array of random numbers from [5, 0):
>> 5 * np.random.random_sample((3, 2))  5 array([[3.99149989, 0.52338984],
[2.99091858, 0.79479508], [1.23204345, 1.75224494]])

dask.array.random.
random_sample
(size=None)¶ Return random floats in the halfopen interval [0.0, 1.0).
Results are from the “continuous uniform” distribution over the stated interval. To sample \(Unif[a, b), b > a\) multiply the output of random_sample by (ba) and add a:
(b  a) * random_sample() + a
Parameters: size : int or tuple of ints, optional
Defines the shape of the returned array of random floats. If None (the default), returns a single float.
Returns: out : float or ndarray of floats
Array of random floats of shape size (unless
size=None
, in which case a single float is returned).Examples
>> np.random.random_sample() 0.47108547995356098 >> type(np.random.random_sample()) <type ‘float’> >> np.random.random_sample((5,)) array([ 0.30220482, 0.86820401, 0.1654503 , 0.11659149, 0.54323428])
Threebytwo array of random numbers from [5, 0):
>> 5 * np.random.random_sample((3, 2))  5 array([[3.99149989, 0.52338984],
[2.99091858, 0.79479508], [1.23204345, 1.75224494]])

dask.array.random.
rayleigh
(scale=1.0, size=None)¶ Draw samples from a Rayleigh distribution.
The \(\chi\) and Weibull distributions are generalizations of the Rayleigh.
Parameters: scale : scalar
Scale, also equals the mode. Should be >= 0.
size : int or tuple of ints, optional
Shape of the output. Default is None, in which case a single value is returned.
Notes
The probability density function for the Rayleigh distribution is
\[P(x;scale) = \frac{x}{scale^2}e^{\frac{x^2}{2 \cdotp scale^2}}\]The Rayleigh distribution arises if the wind speed and wind direction are both gaussian variables, then the vector wind velocity forms a Rayleigh distribution. The Rayleigh distribution is used to model the expected output from wind turbines.
References
[R135] Brighton Webs Ltd., Rayleigh Distribution, http://www.brightonwebs.co.uk/distributions/rayleigh.asp [R136] Wikipedia, “Rayleigh distribution” http://en.wikipedia.org/wiki/Rayleigh_distribution Examples
Draw values from the distribution and plot the histogram
>> values = hist(np.random.rayleigh(3, 100000), bins=200, normed=True)
Wave heights tend to follow a Rayleigh distribution. If the mean wave height is 1 meter, what fraction of waves are likely to be larger than 3 meters?
>> meanvalue = 1 >> modevalue = np.sqrt(2 / np.pi) * meanvalue >> s = np.random.rayleigh(modevalue, 1000000)
The percentage of waves larger than 3 meters is:
>> 100.*sum(s>3)/1000000. 0.087300000000000003

dask.array.random.
standard_cauchy
(size=None)¶ Standard Cauchy distribution with mode = 0.
Also known as the Lorentz distribution.
Parameters: size : int or tuple of ints
Shape of the output.
Returns: samples : ndarray or scalar
The drawn samples.
Notes
The probability density function for the full Cauchy distribution is
\[P(x; x_0, \gamma) = \frac{1}{\pi \gamma \bigl[ 1+ (\frac{xx_0}{\gamma})^2 \bigr] }\]and the Standard Cauchy distribution just sets \(x_0=0\) and \(\gamma=1\)
The Cauchy distribution arises in the solution to the driven harmonic oscillator problem, and also describes spectral line broadening. It also describes the distribution of values at which a line tilted at a random angle will cut the x axis.
When studying hypothesis tests that assume normality, seeing how the tests perform on data from a Cauchy distribution is a good indicator of their sensitivity to a heavytailed distribution, since the Cauchy looks very much like a Gaussian distribution, but with heavier tails.
References
[R137] NIST/SEMATECH eHandbook of Statistical Methods, “Cauchy Distribution”, http://www.itl.nist.gov/div898/handbook/eda/section3/eda3663.htm [R138] Weisstein, Eric W. “Cauchy Distribution.” From MathWorld–A Wolfram Web Resource. http://mathworld.wolfram.com/CauchyDistribution.html [R139] Wikipedia, “Cauchy distribution” http://en.wikipedia.org/wiki/Cauchy_distribution Examples
Draw samples and plot the distribution:
>> s = np.random.standard_cauchy(1000000) >> s = s[(s>25) & (s<25)] # truncate distribution so it plots well >> plt.hist(s, bins=100) >> plt.show()

dask.array.random.
standard_exponential
(size=None)¶ Draw samples from the standard exponential distribution.
standard_exponential is identical to the exponential distribution with a scale parameter of 1.
Parameters: size : int or tuple of ints
Shape of the output.
Returns: out : float or ndarray
Drawn samples.
Examples
Output a 3x8000 array:
>> n = np.random.standard_exponential((3, 8000))

dask.array.random.
standard_gamma
(shape, size=None)¶ Draw samples from a Standard Gamma distribution.
Samples are drawn from a Gamma distribution with specified parameters, shape (sometimes designated “k”) and scale=1.
Parameters: shape : float
Parameter, should be > 0.
size : int or tuple of ints
Output shape. If the given shape is, e.g.,
(m, n, k)
, thenm * n * k
samples are drawn.Returns: samples : ndarray or scalar
The drawn samples.
See also
scipy.stats.distributions.gamma
 probability density function, distribution or cumulative density function, etc.
Notes
The probability density for the Gamma distribution is
\[p(x) = x^{k1}\frac{e^{x/\theta}}{\theta^k\Gamma(k)},\]where \(k\) is the shape and \(\theta\) the scale, and \(\Gamma\) is the Gamma function.
The Gamma distribution is often used to model the times to failure of electronic components, and arises naturally in processes for which the waiting times between Poisson distributed events are relevant.
References
[R140] Weisstein, Eric W. “Gamma Distribution.” From MathWorld–A Wolfram Web Resource. http://mathworld.wolfram.com/GammaDistribution.html [R141] Wikipedia, “Gammadistribution”, http://en.wikipedia.org/wiki/Gammadistribution Examples
Draw samples from the distribution:
>> shape, scale = 2., 1. # mean and width >> s = np.random.standard_gamma(shape, 1000000)
Display the histogram of the samples, along with the probability density function:
>> import matplotlib.pyplot as plt >> import scipy.special as sps >> count, bins, ignored = plt.hist(s, 50, normed=True) >> y = bins**(shape1) * ((np.exp(bins/scale))/ .. (sps.gamma(shape) * scale**shape)) >> plt.plot(bins, y, linewidth=2, color=’r’) >> plt.show()

dask.array.random.
standard_normal
(size=None)¶ Returns samples from a Standard Normal distribution (mean=0, stdev=1).
Parameters: size : int or tuple of ints, optional
Output shape. Default is None, in which case a single value is returned.
Returns: out : float or ndarray
Drawn samples.
Examples
>> s = np.random.standard_normal(8000) >> s array([ 0.6888893 , 0.78096262, 0.89086505, .., 0.49876311, #random
0.38672696, 0.4685006 ]) #random>> s.shape (8000,) >> s = np.random.standard_normal(size=(3, 4, 2)) >> s.shape (3, 4, 2)

dask.array.random.
standard_t
(df, size=None)¶ Standard Student’s t distribution with df degrees of freedom.
A special case of the hyperbolic distribution. As df gets large, the result resembles that of the standard normal distribution (standard_normal).
Parameters: df : int
Degrees of freedom, should be > 0.
size : int or tuple of ints, optional
Output shape. Default is None, in which case a single value is returned.
Returns: samples : ndarray or scalar
Drawn samples.
Notes
The probability density function for the t distribution is
\[P(x, df) = \frac{\Gamma(\frac{df+1}{2})}{\sqrt{\pi df} \Gamma(\frac{df}{2})}\Bigl( 1+\frac{x^2}{df} \Bigr)^{(df+1)/2}\]The t test is based on an assumption that the data come from a Normal distribution. The t test provides a way to test whether the sample mean (that is the mean calculated from the data) is a good estimate of the true mean.
The derivation of the tdistribution was forst published in 1908 by William Gisset while working for the Guinness Brewery in Dublin. Due to proprietary issues, he had to publish under a pseudonym, and so he used the name Student.
References
[R142] (1, 2) Dalgaard, Peter, “Introductory Statistics With R”, Springer, 2002. [R143] Wikipedia, “Student’s tdistribution” http://en.wikipedia.org/wiki/Student’s_tdistribution Examples
From Dalgaard page 83 [R142], suppose the daily energy intake for 11 women in Kj is: