tensor.io – Tensor IO Ops

File operation

MPI operation

Details

class theano.tensor.io.LoadFromDisk(dtype, broadcastable, mmap_mode=None)[source]

An operation to load an array from disk.

See also

load

Notes

Non-differentiable.

make_node(path)[source]

Construct an Apply node that represent the application of this operation to the given inputs.

This must be implemented by sub-classes.

Returns:node – The constructed Apply node.
Return type:Apply
perform(node, inp, out)[source]

Calculate the function on the inputs and put the variables in the output storage.

Parameters:
  • node (Apply) – The symbolic Apply node that represents this computation.
  • inputs (Sequence) – Immutable sequence of non-symbolic/numeric inputs. These are the values of each Variable in node.inputs.
  • output_storage (list of list) – List of mutable single-element lists (do not change the length of these lists). Each sub-list corresponds to value of each Variable in node.outputs. The primary purpose of this method is to set the values of these sub-lists.
  • params (tuple) – A tuple containing the values of each entry in __props__.

Notes

The output_storage list might contain data. If an element of output_storage is not None, it has to be of the right type, for instance, for a TensorVariable, it has to be a NumPy ndarray with the right number of dimensions and the correct dtype. Its shape and stride pattern can be arbitrary. It is not guaranteed that such pre-set values were produced by a previous call to this Op.perform; they could’ve been allocated by another Op’s perform method. A Op is free to reuse output_storage as it sees fit, or to discard it and allocate new memory.

class theano.tensor.io.MPIRecv(source, tag, shape, dtype)[source]

An operation to asynchronously receive an array to a remote host using MPI.

See also

MPIRecv, MPIWait

Notes

Non-differentiable.

do_constant_folding(fgraph, node)[source]

Determine whether or not constant folding should be performed for the given node.

This allows each Op to determine if it wants to be constant folded when all its inputs are constant. This allows it to choose where it puts its memory/speed trade-off. Also, it could make things faster as constants can’t be used for in-place operations (see *IncSubtensor).

Parameters:node (Apply) – The node for which the constant folding determination is made.
Returns:res
Return type:bool
make_node()[source]

Construct an Apply node that represent the application of this operation to the given inputs.

This must be implemented by sub-classes.

Returns:node – The constructed Apply node.
Return type:Apply
perform(node, inp, out)[source]

Calculate the function on the inputs and put the variables in the output storage.

Parameters:
  • node (Apply) – The symbolic Apply node that represents this computation.
  • inputs (Sequence) – Immutable sequence of non-symbolic/numeric inputs. These are the values of each Variable in node.inputs.
  • output_storage (list of list) – List of mutable single-element lists (do not change the length of these lists). Each sub-list corresponds to value of each Variable in node.outputs. The primary purpose of this method is to set the values of these sub-lists.
  • params (tuple) – A tuple containing the values of each entry in __props__.

Notes

The output_storage list might contain data. If an element of output_storage is not None, it has to be of the right type, for instance, for a TensorVariable, it has to be a NumPy ndarray with the right number of dimensions and the correct dtype. Its shape and stride pattern can be arbitrary. It is not guaranteed that such pre-set values were produced by a previous call to this Op.perform; they could’ve been allocated by another Op’s perform method. A Op is free to reuse output_storage as it sees fit, or to discard it and allocate new memory.

class theano.tensor.io.MPIRecvWait(tag)[source]

An operation to wait on a previously received array using MPI.

See also

MPIRecv

Notes

Non-differentiable.

make_node(request, data)[source]

Construct an Apply node that represent the application of this operation to the given inputs.

This must be implemented by sub-classes.

Returns:node – The constructed Apply node.
Return type:Apply
perform(node, inp, out)[source]

Calculate the function on the inputs and put the variables in the output storage.

Parameters:
  • node (Apply) – The symbolic Apply node that represents this computation.
  • inputs (Sequence) – Immutable sequence of non-symbolic/numeric inputs. These are the values of each Variable in node.inputs.
  • output_storage (list of list) – List of mutable single-element lists (do not change the length of these lists). Each sub-list corresponds to value of each Variable in node.outputs. The primary purpose of this method is to set the values of these sub-lists.
  • params (tuple) – A tuple containing the values of each entry in __props__.

Notes

The output_storage list might contain data. If an element of output_storage is not None, it has to be of the right type, for instance, for a TensorVariable, it has to be a NumPy ndarray with the right number of dimensions and the correct dtype. Its shape and stride pattern can be arbitrary. It is not guaranteed that such pre-set values were produced by a previous call to this Op.perform; they could’ve been allocated by another Op’s perform method. A Op is free to reuse output_storage as it sees fit, or to discard it and allocate new memory.

class theano.tensor.io.MPISend(dest, tag)[source]

An operation to asynchronously Send an array to a remote host using MPI.

See also

MPIRecv, MPISendWait

Notes

Non-differentiable.

make_node(data)[source]

Construct an Apply node that represent the application of this operation to the given inputs.

This must be implemented by sub-classes.

Returns:node – The constructed Apply node.
Return type:Apply
perform(node, inp, out)[source]

Calculate the function on the inputs and put the variables in the output storage.

Parameters:
  • node (Apply) – The symbolic Apply node that represents this computation.
  • inputs (Sequence) – Immutable sequence of non-symbolic/numeric inputs. These are the values of each Variable in node.inputs.
  • output_storage (list of list) – List of mutable single-element lists (do not change the length of these lists). Each sub-list corresponds to value of each Variable in node.outputs. The primary purpose of this method is to set the values of these sub-lists.
  • params (tuple) – A tuple containing the values of each entry in __props__.

Notes

The output_storage list might contain data. If an element of output_storage is not None, it has to be of the right type, for instance, for a TensorVariable, it has to be a NumPy ndarray with the right number of dimensions and the correct dtype. Its shape and stride pattern can be arbitrary. It is not guaranteed that such pre-set values were produced by a previous call to this Op.perform; they could’ve been allocated by another Op’s perform method. A Op is free to reuse output_storage as it sees fit, or to discard it and allocate new memory.

class theano.tensor.io.MPISendWait(tag)[source]

An operation to wait on a previously sent array using MPI.

See also

MPISend

Notes

Non-differentiable.

make_node(request, data)[source]

Construct an Apply node that represent the application of this operation to the given inputs.

This must be implemented by sub-classes.

Returns:node – The constructed Apply node.
Return type:Apply
perform(node, inp, out)[source]

Calculate the function on the inputs and put the variables in the output storage.

Parameters:
  • node (Apply) – The symbolic Apply node that represents this computation.
  • inputs (Sequence) – Immutable sequence of non-symbolic/numeric inputs. These are the values of each Variable in node.inputs.
  • output_storage (list of list) – List of mutable single-element lists (do not change the length of these lists). Each sub-list corresponds to value of each Variable in node.outputs. The primary purpose of this method is to set the values of these sub-lists.
  • params (tuple) – A tuple containing the values of each entry in __props__.

Notes

The output_storage list might contain data. If an element of output_storage is not None, it has to be of the right type, for instance, for a TensorVariable, it has to be a NumPy ndarray with the right number of dimensions and the correct dtype. Its shape and stride pattern can be arbitrary. It is not guaranteed that such pre-set values were produced by a previous call to this Op.perform; they could’ve been allocated by another Op’s perform method. A Op is free to reuse output_storage as it sees fit, or to discard it and allocate new memory.

theano.tensor.io.irecv(shape, dtype, source, tag)[source]

Non-blocking receive.

theano.tensor.io.isend(var, dest, tag)[source]

Non blocking send.

theano.tensor.io.load(path, dtype, broadcastable, mmap_mode=None)[source]

Load an array from an .npy file.

Parameters:
  • path – A Generic symbolic variable, that will contain a string
  • dtype (data-type) – The data type of the array to be read.
  • broadcastable – The broadcastable pattern of the loaded array, for instance, (False,) for a vector, (False, True) for a column, (False, False) for a matrix.
  • mmap_mode – How the file will be loaded. None means that the data will be copied into an array in memory, ‘c’ means that the file will be mapped into virtual memory, so only the parts that are needed will be actually read from disk and put into memory. Other modes supported by numpy.load (‘r’, ‘r+’, ‘w+’) cannot be supported by Theano.

Examples

>>> from theano import *
>>> path = Variable(Generic())
>>> x = tensor.load(path, 'int64', (False,))
>>> y = x*2
>>> fn = function([path], y)
>>> fn("stored-array.npy")  
array([0, 2, 4, 6, 8], dtype=int64)
theano.tensor.io.mpi_send_wait_key(a)[source]

Wait as long as possible on Waits, Start Send/Recvs early.

theano.tensor.io.mpi_tag_key(a)[source]

Break MPI ties by using the variable tag - prefer lower tags first.

theano.tensor.io.recv(shape, dtype, source, tag)[source]

Blocking receive.

theano.tensor.io.send(var, dest, tag)[source]

Blocking send.