Core

Sequences:

Sequence(*args)	Sequence of elements, such that next takes input from the previous during run.
Source(*args)	Sequence with no input flow.
FillComputeSeq(*args)	Sequence with one FillCompute element.
FillRequestSeq(*args, **kwargs)	Не рекомендуется, начиная с версии 0.6.
Split(seqs[, bufsize, copy_buf])	Split data flow and run analysis in parallel.

Adapters:

Call(el[, call])	Adapter to provide __call__(value) method.
FillCompute(el[, fill, compute])	Adapter for a FillCompute element.
FillInto(el[, fill_into, explicit])	Adapter for a FillInto element.
FillRequest(el[, bufsize, reset, ...])	Не рекомендуется, начиная с версии 0.6.
Run(el[, run])	Adapter for a Run element.
SourceEl(el[, call])	Adapter to provide __call__() method.

Exceptions:

LenaAttributeError
LenaEnvironmentError	The base class for exceptions that can occur outside the Python system, like IOError or OSError.
LenaException	Base class for all Lena exceptions.
LenaIndexError
LenaKeyError
LenaRuntimeError	Raised when an error does not belong to other categories.
LenaStopFill	Signal that no more fill is accepted.
LenaTypeError	Incorrect type.
LenaValueError	Wrong value.

Sequences

Lena combines calculations using sequences. Sequences consist of elements. Basic Lena sequences and element types are defined in this module.

class Sequence(*args)

Sequence of elements, such that next takes input from the previous during run.

Sequence.run() must accept input flow. For sequence with no input data use Source.

args are objects which implement a method run(flow) or callables.

args can be a single tuple of such elements. In this case one doesn’t need to check argument type when initializing a Sequence in a general function.

For more information about the run method and callables, see Run.

run(flow)

Generator that transforms the incoming flow.

If this Sequence is empty, the flow passes unaltered, but with a small change. This function converts input flow to an iterator, so that it always contains both iter and next methods. This is done for the flow entering the first sequence element and exiting from the sequence.

class Source(*args)

Sequence with no input flow.

First argument is the initial element with no input flow. It can be an an object with a generator function __call__() or an iterable. Following arguments (if present) form a sequence of elements, each accepting computational flow from the previous element.

>>> from lena.flow import CountFrom, Slice
>>> s = Source(CountFrom(), Slice(5))
>>> # iterate in a cycle
>>> for i in s():
...     if i == 5:
...         break
...     print(i, end=" ")
0 1 2 3 4
>>> # if called twice, results depend on the generator
>>> list(s()) == list(range(5, 10))
True

For a sequence that transforms the incoming flow use Sequence.

__call__()

Generate flow.

class FillComputeSeq(*args)

Sequence with one FillCompute element.

Input flow is preprocessed with the Sequence before the FillCompute element, then it fills the FillCompute element.

When the results are computed, they are postprocessed with the Sequence after that element.

args form a sequence with a FillCompute element.

If args contain several FillCompute elements, only the first one is chosen (the subsequent ones are used as simple Run elements). To change that, explicitly cast the first element to FillInto.

If FillCompute element was not found, or if the sequences before and after that could not be correctly initialized, LenaTypeError is raised.

compute()

Compute the results and yield.

If the sequence after FillCompute is not empty, it postprocesses the results yielded from FillCompute element.

fill(value)

Fill self with value.

If the sequence before FillCompute is not empty, it preprocesses the value before filling FillCompute.

class FillRequestSeq(*args, **kwargs)

Не рекомендуется, начиная с версии 0.6: inside a Split element this sequence is a subtype of a simple Sequence.

Sequence with one FillRequest element.

Input flow is preprocessed with the sequence before the FillRequest element, then it fills the FillRequest element.

When the results are yielded from the FillRequest, they are postprocessed with the elements that follow it.

args form a sequence with a FillRequest element.

If args contains several FillRequest elements, only the first one is chosen (the subsequent ones are used as simple Run elements). To change that, explicitly cast the first element to FillInto.

kwargs can contain bufsize or reset. See FillRequest for more information on them. By default bufsize is 1.

If FillRequest element was not found, the sequences could not be correctly initialized, or unknown keyword arguments were received, LenaTypeError is raised.

fill(value)

Fill self with value.

If the sequence before FillRequest is not empty, it preprocesses the value before filling FillRequest.

request()

Request the results and yield.

If the sequence after FillRequest is not empty, it postprocesses the results yielded from the FillRequest element.

reset()

Reset the FillRequest element.

class Split(seqs, bufsize=1000, copy_buf=True)

Split data flow and run analysis in parallel.

seqs must be a list of Sequence, Source, FillComputeSeq or FillRequestSeq sequences. If seqs is empty, Split acts as an empty Sequence and yields all values it receives.

bufsize is the size of the buffer for the input flow. If bufsize is None, whole input flow is materialized in the buffer. bufsize must be a natural number or None.

copy_buf sets whether the buffer should be copied during run(). This is important if different sequences can change input data and thus interfere with each other.

Common type:

If each sequence from seqs has a common type, Split creates methods corresponding to this type. For example, if each sequence is FillCompute, Split creates methods fill and compute and can be used as a FillCompute sequence. fill fills all its subsequences (with copies if copy_buf is True), and compute yields values from all sequences in turn (as would also do request or Source.__call__). Common type is not implemented for Call element.

In case of wrong initialization arguments, LenaTypeError or LenaValueError is raised.

__call__()

Each initialization sequence generates flow. After its flow is empty, next sequence is called, etc.

This method is available only if each self sequence is a Source, otherwise runtime LenaAttributeError is raised.

run(flow)

Iterate input flow and yield results.

The flow is divided into subslices of bufsize. Each subslice is processed by sequences in the order of their initializer list.

If a sequence is a Source, it doesn’t accept the incoming flow, but produces its own complete flow and becomes inactive (is not called any more).

A FillRequestSeq is filled with the buffer contents. After the buffer is finished, it yields all values from request().

A FillComputeSeq is filled with values from each buffer, but yields values from compute only after the whole flow is finished.

A Sequence is called with run(buffer) instead of the whole flow. The results are yielded for each buffer (and also if the flow was empty). If the whole flow must be analysed at once, don’t use such a sequence in Split.

If the flow was empty, each call, compute, request or run is called nevertheless.

If copy_buf is True, then the buffer for each sequence except the last one is a deep copy of the current buffer.

Adapters

Adapters allow to use existing objects as Lena core elements.

Adapters can be used for several purposes:

• provide an unusual name for a method (Run(my_obj, run=»my_run»)).

• hide unused methods to prevent ambiguity.

• automatically convert objects of one type to another in sequences (FillCompute to Run).

• explicitly cast object of one type to another (FillRequest to FillCompute).

Example:

>>> class MyEl(object):
...     def my_run(self, flow):
...         for val in flow:
...             yield val
...
>>> my_run = Run(MyEl(), run="my_run")
>>> list(my_run.run([1, 2, 3]))
[1, 2, 3]

class Call(el, call=<object object>)

Adapter to provide __call__(value) method.

Name of the actually called method can be customized during the initialization.

The method __call__(value) is a simple (preferably pure) function, which accepts a value and returns its transformation.

Element el must contain a callable method call or be callable itself.

If call method name is not provided, it is checked whether el is callable itself.

If Call failed to instantiate with el and call, LenaTypeError is raised.

__call__(value)

Transform the value and return.

class FillCompute(el, fill='fill', compute='compute')

Adapter for a FillCompute element.

A FillCompute element has methods fill(value) and compute().

Method names can be customized through fill and compute keyword arguments during the initialization.

FillCompute can be explicitly cast from FillRequest. In this case compute is request.

If callable methods fill and compute or request were not found, LenaTypeError is raised.

compute()

Yield computed values.

fill(value)

Fill self with value.

class FillInto(el, fill_into=<object object>, explicit=True)

Adapter for a FillInto element.

Element el must implement fill_into method, be callable or be a Run element.

If no fill_into argument is provided, then fill_into method is searched, then __call__, then run. If none of them is found and callable, LenaTypeError is raised.

Note that callable elements and elements with fill_into method have different interface. If the el is callable, it is assumed to be a simple function, which accepts a single value and transforms that, and the result is filled into the element by this adapter. fill_into method, on the contrary, takes two arguments (element and value) and fills the element itself. This allows to use lambdas directly in FillInto.

A Run element is converted to FillInto this way: for each value the el runs a flow consisting of this one value and fills the results into the output element. This can be done only if explicit is True.

fill_into(element, value)

Fill value into an element.

Value is transformed by the initialization element before filling el.

Element must provide a fill method.

class FillRequest(el, bufsize=1, reset=None, buffer_input=None, buffer_output=None, yield_on_remainder=False, fill='fill', request='request', reset_name='reset')

Не рекомендуется, начиная с версии 0.6: inside Split this element can be implemented by a simple Run element.

Adapter for a FillRequest element.

A FillRequest element slices the flow during fill and yields results for each chunk during request. It can also call reset after each request.

Names for actual fill, request and reset methods can be provided during initialization (the latter is set through reset_name).

FillRequest can be initialized from a FillCompute element. If a callable request method was not found, el must have a method compute. request in this case is compute.

FillRequest can also be initialized from a Run element. In that case el is not required to have fill, compute or reset methods (and FillRequest will not have such missing methods). FillRequest implements run method that splits the flow into subslices of bufsize values and feeds them to the run method of el.

Since we require no less than bufsize values (except yield_on_remainder is True), we need to store either bufsize values of the incoming flow or all values produced by el.run or el.request for each slice. This is set by buffer_input or buffer_output. One and only one of them must be True. For example, if the element receives file names and produces data from them, it would be wise to buffer input. If the element receives much data and produces a histogram, one should buffer output.

If a keyword argument reset is True, el must have a method reset_name, and in this case reset() is called after each request() (including those during run()). In general, Run elements have no reset methods, but for FillCompute elements reset must be set explicitly.

If yield_on_remainder is True, then the output will be yielded even if the element was filled less than bufsize times (but at least once). In that case no internal buffers are used during run() and corresponding attributes are not checked.

Attributes

bufsize is the maximum size of subslices during run.

bufsize must be a natural number, otherwise LenaValueError is raised. If callable fill and request methods were not found, or FillRequest could not be derived from FillCompute, or if reset is True, but el has no method reset, LenaTypeError is raised.

Изменено в версии 0.5: add keyword arguments yield_on_remainder, buffer_input, buffer_output, reset_name. Require explicit reset for FillCompute elements.

fill(value)

Fill el with value.

If more than bufsize values were filled, incoming values are stored in a buffer (if buffer_input is True) or, otherwise, the output of el.request is stored in a buffer, until it is requested.

request()

Yield results (if they are available) and possibly reset.

If input or output buffers were filled, all their contents are processed and yielded.

reset()

Reset el (ignoring the initialization setting).

run(flow)

Process the flow slice by slice.

fill each value from a subslice of flow of bufsize length, then yield results from request. Repeat until the flow is exhausted.

If fill was not called even once (flow was empty), nothing is yielded, because bufsize values were not obtained (in contrast to FillCompute, for which output for an empty flow is reasonable). The last slice may contain less than bufsize values. If there were any and if yield_on_remainder is True, request will be called for that.

class Run(el, run=<object object>)

Adapter for a Run element.

Name of the method run can be customized during initialization.

If run argument is supplied, el must be None or it must have a callable method with name given by run.

If run keyword argument is missing, then el is searched for a method run. If that is not found, a type cast is attempted.

A Run element can be initialized from a Call or a FillCompute element.

A callable element is run as a transformation function, which accepts single values from the flow and returns their transformations for each value.

A FillCompute element is run the following way: first, el.fill(value) is called for the whole flow. After the flow is exhausted, el.compute() is called.

It is possible to initialize Run using a generator function without an element. To do that, set the element to None: Run(None, run=<my_function>).

If the initialization failed, LenaTypeError is raised.

Run is used implicitly during the initialization of Sequence.

run(flow)

Yield transformed values from the incoming flow.

class SourceEl(el, call=<object object>)

Adapter to provide __call__() method. Name of the actually called method can be customized during the initialization.

The __call__() method is a generator, which yields values. It doesn’t accept any input flow.

Element el must be callable or iterable, or contain a callable method call.

If SourceEl failed to instantiate with el and call, LenaTypeError is raised.

__call__()

Yield generated values.

Exceptions

All Lena exceptions are subclasses of LenaException and corresponding Python exceptions (if they exist).

exception LenaAttributeError

Базовые классы: LenaException, AttributeError

exception LenaEnvironmentError

Базовые классы: LenaException, OSError

The base class for exceptions that can occur outside the Python system, like IOError or OSError.

exception LenaException

Базовые классы: Exception

Base class for all Lena exceptions.

exception LenaIndexError

Базовые классы: LenaException, IndexError

exception LenaKeyError

Базовые классы: LenaException, KeyError

exception LenaNotImplementedError

Базовые классы: LenaException, NotImplementedError

exception LenaRuntimeError

Базовые классы: LenaException, RuntimeError

Raised when an error does not belong to other categories.

exception LenaStopFill

Базовые классы: LenaException

Signal that no more fill is accepted.

Analogous to StopIteration, but control flow is reversed.

exception LenaTypeError

Базовые классы: LenaException, TypeError

Incorrect type.

Typically used during initialization of Lena elements. Use LenaValueError for errors from values from the flow.

exception LenaValueError

Базовые классы: LenaException, ValueError

Wrong value.

It is also used for values from the flow, even when they have a wrong type.

exception LenaZeroDivisionError

Базовые классы: LenaException, ZeroDivisionError