Core¶
Sequences:
Sequence (*args) |
Sequence of elements, such that next takes input from the previous during run. |
Source (*args) |
Sequence with no input flow. |
FillComputeSeq (*args) |
Sequence with one FillCompute element. |
FillRequestSeq (*args, **kwargs) |
Sequence with one FillRequest element. |
Split (seqs[, bufsize, copy_buf]) |
Split data flow and run analysis in parallel. |
Adapters:
Call (el[, call]) |
Adapter to provide __call__(value) method. |
FillCompute (el[, fill, compute]) |
Adapter for a FillCompute element. |
FillInto (el[, fill_into, explicit]) |
Adapter for a FillInto element. |
FillRequest (el[, fill, request, reset, bufsize]) |
Adapter for a FillRequest element. |
Run (el[, run]) |
Adapter for a Run element. |
SourceEl (el[, call]) |
Adapter to provide __call__() method. |
Exceptions:
LenaAttributeError |
|
LenaEnvironmentError |
The base class for exceptions that can occur outside the Python system, like IOError or OSError. |
LenaException |
Base class for all Lena exceptions. |
LenaIndexError |
|
LenaKeyError |
|
LenaRuntimeError |
Raised when an error does not belong to other categories. |
LenaStopFill |
Signal that no more fill is accepted. |
LenaTypeError |
|
LenaValueError |
Sequences¶
Lena combines calculations using sequences. Sequences consist of elements. Basic Lena sequences and element types are defined in this module.
-
class
Sequence
(*args)[исходный код]¶ Sequence of elements, such that next takes input from the previous during run.
Sequence.run()
must accept input flow. For sequence with no input data useSource
.args are objects which implement a method run(flow) or callables.
args can be a single tuple of such elements. In this case one doesn’t need to check argument type when initializing a Sequence in a general function.
For more information about the run method and callables, see
Run
.-
run
(flow)[исходный код]¶ Generator, which transforms the incoming flow.
If this
Sequence
is empty, the flow passes untransformed, with a small change. This function converts input flow to an iterator, so that it always contains both iter and next methods. This is done for the flow entering the first sequence element and exiting from the sequence.
-
-
class
Source
(*args)[исходный код]¶ Sequence with no input flow.
First argument is the initial element with no input flow. Following arguments (if present) form a sequence of elements, each accepting computational flow from the previous element.
>>> from lena.flow import CountFrom >>> s = Source(CountFrom()) >>> for i in s(): ... if i == 5: ... break ... print(i, end=" ") 0 1 2 3 4
For a sequence which transforms the incoming flow, use
Sequence
.-
__call__
()[исходный код]¶ Generate flow.
-
-
class
FillComputeSeq
(*args)[исходный код]¶ Sequence with one
FillCompute
element.Input flow is preprocessed with the Sequence before the FillCompute element, then it fills the FillCompute element.
When the results are computed, they are postprocessed with the Sequence after that element.
args form a sequence with a FillCompute element.
If args contain several FillCompute elements, only the first one is chosen (the subsequent ones are used as simple Run elements). To change that, explicitly cast the first element to
FillInto
.If FillCompute element was not found, or if the sequences before and after that could not be correctly initialized,
LenaTypeError
is raised.-
compute
()[исходный код]¶ Compute the results and yield.
If the sequence after FillCompute is not empty, it postprocesses the results yielded from FillCompute element.
-
fill
(value)[исходный код]¶ Fill self with value.
If the sequence before FillCompute is not empty, it preprocesses the value before filling FillCompute.
-
-
class
FillRequestSeq
(*args, **kwargs)[исходный код]¶ Sequence with one
FillRequest
element.Input flow is preprocessed with the Sequence before the FillRequest element, then it fills the FillRequest element.
When the results are yielded from the FillRequest, they are postprocessed with the Sequence after that element.
args form a sequence with a FillRequest element.
If args contains several FillRequest elements, only the first one is chosen (the subsequent ones are used as simple Run elements). To change that, explicitly cast the first element to
FillInto
.kwargs can contain bufsize, which is used during run. See
FillRequest
for more information on run. By default bufsize is 1. Other kwargs raiseLenaTypeError
.If FillRequest element was not found, or if the sequences before or after that could not be correctly initialized,
LenaTypeError
is raised.-
fill
(value)[исходный код]¶ Fill self with value.
If the sequence before FillRequest is not empty, it preprocesses the value before filling FillRequest.
-
request
()[исходный код]¶ Request the results and yield.
If the sequence after FillRequest is not empty, it postprocesses the results yielded from the FillRequest element.
-
reset
()[исходный код]¶ Reset the FillRequest element.
-
-
class
Split
(seqs, bufsize=1000, copy_buf=True)[исходный код]¶ Split data flow and run analysis in parallel.
seqs must be a list of Sequence, Source, FillComputeSeq or FillRequestSeq sequences (any other container will raise
LenaTypeError
). If seqs is empty, Split acts as an empty Sequence and yields all values it receives.bufsize is the size of the buffer for the input flow. If bufsize is
None
, whole input flow is materialized in the buffer. bufsize must be a natural number orNone
, otherwiseLenaValueError
is raised.copy_buf sets whether the buffer should be copied during run. This is important if different sequences can change input data and interfere with each other.
- Common type:
- If each sequence from seqs has a common type, Split creates methods corresponding to this type. For example, if each sequence is FillCompute, Split creates methods fill and compute and can be used as a FillCompute sequence. fill fills all its subsequences (with copies if copy_buf is True), and compute yields values from all sequences in turn (as would also do request or Source.__call__).
-
run
(flow)[исходный код]¶ Iterate input flow and yield results.
The flow is divided into subslices of bufsize. Each subslice is processed by sequences in the order of their initializer list.
If a sequence is a Source, it doesn’t accept the incoming flow, but produces its own complete flow and becomes inactive (is not called any more).
A FillRequestSeq is filled with the buffer contents. After the buffer is finished, it yields all values from request().
A FillComputeSeq is filled with values from each buffer, but yields values from compute only after the whole flow is finished.
A Sequence is called with run(buffer) instead of the whole flow. The results are yielded for each buffer (and also if the flow was empty). If the whole flow must be analysed at once, don’t use such a sequence in Split.
If the flow was empty, each call, compute, request or run is called nevertheless.
If copy_buf is True, then the buffer for each sequence except the last one is a deep copy of the current buffer.
Adapters¶
Adapters allow to use existing objects as Lena core elements.
Adapters can be used for several purposes:
- provide an unusual name for a method (Run(my_obj, run=»my_run»)).
- hide unused methods to prevent ambiguity.
- automatically convert objects of one type to another in sequences (FillCompute to Run).
- explicitly cast object of one type to another (FillRequest to FillCompute).
Example:
>>> class MyEl(object):
... def my_run(self, flow):
... for val in flow:
... yield val
...
>>> my_run = Run(MyEl(), run="my_run")
>>> list(my_run.run([1, 2, 3]))
[1, 2, 3]
-
class
Call
(el, call=<object object>)[исходный код]¶ Adapter to provide __call__(value) method.
Name of the actually called method can be customized during the initialization.
The method __call__(value) is a simple (preferably pure) function, which accepts a value and returns its transformation.
Element el must contain a callable method call or be callable itself.
If call method name is not provided, it is checked whether el is callable itself.
If
Call
failed to instantiate with el and call,LenaTypeError
is raised.-
__call__
(value)[исходный код]¶ Transform the value and return.
-
-
class
FillCompute
(el, fill='fill', compute='compute')[исходный код]¶ Adapter for a FillCompute element.
A FillCompute element has methods fill(value) and compute().
Method names can be customized through fill and compute keyword arguments during the initialization.
FillCompute can be explicitly cast from FillRequest. In this case compute is request.
If callable methods fill and compute or request were not found,
LenaTypeError
is raised.-
compute
()[исходный код]¶ Yield computed values.
-
fill
(value)[исходный код]¶ Fill self with value.
-
-
class
FillInto
(el, fill_into=<object object>, explicit=True)[исходный код]¶ Adapter for a FillInto element.
Element el must implement fill_into method, be callable or be a Run element.
If no fill_into argument is provided, then fill_into method is searched, then __call__, then run. If none of them is found and callable,
LenaTypeError
is raised.Note that callable elements and elements with fill_into method have different interface. If the el is callable, it is assumed to be a simple function, which accepts a single value and transforms that, and the result is filled into the element by this adapter. fill_into method, on the contrary, takes two arguments (element and value) and fills the element itself. This allows to use lambdas directly in FillInto.
A Run element is converted to FillInto this way: for each value the el runs a flow consisting of this one value and fills the results into the output element. This can be done only if explicit is True.
-
fill_into
(element, value)[исходный код]¶ Fill value into an element.
Value is transformed by the initialization element before filling el.
Element must provide a fill method.
-
-
class
FillRequest
(el, fill='fill', request='request', reset=True, bufsize=1)[исходный код]¶ Adapter for a FillRequest element.
A FillRequest element has methods fill(value) and request().
Names for fill and request can be customized during initialization.
FillRequest can be initialized from a FillCompute element. If a callable request method was not found, el must have a callable compute method. request in this case is compute.
By default, FillRequest implements run method that splits the flow into subslices of bufsize elements. If el has a callable run method, it is used instead of the default one.
If a keyword argument reset is
True
(default), el must have a method reset, and in this case :meth:`reset` is called after each :meth:`request` (including those during :meth:`run`). If *reset isFalse
,reset()
is never called.Attributes
bufsize
is the maximum size of subslices during run.bufsize must be a natural number, otherwise
LenaValueError
is raised. If callable fill and request methods were not found, or FillRequest could not be derived from FillCompute, or if reset isTrue
, but el has no method reset,LenaTypeError
is raised.-
fill
(value)[исходный код]¶ Fill self with value.
-
request
()[исходный код]¶ Yield computed values.
May be called at any time, the flow may still contain zero or more items.
-
reset
()[исходный код]¶ Reset the element el.
-
run
(flow)[исходный код]¶ Implement run method.
First, fill is called for each value in a subslice of flow of self.bufsize size. After that, results are yielded from self.request(). This repeats until the flow is exhausted.
If fill was not called even once (flow is empty), the results for a general FillRequest are undefined (for example, it can run request or raise an exception). This adapter runs request in this case. If the last slice is empty, request is not run for that. Note that the last slice may contain less than bufsize values. If that is important, implement your own method.
A slice is a non-materialized list, which means that it will not take place of bufsize in memory.
-
-
class
Run
(el, run=<object object>)[исходный код]¶ Adapter for a Run element.
Name of the method run can be customized during initialization.
If run argument is supplied, el must be None or it must have a callable method with name given by run.
If run keyword argument is missing, then el is searched for a method run. If that is not found, a type cast is attempted.
A Run element can be initialized from a Call or a FillCompute element.
A callable element is run as a transformation function, which accepts single values from the flow and returns their transformations for each value.
A FillCompute element is run the following way: first, el.fill(value) is called for the whole flow. After the flow is exhausted, el.compute() is called.
It is possible to initialize
Run
using a generator function without an element. To do that, set the element toNone
: Run(None, run=<my_function>).If the initialization failed,
LenaTypeError
is raised.Run
is used implicitly during the initialization ofSequence
.-
run
(flow)[исходный код]¶ Yield transformed elements from the incoming flow.
-
-
class
SourceEl
(el, call=<object object>)[исходный код]¶ Adapter to provide __call__() method. Name of the actually called method can be customized during the initialization.
The
__call__()
method is a generator, which yields values. It doesn’t accept any input flow.Element el must contain a callable method __call__ or be callable itself.
If call function or method name is not provided, it is checked whether el is callable itself.
If
SourceEl
failed to instantiate with el and call,LenaTypeError
is raised.-
__call__
()[исходный код]¶ Yield generated values.
-
Exceptions¶
All Lena exceptions are subclasses of LenaException
and corresponding Python exceptions (if they exist).
-
exception
LenaAttributeError
[исходный код]¶ - Базовые классы:
lena.core.exceptions.LenaException
,AttributeError
-
exception
LenaEnvironmentError
[исходный код]¶ - Базовые классы:
lena.core.exceptions.LenaException
,OSError
The base class for exceptions that can occur outside the Python system, like IOError or OSError.
-
exception
LenaException
[исходный код]¶ - Базовые классы:
Exception
Base class for all Lena exceptions.
-
exception
LenaIndexError
[исходный код]¶ - Базовые классы:
lena.core.exceptions.LenaException
,IndexError
-
exception
LenaKeyError
[исходный код]¶ - Базовые классы:
lena.core.exceptions.LenaException
,KeyError
-
exception
LenaNotImplementedError
[исходный код]¶ - Базовые классы:
lena.core.exceptions.LenaException
,NotImplementedError
-
exception
LenaRuntimeError
[исходный код]¶ - Базовые классы:
lena.core.exceptions.LenaException
,RuntimeError
Raised when an error does not belong to other categories.
-
exception
LenaStopFill
[исходный код]¶ - Базовые классы:
lena.core.exceptions.LenaException
Signal that no more fill is accepted.
Analogous to StopIteration, but control flow is reversed.
-
exception
LenaTypeError
[исходный код]¶ - Базовые классы:
lena.core.exceptions.LenaException
,TypeError
-
exception
LenaValueError
[исходный код]¶ - Базовые классы:
lena.core.exceptions.LenaException
,ValueError
-
exception
LenaZeroDivisionError
[исходный код]¶ - Базовые классы:
lena.core.exceptions.LenaException
,ZeroDivisionError