Releases: bpolaszek/bentools-etl
Releases · bpolaszek/bentools-etl
Major design refactoring
3.0 alpha is out! This is a complete redesign of the library, so don't upgrade from 2.5 if you don't intend to refactor your whole app, since almost everything has changed. This is now a lot simpler:
- No more useless extractors: if your input data is already an
iterable
, you don't need one. - Removed
ContextElement
object to carry data during the ETL lifecycle - no need for that wrapper anymore. - Transformers now MUST return a
\Generator
even if they just transform a single value. Because transformers can now transform 1 thing into multiple others. - As a consequence, a
\Generator
is expected to be injected into all loaders. - Redesigned the built-in event dispatcher, in favor of a PSR-14 compliant one.
- Events are now read-only, they can't modify data anymore. They still allow skipping the current item / stopping the process, and ignore exceptions at the item level. Way less difficult to maintain. Implement your own extractor / transformer and/or loader instead, if you want a fine control of what goes in and what goes out.
- Dropped support for asynchronous ETL processing.
- More decoupling.
- New ETL builder with its recipes.
- Newcomers can get started way more easier since there are less concepts to understand.
Transformer workflow (multiple transformers)
v2.5
Exceptions handling + CSV Iterators
Exception Handling
The extract, transform, or load part of the ETL process may sometimes throw exceptions.
Before throwing it, we should be able to hook on the exception, handle it, and decide wether or not to continue the process.
CSV Iterators
You can now use CsvFileIterator
or CsvStringIterator
to iterate over your CSVs.
Also check KeysAwareCsvIterator
to automatically assign the keys of each CSV row (you can turn an indexed array into an associative one). Check out the Iterators documentation.