Create unified fake datasets #1324
Labels
developer experience
Resolving this issue should improve development experience for the library users.
testing
Milestone
I think we can create some very generic fake extractors and define a schema for them, for example
We should make sure all of those datasets keep a consistent schema and that they are using all possible entry types. Those virtual datasets would need to follow a very strict backward compatibility policy and proper schema evolution.
This would make it so much easier and more realistic to test not only the entire pipeline but also stand-alone scalar functions as we can also create helpers that would just give us one row (and use them inside fake extractors)
I would put those into
src/core/etl/tests/Flow/ETL/Tests/Double/Fake/Dataset
The text was updated successfully, but these errors were encountered: