#27 Fix obj serialization for saving #29
Conversation
Codecov ReportAttention: Patch coverage is ✅ All tests successful. No failed tests found.
|
…eten extras & pyproject build ignore img dir
…eten extras & pyproject build ignore img dir
|
Added stronger, more defensive input checks for the result_writer and unit tests. |
| - Ensures non-ASCII characters are preserved in the output. | ||
| """ | ||
| if len(eval_inputs) != len(eval_outputs): | ||
| raise ValueError("eval_inputs and eval_outputs must have the same length") |
There was a problem hiding this comment.
This will likely raise an error if there have been downstream errors with outputs. Eval outputs can be <= eval_inputs
There was a problem hiding this comment.
Thanks Minaam, it's the zip function below that needs them to be the same lengths. This check is just to throw the right error type.
I'll see what I could do.
|
Great updates! |
|
I've pushed 'append' style result_writing for batches and end-to-end test for result writing with Llama-Index. I still need to test a bit more and understand what makes the most sense. |
flow_judge/flow_judge.py
Outdated
| def _handle_batch_result( | ||
| self, batch_result: BatchResult, batch_len: int, fail_on_parse_error: bool | ||
| ) -> list[EvalOutput]: | ||
| """Handle output parsing for batched results from Baseten. |
There was a problem hiding this comment.
This is probably for all model instances not just for baseten right?
There was a problem hiding this comment.
True, updated!
|
@sariola Pushed the update for downstream errors! |
Fixes #27