You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Remove the limitation that prevents using streamers with beam search. Currently, there's an error in the generation code:
ifstreamerisnotNoneand (generation_config.num_beams>1):
raiseValueError(
"`streamer` cannot be used with beam search (yet!). Make sure that `num_beams` is set to 1."
)
### Motivation### MotivationWhenworkingwithbeamsearchgeneration, it's often difficult to understand why the model makes certain choices or how the beams evolve during generation. While debugging some beam search behavior, I discovered the `MultiBeamTextStreamer` class, but it took me some time to find it as it wasn'tprominentlyfeaturedinthedocumentation.
Fromaneducationalperspective, thistoolisextremelyvaluable. Beingabletovisualizemultiplebeamsinreal-timeprovidesanexcellentwaytoteachandunderstandhowbeamsearchactuallyworks. Studentsandeducatorscouldseethealgorithm'sdecision-makingprocessstepbystep, makingabstractconceptsconcreteandinteractive.
Makingthisfeaturemorevisibleandprovidingbetterexampleswouldhelpuserswhoneedto:
-Learn/teachbeamsearchconceptsinteractively-Debugbeamsearchissues-Understandmodeldecision-making-CreateinteractivedemosThisimprovementwouldmakebeamsearchlessofa"black box"andprovideapowerfuleducationaltoolfortheNLPcommunity.
### Your contribution### Your contributionIhaveimplementedtheMultiBeamTextStreamerclasswithtestsanddocumentation. IplantosubmitaPRforreviewafterfinalcleanup. Wouldloveearlyfeedbackonthisapproach.
Exampleofusage:
https://huggingface.co/spaces/mosheofer1/multi_beam_text_streamer
The text was updated successfully, but these errors were encountered:
Feature request
Feature request
Remove the limitation that prevents using streamers with beam search. Currently, there's an error in the generation code:
The text was updated successfully, but these errors were encountered: