Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Feature Request] Add beam search text streaming visualization feature #35451

Open
MosheOfer1 opened this issue Dec 29, 2024 · 1 comment
Open
Labels
Feature request Request for a new feature

Comments

@MosheOfer1
Copy link

Feature request

Feature request

Remove the limitation that prevents using streamers with beam search. Currently, there's an error in the generation code:

if streamer is not None and (generation_config.num_beams > 1):
   raise ValueError(
       "`streamer` cannot be used with beam search (yet!). Make sure that `num_beams` is set to 1."
   )

### Motivation

### Motivation
When working with beam search generation, it's often difficult to understand why the model makes certain choices or how the beams evolve during generation. While debugging some beam search behavior, I discovered the `MultiBeamTextStreamer` class, but it took me some time to find it as it wasn't prominently featured in the documentation.

From an educational perspective, this tool is extremely valuable. Being able to visualize multiple beams in real-time provides an excellent way to teach and understand how beam search actually works. Students and educators could see the algorithm's decision-making process step by step, making abstract concepts concrete and interactive.

Making this feature more visible and providing better examples would help users who need to:
- Learn/teach beam search concepts interactively
- Debug beam search issues
- Understand model decision-making
- Create interactive demos

This improvement would make beam search less of a "black box" and provide a powerful educational tool for the NLP community.

### Your contribution

### Your contribution
I have implemented the MultiBeamTextStreamer class with tests and documentation. I plan to submit a PR for review after final cleanup. Would love early feedback on this approach.

Example of usage:
https://huggingface.co/spaces/mosheofer1/multi_beam_text_streamer
@MosheOfer1 MosheOfer1 added the Feature request Request for a new feature label Dec 29, 2024
@LysandreJik
Copy link
Member

cc @gante @zucchini-nlp

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Feature request Request for a new feature
Projects
None yet
Development

No branches or pull requests

2 participants