Add strudel fft to hydra

[reckter]

This also wires through the instance of the StrudelWrapper, just like #309 .

• Patches strudels webaudio module through to be accessible from anywhere.
• Adds fft(index: number, buckets: number = 8, options?: { min?: number; max?: number, scale?: number }): number.
  Currently this hooks into the webaudio module from strudel and takes fft-data from there. In the future, we could add different fft sources, and make the source be switchable (or make it use all per default).
• Automatically adds an .analyze('flok-master') to all strudel patterns, if fft is used an a hydra pane

Documentation

fft(index: number, buckets: number = 8, options?: { min?: number; max?: number, scale?: number, analyzerId?: string }): number

• index: number : the index of the bucket to return the value from
• buckets: number: The number of buckets to combine the underlying fft data too. defaults to 8.
• options?: { min?: number; max?: number, scale?: number }:
  • min?: number: minimum clamp value of the underlying data, defaults to -150
  • max?: number: maximum clamp value of the underlying data, defaults to 0
  • scale?: number: Scale of the outpur, defaults to 1 (so the output is from 0 to 1
  • analyzerId?: string: Which strudel analyser to listen to. defaults to flok-master, which is also automatically added to al strudel patterns. Can be used to route different patterns to different parts of the hydra visualiser

Examples

 solid(() => fft(0,1), 0)
  .mask(shape(5,.05))
  .rotate(() => 50 *  fft(0, 40)) // we need to supply a function for the parameter, for it to update automaticaly

Caveats

• there probably are performance impacts, as we do need to analyze all strudel patterns.
• because of how we setup the analzye node on strudel, every strudel pane needs a re-eval after the hydra code decides that we need to get the fft data. This does not happen automatically, manual re eval is necessary

Open questions:

• Where to put the documntation? (also good to know for Add integration from Strudel patterns and canvas into hydra #309)

[felixroos]

If someone has a workaround idea, maybe we can make it more user friendly!

strudel's superdough could theoretically add an analyzer to the destinationGain node, which is the last node before the output, so no .scope / .analyze would be needed. but that would only work after the next strudel release

as a workaround it might be easier to automatically add a .analyze('flok-master') to the strudel stack of all panes

[reckter]

@felixroos

as a workaround it might be easier to automatically add a .analyze('flok-master') to the strudel stack of all panes
Uh nice!
That works. It currently adds the analysers allways even if they are not needed, maybe an opt in would be nice.

Also added the ability to change the analzer of the fft call, so it is possible now to route different strudel patterns to different fft calls in hydra :D

Edit: I just relized that the autmatic adding of the .analyze('flok-master') overrides any custom .analyze('test') on the patterns.

So that would lock us in, in always having all patterns pass through. (which tbh is not that bad imo, having the option of custom routing would be very nice, but i don't think its a necessity for this feature)

[felixroos]

yes opt in would be good, as fft is not very cheap, so it should be avoided if it's not needed. not sure how tho. maybe we can somehow add an allPanes function that runs on the stack of all panes

[TodePond]

opt in would be goo

it can be different for different clients right? we could chuck it into the settings I guess.

we could probably do a better job if surfacing the settings though

[reckter]

I guess the fft function could toggle something that enables the automatic analyzer adding in strudel 🤔
Disabling per client is probably needed, even though the visuals will look different, but performance wise we probably need it.

I think I would do "opt out" though, just so the default experience is more what the code / coders want.

[reckter]

@felixroos @TodePond added setting to opt out, and only add the analyze node in strudel, when it is necessery (because fft is used in hydra).
It will only reset on reload though.
Also some re-evals might be necessary for it to properly be picked up, but I think that is ok.

Feel free to request text changes, writing texts is not my strong suit xD

[TodePond]

The text copy looks good to me!

I'm not sure of quite how intensive it actually is so I can't comment on whether it should be opt-in or opt-out. but i think it's good it only kicks in when fft is called.

Either way, I think a future PR could surface some of these emerging "performance" settings more clearly.

can you confirm if it still needs the "scope" function to be called, as stated in the PR description?

[reckter]

@TodePond Can confirm that it is no longer necessary, updated the PR description :D (credit to @felixroos for giving me the hints necessary for the automation here)

[munshkr]

Thank you @reckter, this looks great!

About documentation, I've added some of it in the README, inside the Hydra section. Feel free to modify it if you want or think I missed something. I've been putting documentation there for the time being.

Offtopic, but I'm thinking we should probably update the docs a bit, Flok's original focus was on REPL-based languages, and I feel this is starting to shift towards web-based targets.

[reckter]

@munshkr I've added a sentence about the FFT settings, so it at least points there :)

Yeah, this is pretty much a web-only feature (and really, a web only "problem" to beginn with).
There might be an argument about how to share certain data / hooks in general, that would also solve this in other ways.

But my understanding of flok isn't deep enough for that yet.

[munshkr]

Thank you all!