USB fails to enumerate when running CPU at 480MHz

[ryan-summers]

It was noted today that USB devices appear to not function if the CPU SYSCLK is set to 480MHz. However, if the CPU is dropped to 400MHz, USB works fine.

It's unknown if this is a HAL-related issue, USB-device issue, or silicon defect at this time.

I'm opening this just so we can get an initial read on things and get this documented somewhere.

[ryan-summers]

@jamesmunns noted that he fixed an errata around this in embassy in embassy-rs/embassy#2823 that relates to USB being used in multiple ISR contexts

[olback]

I think I ran in to this when I started my Arduino H7 project. I'm running at 480MHz but I'm specifying that USB should use the HSI48 clock source which seems to work fine.

https://github.com/olback/h7/blob/907250a846aacf50beab726f9387dd30a5ee3129/h7-cm7/src/main.rs#L123-L125

[ryan-summers]

@olback That's interesting to me because we're running with very similar clock configurations: https://github.com/quartiq/stabilizer/blob/main/src/hardware/setup.rs#L292

Can you confirm that you're not seeing any USB-related issues with the cod eyou just referenced? I'm confused because yous aid you ran into this issue, but also that it seems to work fine now

[rockboynton]

I experienced this issue. The USB peripheral did work ~1 out of 5 flashing attempts. Also, I noticed my code specifically hang in the usb_device::device_builder::UsbDeviceBuilder::build() call

[jamesmunns]

Noting that ::build() is where the USB core is enabled via RCC, meaning it might actually be closer to embassy-rs/embassy#2677, where it takes a little bit before the RCC is actually active, which the software may be racing.

My suggestion is to insert a delay here: https://docs.rs/synopsys-usb-otg/latest/src/synopsys_usb_otg/bus.rs.html#366-517 after USB::enable().

[ryan-summers]

On Matrix, it was noted that a user was able to resolve this by adding a ~1ms delay after the FDMOD register is configured. The datasheet specifies this needs to be at least 25ms:

This is likely a defect in synopsys-usb-otg.

Rock Boynton
it actually seems like I need to wait until after the // Configure USB PHY section to do the delay, otherwise I still see the issue. I can't just do the delay after the the FDMOD pin being set

[mattthebaker]

I've been grappling with this issue trying to get to a reliable fix.

Some of the above fixes work, sometimes, but changing code elsewhere in the project seems to arbitrarily break the build for all of them.

The core reset is where the hang occurs: https://github.com/stm32-rs/synopsys-usb-otg/blob/master/src/bus.rs#L446

I've observed that adding a random count of discrete nop instructions before the core reset will arbitrarily cause it to hang.

Issuing a full data/instruction synchronization (isb + dsb + dmb instructions) before polling AHBIDL on L446 appears that it may fix it. At least so far I haven't been able to cause it to hang, but it would help if some others could confirm given how arbitrary it is.

In my case I'm on an h723 and it occurs above 400 MHz.

Here is the proposed change in a fork to more easily point a project to for test: https://github.com/AetherWareFoundation/synopsys-usb-otg/tree/work-h7coreresethang

[rockboynton]

I also noticed it hanging at that point. I worked around it by just adding a cortex_m::asm::delay(12_000_000) right before, obviously not a proper solution. That said, those instructions for d/i sync should not be needed according to this and I wonder if those instructions just serve as enough delay

[mattthebaker]

I am able to cause the hang with nops instead of barriers, and with a large number of them (arbitrary counts of them up to 32, I didn't go further due to time required) that I imagine have more total delay. I am so far unable to cause the hang with the barrier which is why I'm asking for 3rd party testing.

I'm not sure that link is really relevant, it seems to describe a case of clearing an interrupt flag. Here one possible explanation is that the preceding write and the AHBIDL bit check read are re-ordered because the M7 is super scalar, letting the core reset attempt proceed in a state we're supposed to prevent. That should be something covered by the memory barrier instructions, though in my testing the ISB was required as well. Or maybe its something with cache lines, I'm not really sure, but in my testing barriers before the AHBIDL read are the only thing that work reliably.

20ms in critical section is too much for my application, though it does seem to achieve the result. If only a handful of instructions are required that can remove whatever race/sync issue is inherent it would be reasonable enough to get upstreamed.