[WASM-Export] Unable to use multiple rust GDExtensions at the same time

[Guusggg]

Hi,

Lately I've been working on a bigger game, and completed the refactor from C# to C++ and then I found this amazing library so I switched one more time to Rust.

Now, what I've had in mind was that I'd create a Shared, Client and Server project in Rust. Shared would have the Nodes that both projects need and Client and Server would be implementing the multiplayer features.

I created the following setup:

• godot (Godot Project)
• rust (Rust workspace)
  • client (Rust project)
  • server (Rust project)
  • shared (Rust project)

Note: The client nor server projects do not depend on shared in Cargo.toml.

This setup is working terrific! I'm able to implement client specific features, and server specific features which will be connected using the shared nodes.

I can run this on my Linux machine, and confirm that it is working. However, I also target wasm so I tried getting that to work. Now, this isn't really working out.

I compile the workspace with cargo +nightly build -Zbuild-std --target wasm32-unknown-emscripten to WebAssembly, and this is going really well. But when I run the project, on Firefox I get an out of memory emscripten error (which it also does with a single Rust project, but only on Debug builds). When I run it on Chromium however, I get a big list of errors:

RuntimeError: Aborted(undefined). Build with -sASSERTIONS for more info.
    at abort (tmp_js_export.js:9:8604)
    at ___cxa_throw (tmp_js_export.js:9:425328)
    at shared.wasm.panic_unwind::imp::panic::h7a89a72211144e04 (shared.wasm-10a1508e:0x42b504)
    at shared.wasm.__rust_start_panic (shared.wasm-10a1508e:0x42b367)
    at shared.wasm.rust_panic (shared.wasm-10a1508e:0x41d2e3)
    at shared.wasm.std::panicking::rust_panic_with_hook::hc3a8cd21f77927d3 (shared.wasm-10a1508e:0x41d1d9)
    at shared.wasm.std::panicking::begin_panic_handler::_$u7b$$u7b$closure$u7d$$u7d$::hb19be6fa9857ee61 (shared.wasm-10a1508e:0x41a5f5)
    at shared.wasm.std::sys::backtrace::__rust_end_short_backtrace::h585b6ab617e4600a (shared.wasm-10a1508e:0x41a4bd)
    at shared.wasm.rust_begin_unwind (shared.wasm-10a1508e:0x41c2a1)
    at shared.wasm.core::panicking::panic_fmt::he773db1f33281f18 (shared.wasm-10a1508e:0x49a208)

  | onPrintError | @ | tmp_js_export.js:477
-- | -- | -- | --
  | abort | @ | tmp_js_export.js:9
  | ___cxa_throw | @ | tmp_js_export.js:9
  | $panic_unwind::imp::panic::h7a89a72211144e04 | @ | shared.wasm-10a1508e:0x42b504
  | $__rust_start_panic | @ | shared.wasm-10a1508e:0x42b367
  | $rust_panic | @ | shared.wasm-10a1508e:0x41d2e3
  | $std::panicking::rust_panic_with_hook::hc3a8cd21f77927d3 | @ | shared.wasm-10a1508e:0x41d1d9
  | $std::panicking::begin_panic_handler::_$u7b$$u7b$closure$u7d$$u7d$::hb19be6fa9857ee61 | @ | shared.wasm-10a1508e:0x41a5f5
  | $std::sys::backtrace::__rust_end_short_backtrace::h585b6ab617e4600a | @ | shared.wasm-10a1508e:0x41a4bd
  | $rust_begin_unwind | @ | shared.wasm-10a1508e:0x41c2a1
  | $core::panicking::panic_fmt::he773db1f33281f18 | @ | shared.wasm-10a1508e:0x49a208
  | $godot_ffi::binding::single_threaded::BindingStorage::set_initialized::haf88d1160aa08dd6 | @ | shared.wasm-10a1508e:0x3a7ebd
  | $godot_ffi::binding::single_threaded::BindingStorage::initialize::h06c546cc4398088b | @ | shared.wasm-10a1508e:0x3a7f1e
  | $godot_ffi::binding::initialize_binding::hec449e30c65edce2 | @ | shared.wasm-10a1508e:0x3b4174
  | $godot_ffi::initialize::hd3864843f056cfd2 | @ | shared.wasm-10a1508e:0x3b53dc
  | $godot_core::init::__gdext_load_library::_$u7b$$u7b$closure$u7d$$u7d$::h80e68cd7c7203dc9 | @ | shared.wasm-10a1508e:0x66696
  | $core::ops::function::FnOnce::call_once::h50d84be8fa3c9798 | @ | shared.wasm-10a1508e:0x70136
  | $std::panicking::try::do_call::h64bb311ea799ec2a | @ | shared.wasm-10a1508e:0x741fb
  | $__rust_try | @ | shared.wasm-10a1508e:0x734d8
  | $std::panicking::try::hcbde3b6f2bb9e296 | @ | shared.wasm-10a1508e:0x74088
  | $std::panic::catch_unwind::h4d4f243b6b7cf82f | @ | shared.wasm-10a1508e:0x72b00
  | $godot_core::private::handle_panic_with_print::h28362aa50547f00d | @ | shared.wasm-10a1508e:0x596e6
  | $godot_core::private::handle_panic::h7e38a629003b6b68 | @ | shared.wasm-10a1508e:0x59400
  | $godot_core::init::__gdext_load_library::h65f7301edd9aa8a6 | @ | shared.wasm-10a1508e:0x66521
  | $gdext_rust_init | @ | shared.wasm-10a1508e:0x5616f
  | $func58011 | @ | 0aaeb74a:0x2599747
  | $func58034 | @ | 0aaeb74a:0x259dff6
  | $func51263 | @ | 0aaeb74a:0x2466843
  | $func51313 | @ | 0aaeb74a:0x246c885
  | $func34708 | @ | 0aaeb74a:0x19938e1
  | $func1310 | @ | 0aaeb74a:0x4c7148
  | $func1212 | @ | 0aaeb74a:0x431bc4
  | $_Z14godot_web_mainiPPc | @ | 0aaeb74a:0x2cfd7e
  | __Z14godot_web_mainiPPc | @ | tmp_js_export.js:9
  | $__main_argc_argv | @ | 006438ee:0xa5975
  | callMain | @ | tmp_js_export.js:9
  | (anonymous) | @ | tmp_js_export.js:804
  | (anonymous) | @ | tmp_js_export.js:799
  | Promise.then |   |  
  | start | @ | tmp_js_export.js:778
  | (anonymous) | @ | tmp_js_export.js:837
  | Promise.then |   |  
  | startGame | @ | tmp_js_export.js:836
  | (anonymous) | @ | tmp_js_export.html:181
  | (anonymous)

I've tried using different entry point names using

struct SharedExtension;

#[gdextension(entry_symbol = gdext_shared_rust_init)]
unsafe impl ExtensionLibrary for SharedExtension {}

Which doesn't work. As soon as I remove the other extensions (so I leave shared.gdextension) it starts working again.

I use these Emscripten compile flags:

[target.wasm32-unknown-emscripten]
rustflags = [
    "-C", "link-args=-sSIDE_MODULE=2",
    "-C", "link-args=-pthread", # was -sUSE_PTHREADS=1 in earlier emscripten versions
    "-C", "target-feature=+atomics,+bulk-memory,+mutable-globals",
    "-Zlink-native-libraries=no",
    "-Cllvm-args=-enable-emscripten-cxx-exceptions=0",
]

I also created two projects to recreate the issue:
single-rust-project.zip
multiple-rust-project.zip

I'm really eager to help this project move forward, and I would say I'm quite well-versed in WebAssembly, however I have no idea where to get started. I tried enabling -sASSERTIONS and -sASSERTIONS=2 but this gives me a resolved is not a function error so no dice...

Any tips are very welcome, and if you need more information please let me know!

Thank you for the amazing work on this library, it's awesome!
Kind regards,

[Bromeon]

Thanks a lot for the detailed report and the kind words 🙂

I tried enabling -sASSERTIONS and -sASSERTIONS=2 but this gives me a resolved is not a function error so no dice...

Yes, this is a known behavior, I had the same experience (Discord thread). It should be correct without those.

---

Looking at your stack trace, problem seems to come from here:

gdext/godot-ffi/src/binding/single_threaded.rs

Lines 72 to 92 in 4bc92e8

	unsafe fn set_initialized(&self, initialized: bool) {
	if initialized == self.initialized() {
	if initialized {
	panic!("already initialized");
	} else {
	panic!("deinitialize without prior initialize");
	}
	}
	// 'std::thread::current()' fails when linking to a Godot web build without threads. When compiling to wasm-nothreads,
	// we assume it is impossible to have multi-threading, so checking if we are in the main thread is not needed.
	// Therefore, we don't store the thread ID, but rather just whether initialization already occurred.
	#[cfg(wasm_nothreads)]
	self.initialized.set(initialized);
	#[cfg(not(wasm_nothreads))]
	{
	let thread_id = initialized.then(|| std::thread::current().id());
	self.main_thread_id.set(thread_id);
	}
	}

I don't know which panic is exactly triggered, is there maybe a way to log/debug the message and/or line number?

Since this is also about threading, one check is to make sure your threading settings are in sync.
You need (mutually exclusive):

• Either:
  • Export options in Godot with thread support
  • Enable experimental-wasm feature
  • Emcc config -C pthread (like you have now)
• Or:
  • Export options in Godot without thread support
  • Enable both experimental-wasm and experimental-wasm-nothreads
  • Remove the -C pthread option

---

Another few points you can try that have caused problems in the past:

1. Do you use the lazy-function-tables feature?
2. Can you check the .wasm file names of the different libraries, and make sure they're distinct?
   (We had an issue with renamed .wasm files that weren't found in Add helpful error for renamed Wasm module #799, although not directly related).
3. Someone added other emcc flags:

   rustflags = [
       ...
       "-Clink-arg=-fwasm-exceptions",
       "-Clink-args=-sDISABLE_EXCEPTION_CATCHING=1",
       "-Clink-args=-sEXPORT_ALL=1",
       "-Clink-args=-sSUPPORT_LONGJMP=wasm",
       "-Cllvm-args=-enable-emscripten-cxx-exceptions=0",
       "-Cllvm-args=-wasm-enable-sjlj",
   ]

4. Emscripten version: I personally couldn't find any difference, but some people tested 3.1.39 last year. For me things worked with latest (but my setup was simple).

There is also quite a bit of knowledge in #438. Unfortunately it's still quite scattered, there's some work to be done in improving the web-export page in the book.

[Guusggg]

Hi,

Thank you so much for the quick reply and the helpful links!

Export options in Godot with thread support
Enable experimental-wasm feature
Emcc config -C pthread (like you have now)

This is how I've been doing it when I submitted the projects, so I guess this isn't it.
(Edit: I notice now how it says godot_ffi::bindings::single_threaded in the original post, which makes me wonder if I really did this...)

Export options in Godot without thread support
Enable both experimental-wasm and experimental-wasm-nothreads
Remove the -C pthread option

This also causes the same error, however, exporting using Release now gives me the error: already initialized. So I guess where are somewhere on track here.

For both approaches, removing the second gdextension makes it run again (although with errors about missing classes, obviously).

1. Currently I'm not using the "lazy-function-tables" option, but I have tried it in the past to get faster compile times.
2. In my export folder, I see client.wasm and shared.wasm as (should be right?) expected.
3. With all of these, same thing. Unable to load the program. I've tried with both -pthread and without.

Aborted()
onPrintError	@	index.js:474
abort	@	index.js:9
__abort_js	@	index.js:9
$abort	@	0062dbd6:0xa3fc5
$std::sys::pal::unix::abort_internal::h5ffb198ad2b8db0d	@	client.wasm-0f92f976:0x3c3083
$std::panicking::rust_panic_with_hook::hc3a8cd21f77927d3	@	client.wasm-0f92f976:0x3ce88c
$std::panicking::begin_panic_handler::_$u7b$$u7b$closure$u7d$$u7d$::hb19be6fa9857ee61	@	client.wasm-0f92f976:0x3cbbe9
$std::sys::backtrace::__rust_end_short_backtrace::h585b6ab617e4600a	@	client.wasm-0f92f976:0x3cbab1
$rust_begin_unwind	@	client.wasm-0f92f976:0x3cd895
$core::panicking::panic_nounwind_fmt::runtime::h9938091b05990098	@	client.wasm-0f92f976:0x44c31b
$core::panicking::panic_nounwind_fmt::h6e7e6d2ffc89c5c5	@	client.wasm-0f92f976:0x44c2b2
$core::panicking::panic_nounwind::h99d9192d3d1b4d38	@	client.wasm-0f92f976:0x44c3c4

$core::ptr::replace::precondition_check::h75dfa671cd82b880	@	client.wasm-0f92f976:0x3c1167
$core::ptr::replace::h9395aa21c0716916	@	client.wasm-0f92f976:0x3c12dd
$std::sys::thread_local::native::lazy::Storage$LT$T$C$D$GT$::initialize::h4975e865484cb4ea	@	client.wasm-0f92f976:0x3a0094
$std::sys::thread_local::native::lazy::Storage$LT$T$C$D$GT$::get_or_init::hd146c2b9dcadca57	@	client.wasm-0f92f976:0x39fe95
$std::hash::random::RandomState::new::KEYS::_$u7b$$u7b$constant$u7d$$u7d$::_$u7b$$u7b$closure$u7d$$u7d$::h259f534f630663f1	@	client.wasm-0f92f976:0x3a036d
$core::ops::function::FnOnce::call_once::hf0fb143070654117	@	client.wasm-0f92f976:0x35457c
$std::thread::local::LocalKey$LT$T$GT$::try_with::h531f46e8c61644d2	@	client.wasm-0f92f976:0x35de7f
$std::thread::local::LocalKey$LT$T$GT$::with::h366e74810743984b	@	client.wasm-0f92f976:0x35ddc8
$std::hash::random::RandomState::new::h5ad2fb59cf112d55	@	client.wasm-0f92f976:0x353b3e
$_$LT$std..hash..random..RandomState$u20$as$u20$core..default..Default$GT$::default::h8ff89411a43748c1	@	client.wasm-0f92f976:0x35b3b6
$_$LT$std..collections..hash..map..HashMap$LT$K$C$V$C$S$GT$$u20$as$u20$core..default..Default$GT$::default::hd24fc6464a267ba0	@	client.wasm-0f92f976:0x35339f
$std::collections::hash::map::HashMap$LT$K$C$V$GT$::new::h1f4c9b7444ff5bb3	@	client.wasm-0f92f976:0x35336e

$godot_ffi::string_cache::StringCache::new::h62ad0e819dbe3e71	@	client.wasm-0f92f976:0x364053

$godot_ffi::initialize::hc53ec13c08f5642d	@	client.wasm-0f92f976:0x35d3d3
$godot_core::init::__gdext_load_library::_$u7b$$u7b$closure$u7d$$u7d$::h91332e9a4c846654	@	client.wasm-0f92f976:0x60b07
$core::ops::function::FnOnce::call_once::h64128a8d9be8ca9e	@	client.wasm-0f92f976:0x5e867
$std::panicking::try::do_call::h8565294c672b6171	@	client.wasm-0f92f976:0x68466
$__rust_try	@	client.wasm-0f92f976:0x681fc
$std::panicking::try::h129b92da609d815c	@	client.wasm-0f92f976:0x682f3
$std::panic::catch_unwind::hdd122ce3867bdf42	@	client.wasm-0f92f976:0x4ea98
$godot_core::private::handle_panic_with_print::h6d40b7b7e0423e98	@	client.wasm-0f92f976:0x517f1
$godot_core::private::handle_panic::h2142ae78a69be744	@	client.wasm-0f92f976:0x51517
$godot_core::init::__gdext_load_library::hb155d0c815240a5c	@	client.wasm-0f92f976:0x609db

$gdext_rust_init

A couple things that I find interesting in the stacktrace is:

• $core::ptr::replace::precondition_check::h75dfa671cd82b880 @ client.wasm-0f92f976:0x3c1167 This is the last function being called before something with unwind, it's also suspicious that it has _check in the name. Could this be triggering the actual panic?
• $godot_ffi::string_cache::StringCache::new::h62ad0e819dbe3e71 @ client.wasm-0f92f976:0x364053 This is the last function being called from code that is coming from this repository. Could it be that WebAssembly has a hard time with "random" things? I know using WASM before in Rust that people recommended using the Javascript functions for random as it is not in std in Rust.

I'm browsing the source code of the godot-rust crates a bit, to see if there's something that I think that could cause a panic. (Edit: I've now read the rust docs, and it should be enabled by default when compiling without optimizations) I'm not sure how to get a better stacktrace, but I did see debug-assertions in the godot-core crate. Is there a way to turn those on or rather, is it as simple as adding "debug_assertions" to features?

Kind regards,

[Bromeon]

debug-assertions is enabled by default (in Debug builds, so when you don't compile with --release or change build flags). I'm not sure how exactly it works in Wasm, maybe the target handles things a bit differently?

---

Your use case of combining multiple extensions is quite new and this setup isn't well-tested yet. We recently discussed a similar dependency model (but outside Wasm) in #951.

godot-rust generally assumes there's only one instance in its address space -- which may no longer be true in your case. Since it panics near some initialization routines in

gdext/godot-ffi/src/binding/single_threaded.rs

Lines 72 to 92 in 4bc92e8

	unsafe fn set_initialized(&self, initialized: bool) {
	if initialized == self.initialized() {
	if initialized {
	panic!("already initialized");
	} else {
	panic!("deinitialize without prior initialize");
	}
	}
	// 'std::thread::current()' fails when linking to a Godot web build without threads. When compiling to wasm-nothreads,
	// we assume it is impossible to have multi-threading, so checking if we are in the main thread is not needed.
	// Therefore, we don't store the thread ID, but rather just whether initialization already occurred.
	#[cfg(wasm_nothreads)]
	self.initialized.set(initialized);
	#[cfg(not(wasm_nothreads))]
	{
	let thread_id = initialized.then(|| std::thread::current().id());
	self.main_thread_id.set(thread_id);
	}
	}

, it might just be that the "singleton" responsible for loading the library is instantiated more than once (in one address space), and the assertion "already initialized" fails, because it accesses the same global variable.

Maybe you could clone the repo, point to the dependency locally and try changing this part -- so that instead of panicking on 2nd initialization, it would return and skip this part? Calling this only once is a safety precondition, so you'd likely need some further changes or run into UB, but it could already be a start to test the hypothesis.

---

In your multiple-rust-project.zip, you have this code twice:

#[gdextension]
unsafe impl ExtensionLibrary for ClientExtension {}

And you mentioned that leaving this only once (in the library shared) would make it work? If yes, that would be another indicator for the above behavior.

[Guusggg]

Thank you so much again for the quick reply! You got me tinkering...!

I'm unable to finish the research tonight but I've learned quite a bit already. For one thing, I'm almost certain that gdext is being initialized twice, as when I compile it with "__debug-log", I see the message Initialize gdext... twice.

I don't really understand the impact of this yet, but I'm sure it's not a good thing :P. I suspect it certainly has to do something with the errors I'm getting, now to find the culprit of how to fix it. I'm sure I don't have enough knowledge to really fix it, but I hope I could make a proposal and with you're help, refine it into a nice fix.

Thanks again for your work, I really appreciate it. I'll see if I can get a hacky fix, and I'll let you know. I'd be really happy if you'd be able to help me along then too!

Edit: Just a quick comment; I'm thinking that some errors occur because it's using the Shared Memory feature of WASM? It might be that the second initialization overwrites some stuff... Food for thought.. Lemme get into it.

[Bromeon]

I think a proper fix would be to add an attribute to the gdextension proc macro:

#[gdextension(dependency)]
unsafe impl ExtensionLibrary for ClientExtension {}

And based on that, certain initialization code would not be generated and/or not run.

In case you want to tinker, you could search for handle_alone in the codebase. That allows you to detect the key dependency in the above macro 🙂

[Guusggg]

@Bromeon Hi, I created a pullrequest at #973. I know it's not of high quality, and I was just looking to get it to work.

Do not merge it! :P

Could you take a look? I've encountered some strange things that I'll be detailing in the pull request.