MAISI Inference Quality Issue

[johnson111788]

Hi,

I've tried using maisi_diff_unet_training_tutorial.ipynb to inference the diffusion model but the result doesn't seem good.

• Inference Log

• Result

unet_3d_seed20070_size128x128x128_spacing1.00x1.25x0.75_20240817231813.nii.gz

---

I've also tried using spacing [1.0, 1.0, 0.75], but the result still doesn't seem good.

• Inference Log

• Result

unet_3d_seed3966_size128x128x128_spacing1.00x1.00x0.75_20240820010733.nii.gz

---

It would be great if you could let me know if there's any issue in my inference steps.

Best,
Yu-Cheng

[Can-Zhao]

Thank you for reaching out! The overall FOV, which is decided by output_size and spacing, is small in the given setting. The FOV here only covers part of the body region in xy-plane, while our training data covers the whole body in xy-plane. We could try a larger output_size or a larger spacing.

[Can-Zhao]

We added description on recommended FOV in a recent PR #1789

[johnson111788]

Hi Can,

It works for [256, 256, 256] with [2, 2, 2.5] spacing (FOV: [512mm, 512mm, 640mm]):

unet_3d_seed97781_size256x256x256_spacing2.00x2.00x2.50_20240820132330.nii.gz

It also works for [256, 256, 256] with [1, 1, 1.5] spacing (FOV: [256mm, 256mm, 384mm]):

unet_3d_seed22680_size256x256x256_spacing1.00x1.00x1.50_20240820163117.nii.gz

But not works for [128, 128, 128] with [4, 4, 4.5] spacing (FOV: [512mm, 512mm, 576mm]):

unet_3d_seed52973_size128x128x128_spacing4.00x4.00x4.50_20240820113857.nii.gz

I believe the failure might not only result from the FOV. The model seems to only work when the output size is set to [256, 256, 256].

Best,
Yu-Cheng

[johnson111788]

I think it's ok for me to continue my work with the quality check function from PR #1789. I'll set the output size as [256, 256, 256]

[dongyang0122]

@johnson111788 Thank you for your feedback. We will further verify this issue. In our experience, the settings of 512 x 512 x 512 and 256 x 256 x 256 are more stable in terms of the quality of the generated results.

[Can-Zhao]

@guopengf @dongyang0122 It seems to me that [128,128,128] is a setting that tends to produce low quality results. Do we have any thoughts on this?

[dongyang0122]

Let's verify this issue and update the model weights in a future release.

[guopengf]

Yeah, agree with @dongyang0122. Shall we also consider limiting the user input for small FOV (e.g., 128^3 with 1mm^3 spacing)? From my experience, I would suggest the FOV should be at least larger than 256x256x128 with 1.5mm^3. We can add an additional input check for the user's input output_szie and spacing before we fix this issue in the next release.

[Can-Zhao]

Yeah, agree with @dongyang0122. Shall we also consider limiting the user input for small FOV (e.g., 128^3 with 1mm^3 spacing)? From my experience, I would suggest the FOV should be at least larger than 256x256x128 with 1.5mm^3. We can add an additional input check for the user's input output_szie and spacing before we fix this issue in the next release.

I already put a FOV check in #1789, but with a loose limit, only need it to be at least 128mm. Do you think we need to make it more strict? I was thinking brain image is usually smaller. I also added in readme that we suggest FOV to be at least 256mm for brain and 384mm for other regions

[guopengf]

I prefer to be stricter for now. It seems like users start playing the model from the smallest images that we support. It will not affect the user's input label or body region. As long as there are suitable masks in the pool, the sampler will automatically resample to the given shape and spacing.

[Can-Zhao]

OK, let's make it stricter for now