Question: how to trace slow UI in a XAML app (e.g. Windows Terminal) #2648
Comments
msft-github-bot
added
the
needs-triage
|
@MikeHillberg @codendone @Austin-Lamb do one of you know the proper route for this question? @vadimkantorov does your performance issue persist even when no debugger is attached to the process? What about release mode? |
|
I haven't tried attaching debugger to WT. I can try using perf.exe / procmon / Event Tracing / some other tracing if you have some suggestions (I never used them before for perf investigations, I just know they exist). I think the problem is somehow related to slow disk, and that something is getting loaded from relatively slow HDD. Maybe Windows still swap some pages even though I have 20% memory free as reported by Task Manager, maybe some shared libraries are getting loaded back into memory, maybe some configs are re-read from disk? Maybe some memory compression at fault? After I use these buttons once, they become much faster, so some caching effects are definitely in play. Ideally it would be good to first get some wall-clock and system-clock timing logs for e.g. button click perceived by XAML and menu drawn/tab created. This should confirm the effect I assume WT is built in release mode, it's 1.0rc2 version downloaded from Releases, I didn't compile it myself. I'll upgrade it to 1.0 release, just in case. |
|
I've done a quick perfmon run on "fast" speed of just showing the New Tab menu, and I see a lot of language/i18n-related registry operations, some font loads and some pagefile reads. Probably Windows swaps things to disk even though WT has 7mb working set (reported by Task Manager). A question is: why is there a pagefile read to just show a menu?
I'll do a similar run for on "slow" speed as well Pagefile read details are: it's only a 32k read - probably a slow one, I wish it could be avoided |
|
I did two "slow" procmon traces and attach them in native procmon format inside the zip. They confirm that the problem is loading from pagefile.sys. The question is: can you figure out what is it loading from pagefile.sys? It seems that it's just the app DLLs. Do you see some other things? Process Explorer shows that WT uses a lot of dynamic shared libraries including a dozen XAML ones. Procexp shows that real WorkingSet is around 65Mb and PrivateBytes 250Mb (just one open tab inside Terminal). Should we just ask WT to reduce WorkingSet and used shared libraries / fonts? Does XAML framework have some guidelines for apps to follow that want to minimize resource usage? |
Its usually bad to let used physical memory get such high because it leaves too little room for file caching. That means Windows has more often to take the significantly slower path to the disk instead of reading from caches. Depending on what you do you want a few GB of unused memory for file caching.
Windows backs (almost) everyhing by the pagefile, this is necessary because it is overcommitting memory, allowing you to run more applications than you actually have memory for, knowing that most application will not be using all the memory they reserved. It is normal that with high physical memory load Windows will swap out currently unused memory and swap it back in when its used again. This means with higher memory load you get more disk access and generally things become slower, long before you actually hit "out of memory" errors. Try the RamMap tool from sysinternals and "Empty Working Sets" - it forces to flush out memory from currently unused applications when Windows isn't doing its job good enough, leaving more space for file caches etc. It helps a lot for my 8 GB notebook, without manually flushing from time to time my notebook starts swapping (and slowing down) like crazy. If that helps you are just observing your machine approaching the out of memory barrier, and Windows trying to keep the system responsive by doing more swapping, which ironically reduces responsiveness. I think the OS heuristics for the pagefile have been tuned for more modern machines, thats probably why the slowdown on older ones with less RAM are so exaggerated. PS: just commenting based on research I did on my own notebook slowdowns, may or may not be related to yours |
|
@weltkante RAMMap "Empty *" seems to have helped considerably |
|
Then its probably an OS level issue and its unlikely WinUI by itself can do much about it, not sure what the appropriate way is to handle this, lets wait for a response from the team. I fully agree that the OS should handle older hardware with limited RAM better, you shouldn't have to do a manually forced flush to get a responsive system. |
|
@welkante I wonder if XAML controls could benefit from IL linker / symbol trimming in general |
|
@StephenLPeters I think a summary for my asks of WinRT / XAML / WinUI side would be:
Terminal by itself is not very keen on doing these investigations, reason being roughly "we are not doing anything special, so it perf should be fixed upstream so that everyone benefits" (speaking from my original issue in the repo) |
|
I think @Austin-Lamb @MikeHillberg or @codendone would have more context or be able to point to someone who does. Could one of you weigh in here? |
|
@bartekk8 as FYI |
|
Hello, Let's start with getting a trace of your sceanrio. To trace XAML app, follow these steps:
I hope this helps! |
|
I kind of confirmed it’s low memory / swapping that is the problem. And I could see in the procmon taking time to load libraries from disk. When I have more memory available, Terminal works faster. Speaking more generally, it would be great if these tools could be easily installed from Store or shipped with windows. Having perf kit shipped only with adk seems s big stumbling part. Ideally, there would be some library that a client app like Terminal could use to do some trace of perf, and e.g. use it in Telemetry or in issue reports. Questions about libraries and reducing working set / linker still hold. Also maybe this issue should be moved to https://github.com/microsoft/ProjectReunion, please feel free to move it if it better fits there :) |
StephenLPeters
removed
the
needs-triage
|
One example that would also benefit from super-easy perf recording (by non windev-experienced end users) is slow Calculator start-up microsoft/calculator#209 |
|
Hi @vadimkantorov, For your comment:
If your running Windows 10;
Hopefully these can get you started without the ADK. |
|
Thanks for advice about WPA. I'll try it next time! Unfortunately, these are "advanced" tools enough so that Microsoft Terminal devs sent me to this repo and suggested asking a question as an issue here. Maybe they are not accustomed to using these tools either :( They didn't communicate to me any prior attempts to measure Working Set memory or number of used shared libraries or testing the app in low-memory conditions and continued to refer that I should investigate this myself (with your help) in this repo :( The general questions about "why Terminal (that uses winrt/winui?)" uses so many assemblies and if it can make use of some linker to strip unused controls holds. |
|
For the record, the Terminal team is plenty accustomed to using those tools, we've just focused our efforts primarily on CPU performance, because the memory impact of the actual terminal bits of the Terminal is pretty minimal. We've kicked this thread over to the WinUI repo because the people who are best equipped to analyze the memory impact of WinUI are the people who own WinUI. |
|
On my side, I'll incur my machine in low-memory mode and will provide you a wprp file. About tools / WinUI: even a simple procmon run sufficed to find low-memory / swapping problems. And procexp.exe showed the problem of a lot of (seemingly unrelated) shared libraries. If you had adviced directly to download WPA (also completely generic, not WinUI-specific) from windows store and how to configure it to run on Terminal, I would have tried to use it from the get-go. @zadjii-msft Sorry for a lot of mis-communication. |
|
Maybe this issue now would be better transferred to https://github.com/microsoft/windev (as per too many libraries used by winrt and for optimizing down the working set of winrt/winui apps including Terminal) |
|
IMO It would probably just get punted back to the experts over here, or to Kenny over in microsoft/xlang / microsoft/cppwinrt, but I doubt the language itself is at fault here - I'd imagine that those libraries are getting regardless if XAML is being used via c++/winrt, c#, rust, cxx, or whatever other language the xlang team adds WinRT support to next. |
|
This issue is stale because it has been open 180 days with no activity. Remove stale label or comment or this will be closed in 5 days. |
I have slow reaction (up to 5 seconds) reaction of New Tab buttons in Windows Terminal (context menu takes a few seconds to appear; new tab button freezes for a few seconds before even a new tab is created). I reported the issue in WT repo: microsoft/terminal#6409
I asked how to enable tracing of WT UI handling. I was directed by @zadjii-msft to ask this question in this repo, and @zadjii-msft strongly suggested that the slowness comes not from WT code but from XAML framework itself.
Are there generic ways to trace what is happening? (e.g. what events are emitted and how long do their handling takes). I have zero context about how WT uses XAML or what it does on handling the button clicks. Unfortunately, I don't have experience with using
perf.exeor Event Tracing, but I'll be happy to try any instructions you suggest.My system has about 80% of used physical memory and HDD (it's a L470 lenovo laptop).
The text was updated successfully, but these errors were encountered: