I split out the init and shutdown stuff from ui.cpp into init.cpp, so now ui.cpp is pure UI. ui.h provides inline stubs if wxUSE_GUI=0. We only have four functions that interface from the node to the UI. In the bitcoind build, we don't link ui.o or uibase.o.
It started increasing right away. I'll see if valgrind can help me.Sure feels like it could be something in wxWidgets retrying endlessly because some UI thing failed or something wasn't inited correctly. Our hack to ignore the initialize failure and run anyway means we're in uncharted territory. We're relying on the fact that we hardly use wx in this mode. We do still use a few things like wxGetTranslation and wxMutex.
Another way to debug would be to run in gdb, wait until everything is quiet and all threads should be idle, and break it and see which thread is busily doing something and what it's doing.
I suspect bitcoind will probably work fine, but I hope you can still debug the problem.