On Sat, 2012-06-09 at 09:21 +0200, Wojciech Puchar wrote:
> top reports wired memory 128MB
> WHERE it is used? below results of vmstat -m and vmstat -z
> values does not sum up even to half of it
> FreeBSD 9 - few days old.
> What i am missing and why there are SO MUCH wired memory on 1GB machine
> without X11 or virtualbox
> [vmstat output snipped]
I have been struggling to answer the same question for about a week on
our embedded systems (running 8.2). We have systems with 64MB ram which
have 20MB wired, and I couldn't find any way to directly view what that
wired memory is being used for. I also discovered that the vmstat
output accounted for only a tiny fraction of the 20MB.
What I eventually determined is that there is some sort of correlation
between vfs buffer space and wired memory. Our embedded systems
typically do very little disk IO, but during some testing we were
spewing debug output to /var/log/messages at the rate of several lines
per second for hours. Under these conditions the amount of wired memory
would climb from its usual of about 8MB to around 20MB, and once it
climbed that high it pretty much never went down, or only went down a
couple MB. The resulting memory pressure caused our apps to get killed
over and over again with "out of swap space" (we have no swap on these
The kernel auto-tunes the vfs buffer space using the formula "for the
first 64 MB of ram use 1/4 for buffers, plus 1/10 of the ram over 64
MB." Using 16 of 64 MB of ram for buffer space seems insane to me, but
maybe it makes sense on certain types of servers or something. I added
"option NBUF=128" to our kernel config and that dropped the buffer space
to under 2 MB and since doing that I haven't seen the amount of wired
memory ever go above 8 MB. I wonder whether my tuning of NBUF is
affecting wired memory usage by indirectly tuning the 'nswbuf' value; I
can't tune nswbuf directly because the embedded system is ARM-based and
we have no loader(8) for setting tunablables.
I'm not sure NBUF=128 is a good setting even for a system that doesn't
do much IO, so I consider it experimental and we're testing under a
variety of conditions to see if it leads to any unexpected behaviors.
I'm certainly not suggesting anyone else rush to add this option to
their kernel config.
I am VERY curious about the nature of this correlation between vfs
buffer space and wired memory. For the VM gurus: Is the behavior I'm
seeing expected? Why would memory become wired and seemingly never get
released back to one of the page queues after the IO is done?