ZFS vnode starvation with large kmem_size

Thu Apr 12 04:55:02 UTC 2007

Hi Pawel,

I observed vnode starvation with large kmem_size in loader.conf.
It seems that even if zfs eats vnodes up to maxvnodes,
vnodes cannot be reclaimed and vnlru kthread will give up.
As far as I understand, the problem comes from difference
of name cache mechanism. I attached a patch for a workaround.

How to repeat:
  arch: amd64 or i386
  physical memory: 512MB
  vm.kmem_size=671088640
    ( In this setting:
       kern.maxvnodes: 35077
       vfs.zfs.dnlc.ncsize: 33095 )

  Create zfs filesystem and extract ports.tar.gz(many files) on it.
  After a while you can see that the system is not responsive because of
  vnode starvation.

Workarounds:

1. reduce vfs.zfs.dnlc.ncsize
 or
2. apply the following patch

Index: dnlc.c
===================================================================
RCS file: /home/ncvs/src/sys/contrib/opensolaris/uts/common/fs/dnlc.c,v
retrieving revision 1.1
diff -u -r1.1 dnlc.c

--- dnlc.c	6 Apr 2007 01:09:02 -0000	1.1
+++ dnlc.c	12 Apr 2007 04:18:34 -0000
@@ -703,6 +703,9 @@
 dnlc_get(uchar_t namlen)
 {
 	ncache_t *ncp;
+	extern int desiredvnodes;
+	extern int numvnodes;
+	static uint_t wantvnodes_reduce_percent = 10;
 
 	if (dnlc_nentries > dnlc_max_nentries) {
 		dnlc_max_nentries_cnt++; /* keep a statistic */
@@ -714,7 +717,10 @@
 	}
 	ncp->namlen = namlen;
 	atomic_add_32(&dnlc_nentries, 1);
-	dnlc_reduce_cache(NULL);
+	if (numvnodes > desiredvnodes)
+		dnlc_reduce_cache((void *)&wantvnodes_reduce_percent);
+	else
+		dnlc_reduce_cache(NULL);
 	return (ncp);
 }
 


I know this kmem_size must be too large but I think zfs should
cooperate with existent vnode management.

Except this problem, zfs works very well on my machines. 
Thank you for your great work!

/\ Hidetoshi Shimokawa
\/  simokawa at FreeBSD.ORG
