Well, it's 256 GB of ram each for 13 servers. I do wonder if the second half of that RAM, with the money used for SSDs, would result in better performance -- as your cache could be larger for the same budget. I think it'd really come down to your cache hit rate / cache size.
memcached is standalone, no one server knows about any other server, so at a simple level, no. It also can't go out and fetch data from else where, it's either got the data or it hasn't.
You could write a more complex cacheing layer on top of memcached that looked first in memcached, then fetched from solid state storage if it wasn't there. That's kind of what we will be using it for by wrapping Rails around it.
> You could write a more complex cacheing layer on top of memcached that looked first in memcached, then fetched from solid state storage if it wasn't there.
I wonder how that would compare to just setting up an SSD as swap space.
Memcached is designed to be fast. Anything that makes it slower that doesn't need to be there isn't there. For example: authentication (anyone who can access the port can fetch data), indexing (you need to know the key), deallocating memory (you configure memcached with an upper bound; it keeps allocating memory as needed until it gets there), etc.
Virtual memory (I assume you actually mean 'does memcached not page data out to disk') would make it much slower. Since memcached is just that - a memory cache - if you're out of memory it just expires the least-recently-used data. In your application, you fetch the key, and if that fails you fetch it from the primary data store (or wherever else you can find it).
In theory it probably could, but you're losing all benefit at that point. The big selling point of memcahce is that it's VERY fast. Our memcache server (Ours is only 8GB, not nearly as impressive as some) averages under 1ms object fetch times, even accounting for network overhead.
