customize memcache expiration

[stephanos]

I'm about to integrate my hrd library with nds and noticed that it is currently not possible to specify a custom memcache expiration time.

Is this something you think might be worthwhile?

[jongillham]

I haven't done this because I can't think of a case where I would want anything other than the max cache expiry time.

What's your usecase?

[stephanos]

Now that I think about it you are right, if everything works as expected in nds it there should not be reason for it. At work we use Objectify which allows to customize the memcache expiration per entity. Apparently I've never questioned this strategy until now, but I can't come up with a good reason either (there is one entity with a 5 minute expiration I change maybe once a month through the datastore viewer since we didn't bother to create a UI for it yet, but that's not a good reason).

[stephanos]

Well, the only reason might be to remove an entity from memcache before the memory pressure needs to do it (the cache size is rather limited unless you have a dedicated memcache), for entities where you know that this will not be needed after x minutes. But that's probably premature optimization.

[stephanos]

On a related note: do you think there is a use case for ignoring the cache for specific datastore requests?

In Objectify:

ofy().cache(false).load().key(thingKey).now();

[jongillham]

Does Objectify have a strongly consistent cache strategy? If it doesn't then it would definitely need an option to specify cache expiry as a client could potentially fetch stale results forever. It would also need the option to fetch straight from the datastore.

If there is strongly consistent cache then there is no reason to not always use the cache for Get, Put and Delete methods assuming the memcache API is working correctly.

Note that NDS only adds entities to the cache on the first Get request. Therefore just using Put on thousands of entities does not affect memcache and memory pressure.

I am only aware of Python's ndb and this library that implements strong cache consistency. Do you know of any Java ones?

[stephanos]

From Objectify's documentation:

Objectify's global cache provides near-transactional consistency with the datastore, even under heavy contention. There is still, however, one circumstance in which the cache could go out of synchronization with the datastore: If your requests are cut off by DeadlineExceededException.

But I can certainly see your point about always using the cache. I don't know what the motivation in Objectify was.

---

One use case I could think of for skipping the cache for specific Get operations though is when a background task that runs maybe once a day iterates over all entities (e.g. to create robots.txt). Then it would fill the cache pretty quickly and possibly throw out entities that are accessed by the actual user.

[jongillham]

Thanks for the info. I am surprised that Objectify doesn't have a strongly consistent caching strategy! I wonder what Java library does. Libraries like Goon and, as you just showed, Objectify work for the majority of cases though.

I guess it depends on what type of app/functionality you are building and if you don't mind some junk being returned from cache requests every so often. App Engine instances frequently die/misbehave in all sorts of ways. I don't ever want to have to think about this in the code I write as cache consistency bugs always seem the hardest to find and potentially have devastating effects. This is why I created NDS.

As for your background task usecase. If you are just getting datastore entities and not mutating them then it would be easiest just to replace nds.Get with datastore.Get. They have the same signature. datastore.Query also doesn't touch the cache. Keep trying though - maybe there is a reason to have the option to avoid memcache or set the expiry! :-)

[stephanos]

Agreed. Great discussion, thanks :)

[jayp]

A max expiration would eventually self-heal #31 and #33. Otherwise, stale results can handarojnd forever if not changed/evicted.

I think it makes sense.

[jongillham]

You are correct, issues #31 and #33 would eventually heal. However, here's why I don't think adding a max expiration adds value for the majority of cases:

• Memcache doesn't stay around permanently. An entry can stay for days, maybe even weeks but it always gets evicted at some point in the future. Although I have to say there are no guarantees without specifying an eviction time.
• I can't think of a case where someone would expect strong cache consistency but be fine with that failing. Most people either want guaranteed cache consistency (which you get with using only nds.Get and nds.Put) or can tolerate not having it.
• Adding an extra setting increases complexity when there may only be a handful of uses.

[jayp]

Reason #2 resonates with me the most. That's really why this a forced expiration approach would be a hack.

With that said, for those that would like to deploy this hack: a forced reload every X minutes (or whatever), the extra setting would help. We could set it as an optional parameter, so no additional complexity for anyone who doesn't want to use it.