Re: using nscd (ldap) makes passwd/group disappearing while installing ports

Am 01.02.2012 01:03, schrieb Benjamin Lee:
> What's going on is:
> 
> 1) The port checks if the group exists
> 2) nscd caches that the group does not exist in its negative cache
> 3) pw(8) creates the group then checks if it exists
> 4) nscd returns the negative cache entry (group does not exist)
> 
> This causes pw(8) to error since it expects the group that it just
> created to exist.

I had suggested before, that nscd be changed to only rely on cached
negative entries, if they have repeatedly been seen in the underlying
files/data-bases.

E.g. only consider negative entries valid after they have been
repeatedly stored into the cache (I'd think 3 times are a reasonable
number). That way, the first lookup of an account or group will lead to
an entry with count=1, which is found during cache lookup but which is
ignored due to a too low "verification count". If the negative lookup is
repeated 3 times within some reasonable time window (say 60 seconds),
then it is to be considered verified.

This will make the above sequence of queries and modifications of the
passwd or group databases work with caching.

This concept does not protect against scenarios where the negative cache
entry is made active by several queries (e.g. manual checking for the
presence of an account or group before the software installation repeats
these tests). That is the reason to ask for more than 2 negative replies
(3, perhaps better 5) before a negative cache entry is trusted. The main
purpose of the cache (reduced latency for positive queries, limited load
due to negative queries) will still be maintained.

>> I also have this error very often when rebuilding/updating or even
>> installing cups when "nscd" is enabled. A simple restart of nscd helps
>> in most cases, most times I need to disable "cache" tag in
>> /etc/nsswitch.conf, then everything runs smooth.
>>
>> Well, this behaviour is since a couple of years now, occurs sporadic. I
>> have had in FreeBSD 7, 8, 9 and I see it in 10. What is it?
>>
>> I like the cache facility, since in domains with a lot of users
>> searching LDAP takes some time and caching help keeping traffic and
>> latency short. But the namservice caching mechanism seems to be
>> unreliable. What is up there?
> 
> You should put "files" before "cache" in /etc/nsswitch.conf, e.g.:
> 
> group: files cache ldap
> passwd: files cache ldap
> 
> The problem is that tools that modify the passwd and group files, like
> pw(8), don't invalidate nscd's negative cache entries when making
> changes.

You point out an alternative to making negative entries trusted only
after they have been repeatedly entered into the cache: Tools that are
used to modify the passwd/group databases might signal their changes to
nscd. They could either purge the modified caches for the current or for
all users, or they could just clear the single affected entry. In each
case, nscd needs to re-fetch the modified (and possibly all other
cached) entries. A simple implementation of such invalidation could be
to invoke "nscd -I" after each modification performed by "pw".

Still I think that the reduced trust in negative entries that have not
been repeatedly tested is the best solution.

I had looked into implementing such a logic in the cache, a few months
back. It took more effort than I had hoped due to the way the cache is
implemented, but I still think it should be possible without major
changes to nscd.

Regards, STefan