Re: My project wish-list for the next 12 months

Scott Long wrote:
> Stephan Uphoff wrote:
> 
>> On Thu, 2004-12-02 at 09:41, Andre Oppermann wrote:
>>
>>> The holy grail of course is to mount
>>> the same filesystem 'rw' on more than one box, preferrably more than 
>>> two.
>>> This requires some more involved synchronization and locking on top 
>>> of the
>>> cache invalidation.  And make sure that the multi-'rw' cluster stays 
>>> alive
>>> if one of the participants freezes and doesn't respond anymore.
>>>
>>> Scrolling through the UFS/FFS code I think the first one is 2-3 days of
>>> work.  The second 2-4 weeks and the third 2-3 month to get it right.
>>> If someone would throw up the money...
> 
> Although I don't know the specifics of your experience, I can easily
> imagine how hard it would be to make this work on UFS.  Common
> operations like walking a file path to the root are nearly impossible to
> do reliably without an overbearing amount of synchronization.  Then you
> have all of the problems synchronizing buffered data and metadata.

You don't do it the fully synchronized way.  The semantics will be somewhat
like with NFS.  Reading in the filesystem does not invoke synchronization.
Only writing, or actually intending to write something, requires synchro-
nization with all other machines in the filesystem cluster.  Synchronization
ensures that writes to a directory or file are properly serialized and that
caches are invalidated at the same time.

> Softupdates would be a nightmare, if not impossible.

To the contrary.  It provides a nice place to link code into and it helps
to keep performance up to good levels.  Any inode changes entering the
softdep code would cause a message to all other machines informing them
of the change.  If another one wants to update the same inode or something
dependend on it it has to wait until you have written it to disk.  His
backnotification would of course move this inode to the front of the softdep
work queue to not stall your request.

As long as you don't want to have mmap() work across machines with contents
in memory but not yet written to disk things stay pretty much sane.

-- 
Andre