Re: My project wish-list for the next 12 months

Stephan Uphoff wrote:
> On Thu, 2004-12-02 at 09:41, Andre Oppermann wrote:
> 
>>Scott Long wrote:
>>
>>>5.  Clustered FS support.  SANs are all the rage these days, and
>>>clustered filesystems that allow data to be distributed across many
>>>storage enpoints and accessed concurrently through the SAN are very
>>>powerful.  RedHat recently bought Sistina and re-opened the GFS source
>>>code, so exploring this would be very interesting.
>>
>>There are certain steps that can be be taken one at a time.  For example
>>it should be relatively easy to mount snapshots (ro) from more than one
>>machine.  Next step would be to mount a full 'rw' filesystem as 'ro' on
>>other boxes.  This would require cache and sector invalidation broadcasting
>>from the 'rw' box to the 'ro' mounts.  
> 
> Mhhh .. if you plan to invalidate at the disk block cache layer then you
> will run into race conditions with UFS/FFS (Especially with remove
> operations)
> I was once called in to evaluate such a multiple reader/single writer
> system based on an UFS like file system and block layer invalidation and
> had to convince management to kill it. (It appeared to work and actually
> made it though internal and customer acceptance testing before failing
> horrible in the field).
> 
> If you send me more details on your proposed cache and sector
> invalidation/cluster design I will be happy to review it.

It obviously doesn't work only the disk block level.  For the rw+ro(n) case
you have to invalidate updated/changed inodes (file and directory inodes)
as well as associated data blocks.  Alternatively you could invalidate all
related data blocks together with the inode but that would kill performance
for files that are simply appended (like log files).

-- 
Andre