Re: Data authentication for geli(8) committed to HEAD.

That's kinda of hard problem with any software-based disk block data 
encryption. Higher layers (UFS for example) expect one-sector writes to 
be atomic, so that there are basically 3 possibilities:

1. Stick with 512-bytes encrypted sector size. Not very good for 
performance reasons and possibly security reasons. Is not even possible 
in the cases when each write of one encrypted sector must also update 
some metadata (e.g. geom).

2. Live with the possibility that an unexpected system crash can leave 
the filesystem system in the unrecoverable state. My own experience with 
geom, which exploits this approach, however suggests that the chance is 
pretty slim even when actively used filesystem (PostgreSQL DB in my 
case) is placed on top of encrypted disk.

3. Implement some kind of sector-level journaling, again the performance 
is likely to suffer a great deal due to seeks.

-Maxim

Ulrich Spoerlein wrote:
> Pawel Jakub Dawidek wrote:
>> One of the main design goals was to make it reliable and resistant to
>> power failures or system crashes. This was very important to commit both
>> data update and HMAC update as an atomic operation to the disk, so users
>> don't have to fight with false positives.
> 
> Great stuff. I take this opportunity to hijack this thread and ask I
> question that has been bothering me a long time now.
> 
> I have an external HDD that I initially attached via Firewire, but I've
> since switched to USB, as our firewire subsystem is less than rock
> solid. When configuring geli encryption on the drive, I made the folly
> of using 4kB sectors, as this should improve performance. But, it has a
> disastrous impact on data integrity if your system crashes.
> 
> Now, I don't know if I'm right with the following thoughts, so please
> correct me if I'm wrong.
> 
> Consider an mtime or atime update. You need to update 4 bytes on the
> disk, which will be accomplished by writes of 512 bytes. If the system
> crashes in between (I'm not talking about power failures), the disk has
> either updated the 512 bytes, or not. No harm done.
> 
> With 4096 byte sectors and unencrypted data, the hard disk will have
> written/updated 512 or 1024, ... or the whole 4096 bytes. Since only 4
> bytes needed to be changed, you either got these changes or not. No harm
> done.
> 
> Now consider encryption with 4096 byte sectors. Whenever you twiddle one
> bit, the whole 4096 byte sector changes completely. To leave the disk in
> a consistent state, you always have to make sure, that the whole 8
> 512-byte blocks are written to disk. If only a single 512-byte block is
> not updated (due to kernel crashing because of that !_at_#^#$_at_ firewire),
> the whole 4096 are useless. SoL.
> 
> The question really is, are 512 byte disk writes considered to be some
> kind of "atomic" as it is the smallest disk block size? What does the
> ATA subsystem do with writes of 4096? Are they completed atomically too,
> or not?
> 
> Due to various kernel panics (no power failures involved!) I lost
> several blocks of inodes on a 4096 byte sector geli mount. It's really
> no fun, if fsck tells you that inodes 102340 - 109329 are lost.
> 
> Please note that I'm not asking any action from you, I'd simply
> appreciated it, if someone could confirm or dispute my claims. Thanks.
> 
> Ulrich Spoerlein