Re: Journaling UFS with gjournal.

From: Pawel Jakub Dawidek <pjd_at_FreeBSD.org>
Date: Fri, 23 Jun 2006 21:38:57 +0200
On Fri, Jun 23, 2006 at 10:20:38AM -0500, Eric Anderson wrote:
+> Pawel Jakub Dawidek wrote:
+> >Hello.
+> >For the last few months I have been working on gjournal project.
+> >To stop confusion right here, I want to note, that this project is not
+> >related to gjournal project on which Ivan Voras was working on the
+> >last SoC (2005).
+> >The lack of journaled file system in FreeBSD was a tendon of achilles
+> >for many years. We do have many file systems, but none with journaling:
+> >- ext2fs (journaling is in ext3fs),
+> >- XFS (read-only),
+> >- ReiserFS (read-only),
+> >- HFS+ (read-write, but without journaling),
+> >- NTFS (read-only).
+> >GJournal was designed to journal GEOM providers, so it actually works
+> >below file system layer, but it has hooks which allow to work with
+> >file systems. In other words, gjournal is not file system-depended,
+> >it can work probably with any file system with minimum knowledge
+> >about it. I implemented only UFS support.
+> >The patches are here:
+> >	http://people.freebsd.org/~pjd/patches/gjournal.patch (for HEAD)
+> >	http://people.freebsd.org/~pjd/patches/gjournal6.patch (for RELENG_6)
+> >To patch your sources you need to:
+> >	# cd /usr/src
+> >	# mkdir sbin/geom/class/journal sys/geom/journal sys/modules/geom/geom_journal
+> >	# patch < /path/to/gjournal.patch
+> >Add 'options UFS_GJOURNAL' to your kernel configuration file and
+> >recompile kernel and world.
+> >How it works (in short). You may define one or two providers which
+> >gjournal will use. If one provider is given, it will be used for both -
+> >data and journal. If two providers are given, one will be used for data
+> >and one for journal.
+> >Every few seconds (you may define how many) journal is terminated and
+> >marked as consistent and gjournal starts to copy data from it to the
+> >data provider. In the same time new data are stored in new journal.
+> 
+> I'm not sure this is happening the way you describe exactly.  On my laptop, while rsyncing my /home partition to a newly created external disk (400G), I see 20MB/s writing 
+> to the journaled UFS2 device (/dev/label/backup.journal) passing through to the journal device (/dev/label/journal), then it switches to no writes to the journaled UFS2 
+> device (/dev/label/backup.journal) (my rsync pauses) while the journaled  device (/dev/label/backup) writes at 20MB/s for about 3-10 seconds.

When it is time for journal switch, we cannot switch the journals if we
still copy data from the inactive journal, so we wait then.
You can tune it a bit using those two sysctls:

kern.geom.journal.parallel_flushes - Number of flush I/O requests send
				     in parallel
kern.geom.journal.parallel_copies - Number of copy I/O requests send
				    in parallel

By default those are equal, you may increase the second one or decrease
the first one to tell gjournal to focus more on copying the data from
the inactive journal, so when journal switch time arrives, it doesn't
have to wait.
Before you do it, please consult kern.geom.journal.stats.wait_for_copy
sysctl variable, which will tell you how many times journal switch was
delayed because of inactive journal not beeing fully copied.

More waiting is because a lot of data is only in memory and when I call
file system synchronization all the data go to gjournal provider.

All modes in which UFS can operate are not optimal for gjournal - I mean
here sync, async and SU. The most optimal mode for gjournal will be
something like: send write request immediatelly and don't wait for an
answer. GJournal will take care of reordering write request to get
optimal throughput and this will allow for more balanced load.
For example SU send write requests in picks, which is bad for gjournal.

+> >Let's call the moment in which journal is terminated as "journal switch".
+> >Journal switch looks as follows:
+> >1. Start journal switch if we have timeout or if we run out of cache.
+> >   Don't perform journal switch if there were no write requests.
+> >2. If we have file system, synchronize it.
+> >3. Mark file system as clean.
+> >4. Block all write requests to the file system.
+> >5. Terminate the journal.
+> >6. Eventually wait if copying of the previous journal is not yet
+> >   finished.
+> 
+> Seems like this is the point we are busy in.
+> 
+> >7. Send BIO_FLUSH request (if the given provider supports it).
+> >8. Mark new journal position on the journal provider.
+> >9. Unblock write requests.
+> >10. Start copying data from the terminated journal to the data provider.
+> 
+> And it seems that 10 is happening earlier on..

The point number 10 is actually after the journal switch. It is when the
active journal was turned into an inactive journal and the copy starts.

Don't take this order to strict, I more wanted to show what steps are
performed.

-- 
Pawel Jakub Dawidek                       http://www.wheel.pl
pjd_at_FreeBSD.org                           http://www.FreeBSD.org
FreeBSD committer                         Am I Evil? Yes, I Am!

Received on Fri Jun 23 2006 - 17:42:24 UTC

This archive was generated by hypermail 2.4.0 : Wed May 19 2021 - 11:38:57 UTC