mirror of https://gitee.com/openkylin/linux.git
doc: Update doc about journalling layer
Documentation of journalling layer in Documentation/DocBook/filesystems.tmpl speaks about JBD layer. Since that is going away, update the documentation to speak about JBD2. Also update the parts that have changed since someone last touched the document and remove some parts which are just misleading and outdated. Signed-off-by: Jan Kara <jack@suse.com>
This commit is contained in:
parent
acc84b05b1
commit
82ff50b222
|
@ -146,36 +146,30 @@
|
||||||
The journalling layer is easy to use. You need to
|
The journalling layer is easy to use. You need to
|
||||||
first of all create a journal_t data structure. There are
|
first of all create a journal_t data structure. There are
|
||||||
two calls to do this dependent on how you decide to allocate the physical
|
two calls to do this dependent on how you decide to allocate the physical
|
||||||
media on which the journal resides. The journal_init_inode() call
|
media on which the journal resides. The jbd2_journal_init_inode() call
|
||||||
is for journals stored in filesystem inodes, or the journal_init_dev()
|
is for journals stored in filesystem inodes, or the jbd2_journal_init_dev()
|
||||||
call can be use for journal stored on a raw device (in a continuous range
|
call can be used for journal stored on a raw device (in a continuous range
|
||||||
of blocks). A journal_t is a typedef for a struct pointer, so when
|
of blocks). A journal_t is a typedef for a struct pointer, so when
|
||||||
you are finally finished make sure you call journal_destroy() on it
|
you are finally finished make sure you call jbd2_journal_destroy() on it
|
||||||
to free up any used kernel memory.
|
to free up any used kernel memory.
|
||||||
</para>
|
</para>
|
||||||
|
|
||||||
<para>
|
<para>
|
||||||
Once you have got your journal_t object you need to 'mount' or load the journal
|
Once you have got your journal_t object you need to 'mount' or load the journal
|
||||||
file, unless of course you haven't initialised it yet - in which case you
|
file. The journalling layer expects the space for the journal was already
|
||||||
need to call journal_create().
|
allocated and initialized properly by the userspace tools. When loading the
|
||||||
|
journal you must call jbd2_journal_load() to process journal contents. If the
|
||||||
|
client file system detects the journal contents does not need to be processed
|
||||||
|
(or even need not have valid contents), it may call jbd2_journal_wipe() to
|
||||||
|
clear the journal contents before calling jbd2_journal_load().
|
||||||
</para>
|
</para>
|
||||||
|
|
||||||
<para>
|
<para>
|
||||||
Most of the time however your journal file will already have been created, but
|
Note that jbd2_journal_wipe(..,0) calls jbd2_journal_skip_recovery() for you if
|
||||||
before you load it you must call journal_wipe() to empty the journal file.
|
it detects any outstanding transactions in the journal and similarly
|
||||||
Hang on, you say , what if the filesystem wasn't cleanly umount()'d . Well, it is the
|
jbd2_journal_load() will call jbd2_journal_recover() if necessary. I would
|
||||||
job of the client file system to detect this and skip the call to journal_wipe().
|
advise reading ext4_load_journal() in fs/ext4/super.c for examples on this
|
||||||
</para>
|
stage.
|
||||||
|
|
||||||
<para>
|
|
||||||
In either case the next call should be to journal_load() which prepares the
|
|
||||||
journal file for use. Note that journal_wipe(..,0) calls journal_skip_recovery()
|
|
||||||
for you if it detects any outstanding transactions in the journal and similarly
|
|
||||||
journal_load() will call journal_recover() if necessary.
|
|
||||||
I would advise reading fs/ext3/super.c for examples on this stage.
|
|
||||||
[RGG: Why is the journal_wipe() call necessary - doesn't this needlessly
|
|
||||||
complicate the API. Or isn't a good idea for the journal layer to hide
|
|
||||||
dirty mounts from the client fs]
|
|
||||||
</para>
|
</para>
|
||||||
|
|
||||||
<para>
|
<para>
|
||||||
|
@ -189,41 +183,41 @@ You still need to actually journal your filesystem changes, this
|
||||||
is done by wrapping them into transactions. Additionally you
|
is done by wrapping them into transactions. Additionally you
|
||||||
also need to wrap the modification of each of the buffers
|
also need to wrap the modification of each of the buffers
|
||||||
with calls to the journal layer, so it knows what the modifications
|
with calls to the journal layer, so it knows what the modifications
|
||||||
you are actually making are. To do this use journal_start() which
|
you are actually making are. To do this use jbd2_journal_start() which
|
||||||
returns a transaction handle.
|
returns a transaction handle.
|
||||||
</para>
|
</para>
|
||||||
|
|
||||||
<para>
|
<para>
|
||||||
journal_start()
|
jbd2_journal_start()
|
||||||
and its counterpart journal_stop(), which indicates the end of a transaction
|
and its counterpart jbd2_journal_stop(), which indicates the end of a
|
||||||
are nestable calls, so you can reenter a transaction if necessary,
|
transaction are nestable calls, so you can reenter a transaction if necessary,
|
||||||
but remember you must call journal_stop() the same number of times as
|
but remember you must call jbd2_journal_stop() the same number of times as
|
||||||
journal_start() before the transaction is completed (or more accurately
|
jbd2_journal_start() before the transaction is completed (or more accurately
|
||||||
leaves the update phase). Ext3/VFS makes use of this feature to simplify
|
leaves the update phase). Ext4/VFS makes use of this feature to simplify
|
||||||
quota support.
|
handling of inode dirtying, quota support, etc.
|
||||||
</para>
|
</para>
|
||||||
|
|
||||||
<para>
|
<para>
|
||||||
Inside each transaction you need to wrap the modifications to the
|
Inside each transaction you need to wrap the modifications to the
|
||||||
individual buffers (blocks). Before you start to modify a buffer you
|
individual buffers (blocks). Before you start to modify a buffer you
|
||||||
need to call journal_get_{create,write,undo}_access() as appropriate,
|
need to call jbd2_journal_get_{create,write,undo}_access() as appropriate,
|
||||||
this allows the journalling layer to copy the unmodified data if it
|
this allows the journalling layer to copy the unmodified data if it
|
||||||
needs to. After all the buffer may be part of a previously uncommitted
|
needs to. After all the buffer may be part of a previously uncommitted
|
||||||
transaction.
|
transaction.
|
||||||
At this point you are at last ready to modify a buffer, and once
|
At this point you are at last ready to modify a buffer, and once
|
||||||
you are have done so you need to call journal_dirty_{meta,}data().
|
you are have done so you need to call jbd2_journal_dirty_{meta,}data().
|
||||||
Or if you've asked for access to a buffer you now know is now longer
|
Or if you've asked for access to a buffer you now know is now longer
|
||||||
required to be pushed back on the device you can call journal_forget()
|
required to be pushed back on the device you can call jbd2_journal_forget()
|
||||||
in much the same way as you might have used bforget() in the past.
|
in much the same way as you might have used bforget() in the past.
|
||||||
</para>
|
</para>
|
||||||
|
|
||||||
<para>
|
<para>
|
||||||
A journal_flush() may be called at any time to commit and checkpoint
|
A jbd2_journal_flush() may be called at any time to commit and checkpoint
|
||||||
all your transactions.
|
all your transactions.
|
||||||
</para>
|
</para>
|
||||||
|
|
||||||
<para>
|
<para>
|
||||||
Then at umount time , in your put_super() you can then call journal_destroy()
|
Then at umount time , in your put_super() you can then call jbd2_journal_destroy()
|
||||||
to clean up your in-core journal object.
|
to clean up your in-core journal object.
|
||||||
</para>
|
</para>
|
||||||
|
|
||||||
|
@ -231,82 +225,74 @@ to clean up your in-core journal object.
|
||||||
Unfortunately there a couple of ways the journal layer can cause a deadlock.
|
Unfortunately there a couple of ways the journal layer can cause a deadlock.
|
||||||
The first thing to note is that each task can only have
|
The first thing to note is that each task can only have
|
||||||
a single outstanding transaction at any one time, remember nothing
|
a single outstanding transaction at any one time, remember nothing
|
||||||
commits until the outermost journal_stop(). This means
|
commits until the outermost jbd2_journal_stop(). This means
|
||||||
you must complete the transaction at the end of each file/inode/address
|
you must complete the transaction at the end of each file/inode/address
|
||||||
etc. operation you perform, so that the journalling system isn't re-entered
|
etc. operation you perform, so that the journalling system isn't re-entered
|
||||||
on another journal. Since transactions can't be nested/batched
|
on another journal. Since transactions can't be nested/batched
|
||||||
across differing journals, and another filesystem other than
|
across differing journals, and another filesystem other than
|
||||||
yours (say ext3) may be modified in a later syscall.
|
yours (say ext4) may be modified in a later syscall.
|
||||||
</para>
|
</para>
|
||||||
|
|
||||||
<para>
|
<para>
|
||||||
The second case to bear in mind is that journal_start() can
|
The second case to bear in mind is that jbd2_journal_start() can
|
||||||
block if there isn't enough space in the journal for your transaction
|
block if there isn't enough space in the journal for your transaction
|
||||||
(based on the passed nblocks param) - when it blocks it merely(!) needs to
|
(based on the passed nblocks param) - when it blocks it merely(!) needs to
|
||||||
wait for transactions to complete and be committed from other tasks,
|
wait for transactions to complete and be committed from other tasks,
|
||||||
so essentially we are waiting for journal_stop(). So to avoid
|
so essentially we are waiting for jbd2_journal_stop(). So to avoid
|
||||||
deadlocks you must treat journal_start/stop() as if they
|
deadlocks you must treat jbd2_journal_start/stop() as if they
|
||||||
were semaphores and include them in your semaphore ordering rules to prevent
|
were semaphores and include them in your semaphore ordering rules to prevent
|
||||||
deadlocks. Note that journal_extend() has similar blocking behaviour to
|
deadlocks. Note that jbd2_journal_extend() has similar blocking behaviour to
|
||||||
journal_start() so you can deadlock here just as easily as on journal_start().
|
jbd2_journal_start() so you can deadlock here just as easily as on
|
||||||
|
jbd2_journal_start().
|
||||||
</para>
|
</para>
|
||||||
|
|
||||||
<para>
|
<para>
|
||||||
Try to reserve the right number of blocks the first time. ;-). This will
|
Try to reserve the right number of blocks the first time. ;-). This will
|
||||||
be the maximum number of blocks you are going to touch in this transaction.
|
be the maximum number of blocks you are going to touch in this transaction.
|
||||||
I advise having a look at at least ext3_jbd.h to see the basis on which
|
I advise having a look at at least ext4_jbd.h to see the basis on which
|
||||||
ext3 uses to make these decisions.
|
ext4 uses to make these decisions.
|
||||||
</para>
|
</para>
|
||||||
|
|
||||||
<para>
|
<para>
|
||||||
Another wriggle to watch out for is your on-disk block allocation strategy.
|
Another wriggle to watch out for is your on-disk block allocation strategy.
|
||||||
why? Because, if you undo a delete, you need to ensure you haven't reused any
|
Why? Because, if you do a delete, you need to ensure you haven't reused any
|
||||||
of the freed blocks in a later transaction. One simple way of doing this
|
of the freed blocks until the transaction freeing these blocks commits. If you
|
||||||
is make sure any blocks you allocate only have checkpointed transactions
|
reused these blocks and crash happens, there is no way to restore the contents
|
||||||
listed against them. Ext3 does this in ext3_test_allocatable().
|
of the reallocated blocks at the end of the last fully committed transaction.
|
||||||
|
|
||||||
|
One simple way of doing this is to mark blocks as free in internal in-memory
|
||||||
|
block allocation structures only after the transaction freeing them commits.
|
||||||
|
Ext4 uses journal commit callback for this purpose.
|
||||||
</para>
|
</para>
|
||||||
|
|
||||||
<para>
|
<para>
|
||||||
Lock is also providing through journal_{un,}lock_updates(),
|
With journal commit callbacks you can ask the journalling layer to call a
|
||||||
ext3 uses this when it wants a window with a clean and stable fs for a moment.
|
callback function when the transaction is finally committed to disk, so that
|
||||||
eg.
|
you can do some of your own management. You ask the journalling layer for
|
||||||
|
calling the callback by simply setting journal->j_commit_callback function
|
||||||
|
pointer and that function is called after each transaction commit. You can also
|
||||||
|
use transaction->t_private_list for attaching entries to a transaction that
|
||||||
|
need processing when the transaction commits.
|
||||||
|
</para>
|
||||||
|
|
||||||
|
<para>
|
||||||
|
JBD2 also provides a way to block all transaction updates via
|
||||||
|
jbd2_journal_{un,}lock_updates(). Ext4 uses this when it wants a window with a
|
||||||
|
clean and stable fs for a moment. E.g.
|
||||||
</para>
|
</para>
|
||||||
|
|
||||||
<programlisting>
|
<programlisting>
|
||||||
|
|
||||||
journal_lock_updates() //stop new stuff happening..
|
jbd2_journal_lock_updates() //stop new stuff happening..
|
||||||
journal_flush() // checkpoint everything.
|
jbd2_journal_flush() // checkpoint everything.
|
||||||
..do stuff on stable fs
|
..do stuff on stable fs
|
||||||
journal_unlock_updates() // carry on with filesystem use.
|
jbd2_journal_unlock_updates() // carry on with filesystem use.
|
||||||
</programlisting>
|
</programlisting>
|
||||||
|
|
||||||
<para>
|
<para>
|
||||||
The opportunities for abuse and DOS attacks with this should be obvious,
|
The opportunities for abuse and DOS attacks with this should be obvious,
|
||||||
if you allow unprivileged userspace to trigger codepaths containing these
|
if you allow unprivileged userspace to trigger codepaths containing these
|
||||||
calls.
|
calls.
|
||||||
</para>
|
|
||||||
|
|
||||||
<para>
|
|
||||||
A new feature of jbd since 2.5.25 is commit callbacks with the new
|
|
||||||
journal_callback_set() function you can now ask the journalling layer
|
|
||||||
to call you back when the transaction is finally committed to disk, so that
|
|
||||||
you can do some of your own management. The key to this is the journal_callback
|
|
||||||
struct, this maintains the internal callback information but you can
|
|
||||||
extend it like this:-
|
|
||||||
</para>
|
|
||||||
<programlisting>
|
|
||||||
struct myfs_callback_s {
|
|
||||||
//Data structure element required by jbd..
|
|
||||||
struct journal_callback for_jbd;
|
|
||||||
// Stuff for myfs allocated together.
|
|
||||||
myfs_inode* i_commited;
|
|
||||||
|
|
||||||
}
|
|
||||||
</programlisting>
|
|
||||||
|
|
||||||
<para>
|
|
||||||
this would be useful if you needed to know when data was committed to a
|
|
||||||
particular inode.
|
|
||||||
</para>
|
</para>
|
||||||
|
|
||||||
</sect2>
|
</sect2>
|
||||||
|
@ -319,36 +305,6 @@ being each mount, each modification (transaction) and each changed buffer
|
||||||
to tell the journalling layer about them.
|
to tell the journalling layer about them.
|
||||||
</para>
|
</para>
|
||||||
|
|
||||||
<para>
|
|
||||||
Here is a some pseudo code to give you an idea of how it works, as
|
|
||||||
an example.
|
|
||||||
</para>
|
|
||||||
|
|
||||||
<programlisting>
|
|
||||||
journal_t* my_jnrl = journal_create();
|
|
||||||
journal_init_{dev,inode}(jnrl,...)
|
|
||||||
if (clean) journal_wipe();
|
|
||||||
journal_load();
|
|
||||||
|
|
||||||
foreach(transaction) { /*transactions must be
|
|
||||||
completed before
|
|
||||||
a syscall returns to
|
|
||||||
userspace*/
|
|
||||||
|
|
||||||
handle_t * xct=journal_start(my_jnrl);
|
|
||||||
foreach(bh) {
|
|
||||||
journal_get_{create,write,undo}_access(xact,bh);
|
|
||||||
if ( myfs_modify(bh) ) { /* returns true
|
|
||||||
if makes changes */
|
|
||||||
journal_dirty_{meta,}data(xact,bh);
|
|
||||||
} else {
|
|
||||||
journal_forget(bh);
|
|
||||||
}
|
|
||||||
}
|
|
||||||
journal_stop(xct);
|
|
||||||
}
|
|
||||||
journal_destroy(my_jrnl);
|
|
||||||
</programlisting>
|
|
||||||
</sect2>
|
</sect2>
|
||||||
|
|
||||||
</sect1>
|
</sect1>
|
||||||
|
@ -357,13 +313,13 @@ an example.
|
||||||
<title>Data Types</title>
|
<title>Data Types</title>
|
||||||
<para>
|
<para>
|
||||||
The journalling layer uses typedefs to 'hide' the concrete definitions
|
The journalling layer uses typedefs to 'hide' the concrete definitions
|
||||||
of the structures used. As a client of the JBD layer you can
|
of the structures used. As a client of the JBD2 layer you can
|
||||||
just rely on the using the pointer as a magic cookie of some sort.
|
just rely on the using the pointer as a magic cookie of some sort.
|
||||||
|
|
||||||
Obviously the hiding is not enforced as this is 'C'.
|
Obviously the hiding is not enforced as this is 'C'.
|
||||||
</para>
|
</para>
|
||||||
<sect2 id="structures"><title>Structures</title>
|
<sect2 id="structures"><title>Structures</title>
|
||||||
!Iinclude/linux/jbd.h
|
!Iinclude/linux/jbd2.h
|
||||||
</sect2>
|
</sect2>
|
||||||
</sect1>
|
</sect1>
|
||||||
|
|
||||||
|
@ -375,11 +331,11 @@ an example.
|
||||||
manage transactions
|
manage transactions
|
||||||
</para>
|
</para>
|
||||||
<sect2 id="journal_level"><title>Journal Level</title>
|
<sect2 id="journal_level"><title>Journal Level</title>
|
||||||
!Efs/jbd/journal.c
|
!Efs/jbd2/journal.c
|
||||||
!Ifs/jbd/recovery.c
|
!Ifs/jbd2/recovery.c
|
||||||
</sect2>
|
</sect2>
|
||||||
<sect2 id="transaction_level"><title>Transasction Level</title>
|
<sect2 id="transaction_level"><title>Transasction Level</title>
|
||||||
!Efs/jbd/transaction.c
|
!Efs/jbd2/transaction.c
|
||||||
</sect2>
|
</sect2>
|
||||||
</sect1>
|
</sect1>
|
||||||
<sect1 id="see_also">
|
<sect1 id="see_also">
|
||||||
|
|
Loading…
Reference in New Issue