Some engines have transfer size and address alignment restrictions. Add
a per-operation alignment property to struct dma_device that the async
routines and dmatest can use to check alignment capabilities.
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
Channel switching is problematic for some dmaengine drivers as the
architecture precludes separating the ->prep from ->submit. In these
cases the driver can select ASYNC_TX_DISABLE_CHANNEL_SWITCH to modify
the async_tx allocator to only return channels that support all of the
required asynchronous operations.
For example MD_RAID456=y selects support for asynchronous xor, xor
validate, pq, pq validate, and memcpy. When
ASYNC_TX_DISABLE_CHANNEL_SWITCH=y any channel with all these
capabilities is marked DMA_ASYNC_TX allowing async_tx_find_channel() to
quickly locate compatible channels with the guarantee that dependency
chains will remain on one channel. When
ASYNC_TX_DISABLE_CHANNEL_SWITCH=n async_tx_find_channel() may select
channels that lead to operation chains that need to cross channel
boundaries using the async_tx channel switch capability.
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
Some engines optimize operation by reading ahead in the descriptor chain
such that descriptor2 may start execution before descriptor1 completes.
If descriptor2 depends on the result from descriptor1 then a fence is
required (on descriptor2) to disable this optimization. The async_tx
api could implicitly identify dependencies via the 'depend_tx'
parameter, but that would constrain cases where the dependency chain
only specifies a completion order rather than a data dependency. So,
provide an ASYNC_TX_FENCE to explicitly identify data dependencies.
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
This patch adds VMAC (a fast MAC) support into crypto framework.
Signed-off-by: Shane Wang <shane.wang@intel.com>
Signed-off-by: Joseph Cihula <joseph.cihula@intel.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
We have a mechanism where newly registered algorithms of a higher
priority can displace existing instances that use a different
implementation of the same algorithm with a lower priority.
Unfortunately the same mechanism can cause a newly registered
algorithm to displace itself if it depends on an existing version
of the same algorithm.
This patch fixes this by keeping all algorithms that the newly
reigstered algorithm depends on, thus protecting them from being
removed.
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Port drivers/md/raid6test/test.c to use the async raid6 recovery
routines. This is meant as a unit test for raid6 acceleration drivers. In
addition to the 16-drive test case this implements tests for the 4-disk and
5-disk special cases (dma devices can not generically handle less than 2
sources), and adds a test for the D+Q case.
Reviewed-by: Andre Noll <maan@systemlinux.org>
Acked-by: Maciej Sosnowski <maciej.sosnowski@intel.com>
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
async_raid6_2data_recov() recovers two data disk failures
async_raid6_datap_recov() recovers a data disk and the P disk
These routines are a port of the synchronous versions found in
drivers/md/raid6recov.c. The primary difference is breaking out the xor
operations into separate calls to async_xor. Two helper routines are
introduced to perform scalar multiplication where needed.
async_sum_product() multiplies two sources by scalar coefficients and
then sums (xor) the result. async_mult() simply multiplies a single
source by a scalar.
This implemention also includes, in contrast to the original
synchronous-only code, special case handling for the 4-disk and 5-disk
array cases. In these situations the default N-disk algorithm will
present 0-source or 1-source operations to dma devices. To cover for
dma devices where the minimum source count is 2 we implement 4-disk and
5-disk handling in the recovery code.
[ Impact: asynchronous raid6 recovery routines for 2data and datap cases ]
Cc: Yuri Tikhonov <yur@emcraft.com>
Cc: Ilya Yanok <yanok@emcraft.com>
Cc: H. Peter Anvin <hpa@zytor.com>
Cc: David Woodhouse <David.Woodhouse@intel.com>
Reviewed-by: Andre Noll <maan@systemlinux.org>
Acked-by: Maciej Sosnowski <maciej.sosnowski@intel.com>
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
[ Based on an original patch by Yuri Tikhonov ]
This adds support for doing asynchronous GF multiplication by adding
two additional functions to the async_tx API:
async_gen_syndrome() does simultaneous XOR and Galois field
multiplication of sources.
async_syndrome_val() validates the given source buffers against known P
and Q values.
When a request is made to run async_pq against more than the hardware
maximum number of supported sources we need to reuse the previous
generated P and Q values as sources into the next operation. Care must
be taken to remove Q from P' and P from Q'. For example to perform a 5
source pq op with hardware that only supports 4 sources at a time the
following approach is taken:
p, q = PQ(src0, src1, src2, src3, COEF({01}, {02}, {04}, {08}))
p', q' = PQ(p, q, q, src4, COEF({00}, {01}, {00}, {10}))
p' = p + q + q + src4 = p + src4
q' = {00}*p + {01}*q + {00}*q + {10}*src4 = q + {10}*src4
Note: 4 is the minimum acceptable maxpq otherwise we punt to
synchronous-software path.
The DMA_PREP_CONTINUE flag indicates to the driver to reuse p and q as
sources (in the above manner) and fill the remaining slots up to maxpq
with the new sources/coefficients.
Note1: Some devices have native support for P+Q continuation and can skip
this extra work. Devices with this capability can advertise it with
dma_set_maxpq. It is up to each driver how to handle the
DMA_PREP_CONTINUE flag.
Note2: The api supports disabling the generation of P when generating Q,
this is ignored by the synchronous path but is implemented by some dma
devices to save unnecessary writes. In this case the continuation
algorithm is simplified to only reuse Q as a source.
Cc: H. Peter Anvin <hpa@zytor.com>
Cc: David Woodhouse <David.Woodhouse@intel.com>
Signed-off-by: Yuri Tikhonov <yur@emcraft.com>
Signed-off-by: Ilya Yanok <yanok@emcraft.com>
Reviewed-by: Andre Noll <maan@systemlinux.org>
Acked-by: Maciej Sosnowski <maciej.sosnowski@intel.com>
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
We currently walk the parent chain when waiting for a given tx to
complete however this walk may race with the driver cleanup routine.
The routines in async_raid6_recov.c may fall back to the synchronous
path at any point so we need to be prepared to call async_tx_quiesce()
(which calls dma_wait_for_async_tx). To remove the ->parent walk we
guarantee that every time a dependency is attached ->issue_pending() is
invoked, then we can simply poll the initial descriptor until
completion.
This also allows for a lighter weight 'issue pending' implementation as
there is no longer a requirement to iterate through all the channels'
->issue_pending() routines as long as operations have been submitted in
an ordered chain. async_tx_issue_pending() is added for this case.
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
If module_init and module_exit are nops then neither need to be defined.
[ Impact: pure cleanup ]
Reviewed-by: Andre Noll <maan@systemlinux.org>
Acked-by: Maciej Sosnowski <maciej.sosnowski@intel.com>
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
Replace the flat zero_sum_result with a collection of flags to contain
the P (xor) zero-sum result, and the soon to be utilized Q (raid6 reed
solomon syndrome) zero-sum result. Use the SUM_CHECK_ namespace instead
of DMA_ since these flags will be used on non-dma-zero-sum enabled
platforms.
Reviewed-by: Andre Noll <maan@systemlinux.org>
Acked-by: Maciej Sosnowski <maciej.sosnowski@intel.com>
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
As struct skcipher_givcrypt_request includes struct crypto_request
at a non-zero offset, testing for NULL after converting the pointer
returned by crypto_dequeue_request does not work. This can result
in IPsec crashes when the queue is depleted.
This patch fixes it by doing the pointer conversion only when the
return value is non-NULL. In particular, we create a new function
__crypto_dequeue_request that does the pointer conversion.
Reported-by: Brad Bosch <bradbosch@comcast.net>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Return the value we got from crypto_register_alg() instead of
returning 0 in any case.
Signed-off-by: Steffen Klassert <steffen.klassert@secunet.com>
Acked-by: Neil Horman <nhorman@tuxdriver.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
The alignment calculation of xcbc_tfm_ctx uses alg->cra_alignmask
and not alg->cra_alignmask + 1 as it should. This led to frequent
crashes during the selftest of xcbc(aes-asm) on x86_64
machines. This patch fixes this. Also we use the alignmask
of xcbc and not the alignmask of the underlying algorithm
for the alignmnent calculation in xcbc_create now.
Signed-off-by: Steffen Klassert <steffen.klassert@secunet.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
What about something like this? It defaults the CPRNG to m and makes FIPS
dependent on the CPRNG. That way you get a module build by default, but you can
change it to y manually during config and still satisfy the dependency, and if
you select N it disables FIPS as well. I rather like that better than making
FIPS a tristate. I just tested it out here and it seems to work well. Let me
know what you think
Signed-off-by: Neil Horman <nhorman@tuxdriver.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Recently we switched to using eseqiv on SMP machines in preference
over chainiv. However, eseqiv does not support stream ciphers so
they should still default to chainiv.
This patch applies the same check as done by eseqiv to weed out
the stream ciphers. In particular, all algorithms where the IV
size is not equal to the block size will now default to chainiv.
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Raw counter mode only works with chainiv, which is no longer
the default IV generator on SMP machines. This broke raw counter
mode as it can no longer instantiate as a givcipher.
This patch fixes it by always picking chainiv on raw counter
mode. This is based on the diagnosis and a patch by Huang
Ying.
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
This reverts commit 215ccd6f55.
It causes CPRNG and everything selected by it to be built-in
whenever FIPS is enabled. The problem is that it is selecting
a tristate from a bool, which is usually not what is intended.
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Correct a typo in crypto/rng.c
Signed-off-by: Christian Kujau <lists@nerdbynature.de>
Acked-by: Neil Horman <nhorman@tuxdriver.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
cryptd_alloc_ahash() will allocate a cryptd-ed ahash for specified
algorithm name. The new allocated one is guaranteed to be cryptd-ed
ahash, so the shash underlying can be gotten via cryptd_ahash_child().
Signed-off-by: Huang Ying <ying.huang@intel.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Remove the dedicated GHASH implementation in GCM, and uses the GHASH
digest algorithm instead. This will make GCM uses hardware accelerated
GHASH implementation automatically if available.
ahash instead of shash interface is used, because some hardware
accelerated GHASH implementation needs asynchronous interface.
Signed-off-by: Huang Ying <ying.huang@intel.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
GHASH is implemented as a shash algorithm. The actual implementation
is copied from gcm.c. This makes it possible to add
architecture/hardware accelerated GHASH implementation.
Signed-off-by: Huang Ying <ying.huang@intel.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
This patch converts authenc to the new ahash interface.
Signed-off-by: Steffen Klassert <steffen.klassert@secunet.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
* 'next' of git://git.kernel.org/pub/scm/linux/kernel/git/djbw/async_tx:
dmaengine: at_hdmac: add DMA slave transfers
dmaengine: at_hdmac: new driver for the Atmel AHB DMA Controller
dmaengine: dmatest: correct thread_count while using multiple thread per channel
dmaengine: dmatest: add a maximum number of test iterations
drivers/dma: Remove unnecessary semicolons
drivers/dma/fsldma.c: Remove unnecessary semicolons
dmaengine: move HIGHMEM64G restriction to ASYNC_TX_DMA
fsldma: do not clear bandwidth control bits on the 83xx controller
fsldma: enable external start for the 83xx controller
fsldma: use PCI Read Multiple command
This patch uses crypto_shash_export/crypto_shash_import to prehash
ipad/opad to speed up hmac. This is partly based on a similar patch
by Steffen Klassert.
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
It's undefined behaviour in C to write outside the bounds of an array.
The key expansion routine takes a shortcut of creating 8 words at a
time, but this creates 4 additional words which don't fit in the array.
As everyone is hopefully now aware, GCC is at liberty to make any
assumptions and optimisations it likes in situations where it can
detect that UB has occured, up to and including nasal demons, and
as the indices being accessed in the array are trivially calculable,
it's rash to invite gcc to do take any liberties at all.
Signed-off-by: Phil Carmody <ext-phil.2.carmody@nokia.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
crypto_init_shash_ops_async() tests for setkey and not for import
before exporting the algorithms import function to ahash.
This patch fixes this.
Signed-off-by: Steffen Klassert <steffen.klassert@secunet.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
ahash_op_unaligned() and ahash_def_finup() allocate memory atomically,
regardless whether the request can sleep or not. This patch changes
this to use GFP_KERNEL if the request can sleep.
Signed-off-by: Steffen Klassert <steffen.klassert@secunet.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
This patch provides a default export/import function for all
shash algorithms. It simply copies the descriptor context as
is done by sha1_generic.
This in essence means that all existing shash algorithms now
support export/import. This is something that will be depended
upon in implementations such as hmac. Therefore all new shash
and ahash implementations must support export/import.
For those that cannot obtain a partial result, padlock-sha's
fallback model should be used so that a partial result is always
available.
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
This patch replaces the 32-bit counters in sha512_generic with
64-bit counters. It also switches the bit count to the simpler
byte count.
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
This patch renames struct sha512_ctx and exports it as struct
sha512_state so that other sha512 implementations can use it
as the reference structure for exporting their state.
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Although xcbc was converted to shash, it didn't obey the new
requirement that all hash state must be stored in the descriptor
rather than the transform.
This patch fixes this issue and also optimises away the rekeying
by precomputing K2 and K3 within setkey.
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
This patch adds the finup/export/import functions to the cryptd
ahash implementation. We simply invoke the underlying shash
operations.
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
When shash_ahash_finup encounters a null request, we end up not
calling the underlying final function. This patch fixes that.
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
When the alignment check was made unconditional for ahash we
may end up crashing on shash algorithms because we're always
calling alg->setkey instead of tfm->setkey.
This patch fixes it.
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
If cryptd_alloc_instance() fails, the return value is uninitialized.
This patch fixes this by setting the return value.
Signed-off-by: Steffen Klassert <steffen.klassert@secunet.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
This patch exports the finup operation where available and adds
a default finup operation for ahash. The operations final, finup
and digest also will now deal with unaligned result pointers by
copying it. Finally export/import operations are will now be
exported too.
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
We currently use GFP_ATOMIC in the unaligned setkey function
to allocate the temporary aligned buffer. Since setkey must
be called in a sleepable context, we can use GFP_KERNEL instead.
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
When we encounter an unaligned pointer we are supposed to copy
it to a temporary aligned location. However the temporary buffer
isn't aligned properly. This patch fixes that.
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Some unaligned buffers on the stack weren't zapped properly which
may cause secret data to be leaked. This patch fixes them by doing
a zero memset.
It is also possible for us to place random kernel stack contents
in the digest buffer if a digest operation fails. This is fixed
by only copying if the operation succeeded.
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Now that all ahash implementations have been converted to the new
ahash type, we can remove old_ahash_alg and its associated support.
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
This patch changes cryptd to use the new style ahash type. In
particular, the instance is enlarged to encapsulate the new
ahash_alg structure.
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
This patch changes cryptd to use the template->create function
instead of alloc in anticipation for the switch to new style
ahash algorithms.
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
This patch adds a mask parameter to complement the existing type
parameter. This is useful when instantiating algorithms that
require a mask other than the default, e.g., ahash algorithms.
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
This patch converts crypto_ahash to the new style. The old ahash
algorithm type is retained until the existing ahash implementations
are also converted. All ahash users will automatically get the
new crypto_ahash type.
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
As the extsize and init_tfm functions belong to the frontend the
frontend argument is superfluous.
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
This patch exports the async functions so that they can be reused
by cryptd when it switches over to using shash.
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
This patch removes the implementation of hash and digest now that
no algorithms use them anymore. The interface though will remain
until the users are converted across.
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Now that there are no more legacy hash implementations we can
remove the reference to crypto_hash.
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
This patch changes descsize to a run-time attribute so that
implementations can change it in their init functions.
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
This patch moves the run-time null setkey check to shash_prepare_alg
just like we did for finup/digest.
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
This patch adds export/import support to sha256_generic. The exported
type is defined by struct sha256_state, which is basically the entire
descriptor state of sha256_generic.
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
This patch replaces the two 32-bit counter code in sha256_generic
with the simpler 64-bit counter code from sha1.
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
This patch adds export/import support to sha1_generic. The exported
type is defined by struct sha1_state, which is basically the entire
descriptor state of sha1_generic.
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
This patch moves the run-time null finup/digest checks to the
shash_prepare_alg function which is run at registration time.
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
This patch replaces the full descriptor export with an export of
the partial hash state. This allows the use of a consistent export
format across all implementations of a given algorithm.
This is useful because a number of cases require the use of the
partial hash state, e.g., PadLock can use the SHA1 hash state
to get around the fact that it can only hash contiguous data
chunks.
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
This patch allows crypto_drop_spawn to be called on spawns that
have not been initialised or have failed initialisation. This
fixes potential crashes during initialisation without adding
special case code.
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
This patch adds shash_register_instance so that shash instances
can be registered without bypassing the shash checks applied to
normal algorithms.
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
This patch adds the helper shash_attr_alg2 which locates a shash
algorithm based on the information in the given attribute.
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
This patch adds the helper crypto_attr_alg2 which is similar to
crypto_attr_alg but takes an extra frontend argument. This is
intended to be used by new style algorithm types such as shash.
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
This patch adds the functions needed to create and use shash
spawns, i.e., to use shash algorithms in a template.
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
This patch modifies the spawn infrastructure to support new style
algorithms like shash. In particular, this means storing the
frontend type in the spawn and using crypto_create_tfm to allocate
the tfm.
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
This patch adds shash_instance and the associated alloc/free
functions. This is meant to be an instance that with a shash
algorithm under it. Note that the instance itself doesn't have
to be shash.
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
This patch adds a new argument to crypto_alloc_instance which
sets aside some space before the instance for use by algorithms
such as shash that place type-specific data before crypto_alg.
For compatibility the function has been renamed so that existing
users aren't affected.
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
This patch introduces the template->create function intended
to replace the existing alloc function. The intention is for
create to handle the registration directly, whereas currently
the caller of alloc has to handle the registration.
This allows type-specific code to be run prior to registration.
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
As reported by Eric Sesterhenn the re-allocation of the cipher in reset leads
to:
|BUG: sleeping function called from invalid context at kernel/rwsem.c:21
|in_atomic(): 1, irqs_disabled(): 0, pid: 4926, name: modprobe
|INFO: lockdep is turned off.
|Pid: 4926, comm: modprobe Tainted: G M 2.6.31-rc1-22297-g5298976 #24
|Call Trace:
| [<c011dd93>] __might_sleep+0xf9/0x101
| [<c0777aa0>] down_read+0x16/0x68
| [<c048bf04>] crypto_alg_lookup+0x16/0x34
| [<c048bf52>] crypto_larval_lookup+0x30/0xf9
| [<c048c038>] crypto_alg_mod_lookup+0x1d/0x62
| [<c048c13e>] crypto_alloc_base+0x1e/0x64
| [<c04bf991>] reset_prng_context+0xab/0x13f
| [<c04e5cfc>] ? __spin_lock_init+0x27/0x51
| [<c04bfce1>] cprng_init+0x2a/0x42
| [<c048bb4c>] __crypto_alloc_tfm+0xfa/0x128
| [<c048c153>] crypto_alloc_base+0x33/0x64
| [<c04933c9>] alg_test_cprng+0x30/0x1f4
| [<c0493329>] alg_test+0x12f/0x19f
| [<c0177f1f>] ? __alloc_pages_nodemask+0x14d/0x481
| [<d09219e2>] do_test+0xf9d/0x163f [tcrypt]
| [<d0920de6>] do_test+0x3a1/0x163f [tcrypt]
| [<d0926035>] tcrypt_mod_init+0x35/0x7c [tcrypt]
| [<c010113c>] _stext+0x54/0x12c
| [<d0926000>] ? tcrypt_mod_init+0x0/0x7c [tcrypt]
| [<c01398a3>] ? up_read+0x16/0x2b
| [<c0139fc4>] ? __blocking_notifier_call_chain+0x40/0x4c
| [<c014ee8d>] sys_init_module+0xa9/0x1bf
| [<c010292b>] sysenter_do_call+0x12/0x32
because a spin lock is held and crypto_alloc_base() may sleep.
There is no reason to re-allocate the cipher, the state is resetted in
->setkey(). This patches makes the cipher allocation a one time thing and
moves it to init.
Reported-by: Eric Sesterhenn <eric.sesterhenn@lsexperts.de>
Signed-off-by: Sebastian Andrzej Siewior <sebastian@breakpoint.cc>
Acked-by: Neil Horman <nhorman@tuxdriver.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
The current code uses a mix of sping_lock() & spin_lock_irqsave(). This can
lead to deadlock with the correct timming & cprng_get_random() + cprng_reset()
sequence.
I've converted them to bottom half locks since all three user grab just a BH
lock so this runs probably in softirq :)
Signed-off-by: Sebastian Andrzej Siewior <sebastian@breakpoint.cc>
Acked-by: Neil Horman <nhorman@tuxdriver.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
This patch adds the support for testing specific implementations.
This should only be used in very specific situations. Right now
this means specific implementations of random number generators.
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
On HIGHMEM64G systems dma_addr_t is known to be larger than (void *)
which precludes async_xor from performing dma address conversions by
reusing the input parameter address list. However, other parts of the
dmaengine infrastructure do not suffer this constraint, so the
HIGHMEM64G restriction can be down-levelled.
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
As it stands we use chainiv for sync algorithms and eseqiv for
async algorithms. However, when there is more than one CPU
chainiv forces all processing to be serialised which is usually
not what you want. Also, the added overhead of eseqiv isn't that
great.
Therefore this patch changes the default sync geniv on SMP machines
to eseqiv. For the odd situation where the overhead is unacceptable
then chainiv is still available as an option.
Note that on UP machines chainiv is still preferred over eseqiv
for sync algorithms.
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
When a sync givcipher algorithm is requested, if an async version
of the same algorithm already exists, then we will loop forever
without ever constructing the sync version based on a blkcipher.
This is because we did not include the requested type/mask when
getting a larval for the geniv algorithm that is to be constructed.
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Until hash test vectors grow longer than 256 bytes, the only
purpose of the check is to generate a gcc warning.
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
The ANSI CPRNG has no dependence on FIPS support. FIPS support however,
requires the use of the CPRNG. Adjust that depedency relationship in Kconfig.
Signed-off-by: Neil Horman <nhorman@tuxdriver.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
This adds the 'alg' module parameter to be able to test an
algorithm by name. If the algorithm type is not ad-hoc
clear for a algorithm (e.g. pcrypt, cryptd) it is possilbe
to set the algorithm type with the 'type' module parameter.
Signed-off-by: Steffen Klassert <steffen.klassert@secunet.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
FIPS-140 requires that all random number generators implement continuous self
tests in which each extracted block of data is compared against the last block
for repetition. The ansi_cprng implements such a test, but it would be nice if
the hw rng's did the same thing. Obviously its not something thats always
needed, but it seems like it would be a nice feature to have on occasion. I've
written the below patch which allows individual entropy stores to be flagged as
desiring a continuous test to be run on them as is extracted. By default this
option is off, but is enabled in the event that fips mode is selected during
bootup.
Signed-off-by: Neil Horman <nhorman@tuxdriver.com>
Acked-by: Matt Mackall <mpm@selenic.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
The xor tests are run on uninitialized data, because it is doesn't
really matter what the underlying data is. Annotate this false-
positive warning.
Acked-by: Pekka Enberg <penberg@cs.helsinki.fi>
Signed-off-by: Vegard Nossum <vegard.nossum@gmail.com>
async_xor() needs space to perform dma and page address conversions. In
most cases the code can simply reuse the struct page * array because the
size of the native pointer matches the size of a dma/page address. In
order to support archs where sizeof(dma_addr_t) is larger than
sizeof(struct page *), or to preserve the input parameters, we utilize a
memory region passed in by the caller.
Since the code is now prepared to handle the case where it cannot
perform address conversions on the stack, we no longer need the
!HIGHMEM64G dependency in drivers/dma/Kconfig.
[ Impact: don't clobber input buffers for address conversions ]
Reviewed-by: Andre Noll <maan@systemlinux.org>
Acked-by: Maciej Sosnowski <maciej.sosnowski@intel.com>
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
Prepare the api for the arrival of a new parameter, 'scribble'. This
will allow callers to identify scratchpad memory for dma address or page
address conversions. As this adds yet another parameter, take this
opportunity to convert the common submission parameters (flags,
dependency, callback, and callback argument) into an object that is
passed by reference.
Also, take this opportunity to fix up the kerneldoc and add notes about
the relevant ASYNC_TX_* flags for each routine.
[ Impact: moves api pass-by-value parameters to a pass-by-reference struct ]
Signed-off-by: Andre Noll <maan@systemlinux.org>
Acked-by: Maciej Sosnowski <maciej.sosnowski@intel.com>
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
In support of inter-channel chaining async_tx utilizes an ack flag to
gate whether a dependent operation can be chained to another. While the
flag is not set the chain can be considered open for appending. Setting
the ack flag closes the chain and flags the descriptor for garbage
collection. The ASYNC_TX_DEP_ACK flag essentially means "close the
chain after adding this dependency". Since each operation can only have
one child the api now implicitly sets the ack flag at dependency
submission time. This removes an unnecessary management burden from
clients of the api.
[ Impact: clean up and enforce one dependency per operation ]
Reviewed-by: Andre Noll <maan@systemlinux.org>
Acked-by: Maciej Sosnowski <maciej.sosnowski@intel.com>
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
Besdies, for the old code, gcc-4.3.3 produced this warning:
"format not a string literal and no format arguments"
Signed-off-by: Alex Riesen <raa.lkml@gmail.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
As it stands we will each test hash vector both linearly and as
a scatter list if applicable. This means that we cannot have
vectors longer than a page, even with scatter lists.
This patch fixes this by skipping test vectors with np != 0 when
testing linearly.
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
As we cannot guarantee the availability of contiguous pages at
run-time, all test vectors must either fit within a page, or use
scatter lists. In some cases vectors were not checked as to
whether they fit inside a page. This patch adds all the missing
checks.
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
At present, the tcrypt module always exits with an -EAGAIN upon
successfully completing all the tests its been asked to run. In fips
mode, integrity checking is done by running all self-tests from the
initrd, and its much simpler to check the ret from modprobe for
success than to scrape dmesg and/or /proc/crypto. Simply stay
loaded, giving modprobe a retval of 0, if self-tests all pass and
we're in fips mode.
A side-effect of tracking success/failure for fips mode is that in
non-fips mode, self-test failures will return the actual failure
return codes, rather than always returning -EAGAIN, which seems more
correct anyway.
The tcrypt_test() portion of the patch is dependent on my earlier
pair of patches that skip non-fips algs in fips mode, at least to
achieve the fully intended behavior.
Nb: testing this patch against the cryptodev tree revealed a test
failure for sha384, which I have yet to look into...
Signed-off-by: Jarod Wilson <jarod@redhat.com>
Acked-by: Neil Horman <nhorman@tuxdriver.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
If crypto_{,de}compress_{update,final}() succeed, return the actual number of
bytes produced instead of zero, so their users don't have to calculate that
theirselves.
Signed-off-by: Geert Uytterhoeven <Geert.Uytterhoeven@sonycom.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Because all fips-allowed algorithms must be self-tested before they
can be used, they will all have entries in testmgr.c's alg_test_descs[].
Skip self-tests for any algs not flagged as fips_approved and return
-EINVAL when in fips mode.
Signed-off-by: Jarod Wilson <jarod@redhat.com>
Acked-by: Neil Horman <nhorman@tuxdriver.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Set the fips_allowed flag in testmgr.c's alg_test_descs[] for algs
that are allowed to be used when in fips mode.
One caveat: des isn't actually allowed anymore, but des (and thus also
ecb(des)) has to be permitted, because disallowing them results in
des3_ede being unable to properly register (see des module init func).
Also, crc32 isn't technically on the fips approved list, but I think
it gets used in various places that necessitate it being allowed.
This list is based on
http://csrc.nist.gov/groups/STM/cavp/index.html
Important note: allowed/approved here does NOT mean "validated", just
that its an alg that *could* be validated.
Signed-off-by: Jarod Wilson <jarod@redhat.com>
Acked-by: Neil Horman <nhorman@tuxdriver.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Now with multi-block test vectors, all from SP800-38A, Appendix F.5.
Also added ctr(aes) to case 10 in tcrypt.
Signed-off-by: Jarod Wilson <jarod@redhat.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
We currently allocate temporary memory that is used for testing
statically. This renders the testing engine non-reentrant. As
algorithms may nest, i.e., one may construct another in order to
carry out a part of its operation, this is unacceptable. For
example, it has been reported that an AEAD implementation allocates
a cipher in its setkey function, which causes it to fail during
testing as the temporary memory is overwritten.
This patch replaces the static memory with dynamically allocated
buffers. We need a maximum of 16 pages so this slightly increases
the chances of an algorithm failing due to memory shortage.
However, as testing usually occurs at registration, this shouldn't
be a big problem.
Reported-by: Shasi Pulijala <spulijala@amcc.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
According to our FIPS CAVS testing lab guru, when we're in fips mode,
we must print out notices of successful self-test completion for
every alg to be compliant.
New and improved v2, without strncmp crap. Doesn't need to touch a flag
though, due to not moving the notest label around anymore.
Applies atop '[PATCH v2] crypto: catch base cipher self-test failures
in fips mode'.
Personally, I wouldn't mind seeing this info printed out regardless of
whether or not we're in fips mode, I think its useful info, but will
stick with only in fips mode for now.
Signed-off-by: Jarod Wilson <jarod@redhat.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Add ANSI X9.31 Continuous Pseudo-Random Number Generator (AES mode),
aka 'ansi_cprng' test vectors, taken from Appendix B.2.9 and B.2.10
of the NIST RNGVS document, found here:
http://csrc.nist.gov/groups/STM/cavp/documents/rng/RNGVS.pdf
Successfully tested against both the cryptodev-2.6 tree and a Red
Hat Enterprise Linux 5.4 kernel, via 'modprobe tcrypt mode=150'.
The selection of 150 was semi-arbitrary, didn't seem like it should
go any place in particular, so I started a new range for rng tests.
Signed-off-by: Jarod Wilson <jarod@redhat.com>
Acked-by: Neil Horman <nhorman@tuxdriver.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Add some necessary infrastructure to make it possible to run
self-tests for ansi_cprng. The bits are likely very specific
to the ANSI X9.31 CPRNG in AES mode, and thus perhaps should
be named more specifically if/when we grow additional CPRNG
support...
Successfully tested against the cryptodev-2.6 tree and a
Red Hat Enterprise Linux 5.x kernel with the follow-on
patch that adds the actual test vectors.
Signed-off-by: Jarod Wilson <jarod@redhat.com>
Acked-by: Neil Horman <nhorman@tuxdriver.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Add an array of encryption and decryption + verification self-tests
for rfc4309(ccm(aes)).
Test vectors all come from sample FIPS CAVS files provided to
Red Hat by a testing lab. Unfortunately, all the published sample
vectors in RFC 3610 and NIST Special Publication 800-38C contain nonce
lengths that the kernel's rfc4309 implementation doesn't support, so
while using some public domain vectors would have been preferred, its
not possible at this time.
Signed-off-by: Jarod Wilson <jarod@redhat.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Add infrastructure to tcrypt/testmgr to support handling ccm decryption
test vectors that are expected to fail verification.
Signed-off-by: Jarod Wilson <jarod@redhat.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
make C=1:
| crypto/pcompress.c:77:5: warning: symbol 'crypto_register_pcomp' was not declared. Should it be static?
| crypto/pcompress.c:89:5: warning: symbol 'crypto_unregister_pcomp' was not declared. Should it be static?
Signed-off-by: Geert Uytterhoeven <Geert.Uytterhoeven@sonycom.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Because kernel_fpu_begin() and kernel_fpu_end() operations are too
slow, the performance gain of general mode implementation + aes-aesni
is almost all compensated.
The AES-NI support for more modes are implemented as follow:
- Add a new AES algorithm implementation named __aes-aesni without
kernel_fpu_begin/end()
- Use fpu(<mode>(AES)) to provide kenrel_fpu_begin/end() invoking
- Add <mode>(AES) ablkcipher, which uses cryptd(fpu(<mode>(AES))) to
defer cryption to cryptd context in soft_irq context.
Now the ctr, lrw, pcbc and xts support are added.
Performance testing based on dm-crypt shows that cryption time can be
reduced to 50% of general mode implementation + aes-aesni implementation.
Signed-off-by: Huang Ying <ying.huang@intel.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Blkcipher touching FPU need to be enclosed by kernel_fpu_begin() and
kernel_fpu_end(). If they are invoked in cipher algorithm
implementation, they will be invoked for each block, so that
performance will be hurt, because they are "slow" operations. This
patch implements "fpu" template, which makes these operations to be
invoked for each request.
Signed-off-by: Huang Ying <ying.huang@intel.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Use crypto_alloc_base() instead of crypto_alloc_ablkcipher() to
allocate underlying tfm in cryptd_alloc_ablkcipher. Because
crypto_alloc_ablkcipher() prefer GENIV encapsulated crypto instead of
raw one, while cryptd_alloc_ablkcipher needed the raw one.
Signed-off-by: Huang Ying <ying.huang@intel.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Use kzfree() instead of memset() + kfree().
Signed-off-by: Johannes Weiner <hannes@cmpxchg.org>
Reviewed-by: Pekka Enberg <penberg@cs.helsinki.fi>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Applying kernel janitors todos (printk calls need KERN_*
constants on linebeginnings, reduce stack footprint where
possible) to tcrypts test_hash_speed (where stacks
memory footprint was very high (on i386 1184 bytes to
160 now).
Signed-off-by: Frank Seidel <frank@f-seidel.de>
Acked-by: Neil Horman <nhorman@tuxdriver.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
A quirk that we've always supported is having an sg entry that's
bigger than a page, or more generally an sg entry that crosses
page boundaries. Even though it would be better to explicitly have
to sg entries for this, we need to support it for the existing users,
in particular, IPsec.
The new ahash sg walking code did try to handle this, but there was
a bug where we didn't increment the page so kept on walking on the
first page over an dover again.
This patch fixes it.
Tested-by: Martin Willi <martin@strongswan.org>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
* git://git.kernel.org/pub/scm/linux/kernel/git/herbert/crypto-2.6:
crypto: padlock - Revert aes-all alias to aes
crypto: api - Fix algorithm module auto-loading
crypto: eseqiv - Fix IV generation for sync algorithms
crypto: ixp4xx - check firmware for crypto support
The commit a760a6656e (crypto:
api - Fix module load deadlock with fallback algorithms) broke
the auto-loading of algorithms that require fallbacks. The
problem is that the fallback mask check is missing an and which
cauess bits that should be considered to interfere with the
result.
Reported-by: Chuck Ebbert <cebbert@redhat.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
If crypto_ablkcipher_encrypt() returns synchronous,
eseqiv_complete2() is called even if req->giv is already the
pointer to the generated IV. The generated IV is overwritten
with some random data in this case. This patch fixes this by
calling eseqiv_complete2() just if the generated IV has to be
copied to req->giv.
Signed-off-by: Steffen Klassert <steffen.klassert@secunet.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
'zero_sum' does not properly describe the operation of generating parity
and checking that it validates against an existing buffer. Change the
name of the operation to 'val' (for 'validate'). This is in
anticipation of the p+q case where it is a requirement to identify the
target parity buffers separately from the source buffers, because the
target parity buffers will not have corresponding pq coefficients.
Reviewed-by: Andre Noll <maan@systemlinux.org>
Acked-by: Maciej Sosnowski <maciej.sosnowski@intel.com>
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
* 'next' of git://git.kernel.org/pub/scm/linux/kernel/git/djbw/async_tx:
dma: Add SoF and EoF debugging to ipu_idmac.c, minor cleanup
dw_dmac: add cyclic API to DW DMA driver
dmaengine: Add privatecnt to revert DMA_PRIVATE property
dmatest: add dma interrupts and callbacks
dmatest: add xor test
dmaengine: allow dma support for async_tx to be toggled
async_tx: provide __async_inline for HAS_DMA=n archs
dmaengine: kill some unused headers
dmaengine: initialize tx_list in dma_async_tx_descriptor_init
dma: i.MX31 IPU DMA robustness improvements
dma: improve section assignment in i.MX31 IPU DMA driver
dma: ipu_idmac driver cosmetic clean-up
dmaengine: fail device registration if channel registration fails
* git://git.kernel.org/pub/scm/linux/kernel/git/herbert/crypto-2.6:
crypto: ixp4xx - Fix handling of chained sg buffers
crypto: shash - Fix unaligned calculation with short length
hwrng: timeriomem - Use phys address rather than virt
* 'for-linus' of git://neil.brown.name/md: (53 commits)
md/raid5 revise rules for when to update metadata during reshape
md/raid5: minor code cleanups in make_request.
md: remove CONFIG_MD_RAID_RESHAPE config option.
md/raid5: be more careful about write ordering when reshaping.
md: don't display meaningless values in sysfs files resync_start and sync_speed
md/raid5: allow layout and chunksize to be changed on active array.
md/raid5: reshape using largest of old and new chunk size
md/raid5: prepare for allowing reshape to change layout
md/raid5: prepare for allowing reshape to change chunksize.
md/raid5: clearly differentiate 'before' and 'after' stripes during reshape.
Documentation/md.txt update
md: allow number of drives in raid5 to be reduced
md/raid5: change reshape-progress measurement to cope with reshaping backwards.
md: add explicit method to signal the end of a reshape.
md/raid5: enhance raid5_size to work correctly with negative delta_disks
md/raid5: drop qd_idx from r6_state
md/raid6: move raid6 data processing to raid6_pq.ko
md: raid5 run(): Fix max_degraded for raid level 4.
md: 'array_size' sysfs attribute
md: centralize ->array_sectors modifications
...
This makes the includes more explicit, and is preparation for moving
md_k.h to drivers/md/md.h
Remove include/raid/md.h as its only remaining use was to #include
other files.
Signed-off-by: NeilBrown <neilb@suse.de>
When the total length is shorter than the calculated number of unaligned bytes, the call to shash->update breaks. For example, calling crc32c on unaligned buffer with length of 1 can result in a system crash.
Signed-off-by: Yehuda Sadeh <yehuda@hq.newdream.net>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
* git://git.kernel.org/pub/scm/linux/kernel/git/herbert/crypto-2.6: (29 commits)
crypto: sha512-s390 - Add missing block size
hwrng: timeriomem - Breaks an allyesconfig build on s390:
nlattr: Fix build error with NET off
crypto: testmgr - add zlib test
crypto: zlib - New zlib crypto module, using pcomp
crypto: testmgr - Add support for the pcomp interface
crypto: compress - Add pcomp interface
netlink: Move netlink attribute parsing support to lib
crypto: Fix dead links
hwrng: timeriomem - New driver
crypto: chainiv - Use kcrypto_wq instead of keventd_wq
crypto: cryptd - Per-CPU thread implementation based on kcrypto_wq
crypto: api - Use dedicated workqueue for crypto subsystem
crypto: testmgr - Test skciphers with no IVs
crypto: aead - Avoid infinite loop when nivaead fails selftest
crypto: skcipher - Avoid infinite loop when cipher fails selftest
crypto: api - Fix crypto_alloc_tfm/create_create_tfm return convention
crypto: api - crypto_alg_mod_lookup either tested or untested
crypto: amcc - Add crypt4xx driver
crypto: ansi_cprng - Add maintainer
...
To allow an async_tx routine to be compiled away on HAS_DMA=n arch it
needs to be declared __always_inline otherwise the compiler may emit
code and cause a link error.
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
The current "comp" crypto interface supports one-shot (de)compression only,
i.e. the whole data buffer to be (de)compressed must be passed at once, and
the whole (de)compressed data buffer will be received at once.
In several use-cases (e.g. compressed file systems that store files in big
compressed blocks), this workflow is not suitable.
Furthermore, the "comp" type doesn't provide for the configuration of
(de)compression parameters, and always allocates workspace memory for both
compression and decompression, which may waste memory.
To solve this, add a "pcomp" partial (de)compression interface that provides
the following operations:
- crypto_compress_{init,update,final}() for compression,
- crypto_decompress_{init,update,final}() for decompression,
- crypto_{,de}compress_setup(), to configure (de)compression parameters
(incl. allocating workspace memory).
The (de)compression methods take a struct comp_request, which was mimicked
after the z_stream object in zlib, and contains buffer pointer and length
pairs for input and output.
The setup methods take an opaque parameter pointer and length pair. Parameters
are supposed to be encoded using netlink attributes, whose meanings depend on
the actual (name of the) (de)compression algorithm.
Signed-off-by: Geert Uytterhoeven <Geert.Uytterhoeven@sonycom.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
With the mandatory algorithm testing at registration, we have
now created a deadlock with algorithms requiring fallbacks.
This can happen if the module containing the algorithm requiring
fallback is loaded first, without the fallback module being loaded
first. The system will then try to test the new algorithm, find
that it needs to load a fallback, and then try to load that.
As both algorithms share the same module alias, it can attempt
to load the original algorithm again and block indefinitely.
As algorithms requiring fallbacks are a special case, we can fix
this by giving them a different module alias than the rest. Then
it's just a matter of using the right aliases according to what
algorithms we're trying to find.
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
crypto_ahash_show changed to use cra_ahash for digestsize reference.
Signed-off-by: Lee Nipper <lee.nipper@freescale.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
keventd_wq has potential starvation problem, so use dedicated
kcrypto_wq instead.
Signed-off-by: Huang Ying <ying.huang@intel.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Use dedicated workqueue for crypto subsystem
A dedicated workqueue named kcrypto_wq is created to be used by crypto
subsystem. The system shared keventd_wq is not suitable for
encryption/decryption, because of potential starvation problem.
Signed-off-by: Huang Ying <ying.huang@intel.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
As it is an skcipher with no IV escapes testing altogether because
we only test givcipher objects. This patch fixes the bypass logic
to test these algorithms.
Conversely, we're currently testing nivaead algorithms with IVs,
which would have deadlocked had it not been for the fact that no
nivaead algorithms have any test vectors. This patch also fixes
that case.
Both fixes are ugly as hell, but this ugliness should hopefully
disappear once we move them into the per-type code (i.e., the
AEAD test would live in aead.c and the skcipher stuff in ablkcipher.c).
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
When an aead constructed through crypto_nivaead_default fails
its selftest, we'll loop forever trying to construct new aead
objects but failing because it already exists.
The crux of the issue is that once an aead fails the selftest,
we'll ignore it on the next run through crypto_aead_lookup and
attempt to construct a new aead.
We should instead return an error to the caller if we find an
an that has failed the test.
This bug hasn't manifested itself yet because we don't have any
test vectors for the existing nivaead algorithms. They're tested
through the underlying algorithms only.
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
When an skcipher constructed through crypto_givcipher_default fails
its selftest, we'll loop forever trying to construct new skcipher
objects but failing because it already exists.
The crux of the issue is that once a givcipher fails the selftest,
we'll ignore it on the next run through crypto_skcipher_lookup and
attempt to construct a new givcipher.
We should instead return an error to the caller if we find a
givcipher that has failed the test.
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
This is based on a report and patch by Geert Uytterhoeven.
The functions crypto_alloc_tfm and create_create_tfm return a
pointer that needs to be adjusted by the caller when successful
and otherwise an error value. This means that the caller has
to check for the error and only perform the adjustment if the
pointer returned is valid.
Since all callers want to make the adjustment and we know how
to adjust it ourselves, it's much easier to just return adjusted
pointer directly.
The only caveat is that we have to return a void * instead of
struct crypto_tfm *. However, this isn't that bad because both
of these functions are for internal use only (by types code like
shash.c, not even algorithms code).
This patch also moves crypto_alloc_tfm into crypto/internal.h
(crypto_create_tfm is already there) to reflect this.
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
As it stands crypto_alg_mod_lookup will search either tested or
untested algorithms, but never both at the same time. However,
we need exactly that when constructing givcipher and aead so
this patch adds support for that by setting the tested bit in
type but clearing it in mask. This combination is currently
unused.
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
FIPS 140-2 specifies that all access to various cryptographic modules be
prevented in the event that any of the provided self tests fail on the various
implemented algorithms. We already panic when any of the test in testmgr.c
fail when we are operating in fips mode. The continuous test in the cprng here
was missed when that was implmented. This code simply checks for the
fips_enabled flag if the test fails, and warns us via syslog or panics the box
accordingly.
Signed-off-by: Neil Horman <nhorman@tuxdriver.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Pseudo RNGs provide predictable outputs based on input parateters {key, V, DT},
the idea behind them is that only the user should know what the inputs are.
While its nice to have default known values for testing purposes, it seems
dangerous to allow the use of those default values without some sort of safety
measure in place, lest an attacker easily guess the output of the cprng. This
patch forces the NEED_RESET flag on when allocating a cprng context, so that any
user is forced to reseed it before use. The defaults can still be used for
testing, but this will prevent their inadvertent use, and be more secure.
Signed-off-by: Neil Horman <nhorman@redhat.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Intel AES-NI is a new set of Single Instruction Multiple Data (SIMD)
instructions that are going to be introduced in the next generation of
Intel processor, as of 2009. These instructions enable fast and secure
data encryption and decryption, using the Advanced Encryption Standard
(AES), defined by FIPS Publication number 197. The architecture
introduces six instructions that offer full hardware support for
AES. Four of them support high performance data encryption and
decryption, and the other two instructions support the AES key
expansion procedure.
The white paper can be downloaded from:
http://softwarecommunity.intel.com/isn/downloads/intelavx/AES-Instructions-Set_WP.pdf
AES may be used in soft_irq context, but MMX/SSE context can not be
touched safely in soft_irq context. So in_interrupt() is checked, if
in IRQ or soft_irq context, the general x86_64 implementation are used
instead.
Signed-off-by: Huang Ying <ying.huang@intel.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
cryptd_alloc_ablkcipher() will allocate a cryptd-ed ablkcipher for
specified algorithm name. The new allocated one is guaranteed to be
cryptd-ed ablkcipher, so the blkcipher underlying can be gotten via
cryptd_ablkcipher_child().
Signed-off-by: Huang Ying <ying.huang@intel.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
We're currently checking the frontend type in init_tfm. This is
completely pointless because the fact that we're called at all
means that the frontend is ours so the type must match as well.
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
It turns out that LRW has never worked properly on big endian.
This was never discussed because nobody actually used it that
way. In fact, it was only discovered when Geert Uytterhoeven
loaded it through tcrypt which failed the test on it.
The fix is straightforward, on big endian the to find the nth
bit we should be grouping them by words instead of bytes. So
setbit128_bbe should xor with 128 - BITS_PER_LONG instead of
128 - BITS_PER_BYTE == 0x78.
Tested-by: Geert Uytterhoeven <Geert.Uytterhoeven@sonycom.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
It's illegal to call flush_dcache_page on slab pages on a number
of architectures. So this patch avoids doing so if PageSlab is
true.
In future we can move the flush_dcache_page call to those page
cache users that actually need it.
Reported-by: David S. Miller <davem@davemloft.net>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Geert Uytterhoeven pointed out that we're not zeroing all the
memory when freeing a transform. This patch fixes it by calling
ksize to ensure that we zero everything in sight.
Reported-by: Geert Uytterhoeven <Geert.Uytterhoeven@sonycom.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Module reference counting for shash is incorrect: when
a new shash transformation is created the refcount is not
increased as it should.
Signed-off-by: Adrian-Ken Rueegsegger <rueegsegger@swiss-it.ch>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
When we complete a test we'll notify everyone waiting on it, drop
the mutex, and then remove the test larval (after reacquiring the
mutex). If one of the notified parties tries to register another
algorithm with the same driver name prior to the removal of the
test larval, they will fail with EEXIST as only one algorithm of
a given name can be tested at any time.
This broke the initialisation of aead and givcipher algorithms as
they will register two algorithms with the same driver name, in
sequence.
This patch fixes the problem by marking the larval as dead before
we drop the mutex, and also ignoring all dead or dying algorithms
on the registration path.
Tested-by: Andreas Steffen <andreas.steffen@strongswan.org>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Its a valid use case to have null associated data in a ccm vector, but
this case isn't being handled properly right now.
The following ccm decryption/verification test vector, using the
rfc4309 implementation regularly triggers a panic, as will any
other vector with null assoc data:
* key: ab2f8a74b71cd2b1ff802e487d82f8b9
* iv: c6fb7d800d13abd8a6b2d8
* Associated Data: [NULL]
* Tag Length: 8
* input: d5e8939fc7892e2b
The resulting panic looks like so:
Unable to handle kernel paging request at ffff810064ddaec0 RIP:
[<ffffffff8864c4d7>] :ccm:get_data_to_compute+0x1a6/0x1d6
PGD 8063 PUD 0
Oops: 0002 [1] SMP
last sysfs file: /module/libata/version
CPU 0
Modules linked in: crypto_tester_kmod(U) seqiv krng ansi_cprng chainiv rng ctr aes_generic aes_x86_64 ccm cryptomgr testmgr_cipher testmgr aead crypto_blkcipher crypto_a
lgapi des ipv6 xfrm_nalgo crypto_api autofs4 hidp l2cap bluetooth nfs lockd fscache nfs_acl sunrpc ip_conntrack_netbios_ns ipt_REJECT xt_state ip_conntrack nfnetlink xt_
tcpudp iptable_filter ip_tables x_tables dm_mirror dm_log dm_multipath scsi_dh dm_mod video hwmon backlight sbs i2c_ec button battery asus_acpi acpi_memhotplug ac lp sg
snd_intel8x0 snd_ac97_codec ac97_bus snd_seq_dummy snd_seq_oss joydev snd_seq_midi_event snd_seq snd_seq_device snd_pcm_oss snd_mixer_oss ide_cd snd_pcm floppy parport_p
c shpchp e752x_edac snd_timer e1000 i2c_i801 edac_mc snd soundcore snd_page_alloc i2c_core cdrom parport serio_raw pcspkr ata_piix libata sd_mod scsi_mod ext3 jbd uhci_h
cd ohci_hcd ehci_hcd
Pid: 12844, comm: crypto-tester Tainted: G 2.6.18-128.el5.fips1 #1
RIP: 0010:[<ffffffff8864c4d7>] [<ffffffff8864c4d7>] :ccm:get_data_to_compute+0x1a6/0x1d6
RSP: 0018:ffff8100134434e8 EFLAGS: 00010246
RAX: 0000000000000000 RBX: ffff8100104898b0 RCX: ffffffffab6aea10
RDX: 0000000000000010 RSI: ffff8100104898c0 RDI: ffff810064ddaec0
RBP: 0000000000000000 R08: ffff8100104898b0 R09: 0000000000000000
R10: ffff8100103bac84 R11: ffff8100104898b0 R12: ffff810010489858
R13: ffff8100104898b0 R14: ffff8100103bac00 R15: 0000000000000000
FS: 00002ab881adfd30(0000) GS:ffffffff803ac000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
CR2: ffff810064ddaec0 CR3: 0000000012a88000 CR4: 00000000000006e0
Process crypto-tester (pid: 12844, threadinfo ffff810013442000, task ffff81003d165860)
Stack: ffff8100103bac00 ffff8100104898e8 ffff8100134436f8 ffffffff00000000
0000000000000000 ffff8100104898b0 0000000000000000 ffff810010489858
0000000000000000 ffff8100103bac00 ffff8100134436f8 ffffffff8864c634
Call Trace:
[<ffffffff8864c634>] :ccm:crypto_ccm_auth+0x12d/0x140
[<ffffffff8864cf73>] :ccm:crypto_ccm_decrypt+0x161/0x23a
[<ffffffff88633643>] :crypto_tester_kmod:cavs_test_rfc4309_ccm+0x4a5/0x559
[...]
The above is from a RHEL5-based kernel, but upstream is susceptible too.
The fix is trivial: in crypto/ccm.c:crypto_ccm_auth(), pctx->ilen contains
whatever was in memory when pctx was allocated if assoclen is 0. The tested
fix is to simply add an else clause setting pctx->ilen to 0 for the
assoclen == 0 case, so that get_data_to_compute() doesn't try doing
things its not supposed to.
Signed-off-by: Jarod Wilson <jarod@redhat.com>
Acked-by: Neil Horman <nhorman@tuxdriver.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
When we get left-over bits from a slow walk, it means that the
underlying cipher has gone troppo. However, as we're handling
that case we should ensure that the caller terminates the walk.
This patch does this by setting walk->nbytes to zero.
Reported-by: Roel Kluin <roel.kluin@gmail.com>
Reported-by: Huang Ying <ying.huang@intel.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
As it is if an algorithm with a zero-length IV is used (e.g.,
NULL encryption) with authenc, authenc may generate an SG entry
of length zero, which will trigger a BUG check in the hash layer.
This patch fixes it by skipping the IV SG generation if the IV
size is zero.
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Now that clients no longer need to be notified of channel arrival
dma_async_client_register can simply increment the dmaengine_ref_count.
Reviewed-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
async_tx and net_dma each have open-coded versions of issue_pending_all,
so provide a common routine in dmaengine.
The implementation needs to walk the global device list, so implement
rcu to allow dma_issue_pending_all to run lockless. Clients protect
themselves from channel removal events by holding a dmaengine reference.
Reviewed-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
Allowing multiple clients to each define their own channel allocation
scheme quickly leads to a pathological situation. For memory-to-memory
offload all clients can share a central allocator.
This simply moves the existing async_tx allocator to dmaengine with
minimal fixups:
* async_tx.c:get_chan_ref_by_cap --> dmaengine.c:nth_chan
* async_tx.c:async_tx_rebalance --> dmaengine.c:dma_channel_rebalance
* split out common code from async_tx.c:__async_tx_find_channel -->
dma_find_channel
Reviewed-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
Simply, if a client wants any dmaengine channel then prevent all dmaengine
modules from being removed. Once the clients are done re-enable module
removal.
Why?, beyond reducing complication:
1/ Tracking reference counts per-transaction in an efficient manner, as
is currently done, requires a complicated scheme to avoid cache-line
bouncing effects.
2/ Per-transaction ref-counting gives the false impression that a
dma-driver can be gracefully removed ahead of its user (net, md, or
dma-slave)
3/ None of the in-tree dma-drivers talk to hot pluggable hardware, but
if such an engine were built one day we still would not need to notify
clients of remove events. The driver can simply return NULL to a
->prep() request, something that is much easier for a client to handle.
Reviewed-by: Andrew Morton <akpm@linux-foundation.org>
Acked-by: Maciej Sosnowski <maciej.sosnowski@intel.com>
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
async_tx.ko is a consumer of dma channels. A circular dependency arises
if modules in drivers/dma rely on common code in async_tx.ko. It
prevents either module from being unloaded.
Move dma_wait_for_async_tx and async_tx_run_dependencies to dmaeninge.o
where they should have been from the beginning.
Reviewed-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
The tables used by the various AES algorithms are currently
computed at run-time. This has created an init ordering problem
because some AES algorithms may be registered before the tables
have been initialised.
This patch gets around this whole thing by precomputing the tables.
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
The comment for the deflate test vectors says the winbits parameter is 11,
while the deflate module actually uses -11 (a negative window bits parameter
enables the raw deflate format instead of the zlib format).
Correct this, to avoid confusion about the format used.
Signed-off-by: Geert Uytterhoeven <Geert.Uytterhoeven@sonycom.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
ROTATE -> rol32
XOR was always used with the same destination, use ^=
PLUS/PLUSONE use ++ or +=
Signed-off-by: Harvey Harrison <harvey.harrison@gmail.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
While its a slightly insane to bypass the key1 == key2 ||
key2 == key3 check in triple-des, since it reduces it to the
same strength as des, some folks do need to do this from time
to time for backwards compatibility with des.
My own case is FIPS CAVS test vectors. Many triple-des test
vectors use a single key, replicated 3x. In order to get the
expected results, des3_ede_setkey() needs to only reject weak
keys if the CRYPTO_TFM_REQ_WEAK_KEY flag is set.
Also sets a more appropriate RES flag when a weak key is found.
Signed-off-by: Jarod Wilson <jarod@redhat.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
This patch changes sha512 and sha384 to the new shash interface.
Signed-off-by: Adrian-Ken Rueegsegger <ken@codelabs.ch>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
The message schedule W (u64[80]) is too big for the stack. In order
for this algorithm to be used with shash it is moved to a static
percpu area.
Signed-off-by: Adrian-Ken Rueegsegger <ken@codelabs.ch>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
This patch changes michael_mic to the new shash interface.
Signed-off-by: Adrian-Ken Rueegsegger <ken@codelabs.ch>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
This patch changes wp512, wp384 and wp256 to the new shash interface.
Signed-off-by: Adrian-Ken Rueegsegger <ken@codelabs.ch>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
This patch changes tgr192, tgr160 and tgr128 to the new shash interface.
Signed-off-by: Adrian-Ken Rueegsegger <ken@codelabs.ch>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
This patch changes sha256 and sha224 to the new shash interface.
Signed-off-by: Adrian-Ken Rueegsegger <ken@codelabs.ch>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
This patch changes md5 to the new shash interface.
Signed-off-by: Adrian-Ken Rueegsegger <ken@codelabs.ch>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
This patch changes md4 to the new shash interface.
Signed-off-by: Adrian-Ken Rueegsegger <ken@codelabs.ch>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
This patch changes sha1 to the new shash interface.
Signed-off-by: Adrian-Ken Rueegsegger <ken@codelabs.ch>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Since most cryptographic hash algorithms have no keys, this patch
makes the setkey function optional for ahash and shash.
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
When self-testing (de)compression algorithms, make sure the actual size of
the (de)compressed output data matches the expected output size.
Otherwise, in case the actual output size would be smaller than the expected
output size, the subsequent buffer compare test would still succeed, and no
error would be reported.
Signed-off-by: Geert Uytterhoeven <Geert.Uytterhoeven@sonycom.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Base versions handle constant folding just fine.
Signed-off-by: Harvey Harrison <harvey.harrison@gmail.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
This warning:
crypto/testmgr.c: In function ‘test_comp’:
crypto/testmgr.c:829: warning: ‘ret’ may be used uninitialized in this function
triggers because GCC correctly notices that in the ctcount == 0 &&
dtcount != 0 input condition case this function can return an undefined
value, if the second loop fails.
Remove the shadowed 'ret' variable from the second loop that was probably
unintended.
Signed-off-by: Ingo Molnar <mingo@elte.hu>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
The ANSI X9.31 PRNG docs aren't particularly clear on how to increment DT,
but empirical testing shows we're incrementing from the wrong end. A 10,000
iteration Monte Carlo RNG test currently winds up not getting the expected
result.
From http://csrc.nist.gov/groups/STM/cavp/documents/rng/RNGVS.pdf :
# CAVS 4.3
# ANSI931 MCT
[X9.31]
[AES 128-Key]
COUNT = 0
Key = 9f5b51200bf334b5d82be8c37255c848
DT = 6376bbe52902ba3b67c925fa701f11ac
V = 572c8e76872647977e74fbddc49501d1
R = 48e9bd0d06ee18fbe45790d5c3fc9b73
Currently, we get 0dd08496c4f7178bfa70a2161a79459a after 10000 loops.
Inverting the DT increment routine results in us obtaining the expected result
of 48e9bd0d06ee18fbe45790d5c3fc9b73. Verified on both x86_64 and ppc64.
Signed-off-by: Jarod Wilson <jarod@redhat.com>
Acked-by: Neil Horman <nhorman@tuxdriver.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
While working with some FIPS RNGVS test vectors yesterday, I discovered a
little bug in the way the ansi_cprng code works right now.
For example, the following test vector (complete with expected result)
from http://csrc.nist.gov/groups/STM/cavp/documents/rng/RNGVS.pdf ...
Key = f3b1666d13607242ed061cabb8d46202
DT = e6b3be782a23fa62d71d4afbb0e922fc
V = f0000000000000000000000000000000
R = 88dda456302423e5f69da57e7b95c73a
...when run through ansi_cprng, yields an incorrect R value
of e2afe0d794120103d6e86a2b503bdfaa.
If I load up ansi_cprng w/dbg=1 though, it was fairly obvious what was
going wrong:
----8<----
getting 16 random bytes for context ffff810033fb2b10
Calling _get_more_prng_bytes for context ffff810033fb2b10
Input DT: 00000000: e6 b3 be 78 2a 23 fa 62 d7 1d 4a fb b0 e9 22 fc
Input I: 00000000: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
Input V: 00000000: f0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
tmp stage 0: 00000000: e6 b3 be 78 2a 23 fa 62 d7 1d 4a fb b0 e9 22 fc
tmp stage 1: 00000000: f4 8e cb 25 94 3e 8c 31 d6 14 cd 8a 23 f1 3f 84
tmp stage 2: 00000000: 8c 53 6f 73 a4 1a af d4 20 89 68 f4 58 64 f8 be
Returning new block for context ffff810033fb2b10
Output DT: 00000000: e7 b3 be 78 2a 23 fa 62 d7 1d 4a fb b0 e9 22 fc
Output I: 00000000: 04 8e cb 25 94 3e 8c 31 d6 14 cd 8a 23 f1 3f 84
Output V: 00000000: 48 89 3b 71 bc e4 00 b6 5e 21 ba 37 8a 0a d5 70
New Random Data: 00000000: 88 dd a4 56 30 24 23 e5 f6 9d a5 7e 7b 95 c7 3a
Calling _get_more_prng_bytes for context ffff810033fb2b10
Input DT: 00000000: e7 b3 be 78 2a 23 fa 62 d7 1d 4a fb b0 e9 22 fc
Input I: 00000000: 04 8e cb 25 94 3e 8c 31 d6 14 cd 8a 23 f1 3f 84
Input V: 00000000: 48 89 3b 71 bc e4 00 b6 5e 21 ba 37 8a 0a d5 70
tmp stage 0: 00000000: e7 b3 be 78 2a 23 fa 62 d7 1d 4a fb b0 e9 22 fc
tmp stage 1: 00000000: 80 6b 3a 8c 23 ae 8f 53 be 71 4c 16 fc 13 b2 ea
tmp stage 2: 00000000: 2a 4d e1 2a 0b 58 8e e6 36 b8 9c 0a 26 22 b8 30
Returning new block for context ffff810033fb2b10
Output DT: 00000000: e8 b3 be 78 2a 23 fa 62 d7 1d 4a fb b0 e9 22 fc
Output I: 00000000: c8 e2 01 fd 9f 4a 8f e5 e0 50 f6 21 76 19 67 9a
Output V: 00000000: ba 98 e3 75 c0 1b 81 8d 03 d6 f8 e2 0c c6 54 4b
New Random Data: 00000000: e2 af e0 d7 94 12 01 03 d6 e8 6a 2b 50 3b df aa
returning 16 from get_prng_bytes in context ffff810033fb2b10
----8<----
The expected result is there, in the first "New Random Data", but we're
incorrectly making a second call to _get_more_prng_bytes, due to some checks
that are slightly off, which resulted in our original bytes never being
returned anywhere.
One approach to fixing this would be to alter some byte_count checks in
get_prng_bytes, but it would mean the last DEFAULT_BLK_SZ bytes would be
copied a byte at a time, rather than in a single memcpy, so a slightly more
involved, equally functional, and ultimately more efficient way of fixing this
was suggested to me by Neil, which I'm submitting here. All of the RNGVS ANSI
X9.31 AES128 VST test vectors I've passed through ansi_cprng are now returning
the expected results with this change.
Signed-off-by: Jarod Wilson <jarod@redhat.com>
Acked-by: Neil Horman <nhorman@tuxdriver.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
ARRAY_SIZE is more concise to use when the size of an array is divided by
the size of its type or the size of its first element.
The semantic patch that makes this change is as follows:
(http://www.emn.fr/x-info/coccinelle/)
// <smpl>
@i@
@@
#include <linux/kernel.h>
@depends on i using "paren.iso"@
type T;
T[] E;
@@
- (sizeof(E)/sizeof(T))
+ ARRAY_SIZE(E)
// </smpl>
Signed-off-by: Julia Lawall <julia@diku.dk>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
This patch swaps the role of libcrc32c and crc32c. Previously
the implementation was in libcrc32c and crc32c was a wrapper.
Now the code is in crc32c and libcrc32c just calls the crypto
layer.
The reason for the change is to tap into the algorithm selection
capability of the crypto API so that optimised implementations
such as the one utilising Intel's CRC32C instruction can be
used where available.
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
This patch adds a test for the requirement that all crc32c algorithms
shall store the partial result in the first four bytes of the descriptor
context.
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
This patch allows shash algorithms to be used through the old hash
interface. This is a transitional measure so we can convert the
underlying algorithms to shash before converting the users across.
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
This patch makes /proc/crypto call the type-specific show function
if one is present before calling the legacy show functions for
cipher/digest/compress. This allows us to reuse the type values
for those legacy types. In particular, hash and digest will share
one type value while shash is phased in as the default hash type.
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
It is often useful to save the partial state of a hash function
so that it can be used as a base for two or more computations.
The most prominent example is HMAC where all hashes start from
a base determined by the key. Having an import/export interface
means that we only have to compute that base once rather than
for each message.
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
This patch allows shash algorithms to be used through the ahash
interface. This is required before we can convert digest algorithms
over to shash.
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
The shash interface replaces the current synchronous hash interface.
It improves over hash in two ways. Firstly shash is reentrant,
meaning that the same tfm may be used by two threads simultaneously
as all hashing state is stored in a local descriptor.
The other enhancement is that shash no longer takes scatter list
entries. This is because shash is specifically designed for
synchronous algorithms and as such scatter lists are unnecessary.
All existing hash users will be converted to shash once the
algorithms have been completely converted.
There is also a new finup function that combines update with final.
This will be extended to ahash once the algorithm conversion is
done.
This is also the first time that an algorithm type has their own
registration function. Existing algorithm types will be converted
to this way in due course.
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
This patch reintroduces a completely revamped crypto_alloc_tfm.
The biggest change is that we now take two crypto_type objects
when allocating a tfm, a frontend and a backend. In fact this
simply formalises what we've been doing behind the API's back.
For example, as it stands crypto_alloc_ahash may use an
actual ahash algorithm or a crypto_hash algorithm. Putting
this in the API allows us to do this much more cleanly.
The existing types will be converted across gradually.
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
The type exit function needs to undo any allocations done by the type
init function. However, the type init function may differ depending
on the upper-level type of the transform (e.g., a crypto_blkcipher
instantiated as a crypto_ablkcipher).
So we need to move the exit function out of the lower-level
structure and into crypto_tfm itself.
As it stands this is a no-op since nobody uses exit functions at
all. However, all cases where a lower-level type is instantiated
as a different upper-level type (such as blkcipher as ablkcipher)
will be converted such that they allocate the underlying transform
and use that instead of casting (e.g., crypto_ablkcipher casted
into crypto_blkcipher). That will need to use a different exit
function depending on the upper-level type.
This patch also allows the type init/exit functions to call (or not)
cra_init/cra_exit instead of always calling them from the top level.
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
This is a patch that was sent to me by Jarod Wilson, marking off my
outstanding todo to allow the ansi cprng to set/reset the DT counter value in a
cprng instance. Currently crytpo_rng_reset accepts a seed byte array which is
interpreted by the ansi_cprng as a {V key} tuple. This patch extends that tuple
to now be {V key DT}, with DT an optional value during reset. This patch also
fixes a bug we noticed in which the offset of the key area of the seed is
started at DEFAULT_PRNG_KSZ rather than DEFAULT_BLK_SZ as it should be.
Signed-off-by: Neil Horman <nhorman@tuxdriver.com>
Signed-off-by: Jarod Wilson <jarod@redhat.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Remove the private implementation of 32-bit rotation and unaligned
access with byteswapping.
As a bonus, fixes sparse warnings:
crypto/camellia.c:602:2: warning: cast to restricted __be32
crypto/camellia.c:603:2: warning: cast to restricted __be32
crypto/camellia.c:604:2: warning: cast to restricted __be32
crypto/camellia.c:605:2: warning: cast to restricted __be32
crypto/camellia.c:710:2: warning: cast to restricted __be32
crypto/camellia.c:711:2: warning: cast to restricted __be32
crypto/camellia.c:712:2: warning: cast to restricted __be32
crypto/camellia.c:713:2: warning: cast to restricted __be32
crypto/camellia.c:714:2: warning: cast to restricted __be32
crypto/camellia.c:715:2: warning: cast to restricted __be32
crypto/camellia.c:716:2: warning: cast to restricted __be32
crypto/camellia.c:717:2: warning: cast to restricted __be32
[Thanks to Tomoyuki Okazaki for spotting the typo]
Tested-by: Carlo E. Prelz <fluido@fluido.as>
Signed-off-by: Harvey Harrison <harvey.harrison@gmail.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
The FIPS specification requires that should self test for any supported
crypto algorithm fail during operation in fips mode, we need to prevent
the use of any crypto functionality until such time as the system can
be re-initialized. Seems like the best way to handle that would be
to panic the system if we were in fips mode and failed a self test.
This patch implements that functionality. I've built and run it
successfully.
Signed-off-by: Neil Horman <nhorman@tuxdriver.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
If we have at least one algorithm built-in then it no longer makes
sense to have the testing framework, and hence cryptomgr to be a
module. It should be either on or off, i.e., built-in or disabled.
This just happens to stop a potential runaway modprobe loop that
seems to trigger on at least one distro.
With fixes from Evgeniy Polyakov.
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Mapping the destination multiple times is a misuse of the dma-api.
Since the destination may be reused as a source, ensure that it is only
mapped once and that it is mapped bidirectionally. This appears to add
ugliness on the unmap side in that it always reads back the destination
address from the descriptor, but gcc can determine that dma_unmap is a
nop and not emit the code that calculates its arguments.
Cc: <stable@kernel.org>
Cc: Saeed Bishara <saeed@marvell.com>
Acked-by: Yuri Tikhonov <yur@emcraft.com>
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
* 'next' of git://git.kernel.org/pub/scm/linux/kernel/git/djbw/async_tx:
fsldma: allow Freescale Elo DMA driver to be compiled as a module
fsldma: remove internal self-test from Freescale Elo DMA driver
drivers/dma/dmatest.c: switch a GFP_ATOMIC to GFP_KERNEL
dmatest: properly handle duplicate DMA channels
drivers/dma/ioat_dma.c: drop code after return
async_tx: make async_tx_run_dependencies() easier to read
* git://git.kernel.org/pub/scm/linux/kernel/git/herbert/crypto-2.6:
crypto: skcipher - Use RNG interface instead of get_random_bytes
crypto: rng - RNG interface and implementation
crypto: api - Add fips_enable flag
crypto: skcipher - Move IV generators into their own modules
crypto: cryptomgr - Test ciphers using ECB
crypto: api - Use test infrastructure
crypto: cryptomgr - Add test infrastructure
crypto: tcrypt - Add alg_test interface
crypto: tcrypt - Abort and only log if there is an error
crypto: crc32c - Use Intel CRC32 instruction
crypto: tcrypt - Avoid using contiguous pages
crypto: api - Display larval objects properly
crypto: api - Export crypto_alg_lookup instead of __crypto_alg_lookup
crypto: Kconfig - Replace leading spaces with tabs
* Rename 'next' to 'dep'
* Move the channel switch check inside the loop to simplify
termination
Acked-by: Ilya Yanok <yanok@emcraft.com>
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
This reverts commit bd699f2df6,
which causes camellia to fail the included self-test vectors.
It has also been confirmed that it breaks existing encrypted
disks using camellia.
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Should clear the next pointer of the TX if we are sure that the
next TX (say NXT) will be submitted to the channel too. Overwise,
we break the chain of descriptors, because we lose the information
about the next descriptor to run. So next time, when invoke
async_tx_run_dependencies() with TX, it's TX->next will be NULL, and
NXT will be never submitted.
Cc: <stable@kernel.org> [2.6.26]
Signed-off-by: Yuri Tikhonov <yur@emcraft.com>
Signed-off-by: Ilya Yanok <yanok@emcraft.com>
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
This patch makes the IV generators use the new RNG interface so
that the user can pick an RNG other than the default get_random_bytes.
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
This patch adds a random number generator interface as well as a
cryptographic pseudo-random number generator based on AES. It is
meant to be used in cases where a deterministic CPRNG is required.
One of the first applications will be as an input in the IPsec IV
generation process.
Signed-off-by: Neil Horman <nhorman@tuxdriver.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Add the ability to turn FIPS-compliant mode on or off at boot
In order to be FIPS compliant, several check may need to be preformed that may
be construed as unusefull in a non-compliant mode. This patch allows us to set
a kernel flag incating that we are running in a fips-compliant mode from boot
up. It also exports that mode information to user space via a sysctl
(/proc/sys/crypto/fips_enabled).
Tested successfully by me.
Signed-off-by: Neil Horman <nhorman@tuxdriver.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
This patch moves the default IV generators into their own modules
in order to break a dependency loop between cryptomgr, rng, and
blkcipher.
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
As it is we only test ciphers when combined with a mode. That means
users that do not invoke a mode of operations may get an untested
cipher.
This patch tests all ciphers using the ECB mode so that simple cipher
users such as ansi-cprng are also protected.
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
This patch makes use of the new testing infrastructure by requiring
algorithms to pass a run-time test before they're made available to
users.
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
This patch moves the newly created alg_test infrastructure into
cryptomgr. This shall allow us to use it for testing at algorithm
registrations.
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
This patch creates a new interface algorithm testing. A test can
be requested for a particular implementation of an algorithm. This
is achieved by taking both the name of the algorithm and that of
the implementation.
The all-inclusive test has also been rewritten to no longer require
a duplicate listing of all algorithms with tests. In that process
a number of missing tests have also been discovered and rectified.
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
The info printed is a complete waste of space when there is no error
since it doesn't tell us anything that we don't already know. If there
is an error, we can also be more verbose.
In case that there is an error, this patch also aborts the test and
returns the error to the caller. In future this will be used to
algorithms at registration time.
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
From NHM processor onward, Intel processors can support hardware accelerated
CRC32c algorithm with the new CRC32 instruction in SSE 4.2 instruction set.
The patch detects the availability of the feature, and chooses the most proper
way to calculate CRC32c checksum.
Byte code instructions are used for compiler compatibility.
No MMX / XMM registers is involved in the implementation.
Signed-off-by: Austin Zhang <austin.zhang@intel.com>
Signed-off-by: Kent Liu <kent.liu@intel.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
If tcrypt is to be used as a run-time integrity test, it needs to be
more resilient in a hostile environment. For a start allocating 32K
of physically contiguous memory is definitely out.
This patch teaches it to use separate pages instead.
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Rather than displaying larval objects as real objects, this patch
makes them show up under /proc/crypto as of type larval.
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Since the only user of __crypto_alg_lookup is doing exactly what
crypto_alg_lookup does, we can now the latter in lieu of the former.
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Authenc works in two stages for encryption, it first encrypts and
then computes an ICV. The context memory of the request is used
by both operations. The problem is that when an asynchronous
encryption completes, we will compute the ICV and then reread the
context memory of the encryption to get the original request.
It just happens that we have a buffer of 16 bytes in front of the
request pointer, so ICVs of 16 bytes (such as SHA1) do not trigger
the bug. However, any attempt to uses a larger ICV instantly kills
the machine when the first asynchronous encryption is completed.
This patch fixes this by saving the request pointer before we start
the ICV computation.
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
The changeset ca786dc738
crypto: hash - Fixed digest size check
missed one spot for the digest type. This patch corrects that
error.
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
My changeset 4b22f0ddb6
crypto: tcrpyt - Remove unnecessary kmap/kunmap calls
introduced a typo that broke AEAD chunk testing. In particular,
axbuf should really be xbuf.
There is also an issue with testing the last segment when encrypting.
The additional part produced by AEAD wasn't tested. Similarly, on
decryption the additional part of the AEAD input is mistaken for
corruption.
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
* 'for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/djbw/async_tx: (24 commits)
I/OAT: I/OAT version 3.0 support
I/OAT: tcp_dma_copybreak default value dependent on I/OAT version
I/OAT: Add watchdog/reset functionality to ioatdma
iop_adma: cleanup iop_chan_xor_slot_count
iop_adma: document how to calculate the minimum descriptor pool size
iop_adma: directly reclaim descriptors on allocation failure
async_tx: make async_tx_test_ack a boolean routine
async_tx: remove depend_tx from async_tx_sync_epilog
async_tx: export async_tx_quiesce
async_tx: fix handling of the "out of descriptor" condition in async_xor
async_tx: ensure the xor destination buffer remains dma-mapped
async_tx: list_for_each_entry_rcu() cleanup
dmaengine: Driver for the Synopsys DesignWare DMA controller
dmaengine: Add slave DMA interface
dmaengine: add DMA_COMPL_SKIP_{SRC,DEST}_UNMAP flags to control dma unmap
dmaengine: Add dma_client parameter to device_alloc_chan_resources
dmatest: Simple DMA memcpy test client
dmaengine: DMA engine driver for Marvell XOR engine
iop-adma: fix platform driver hotplug/coldplug
dmaengine: track the number of clients using a channel
...
Fixed up conflict in drivers/dca/dca-sysfs.c manually
All callers of async_tx_sync_epilog have called async_tx_quiesce on the
depend_tx, so async_tx_sync_epilog need only call the callback to
complete the operation.
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
Ensure forward progress is made when a dmaengine driver is unable to
allocate an xor descriptor by breaking the dependency chain with
async_tx_quisce() and issue any pending descriptors.
Tested with iop-adma by setting device->max_xor = 2 to force multiple
calls to device_prep_dma_xor for each call to async_xor and limiting the
descriptor slot pool to 5. Discovered that the minimum descriptor pool
size for iop-adma is 2 * iop_chan_xor_slot_cnt(device->max_xor) + 1.
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
When the number of source buffers for an xor operation exceeds the hardware
channel maximum async_xor creates a chain of dependent operations. The result
of one operation is reused as an input to the next to continue the xor
calculation. The destination buffer should remain mapped for the duration of
the entire chain. To provide this guarantee the code must no longer be allowed
to fallback to the synchronous path as this will preclude the buffer from being
unmapped, i.e. the dma-driver will potentially miss the descriptor with
!DMA_COMPL_SKIP_DEST_UNMAP.
Cc: Neil Brown <neilb@suse.de>
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
In the rcu update side, don't use list_for_each_entry_rcu().
Signed-off-by: Li Zefan <lizf@cn.fujitsu.com>
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
All new crypto interfaces should go into individual files as much
as possible in order to ensure that crypto.h does not collapse under
its own weight.
This patch moves the ahash code into crypto/hash.h and crypto/internal/hash.h
respectively.
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
This patch reimplements crc32c using the ahash interface. This
allows one tfm to be used by an unlimited number of users provided
that they all use the same key (which all current crc32c users do).
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
This patch adds the walking helpers for hash algorithms akin to
those of block ciphers. This is a necessary step before we can
reimplement existing hash algorithms using the new ahash interface.
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
This patch adds a cryptographic pseudo-random number generator
based on CTR(AES-128). It is meant to be used in cases where a
deterministic CPRNG is required.
One of the first applications will be as an input in the IPsec IV
generation process.
Signed-off-by: Neil Horman <nhorman@tuxdriver.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
The base field in ahash_tfm appears to have been cut-n-pasted from
ablkcipher. It isn't needed here at all. Similarly, the info field
in ahash_request also appears to have originated from its cipher
counter-part and is vestigial.
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
The digest size check on hash algorithms is incorrect. It's
perfectly valid for hash algorithms to have a digest length
longer than their block size. For example crc32c has a block
size of 1 and a digest size of 4. Rather than having it lie
about its block size, this patch fixes the checks to do what
they really should which is to bound the digest size so that
code placing the digest on the stack continue to work.
HMAC however still needs to check this as it's only defined
for such algorithms.
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Similar to the rmd128.c annotations, significantly cuts down on the
noise.
Signed-off-by: Harvey Harrison <harvey.harrison@gmail.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Remove the private implementation of 32-bit rotation and unaligned
access with byteswapping.
As a bonus, fixes sparse warnings:
crypto/camellia.c:602:2: warning: cast to restricted __be32
crypto/camellia.c:603:2: warning: cast to restricted __be32
crypto/camellia.c:604:2: warning: cast to restricted __be32
crypto/camellia.c:605:2: warning: cast to restricted __be32
crypto/camellia.c:710:2: warning: cast to restricted __be32
crypto/camellia.c:711:2: warning: cast to restricted __be32
crypto/camellia.c:712:2: warning: cast to restricted __be32
crypto/camellia.c:713:2: warning: cast to restricted __be32
crypto/camellia.c:714:2: warning: cast to restricted __be32
crypto/camellia.c:715:2: warning: cast to restricted __be32
crypto/camellia.c:716:2: warning: cast to restricted __be32
crypto/camellia.c:717:2: warning: cast to restricted __be32
Signed-off-by: Harvey Harrison <harvey.harrison@gmail.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Noticed by Neil Horman: we are doing unnecessary kmap/kunmap calls
on kmalloced memory. This patch removes them. For the purposes of
testing SG construction, the underlying crypto code already does plenty
of kmap/kunmap calls anyway.
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Patch to add checking of DES3 test vectors using CBC mode. FIPS-140-2
compliance mandates that any supported mode of operation must include a self
test. This satisfies that requirement for cbc(des3_ede). The included test
vector was generated by me using openssl. Key/IV was generated with the
following command:
openssl enc -des_ede_cbc -P
input and output values were generated by repeating the string "Too many
secrets" a few times over, truncating it to 128 bytes, and encrypting it with
openssl using the aformentioned key. Tested successfully by myself
Signed-off-by: Neil Horman <nhorman@tuxdriver.com>
Acked-by: Adrian-Ken Rueegsegger <rueegsegger@swiss-it.ch>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Dan Williams