After DMA is complete, the omap_sham_finish_req function is called as
a part of the done_task tasklet. During this its atomic and any calls
to pm functions should not assume they wont sleep.
The patch replaces a call to pm_runtime_put_sync (which can sleep) with
pm_runtime_put thus fixing a kernel panic observed on AM33xx SoC during
SHA operation.
Tested on an AM33xx SoC device (beaglebone board).
To reproduce the problem, used the tcrypt kernel module as:
modprobe tcrypt sec=2 mode=403
Signed-off-by: Joel A Fernandes <joelagnel@ti.com>
Cc: David S. Miller <davem@davemloft.net>
Acked-by: Mark A. Greer <mgreer@animalcreek.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Pull crypto update from Herbert Xu:
"Here is the crypto update for 3.9:
- Added accelerated implementation of crc32 using pclmulqdq.
- Added test vector for fcrypt.
- Added support for OMAP4/AM33XX cipher and hash.
- Fixed loose crypto_user input checks.
- Misc fixes"
* git://git.kernel.org/pub/scm/linux/kernel/git/herbert/crypto-2.6: (43 commits)
crypto: user - ensure user supplied strings are nul-terminated
crypto: user - fix empty string test in report API
crypto: user - fix info leaks in report API
crypto: caam - Added property fsl,sec-era in SEC4.0 device tree binding.
crypto: use ERR_CAST
crypto: atmel-aes - adjust duplicate test
crypto: crc32-pclmul - Kill warning on x86-32
crypto: x86/twofish - assembler clean-ups: use ENTRY/ENDPROC, localize jump labels
crypto: x86/sha1 - assembler clean-ups: use ENTRY/ENDPROC
crypto: x86/serpent - use ENTRY/ENDPROC for assember functions and localize jump targets
crypto: x86/salsa20 - assembler cleanup, use ENTRY/ENDPROC for assember functions and rename ECRYPT_* to salsa20_*
crypto: x86/ghash - assembler clean-up: use ENDPROC at end of assember functions
crypto: x86/crc32c - assembler clean-up: use ENTRY/ENDPROC
crypto: cast6-avx: use ENTRY()/ENDPROC() for assembler functions
crypto: cast5-avx: use ENTRY()/ENDPROC() for assembler functions and localize jump targets
crypto: camellia-x86_64/aes-ni: use ENTRY()/ENDPROC() for assembler functions and localize jump targets
crypto: blowfish-x86_64: use ENTRY()/ENDPROC() for assembler functions and localize jump targets
crypto: aesni-intel - add ENDPROC statements for assembler functions
crypto: x86/aes - assembler clean-ups: use ENTRY/ENDPROC, localize jump targets
crypto: testmgr - add test vector for fcrypt
...
Fix the compile errors created by commit 2545e8d
(crypto: omap-sham - Add Device Tree Support)
when CONFIG_OF is not defined. This includes
changing omap_sham_get_res_dev() to omap_sham_get_res_of()
and creating an empty version of omap_sham_of_match[].
Signed-off-by: Mark A. Greer <mgreer@animalcreek.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
We still need to fix up few places for multiplatform support,
but that can proceed separately. Fix the issue by making the
problem drivers depends !ARCH_MULTIPLATFORM for now.
The remaining pieces that are not multiplatform compatible
for omap2+ SoCs are:
1. Some drivers are using custom omap_dm_timer calls
There are two drivers that are directly usign omap hardware
timers for PWM and DSP clocking: drivers/media/rc/ir-rx51.c and
drivers/staging/tidspbridge/core/dsp-clock.c. These can be
fixed for multiplatform by allowing a minimal set of hardware
timers to be accessed, and for some functionality by using the
hrtimer framework.
2. Hardware OMAP4_ERRATA_I688 needs to be fixed up
This can't be enabled for multiplatform configurations in
it's current form. It may be possible to fix it up to do
instruction replacement early on during init. Luckily it
looks like this errata does not seem to get hit with
mainline kernel code alone at least currently.
3. Legacy header needed for omap-sham.c
Looks like it still needs mach/irqs.h for omap1 that
does not exist for multiplatform systems. Just ifdef
it for now.
4. Mailbox is waiting to get moved to drivers
Disable it for now to avoid adding a dependency to the
mailbox patches.
Cc: Timo Kokkonen <timo.t.kokkonen@iki.fi>
Cc: Sean Young <sean@mess.org>
Cc: "Víctor Manuel Jáquez Leal" <vjaquez@igalia.com>
Cc: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Cc: Mauro Carvalho Chehab <mchehab@redhat.com>
Cc: Omar Ramirez Luna <omar.ramirez@ti.com>
Cc: Herbert Xu <herbert@gondor.apana.org.au>
Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Cc: Santosh Shilimkar <santosh.shilimkar@ti.com>
Tested-by: Ezequiel Garcia <ezequiel.garcia@free-electrons.com>
[tony@atomide.com: updated to disable mailbox]
Signed-off-by: Tony Lindgren <tony@atomide.com>
The OMAP4/AM33xx version of the SHAM crypto module
supports SHA224 and SHA256 in addition to MD5 and
SHA1 that the OMAP2 version of the module supports.
To add this support, use the platform_data introduced
in an ealier commit to hold the list of algorithms
supported by the current module. The probe routine
will use that list to register the correct algorithms.
Note: The code being integrated is from the TI AM33xx SDK
and was written by Greg Turner <gkmturner@gmail.com> and
Herman Schuurman (current email unknown) while at TI.
CC: Greg Turner <gkmturner@gmail.com>
CC: Dmitry Kasatkin <dmitry.kasatkin@intel.com>
Signed-off-by: Mark A. Greer <mgreer@animalcreek.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Add support for the OMAP4 version of the SHAM module
that is present on OMAP4 and AM33xx SoCs.
The modules have several differences including register
offsets, hardware XORing, and how DMA is triggered.
To handle these differences, a platform_data structure
is defined and contains routine pointers, register offsets,
bit shifts within registers, and flags to indicate whether
the hardware supports XORing and provides SHA1 results in
big or little endian. OMAP2/OMAP3-specific routines are
suffixed with '_omap2' and OMAP4/AM33xx routines are suffixed
with '_omap4'.
Note: The code being integrated is from the TI AM33xx SDK
and was written by Greg Turner <gkmturner@gmail.com> and
Herman Schuurman (current email unknown) while at TI.
CC: Greg Turner <gkmturner@gmail.com>
CC: Dmitry Kasatkin <dmitry.kasatkin@intel.com>
Signed-off-by: Mark A. Greer <mgreer@animalcreek.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Use the dma_request_slave_channel_compat() call instead of
the dma_request_channel() call to request a DMA channel.
This allows the omap-sham driver use different DMA engines.
CC: Dmitry Kasatkin <dmitry.kasatkin@intel.com>
Signed-off-by: Mark A. Greer <mgreer@animalcreek.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Add Device Tree suport to the omap-sham crypto
driver. Currently, only support for OMAP2 and
OMAP3 is being added but support for OMAP4 will
be added in a subsequent patch.
CC: Dmitry Kasatkin <dmitry.kasatkin@intel.com>
Signed-off-by: Mark A. Greer <mgreer@animalcreek.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Remove usage of the private OMAP DMA API.
The dmaengine API will be used instead.
CC: Russell King <rmk+kernel@arm.linux.org.uk>
CC: Dmitry Kasatkin <dmitry.kasatkin@intel.com>
Signed-off-by: Mark A. Greer <mgreer@animalcreek.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Add code to use the new dmaengine API alongside
the existing DMA code that uses the private
OMAP DMA API. The API to use is chosen by
defining or undefining 'OMAP_SHAM_DMA_PRIVATE'.
This is a transitional change and the code that uses
the private DMA API will be removed in an upcoming
commit.
CC: Russell King <rmk+kernel@arm.linux.org.uk>
CC: Dmitry Kasatkin <dmitry.kasatkin@intel.com>
Signed-off-by: Mark A. Greer <mgreer@animalcreek.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Add suspend/resume support to the OMAP SHAM driver.
CC: Dmitry Kasatkin <dmitry.kasatkin@intel.com>
Signed-off-by: Mark A. Greer <mgreer@animalcreek.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Convert the omap-sham crypto driver to use the
pm_runtime API instead of the clk API.
CC: Kevin Hilman <khilman@deeprootsystems.com>
CC: Dmitry Kasatkin <dmitry.kasatkin@intel.com>
Signed-off-by: Mark A. Greer <mgreer@animalcreek.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Remove the unnecessary pr_info() call in omap_sham_mod_init().
CC: Dmitry Kasatkin <dmitry.kasatkin@intel.com>
Signed-off-by: Mark A. Greer <mgreer@animalcreek.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
CONFIG_HOTPLUG is going away as an option. As a result, the __dev*
markings need to be removed.
This change removes the use of __devinit, __devexit_p, __devinitdata,
and __devexit from these drivers.
Based on patches originally written by Bill Pemberton, but redone by me
in order to handle some of the coding style issues better, by hand.
Cc: Bill Pemberton <wfp5p@virginia.edu>
Cc: Herbert Xu <herbert@gondor.apana.org.au>
Cc: "David S. Miller" <davem@davemloft.net>
Cc: Kent Yoder <key@linux.vnet.ibm.com>
Cc: Jamie Iles <jamie@jamieiles.com>
Cc: Kim Phillips <kim.phillips@freescale.com>
Cc: Shengzhou Liu <Shengzhou.Liu@freescale.com>
Cc: Alex Porosanu <alexandru.porosanu@freescale.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Based on earlier discussions[1] we attempted to find a suitable
location for the omap DMA header in commit 2b6c4e73 (ARM: OMAP:
DMA: Move plat/dma.h to plat-omap/dma-omap.h) until the conversion
to dmaengine is complete.
Unfortunately that was before I was able to try to test compile
of the ARM multiplatform builds for omap2+, and the end result
was not very good.
So I'm creating yet another all over the place patch to cut the
last dependency for building omap2+ for ARM multiplatform. After
this, we have finally removed the driver dependencies to the
arch/arm code, except for few drivers that are being worked on.
The other option was to make the <plat-omap/dma-omap.h> path
to work, but we'd have to add some new header directory to for
multiplatform builds.
Or we would have to manually include arch/arm/plat-omap/include
again from arch/arm/Makefile for omap2+.
Neither of these alternatives sound appealing as they will
likely lead addition of various other headers exposed to the
drivers, which we want to avoid for the multiplatform kernels.
Since we already have a minimal include/linux/omap-dma.h,
let's just use that instead and add a note to it to not
use the custom omap DMA functions any longer where possible.
Note that converting omap DMA to dmaengine depends on
dmaengine supporting automatically incrementing the FIFO
address at the device end, and converting all the remaining
legacy drivers. So it's going to be few more merge windows.
[1] https://patchwork.kernel.org/patch/1519591/#
cc: Russell King <linux@arm.linux.org.uk>
cc: Kevin Hilman <khilman@ti.com>
cc: "Benoît Cousson" <b-cousson@ti.com>
cc: Herbert Xu <herbert@gondor.apana.org.au>
cc: "David S. Miller" <davem@davemloft.net>
cc: Vinod Koul <vinod.koul@intel.com>
cc: Dan Williams <djbw@fb.com>
cc: Mauro Carvalho Chehab <mchehab@infradead.org>
cc: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
cc: Guennadi Liakhovetski <g.liakhovetski@gmx.de>
cc: David Woodhouse <dwmw2@infradead.org>
cc: Kyungmin Park <kyungmin.park@samsung.com>
cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
cc: Tomi Valkeinen <tomi.valkeinen@ti.com>
cc: Florian Tobias Schandinat <FlorianSchandinat@gmx.de>
cc: Hans Verkuil <hans.verkuil@cisco.com>
cc: Vaibhav Hiremath <hvaibhav@ti.com>
cc: Lokesh Vutla <lokeshvutla@ti.com>
cc: Rusty Russell <rusty@rustcorp.com.au>
cc: Artem Bityutskiy <artem.bityutskiy@linux.intel.com>
cc: Afzal Mohammed <afzal@ti.com>
cc: linux-crypto@vger.kernel.org
cc: linux-media@vger.kernel.org
cc: linux-mtd@lists.infradead.org
cc: linux-usb@vger.kernel.org
cc: linux-fbdev@vger.kernel.org
Acked-by: Felipe Balbi <balbi@ti.com>
Signed-off-by: Tony Lindgren <tony@atomide.com>
Drivers should not use cpu_is_omap or cpu_class_is_omap macros,
they should be private to the platform init code. And we'll be
removing plat/cpu.h and only have a private soc.h for the
arch/arm/*omap* code.
This patch is intended as preparation for the core omap changes
and removes the need to include plat/cpu.h from several drivers.
This is needed for the ARM common zImage support.
These changes are OK to do because:
- omap-rng.c does not need plat/cpu.h
- omap-aes.c and omap-sham.c get the proper platform_data
passed to them so they don't need extra checks in the driver
- omap-dma.c and omap-pcm.c can test the arch locally as
omap1 and omap2 cannot be compiled together because of
conflicting compiler flags
Cc: Deepak Saxena <dsaxena@plexity.net>
Cc: Matt Mackall <mpm@selenic.com>
Cc: Herbert Xu <herbert@gondor.apana.org.au>
Cc: David S. Miller <davem@davemloft.net>
Cc: Venkatraman S <svenkatr@ti.com>
Cc: Chris Ball <cjb@laptop.org>
Cc: Vinod Koul <vinod.koul@intel.com>
Cc: Dan Williams <djbw@fb.com>
Acked-by: Peter Ujfalusi <peter.ujfalusi@ti.com>
Acked-by: Jarkko Nikula <jarkko.nikula@bitmer.com>
Cc: Liam Girdwood <lrg@ti.com>
Cc: Mark Brown <broonie@opensource.wolfsonmicro.com>
Cc: linux-crypto@vger.kernel.org
Cc: linux-mmc@vger.kernel.org
Cc: alsa-devel@alsa-project.org
Cc: linux-kernel@vger.kernel.org
[tony@atomide.com: mmc changes folded in to an earlier patch]
Signed-off-by: Tony Lindgren <tony@atomide.com>
Move plat/dma.h to plat-omap/dma-omap.h as part of single
zImage work
Signed-off-by: Lokesh Vutla <lokeshvutla@ti.com>
Signed-off-by: Tony Lindgren <tony@atomide.com>
The added CRYPTO_ALG_KERN_DRIVER_ONLY indicates whether a cipher
is only available via a kernel driver. If the cipher implementation
might be available by using an instruction set or by porting the
kernel code, then it must not be set.
Signed-off-by: Nikos Mavrogiannopoulos <nmav@gnutls.org>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
It could be a situation, that tasklet is executed twice because of
certain delay between dma callback and irq handler execution.
In that case, second tasklet execution could actually corrupt the data
of the new started dma transactions.
This patch improves tasklet logic and prevents above described cases.
Signed-off-by: Dmitry Kasatkin <dmitry.kasatkin@nokia.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Calling omap_sham_handle_queue from "done" tasklet should be done
after irq scheduled tasklet completes.
Having additional tasklet does not solve that issue because it might
be execute before.
So queue handling tasklet has been removed and functionality integrated
into single tasklet.
Signed-off-by: Dmitry Kasatkin <dmitry.kasatkin@nokia.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Couple of context flags have been moved to device flags.
IRQ and tasklet handlers does not need to access request
context anymore.
Signed-off-by: Dmitry Kasatkin <dmitry.kasatkin@nokia.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Some flags are changed in interrupt handlers and verified in the tasklet.
There might be a race condition when tasklet is interrupted or another
cpu/core will run IRQ handler and tasklet in parallel.
Atomic bitops functions are now used instead of bitmask operations.
Signed-off-by: Dmitry Kasatkin <dmitry.kasatkin@nokia.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Flags mask cannot be used with atomic bit operations.
This patch changes masks to bit numbers.
Atomic bit operations will be used by following patches.
Signed-off-by: Dmitry Kasatkin <dmitry.kasatkin@nokia.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Request pointer is already available in the function.
Signed-off-by: Dmitry Kasatkin <dmitry.kasatkin@nokia.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
This patch fixes 2 hmac inter-dependent bugs.
1. "omap-sham: hash-in-progress is stored in hw format" commit introduced
optimization where temporary hash had been stored in OMAP specific format
(big endian).
For SHA1 it is different to real hash format, which is little endian.
Final HMAC value was calculated using incorrect hash.
Because CONFIG_CRYPTO_MANAGER_TESTS was disabled this error remained
unnoticed. After enabling this option, bug has been found.
2. HMAC was calculated using temporrary hash value.
For a single-request updates, temporary hash was the final one and
HMAC result was correct. But in fact only the final hash had to be used.
All crypto tests for HMAC produces only single request and
could not catch the problem. This problem is fixed here.
Signed-off-by: Dmitry Kasatkin <dmitry.kasatkin@nokia.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
EMU chips also have crypto HW as HS chips.
Signed-off-by: Dmitry Kasatkin <dmitry.kasatkin@nokia.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
SHA1 accelerator can also be busy. Add -EBUSY status return option and
return busy status from omap_sham_finup().
Signed-off-by: Markku Kylanpaa <ext-markku.kylanpaa@nokia.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
clk_get() returns a struct clk cookie to the driver and some platforms
may return NULL if they only support a single clock. clk_get() has only
failed if it returns a ERR_PTR() encoded pointer.
Signed-off-by: Jamie Iles <jamie@jamieiles.com>
Reviewed-by: Aaro Koskinen <aaro.koskinen@nokia.com>
Reviewed-by: Dmitry Kasatkin <dmitry.kasatkin@nokia.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Previous commit "removed redundant locking" introduced
a bug in handling backlog.
In certain cases, when async request complete callback will
call complete() on -EINPROGRESS code, it will cause uncompleted requests.
It does not happen in implementation similar to crypto test manager,
but it will happen in implementation similar to dm-crypt.
Backlog needs to be checked before dequeuing next request.
Signed-off-by: Dmitry Kasatkin <dmitry.kasatkin@nokia.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
If scatterlist have more than one entry, current driver uses
aligned buffer to copy data to to accelerator to tackle possible
issues with DMA and SHA buffer alignment.
This commit adds more intelligence to verify SG alignment and
possibility to use DMA directly on the data without using copy
buffer.
Signed-off-by: Dmitry Kasatkin <dmitry.kasatkin@nokia.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
bufcnt is 0 if it was no update requests before,
which is exact meaning of FLAGS_FIRST.
Signed-off-by: Dmitry Kasatkin <dmitry.kasatkin@nokia.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Hash-in-progress is now stored in hw format.
Only on final call, hash is converted to correct format.
Speedup copy procedure and will allow to use OMAP burst mode.
Signed-off-by: Dmitry Kasatkin <dmitry.kasatkin@nokia.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
According to the Herbert Xu, client may not always call
crypto_ahash_final().
In the case of error in hash calculation resources will be
automatically cleaned up.
But if no hash calculation error happens and client will not call
crypto_ahash_final() at all, then internal buffer will not be freed,
and clocks will not be disabled.
This patch provides support for atomic crypto_ahash_update() call.
Clocks are now enabled and disabled per update request.
Data buffer is now allocated as a part of request context.
Client is obligated to free it with crypto_free_ahash().
Signed-off-by: Dmitry Kasatkin <dmitry.kasatkin@nokia.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Locking for queuing and dequeuing is combined.
test_and_set_bit() is also replaced with checking under dd->lock.
Signed-off-by: Dmitry Kasatkin <dmitry.kasatkin@nokia.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Introduces DMA error handling.
DMA error is returned as a result code of the hash request.
Clients needs to handle error codes and may repeat hash calculation attempt.
Also in the case of DMA error, SHAM module is set to be re-initialized again.
It significantly improves stability against possible HW failures.
Signed-off-by: Dmitry Kasatkin <dmitry.kasatkin@nokia.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
DMA parameters for constant data were initialized during driver probe().
It seems that those settings sometimes are lost when devices goes to off mode.
This patch makes DMA initialization just before use.
It solves off mode problems.
Fixes: NB#202786 - Aegis & SHA1 block off mode changes
Signed-off-by: Dmitry Kasatkin <dmitry.kasatkin@nokia.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Currently driver storred digest results in req->results
provided by the client. But some clients do not set it
until final() call. It leads to crash.
Changed to use internal buffer to store temporary digest results.
Signed-off-by: Dmitry Kasatkin <dmitry.kasatkin@nokia.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
DMA is set to use burst mode also for source channel. It should
descrease memory bandwidth needs.
DMA synchronization is set to use prefetch mechanism. SHAM block is behind
L4 bus and it doesn't have fifo. SHAM block is stalling as long as the new
data is available. It takes time to fetch data from memory and transfer it
via L4 bus. With prefetch enabled, data is waiting in DMA fifo and SHAM block
receives new data block faster.
This increases SHA processing speed up to 30 percent depending on the
bus / memory load.
Signed-off-by: Samu Onkalo <samu.p.onkalo@nokia.com>
Signed-off-by: Dmitry Kasatkin <dmitry.kasatkin@nokia.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Earlier kernel contained omap sha1 and md5 driver, which was not maintained,
was not ported to new crypto APIs and removed from the source tree.
- implements async crypto API using dma and cpu.
- supports multiple sham instances if available
- hmac
- concurrent requests
Signed-off-by: Dmitry Kasatkin <dmitry.kasatkin@nokia.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Joel A Fernandes