linux/fs/cifs/cifs_unicode.c

612 lines
16 KiB
C
Raw Normal View History

/*
* fs/cifs/cifs_unicode.c
*
* Copyright (c) International Business Machines Corp., 2000,2009
* Modified by Steve French (sfrench@us.ibm.com)
*
* This program is free software; you can redistribute it and/or modify
* it under the terms of the GNU General Public License as published by
* the Free Software Foundation; either version 2 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See
* the GNU General Public License for more details.
*
* You should have received a copy of the GNU General Public License
* along with this program; if not, write to the Free Software
* Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
*/
#include <linux/fs.h>
include cleanup: Update gfp.h and slab.h includes to prepare for breaking implicit slab.h inclusion from percpu.h percpu.h is included by sched.h and module.h and thus ends up being included when building most .c files. percpu.h includes slab.h which in turn includes gfp.h making everything defined by the two files universally available and complicating inclusion dependencies. percpu.h -> slab.h dependency is about to be removed. Prepare for this change by updating users of gfp and slab facilities include those headers directly instead of assuming availability. As this conversion needs to touch large number of source files, the following script is used as the basis of conversion. http://userweb.kernel.org/~tj/misc/slabh-sweep.py The script does the followings. * Scan files for gfp and slab usages and update includes such that only the necessary includes are there. ie. if only gfp is used, gfp.h, if slab is used, slab.h. * When the script inserts a new include, it looks at the include blocks and try to put the new include such that its order conforms to its surrounding. It's put in the include block which contains core kernel includes, in the same order that the rest are ordered - alphabetical, Christmas tree, rev-Xmas-tree or at the end if there doesn't seem to be any matching order. * If the script can't find a place to put a new include (mostly because the file doesn't have fitting include block), it prints out an error message indicating which .h file needs to be added to the file. The conversion was done in the following steps. 1. The initial automatic conversion of all .c files updated slightly over 4000 files, deleting around 700 includes and adding ~480 gfp.h and ~3000 slab.h inclusions. The script emitted errors for ~400 files. 2. Each error was manually checked. Some didn't need the inclusion, some needed manual addition while adding it to implementation .h or embedding .c file was more appropriate for others. This step added inclusions to around 150 files. 3. The script was run again and the output was compared to the edits from #2 to make sure no file was left behind. 4. Several build tests were done and a couple of problems were fixed. e.g. lib/decompress_*.c used malloc/free() wrappers around slab APIs requiring slab.h to be added manually. 5. The script was run on all .h files but without automatically editing them as sprinkling gfp.h and slab.h inclusions around .h files could easily lead to inclusion dependency hell. Most gfp.h inclusion directives were ignored as stuff from gfp.h was usually wildly available and often used in preprocessor macros. Each slab.h inclusion directive was examined and added manually as necessary. 6. percpu.h was updated not to include slab.h. 7. Build test were done on the following configurations and failures were fixed. CONFIG_GCOV_KERNEL was turned off for all tests (as my distributed build env didn't work with gcov compiles) and a few more options had to be turned off depending on archs to make things build (like ipr on powerpc/64 which failed due to missing writeq). * x86 and x86_64 UP and SMP allmodconfig and a custom test config. * powerpc and powerpc64 SMP allmodconfig * sparc and sparc64 SMP allmodconfig * ia64 SMP allmodconfig * s390 SMP allmodconfig * alpha SMP allmodconfig * um on x86_64 SMP allmodconfig 8. percpu.h modifications were reverted so that it could be applied as a separate patch and serve as bisection point. Given the fact that I had only a couple of failures from tests on step 6, I'm fairly confident about the coverage of this conversion patch. If there is a breakage, it's likely to be something in one of the arch headers which should be easily discoverable easily on most builds of the specific arch. Signed-off-by: Tejun Heo <tj@kernel.org> Guess-its-ok-by: Christoph Lameter <cl@linux-foundation.org> Cc: Ingo Molnar <mingo@redhat.com> Cc: Lee Schermerhorn <Lee.Schermerhorn@hp.com>
2010-03-24 16:04:11 +08:00
#include <linux/slab.h>
Remap reserved posix characters by default (part 3/3) This is a bigger patch, but its size is mostly due to a single change for how we check for remapping illegal characters in file names - a lot of repeated, small changes to the way callers request converting file names. The final patch in the series does the following: 1) changes default behavior for cifs to be more intuitive. Currently we do not map by default to seven reserved characters, ie those valid in POSIX but not in NTFS/CIFS/SMB3/Windows, unless a mount option (mapchars) is specified. Change this to by default always map and map using the SFM maping (like the Mac uses) unless the server negotiates the CIFS Unix Extensions (like Samba does when mounting with the cifs protocol) when the remapping of the characters is unnecessary. This should help SMB3 mounts in particular since Samba will likely be able to implement this mapping with its new "vfs_fruit" module as it will be doing for the Mac. 2) if the user specifies the existing "mapchars" mount option then use the "SFU" (Microsoft Services for Unix, SUA) style mapping of the seven characters instead. 3) if the user specifies "nomapposix" then disable SFM/MAC style mapping (so no character remapping would be used unless the user specifies "mapchars" on mount as well, as above). 4) change all the places in the code that check for the superblock flag on the mount which is set by mapchars and passed in on all path based operation and change it to use a small function call instead to set the mapping type properly (and check for the mapping type in the cifs unicode functions) Signed-off-by: Steve French <smfrench@gmail.com>
2014-09-27 15:19:01 +08:00
#include "cifs_fs_sb.h"
#include "cifs_unicode.h"
#include "cifs_uniupr.h"
#include "cifspdu.h"
#include "cifsglob.h"
#include "cifs_debug.h"
Remap reserved posix characters by default (part 3/3) This is a bigger patch, but its size is mostly due to a single change for how we check for remapping illegal characters in file names - a lot of repeated, small changes to the way callers request converting file names. The final patch in the series does the following: 1) changes default behavior for cifs to be more intuitive. Currently we do not map by default to seven reserved characters, ie those valid in POSIX but not in NTFS/CIFS/SMB3/Windows, unless a mount option (mapchars) is specified. Change this to by default always map and map using the SFM maping (like the Mac uses) unless the server negotiates the CIFS Unix Extensions (like Samba does when mounting with the cifs protocol) when the remapping of the characters is unnecessary. This should help SMB3 mounts in particular since Samba will likely be able to implement this mapping with its new "vfs_fruit" module as it will be doing for the Mac. 2) if the user specifies the existing "mapchars" mount option then use the "SFU" (Microsoft Services for Unix, SUA) style mapping of the seven characters instead. 3) if the user specifies "nomapposix" then disable SFM/MAC style mapping (so no character remapping would be used unless the user specifies "mapchars" on mount as well, as above). 4) change all the places in the code that check for the superblock flag on the mount which is set by mapchars and passed in on all path based operation and change it to use a small function call instead to set the mapping type properly (and check for the mapping type in the cifs unicode functions) Signed-off-by: Steve French <smfrench@gmail.com>
2014-09-27 15:19:01 +08:00
int cifs_remap(struct cifs_sb_info *cifs_sb)
{
int map_type;
if (cifs_sb->mnt_cifs_flags & CIFS_MOUNT_MAP_SFM_CHR)
map_type = SFM_MAP_UNI_RSVD;
else if (cifs_sb->mnt_cifs_flags & CIFS_MOUNT_MAP_SPECIAL_CHR)
map_type = SFU_MAP_UNI_RSVD;
else
map_type = NO_MAP_UNI_RSVD;
return map_type;
}
Allow conversion of characters in Mac remap range. Part 1 This allows directory listings to Mac to display filenames correctly which have been created with illegal (to Windows) characters in their filename. It does not allow converting the other direction yet ie opening files with these characters (followon patch). There are seven reserved characters that need to be remapped when mounting to Windows, Mac (or any server without Unix Extensions) which are valid in POSIX but not in the other OS. : \ < > ? * | We used the normal UCS-2 remap range for this in order to convert this to/from UTF8 as did Windows Services for Unix (basically add 0xF000 to any of the 7 reserved characters), at least when the "mapchars" mount option was specified. Mac used a very slightly different "Services for Mac" remap range 0xF021 through 0xF027. The attached patch allows cifs.ko (the kernel client) to read directories on macs containing files with these characters and display their names properly. In theory this even might be useful on mounts to Samba when the vfs_catia or new "vfs_fruit" module is loaded. Currently the 7 reserved characters look very strange in directory listings from cifs.ko to Mac server. This patch allows these file name characters to be read (requires specifying mapchars on mount). Two additional changes are needed: 1) Make it more automatic: a way of detecting enough info so that we know to try to always remap these characters or not. Various have suggested that the SFM approach be made the default when the server does not support POSIX Unix extensions (cifs mounts to Samba for example) so need to make SFM remapping the default unless mapchars (SFU style mapping) specified on mount or no mapping explicitly requested or no mapping needed (cifs mounts to Samba). 2) Adding a patch to map the characters the other direction (ie UTF-8 to UCS-2 on open). This patch does it for translating readdir entries (ie UCS-2 to UTF-8) Signed-off-by: Steve French <smfrench@gmail.com> Reviewed-by: Shirish Pargaonkar <shirishpargaonkar@gmail.com>
2014-09-26 02:20:05 +08:00
/* Convert character using the SFU - "Services for Unix" remapping range */
static bool
convert_sfu_char(const __u16 src_char, char *target)
{
/*
* BB: Cannot handle remapping UNI_SLASH until all the calls to
* build_path_from_dentry are modified, as they use slash as
* separator.
*/
switch (src_char) {
case UNI_COLON:
*target = ':';
break;
case UNI_ASTERISK:
*target = '*';
break;
case UNI_QUESTION:
*target = '?';
break;
case UNI_PIPE:
*target = '|';
break;
case UNI_GRTRTHAN:
*target = '>';
break;
case UNI_LESSTHAN:
*target = '<';
break;
default:
Allow conversion of characters in Mac remap range. Part 1 This allows directory listings to Mac to display filenames correctly which have been created with illegal (to Windows) characters in their filename. It does not allow converting the other direction yet ie opening files with these characters (followon patch). There are seven reserved characters that need to be remapped when mounting to Windows, Mac (or any server without Unix Extensions) which are valid in POSIX but not in the other OS. : \ < > ? * | We used the normal UCS-2 remap range for this in order to convert this to/from UTF8 as did Windows Services for Unix (basically add 0xF000 to any of the 7 reserved characters), at least when the "mapchars" mount option was specified. Mac used a very slightly different "Services for Mac" remap range 0xF021 through 0xF027. The attached patch allows cifs.ko (the kernel client) to read directories on macs containing files with these characters and display their names properly. In theory this even might be useful on mounts to Samba when the vfs_catia or new "vfs_fruit" module is loaded. Currently the 7 reserved characters look very strange in directory listings from cifs.ko to Mac server. This patch allows these file name characters to be read (requires specifying mapchars on mount). Two additional changes are needed: 1) Make it more automatic: a way of detecting enough info so that we know to try to always remap these characters or not. Various have suggested that the SFM approach be made the default when the server does not support POSIX Unix extensions (cifs mounts to Samba for example) so need to make SFM remapping the default unless mapchars (SFU style mapping) specified on mount or no mapping explicitly requested or no mapping needed (cifs mounts to Samba). 2) Adding a patch to map the characters the other direction (ie UTF-8 to UCS-2 on open). This patch does it for translating readdir entries (ie UCS-2 to UTF-8) Signed-off-by: Steve French <smfrench@gmail.com> Reviewed-by: Shirish Pargaonkar <shirishpargaonkar@gmail.com>
2014-09-26 02:20:05 +08:00
return false;
}
Allow conversion of characters in Mac remap range. Part 1 This allows directory listings to Mac to display filenames correctly which have been created with illegal (to Windows) characters in their filename. It does not allow converting the other direction yet ie opening files with these characters (followon patch). There are seven reserved characters that need to be remapped when mounting to Windows, Mac (or any server without Unix Extensions) which are valid in POSIX but not in the other OS. : \ < > ? * | We used the normal UCS-2 remap range for this in order to convert this to/from UTF8 as did Windows Services for Unix (basically add 0xF000 to any of the 7 reserved characters), at least when the "mapchars" mount option was specified. Mac used a very slightly different "Services for Mac" remap range 0xF021 through 0xF027. The attached patch allows cifs.ko (the kernel client) to read directories on macs containing files with these characters and display their names properly. In theory this even might be useful on mounts to Samba when the vfs_catia or new "vfs_fruit" module is loaded. Currently the 7 reserved characters look very strange in directory listings from cifs.ko to Mac server. This patch allows these file name characters to be read (requires specifying mapchars on mount). Two additional changes are needed: 1) Make it more automatic: a way of detecting enough info so that we know to try to always remap these characters or not. Various have suggested that the SFM approach be made the default when the server does not support POSIX Unix extensions (cifs mounts to Samba for example) so need to make SFM remapping the default unless mapchars (SFU style mapping) specified on mount or no mapping explicitly requested or no mapping needed (cifs mounts to Samba). 2) Adding a patch to map the characters the other direction (ie UTF-8 to UCS-2 on open). This patch does it for translating readdir entries (ie UCS-2 to UTF-8) Signed-off-by: Steve French <smfrench@gmail.com> Reviewed-by: Shirish Pargaonkar <shirishpargaonkar@gmail.com>
2014-09-26 02:20:05 +08:00
return true;
}
Allow conversion of characters in Mac remap range. Part 1 This allows directory listings to Mac to display filenames correctly which have been created with illegal (to Windows) characters in their filename. It does not allow converting the other direction yet ie opening files with these characters (followon patch). There are seven reserved characters that need to be remapped when mounting to Windows, Mac (or any server without Unix Extensions) which are valid in POSIX but not in the other OS. : \ < > ? * | We used the normal UCS-2 remap range for this in order to convert this to/from UTF8 as did Windows Services for Unix (basically add 0xF000 to any of the 7 reserved characters), at least when the "mapchars" mount option was specified. Mac used a very slightly different "Services for Mac" remap range 0xF021 through 0xF027. The attached patch allows cifs.ko (the kernel client) to read directories on macs containing files with these characters and display their names properly. In theory this even might be useful on mounts to Samba when the vfs_catia or new "vfs_fruit" module is loaded. Currently the 7 reserved characters look very strange in directory listings from cifs.ko to Mac server. This patch allows these file name characters to be read (requires specifying mapchars on mount). Two additional changes are needed: 1) Make it more automatic: a way of detecting enough info so that we know to try to always remap these characters or not. Various have suggested that the SFM approach be made the default when the server does not support POSIX Unix extensions (cifs mounts to Samba for example) so need to make SFM remapping the default unless mapchars (SFU style mapping) specified on mount or no mapping explicitly requested or no mapping needed (cifs mounts to Samba). 2) Adding a patch to map the characters the other direction (ie UTF-8 to UCS-2 on open). This patch does it for translating readdir entries (ie UCS-2 to UTF-8) Signed-off-by: Steve French <smfrench@gmail.com> Reviewed-by: Shirish Pargaonkar <shirishpargaonkar@gmail.com>
2014-09-26 02:20:05 +08:00
/* Convert character using the SFM - "Services for Mac" remapping range */
static bool
convert_sfm_char(const __u16 src_char, char *target)
{
switch (src_char) {
case SFM_COLON:
*target = ':';
break;
case SFM_ASTERISK:
*target = '*';
break;
case SFM_QUESTION:
*target = '?';
break;
case SFM_PIPE:
*target = '|';
break;
case SFM_GRTRTHAN:
*target = '>';
break;
case SFM_LESSTHAN:
*target = '<';
break;
case SFM_SLASH:
*target = '\\';
break;
default:
return false;
}
return true;
}
/*
* cifs_mapchar - convert a host-endian char to proper char in codepage
* @target - where converted character should be copied
* @src_char - 2 byte host-endian source character
* @cp - codepage to which character should be converted
* @map_type - How should the 7 NTFS/SMB reserved characters be mapped to UCS2?
*
* This function handles the conversion of a single character. It is the
* responsibility of the caller to ensure that the target buffer is large
* enough to hold the result of the conversion (at least NLS_MAX_CHARSET_SIZE).
*/
static int
cifs_mapchar(char *target, const __u16 *from, const struct nls_table *cp,
Allow conversion of characters in Mac remap range. Part 1 This allows directory listings to Mac to display filenames correctly which have been created with illegal (to Windows) characters in their filename. It does not allow converting the other direction yet ie opening files with these characters (followon patch). There are seven reserved characters that need to be remapped when mounting to Windows, Mac (or any server without Unix Extensions) which are valid in POSIX but not in the other OS. : \ < > ? * | We used the normal UCS-2 remap range for this in order to convert this to/from UTF8 as did Windows Services for Unix (basically add 0xF000 to any of the 7 reserved characters), at least when the "mapchars" mount option was specified. Mac used a very slightly different "Services for Mac" remap range 0xF021 through 0xF027. The attached patch allows cifs.ko (the kernel client) to read directories on macs containing files with these characters and display their names properly. In theory this even might be useful on mounts to Samba when the vfs_catia or new "vfs_fruit" module is loaded. Currently the 7 reserved characters look very strange in directory listings from cifs.ko to Mac server. This patch allows these file name characters to be read (requires specifying mapchars on mount). Two additional changes are needed: 1) Make it more automatic: a way of detecting enough info so that we know to try to always remap these characters or not. Various have suggested that the SFM approach be made the default when the server does not support POSIX Unix extensions (cifs mounts to Samba for example) so need to make SFM remapping the default unless mapchars (SFU style mapping) specified on mount or no mapping explicitly requested or no mapping needed (cifs mounts to Samba). 2) Adding a patch to map the characters the other direction (ie UTF-8 to UCS-2 on open). This patch does it for translating readdir entries (ie UCS-2 to UTF-8) Signed-off-by: Steve French <smfrench@gmail.com> Reviewed-by: Shirish Pargaonkar <shirishpargaonkar@gmail.com>
2014-09-26 02:20:05 +08:00
int maptype)
{
int len = 1;
__u16 src_char;
src_char = *from;
Allow conversion of characters in Mac remap range. Part 1 This allows directory listings to Mac to display filenames correctly which have been created with illegal (to Windows) characters in their filename. It does not allow converting the other direction yet ie opening files with these characters (followon patch). There are seven reserved characters that need to be remapped when mounting to Windows, Mac (or any server without Unix Extensions) which are valid in POSIX but not in the other OS. : \ < > ? * | We used the normal UCS-2 remap range for this in order to convert this to/from UTF8 as did Windows Services for Unix (basically add 0xF000 to any of the 7 reserved characters), at least when the "mapchars" mount option was specified. Mac used a very slightly different "Services for Mac" remap range 0xF021 through 0xF027. The attached patch allows cifs.ko (the kernel client) to read directories on macs containing files with these characters and display their names properly. In theory this even might be useful on mounts to Samba when the vfs_catia or new "vfs_fruit" module is loaded. Currently the 7 reserved characters look very strange in directory listings from cifs.ko to Mac server. This patch allows these file name characters to be read (requires specifying mapchars on mount). Two additional changes are needed: 1) Make it more automatic: a way of detecting enough info so that we know to try to always remap these characters or not. Various have suggested that the SFM approach be made the default when the server does not support POSIX Unix extensions (cifs mounts to Samba for example) so need to make SFM remapping the default unless mapchars (SFU style mapping) specified on mount or no mapping explicitly requested or no mapping needed (cifs mounts to Samba). 2) Adding a patch to map the characters the other direction (ie UTF-8 to UCS-2 on open). This patch does it for translating readdir entries (ie UCS-2 to UTF-8) Signed-off-by: Steve French <smfrench@gmail.com> Reviewed-by: Shirish Pargaonkar <shirishpargaonkar@gmail.com>
2014-09-26 02:20:05 +08:00
if ((maptype == SFM_MAP_UNI_RSVD) && convert_sfm_char(src_char, target))
return len;
else if ((maptype == SFU_MAP_UNI_RSVD) &&
convert_sfu_char(src_char, target))
return len;
Allow conversion of characters in Mac remap range. Part 1 This allows directory listings to Mac to display filenames correctly which have been created with illegal (to Windows) characters in their filename. It does not allow converting the other direction yet ie opening files with these characters (followon patch). There are seven reserved characters that need to be remapped when mounting to Windows, Mac (or any server without Unix Extensions) which are valid in POSIX but not in the other OS. : \ < > ? * | We used the normal UCS-2 remap range for this in order to convert this to/from UTF8 as did Windows Services for Unix (basically add 0xF000 to any of the 7 reserved characters), at least when the "mapchars" mount option was specified. Mac used a very slightly different "Services for Mac" remap range 0xF021 through 0xF027. The attached patch allows cifs.ko (the kernel client) to read directories on macs containing files with these characters and display their names properly. In theory this even might be useful on mounts to Samba when the vfs_catia or new "vfs_fruit" module is loaded. Currently the 7 reserved characters look very strange in directory listings from cifs.ko to Mac server. This patch allows these file name characters to be read (requires specifying mapchars on mount). Two additional changes are needed: 1) Make it more automatic: a way of detecting enough info so that we know to try to always remap these characters or not. Various have suggested that the SFM approach be made the default when the server does not support POSIX Unix extensions (cifs mounts to Samba for example) so need to make SFM remapping the default unless mapchars (SFU style mapping) specified on mount or no mapping explicitly requested or no mapping needed (cifs mounts to Samba). 2) Adding a patch to map the characters the other direction (ie UTF-8 to UCS-2 on open). This patch does it for translating readdir entries (ie UCS-2 to UTF-8) Signed-off-by: Steve French <smfrench@gmail.com> Reviewed-by: Shirish Pargaonkar <shirishpargaonkar@gmail.com>
2014-09-26 02:20:05 +08:00
/* if character not one of seven in special remap set */
len = cp->uni2char(src_char, target, NLS_MAX_CHARSET_SIZE);
if (len <= 0)
goto surrogate_pair;
return len;
surrogate_pair:
/* convert SURROGATE_PAIR and IVS */
if (strcmp(cp->charset, "utf8"))
goto unknown;
len = utf16s_to_utf8s(from, 3, UTF16_LITTLE_ENDIAN, target, 6);
if (len <= 0)
goto unknown;
return len;
unknown:
*target = '?';
len = 1;
Allow conversion of characters in Mac remap range. Part 1 This allows directory listings to Mac to display filenames correctly which have been created with illegal (to Windows) characters in their filename. It does not allow converting the other direction yet ie opening files with these characters (followon patch). There are seven reserved characters that need to be remapped when mounting to Windows, Mac (or any server without Unix Extensions) which are valid in POSIX but not in the other OS. : \ < > ? * | We used the normal UCS-2 remap range for this in order to convert this to/from UTF8 as did Windows Services for Unix (basically add 0xF000 to any of the 7 reserved characters), at least when the "mapchars" mount option was specified. Mac used a very slightly different "Services for Mac" remap range 0xF021 through 0xF027. The attached patch allows cifs.ko (the kernel client) to read directories on macs containing files with these characters and display their names properly. In theory this even might be useful on mounts to Samba when the vfs_catia or new "vfs_fruit" module is loaded. Currently the 7 reserved characters look very strange in directory listings from cifs.ko to Mac server. This patch allows these file name characters to be read (requires specifying mapchars on mount). Two additional changes are needed: 1) Make it more automatic: a way of detecting enough info so that we know to try to always remap these characters or not. Various have suggested that the SFM approach be made the default when the server does not support POSIX Unix extensions (cifs mounts to Samba for example) so need to make SFM remapping the default unless mapchars (SFU style mapping) specified on mount or no mapping explicitly requested or no mapping needed (cifs mounts to Samba). 2) Adding a patch to map the characters the other direction (ie UTF-8 to UCS-2 on open). This patch does it for translating readdir entries (ie UCS-2 to UTF-8) Signed-off-by: Steve French <smfrench@gmail.com> Reviewed-by: Shirish Pargaonkar <shirishpargaonkar@gmail.com>
2014-09-26 02:20:05 +08:00
return len;
}
/*
* cifs_from_utf16 - convert utf16le string to local charset
* @to - destination buffer
* @from - source buffer
* @tolen - destination buffer size (in bytes)
* @fromlen - source buffer size (in bytes)
* @codepage - codepage to which characters should be converted
* @mapchar - should characters be remapped according to the mapchars option?
*
* Convert a little-endian utf16le string (as sent by the server) to a string
* in the provided codepage. The tolen and fromlen parameters are to ensure
* that the code doesn't walk off of the end of the buffer (which is always
* a danger if the alignment of the source buffer is off). The destination
* string is always properly null terminated and fits in the destination
* buffer. Returns the length of the destination string in bytes (including
* null terminator).
*
* Note that some windows versions actually send multiword UTF-16 characters
* instead of straight UTF16-2. The linux nls routines however aren't able to
* deal with those characters properly. In the event that we get some of
* those characters, they won't be translated properly.
*/
int
cifs_from_utf16(char *to, const __le16 *from, int tolen, int fromlen,
Allow conversion of characters in Mac remap range. Part 1 This allows directory listings to Mac to display filenames correctly which have been created with illegal (to Windows) characters in their filename. It does not allow converting the other direction yet ie opening files with these characters (followon patch). There are seven reserved characters that need to be remapped when mounting to Windows, Mac (or any server without Unix Extensions) which are valid in POSIX but not in the other OS. : \ < > ? * | We used the normal UCS-2 remap range for this in order to convert this to/from UTF8 as did Windows Services for Unix (basically add 0xF000 to any of the 7 reserved characters), at least when the "mapchars" mount option was specified. Mac used a very slightly different "Services for Mac" remap range 0xF021 through 0xF027. The attached patch allows cifs.ko (the kernel client) to read directories on macs containing files with these characters and display their names properly. In theory this even might be useful on mounts to Samba when the vfs_catia or new "vfs_fruit" module is loaded. Currently the 7 reserved characters look very strange in directory listings from cifs.ko to Mac server. This patch allows these file name characters to be read (requires specifying mapchars on mount). Two additional changes are needed: 1) Make it more automatic: a way of detecting enough info so that we know to try to always remap these characters or not. Various have suggested that the SFM approach be made the default when the server does not support POSIX Unix extensions (cifs mounts to Samba for example) so need to make SFM remapping the default unless mapchars (SFU style mapping) specified on mount or no mapping explicitly requested or no mapping needed (cifs mounts to Samba). 2) Adding a patch to map the characters the other direction (ie UTF-8 to UCS-2 on open). This patch does it for translating readdir entries (ie UCS-2 to UTF-8) Signed-off-by: Steve French <smfrench@gmail.com> Reviewed-by: Shirish Pargaonkar <shirishpargaonkar@gmail.com>
2014-09-26 02:20:05 +08:00
const struct nls_table *codepage, int map_type)
{
int i, charlen, safelen;
int outlen = 0;
int nullsize = nls_nullsize(codepage);
int fromwords = fromlen / 2;
char tmp[NLS_MAX_CHARSET_SIZE];
__u16 ftmp[3]; /* ftmp[3] = 3array x 2bytes = 6bytes UTF-16 */
/*
* because the chars can be of varying widths, we need to take care
* not to overflow the destination buffer when we get close to the
* end of it. Until we get to this offset, we don't need to check
* for overflow however.
*/
safelen = tolen - (NLS_MAX_CHARSET_SIZE + nullsize);
for (i = 0; i < fromwords; i++) {
ftmp[0] = get_unaligned_le16(&from[i]);
if (ftmp[0] == 0)
break;
if (i + 1 < fromwords)
ftmp[1] = get_unaligned_le16(&from[i + 1]);
else
ftmp[1] = 0;
if (i + 2 < fromwords)
ftmp[2] = get_unaligned_le16(&from[i + 2]);
else
ftmp[2] = 0;
/*
* check to see if converting this character might make the
* conversion bleed into the null terminator
*/
if (outlen >= safelen) {
Allow conversion of characters in Mac remap range. Part 1 This allows directory listings to Mac to display filenames correctly which have been created with illegal (to Windows) characters in their filename. It does not allow converting the other direction yet ie opening files with these characters (followon patch). There are seven reserved characters that need to be remapped when mounting to Windows, Mac (or any server without Unix Extensions) which are valid in POSIX but not in the other OS. : \ < > ? * | We used the normal UCS-2 remap range for this in order to convert this to/from UTF8 as did Windows Services for Unix (basically add 0xF000 to any of the 7 reserved characters), at least when the "mapchars" mount option was specified. Mac used a very slightly different "Services for Mac" remap range 0xF021 through 0xF027. The attached patch allows cifs.ko (the kernel client) to read directories on macs containing files with these characters and display their names properly. In theory this even might be useful on mounts to Samba when the vfs_catia or new "vfs_fruit" module is loaded. Currently the 7 reserved characters look very strange in directory listings from cifs.ko to Mac server. This patch allows these file name characters to be read (requires specifying mapchars on mount). Two additional changes are needed: 1) Make it more automatic: a way of detecting enough info so that we know to try to always remap these characters or not. Various have suggested that the SFM approach be made the default when the server does not support POSIX Unix extensions (cifs mounts to Samba for example) so need to make SFM remapping the default unless mapchars (SFU style mapping) specified on mount or no mapping explicitly requested or no mapping needed (cifs mounts to Samba). 2) Adding a patch to map the characters the other direction (ie UTF-8 to UCS-2 on open). This patch does it for translating readdir entries (ie UCS-2 to UTF-8) Signed-off-by: Steve French <smfrench@gmail.com> Reviewed-by: Shirish Pargaonkar <shirishpargaonkar@gmail.com>
2014-09-26 02:20:05 +08:00
charlen = cifs_mapchar(tmp, ftmp, codepage, map_type);
if ((outlen + charlen) > (tolen - nullsize))
break;
}
/* put converted char into 'to' buffer */
Allow conversion of characters in Mac remap range. Part 1 This allows directory listings to Mac to display filenames correctly which have been created with illegal (to Windows) characters in their filename. It does not allow converting the other direction yet ie opening files with these characters (followon patch). There are seven reserved characters that need to be remapped when mounting to Windows, Mac (or any server without Unix Extensions) which are valid in POSIX but not in the other OS. : \ < > ? * | We used the normal UCS-2 remap range for this in order to convert this to/from UTF8 as did Windows Services for Unix (basically add 0xF000 to any of the 7 reserved characters), at least when the "mapchars" mount option was specified. Mac used a very slightly different "Services for Mac" remap range 0xF021 through 0xF027. The attached patch allows cifs.ko (the kernel client) to read directories on macs containing files with these characters and display their names properly. In theory this even might be useful on mounts to Samba when the vfs_catia or new "vfs_fruit" module is loaded. Currently the 7 reserved characters look very strange in directory listings from cifs.ko to Mac server. This patch allows these file name characters to be read (requires specifying mapchars on mount). Two additional changes are needed: 1) Make it more automatic: a way of detecting enough info so that we know to try to always remap these characters or not. Various have suggested that the SFM approach be made the default when the server does not support POSIX Unix extensions (cifs mounts to Samba for example) so need to make SFM remapping the default unless mapchars (SFU style mapping) specified on mount or no mapping explicitly requested or no mapping needed (cifs mounts to Samba). 2) Adding a patch to map the characters the other direction (ie UTF-8 to UCS-2 on open). This patch does it for translating readdir entries (ie UCS-2 to UTF-8) Signed-off-by: Steve French <smfrench@gmail.com> Reviewed-by: Shirish Pargaonkar <shirishpargaonkar@gmail.com>
2014-09-26 02:20:05 +08:00
charlen = cifs_mapchar(&to[outlen], ftmp, codepage, map_type);
outlen += charlen;
/* charlen (=bytes of UTF-8 for 1 character)
* 4bytes UTF-8(surrogate pair) is charlen=4
* (4bytes UTF-16 code)
* 7-8bytes UTF-8(IVS) is charlen=3+4 or 4+4
* (2 UTF-8 pairs divided to 2 UTF-16 pairs) */
if (charlen == 4)
i++;
else if (charlen >= 5)
/* 5-6bytes UTF-8 */
i += 2;
}
/* properly null-terminate string */
for (i = 0; i < nullsize; i++)
to[outlen++] = 0;
return outlen;
}
/*
* NAME: cifs_strtoUTF16()
*
* FUNCTION: Convert character string to unicode string
*
*/
int
cifs_strtoUTF16(__le16 *to, const char *from, int len,
const struct nls_table *codepage)
{
int charlen;
int i;
wchar_t wchar_to; /* needed to quiet sparse */
/* special case for utf8 to handle no plane0 chars */
if (!strcmp(codepage->charset, "utf8")) {
/*
* convert utf8 -> utf16, we assume we have enough space
* as caller should have assumed conversion does not overflow
* in destination len is length in wchar_t units (16bits)
*/
i = utf8s_to_utf16s(from, len, UTF16_LITTLE_ENDIAN,
(wchar_t *) to, len);
/* if success terminate and exit */
if (i >= 0)
goto success;
/*
* if fails fall back to UCS encoding as this
* function should not return negative values
* currently can fail only if source contains
* invalid encoded characters
*/
}
for (i = 0; len && *from; i++, from += charlen, len -= charlen) {
charlen = codepage->char2uni(from, len, &wchar_to);
if (charlen < 1) {
cifs_dbg(VFS, "strtoUTF16: char2uni of 0x%x returned %d\n",
*from, charlen);
/* A question mark */
wchar_to = 0x003f;
charlen = 1;
}
put_unaligned_le16(wchar_to, &to[i]);
}
success:
put_unaligned_le16(0, &to[i]);
return i;
}
/*
* cifs_utf16_bytes - how long will a string be after conversion?
* @utf16 - pointer to input string
* @maxbytes - don't go past this many bytes of input string
* @codepage - destination codepage
*
* Walk a utf16le string and return the number of bytes that the string will
* be after being converted to the given charset, not including any null
* termination required. Don't walk past maxbytes in the source buffer.
*/
int
cifs_utf16_bytes(const __le16 *from, int maxbytes,
const struct nls_table *codepage)
{
int i;
int charlen, outlen = 0;
int maxwords = maxbytes / 2;
char tmp[NLS_MAX_CHARSET_SIZE];
__u16 ftmp[3];
for (i = 0; i < maxwords; i++) {
ftmp[0] = get_unaligned_le16(&from[i]);
if (ftmp[0] == 0)
break;
if (i + 1 < maxwords)
ftmp[1] = get_unaligned_le16(&from[i + 1]);
else
ftmp[1] = 0;
if (i + 2 < maxwords)
ftmp[2] = get_unaligned_le16(&from[i + 2]);
else
ftmp[2] = 0;
charlen = cifs_mapchar(tmp, ftmp, codepage, NO_MAP_UNI_RSVD);
outlen += charlen;
}
return outlen;
}
/*
* cifs_strndup_from_utf16 - copy a string from wire format to the local
* codepage
* @src - source string
* @maxlen - don't walk past this many bytes in the source string
* @is_unicode - is this a unicode string?
* @codepage - destination codepage
*
* Take a string given by the server, convert it to the local codepage and
* put it in a new buffer. Returns a pointer to the new string or NULL on
* error.
*/
char *
cifs_strndup_from_utf16(const char *src, const int maxlen,
const bool is_unicode, const struct nls_table *codepage)
{
int len;
char *dst;
if (is_unicode) {
len = cifs_utf16_bytes((__le16 *) src, maxlen, codepage);
len += nls_nullsize(codepage);
dst = kmalloc(len, GFP_KERNEL);
if (!dst)
return NULL;
cifs_from_utf16(dst, (__le16 *) src, len, maxlen, codepage,
Allow conversion of characters in Mac remap range. Part 1 This allows directory listings to Mac to display filenames correctly which have been created with illegal (to Windows) characters in their filename. It does not allow converting the other direction yet ie opening files with these characters (followon patch). There are seven reserved characters that need to be remapped when mounting to Windows, Mac (or any server without Unix Extensions) which are valid in POSIX but not in the other OS. : \ < > ? * | We used the normal UCS-2 remap range for this in order to convert this to/from UTF8 as did Windows Services for Unix (basically add 0xF000 to any of the 7 reserved characters), at least when the "mapchars" mount option was specified. Mac used a very slightly different "Services for Mac" remap range 0xF021 through 0xF027. The attached patch allows cifs.ko (the kernel client) to read directories on macs containing files with these characters and display their names properly. In theory this even might be useful on mounts to Samba when the vfs_catia or new "vfs_fruit" module is loaded. Currently the 7 reserved characters look very strange in directory listings from cifs.ko to Mac server. This patch allows these file name characters to be read (requires specifying mapchars on mount). Two additional changes are needed: 1) Make it more automatic: a way of detecting enough info so that we know to try to always remap these characters or not. Various have suggested that the SFM approach be made the default when the server does not support POSIX Unix extensions (cifs mounts to Samba for example) so need to make SFM remapping the default unless mapchars (SFU style mapping) specified on mount or no mapping explicitly requested or no mapping needed (cifs mounts to Samba). 2) Adding a patch to map the characters the other direction (ie UTF-8 to UCS-2 on open). This patch does it for translating readdir entries (ie UCS-2 to UTF-8) Signed-off-by: Steve French <smfrench@gmail.com> Reviewed-by: Shirish Pargaonkar <shirishpargaonkar@gmail.com>
2014-09-26 02:20:05 +08:00
NO_MAP_UNI_RSVD);
} else {
len = strnlen(src, maxlen);
len++;
dst = kmalloc(len, GFP_KERNEL);
if (!dst)
return NULL;
strlcpy(dst, src, len);
}
return dst;
}
static __le16 convert_to_sfu_char(char src_char)
{
__le16 dest_char;
switch (src_char) {
case ':':
dest_char = cpu_to_le16(UNI_COLON);
break;
case '*':
dest_char = cpu_to_le16(UNI_ASTERISK);
break;
case '?':
dest_char = cpu_to_le16(UNI_QUESTION);
break;
case '<':
dest_char = cpu_to_le16(UNI_LESSTHAN);
break;
case '>':
dest_char = cpu_to_le16(UNI_GRTRTHAN);
break;
case '|':
dest_char = cpu_to_le16(UNI_PIPE);
break;
default:
dest_char = 0;
}
return dest_char;
}
static __le16 convert_to_sfm_char(char src_char)
{
__le16 dest_char;
switch (src_char) {
case ':':
dest_char = cpu_to_le16(SFM_COLON);
break;
case '*':
dest_char = cpu_to_le16(SFM_ASTERISK);
break;
case '?':
dest_char = cpu_to_le16(SFM_QUESTION);
break;
case '<':
dest_char = cpu_to_le16(SFM_LESSTHAN);
break;
case '>':
dest_char = cpu_to_le16(SFM_GRTRTHAN);
break;
case '|':
dest_char = cpu_to_le16(SFM_PIPE);
break;
default:
dest_char = 0;
}
return dest_char;
}
/*
* Convert 16 bit Unicode pathname to wire format from string in current code
* page. Conversion may involve remapping up the six characters that are
* only legal in POSIX-like OS (if they are present in the string). Path
* names are little endian 16 bit Unicode on the wire
*/
int
cifsConvertToUTF16(__le16 *target, const char *source, int srclen,
const struct nls_table *cp, int map_chars)
{
int i, charlen;
int j = 0;
char src_char;
__le16 dst_char;
wchar_t tmp;
wchar_t *wchar_to; /* UTF-16 */
int ret;
unicode_t u;
if (map_chars == NO_MAP_UNI_RSVD)
return cifs_strtoUTF16(target, source, PATH_MAX, cp);
wchar_to = kzalloc(6, GFP_KERNEL);
for (i = 0; i < srclen; j++) {
src_char = source[i];
charlen = 1;
/* check if end of string */
if (src_char == 0)
goto ctoUTF16_out;
/* see if we must remap this char */
if (map_chars == SFU_MAP_UNI_RSVD)
dst_char = convert_to_sfu_char(src_char);
else if (map_chars == SFM_MAP_UNI_RSVD)
dst_char = convert_to_sfm_char(src_char);
else
dst_char = 0;
/*
* FIXME: We can not handle remapping backslash (UNI_SLASH)
* until all the calls to build_path_from_dentry are modified,
* as they use backslash as separator.
*/
if (dst_char == 0) {
charlen = cp->char2uni(source + i, srclen - i, &tmp);
dst_char = cpu_to_le16(tmp);
/*
* if no match, use question mark, which at least in
* some cases serves as wild card
*/
if (charlen > 0)
goto ctoUTF16;
/* convert SURROGATE_PAIR */
if (strcmp(cp->charset, "utf8") || !wchar_to)
goto unknown;
if (*(source + i) & 0x80) {
charlen = utf8_to_utf32(source + i, 6, &u);
if (charlen < 0)
goto unknown;
} else
goto unknown;
ret = utf8s_to_utf16s(source + i, charlen,
UTF16_LITTLE_ENDIAN,
wchar_to, 6);
if (ret < 0)
goto unknown;
i += charlen;
dst_char = cpu_to_le16(*wchar_to);
if (charlen <= 3)
/* 1-3bytes UTF-8 to 2bytes UTF-16 */
put_unaligned(dst_char, &target[j]);
else if (charlen == 4) {
/* 4bytes UTF-8(surrogate pair) to 4bytes UTF-16
* 7-8bytes UTF-8(IVS) divided to 2 UTF-16
* (charlen=3+4 or 4+4) */
put_unaligned(dst_char, &target[j]);
dst_char = cpu_to_le16(*(wchar_to + 1));
j++;
put_unaligned(dst_char, &target[j]);
} else if (charlen >= 5) {
/* 5-6bytes UTF-8 to 6bytes UTF-16 */
put_unaligned(dst_char, &target[j]);
dst_char = cpu_to_le16(*(wchar_to + 1));
j++;
put_unaligned(dst_char, &target[j]);
dst_char = cpu_to_le16(*(wchar_to + 2));
j++;
put_unaligned(dst_char, &target[j]);
}
continue;
unknown:
dst_char = cpu_to_le16(0x003f);
charlen = 1;
}
ctoUTF16:
/*
* character may take more than one byte in the source string,
* but will take exactly two bytes in the target string
*/
i += charlen;
put_unaligned(dst_char, &target[j]);
}
ctoUTF16_out:
put_unaligned(0, &target[j]); /* Null terminate target unicode string */
kfree(wchar_to);
return j;
}
#ifdef CONFIG_CIFS_SMB2
/*
* cifs_local_to_utf16_bytes - how long will a string be after conversion?
* @from - pointer to input string
* @maxbytes - don't go past this many bytes of input string
* @codepage - source codepage
*
* Walk a string and return the number of bytes that the string will
* be after being converted to the given charset, not including any null
* termination required. Don't walk past maxbytes in the source buffer.
*/
static int
cifs_local_to_utf16_bytes(const char *from, int len,
const struct nls_table *codepage)
{
int charlen;
int i;
wchar_t wchar_to;
for (i = 0; len && *from; i++, from += charlen, len -= charlen) {
charlen = codepage->char2uni(from, len, &wchar_to);
/* Failed conversion defaults to a question mark */
if (charlen < 1)
charlen = 1;
}
return 2 * i; /* UTF16 characters are two bytes */
}
/*
* cifs_strndup_to_utf16 - copy a string to wire format from the local codepage
* @src - source string
* @maxlen - don't walk past this many bytes in the source string
* @utf16_len - the length of the allocated string in bytes (including null)
* @cp - source codepage
* @remap - map special chars
*
* Take a string convert it from the local codepage to UTF16 and
* put it in a new buffer. Returns a pointer to the new string or NULL on
* error.
*/
__le16 *
cifs_strndup_to_utf16(const char *src, const int maxlen, int *utf16_len,
const struct nls_table *cp, int remap)
{
int len;
__le16 *dst;
len = cifs_local_to_utf16_bytes(src, maxlen, cp);
len += 2; /* NULL */
dst = kmalloc(len, GFP_KERNEL);
if (!dst) {
*utf16_len = 0;
return NULL;
}
cifsConvertToUTF16(dst, src, strlen(src), cp, remap);
*utf16_len = len;
return dst;
}
#endif /* CONFIG_CIFS_SMB2 */