Commit Graph

109 Commits

Author SHA1 Message Date
Serhiy Storchaka c9c4338e2b Add tests for raw-unicode-escape codec. 2013-01-29 11:40:00 +02:00
Serhiy Storchaka 077cb347a9 Clean up escape-decode decoder tests. 2013-01-29 11:06:53 +02:00
Serhiy Storchaka d679377be7 Issue #16979: Fix error handling bugs in the unicode-escape-decode decoder. 2013-01-29 10:20:44 +02:00
Serhiy Storchaka ace3ad3bf7 Issue #16975: Fix error handling bug in the escape-decode bytes decoder. 2013-01-25 23:31:43 +02:00
Serhiy Storchaka 4fb8caee87 Issue #14850: Now a chamap decoder treates U+FFFE as "undefined mapping"
in any mapping, not only in an unicode string.
2013-01-15 14:43:21 +02:00
Ezio Melotti 26ed234052 Enable a broken test and fix it. 2013-01-11 05:54:57 +02:00
Serhiy Storchaka 48e188e573 Issue #11461: Fix the incremental UTF-16 decoder. Original patch by
Amaury Forgeot d'Arc. Added tests for partial decoding of non-BMP
characters.
2013-01-08 23:14:24 +02:00
Ezio Melotti 540da76115 #16336: fix input checking in the surrogatepass error handler. Patch by Serhiy Storchaka. 2012-11-03 23:03:39 +02:00
Philip Jenvey 45c41494bf bounds check for bad data (thanks amaury) 2012-10-26 17:01:53 -07:00
Antoine Pitrou 6f80f5d444 Issue #15379: Fix passing of non-BMP characters as integers for the charmap decoder (already working as unicode strings).
Patch by Serhiy Storchaka.
2012-09-23 19:55:21 +02:00
Antoine Pitrou b4bbee25b1 Issue #14579: Fix CVE-2012-2135: vulnerability in the utf-16 decoder after error handling.
Patch by Serhiy Storchaka.
2012-07-21 00:45:14 +02:00
Antoine Pitrou 2a20f9be70 Backport 0398f07d4827 (fix for weird buildbot failures) 2011-07-27 01:06:07 +02:00
Antoine Pitrou cf9d3c08c8 Issue #1813: Fix codec lookup under Turkish locales. 2011-07-24 02:27:04 +02:00
Victor Stinner b43dd4b8ca Merge 3.1 2011-05-23 14:57:05 +02:00
Victor Stinner 2cca057284 test_codecs now removes the temporay file (created by the test) 2011-05-23 14:51:42 +02:00
Benjamin Peterson 28a4dce6a8 remove (un)transform methods 2010-12-12 01:33:04 +00:00
Victor Stinner 53a9dd776e Issue #10546: UTF-16-LE and UTF-16-BE *do* support non-BMP characters
Fix the doc and add tests.
2010-12-08 22:25:45 +00:00
Georg Brandl 02524629f3 #7475: add (un)transform method to bytes/bytearray and str, add back codecs that can be used with them from Python 2. 2010-12-02 18:06:51 +00:00
Ezio Melotti 19f2aeba67 Merged revisions 86596 via svnmerge from
svn+ssh://pythondev@svn.python.org/python/branches/py3k

........
  r86596 | ezio.melotti | 2010-11-20 21:04:17 +0200 (Sat, 20 Nov 2010) | 1 line

  #9424: Replace deprecated assert* methods in the Python test suite.
........
2010-11-21 01:30:29 +00:00
Ezio Melotti b3aedd4862 #9424: Replace deprecated assert* methods in the Python test suite. 2010-11-20 19:04:17 +00:00
Benjamin Peterson 5a6214afe2 Merged revisions 81499,81506 via svnmerge from
svn+ssh://pythondev@svn.python.org/python/trunk

........
  r81499 | georg.brandl | 2010-05-24 16:29:07 -0500 (Mon, 24 May 2010) | 1 line

  #8016: add the CP858 codec (approved by Benjamin).  (Also add CP720 to the tests, it was missing there.)
........
  r81506 | benjamin.peterson | 2010-05-24 17:04:53 -0500 (Mon, 24 May 2010) | 1 line

  set svn:eol-style
........
2010-06-27 22:41:29 +00:00
Victor Stinner 554f3f0081 Issue #850997: mbcs encoding (Windows only) handles errors argument: strict
mode raises unicode errors. The encoder only supports "strict" and "replace"
error handlers, the decoder only supports "strict" and "ignore" error handlers.
2010-06-16 23:33:54 +00:00
Antoine Pitrou 6107a688ee Merged revisions 81908 via svnmerge from
svn+ssh://pythondev@svn.python.org/python/branches/py3k

................
  r81908 | antoine.pitrou | 2010-06-11 23:46:32 +0200 (ven., 11 juin 2010) | 11 lines

  Merged revisions 81907 via svnmerge from
  svn+ssh://pythondev@svn.python.org/python/trunk

  ........
    r81907 | antoine.pitrou | 2010-06-11 23:42:26 +0200 (ven., 11 juin 2010) | 5 lines

    Issue #8941: decoding big endian UTF-32 data in UCS-2 builds could crash
    the interpreter with characters outside the Basic Multilingual Plane
    (higher than 0x10000).
  ........
................
2010-06-11 21:48:34 +00:00
Antoine Pitrou cc0cfd3576 Merged revisions 81907 via svnmerge from
svn+ssh://pythondev@svn.python.org/python/trunk

........
  r81907 | antoine.pitrou | 2010-06-11 23:42:26 +0200 (ven., 11 juin 2010) | 5 lines

  Issue #8941: decoding big endian UTF-32 data in UCS-2 builds could crash
  the interpreter with characters outside the Basic Multilingual Plane
  (higher than 0x10000).
........
2010-06-11 21:46:32 +00:00
Philip Jenvey ddf0d0383c Merged revisions 79780 via svnmerge from
svn+ssh://pythondev@svn.python.org/python/branches/py3k

................
  r79780 | philip.jenvey | 2010-04-04 20:05:24 -0700 (Sun, 04 Apr 2010) | 9 lines

  Merged revisions 79779 via svnmerge from
  svn+ssh://pythondev@svn.python.org/python/trunk

  ........
    r79779 | philip.jenvey | 2010-04-04 19:51:51 -0700 (Sun, 04 Apr 2010) | 2 lines

    fix escape_encode to return the correct consumed size
  ........
................
2010-06-09 17:56:11 +00:00
Victor Stinner 3dcb5acdb0 Issue #8838, #8339: Remove codecs.charbuffer_encode() and "t#" parsing format
Remove last references to the "char buffer" of the buffer protocol from
Python3.
2010-06-08 22:54:19 +00:00
Victor Stinner b64d0eba50 Merged revisions 81474 via svnmerge from
svn+ssh://pythondev@svn.python.org/python/branches/py3k

................
  r81474 | victor.stinner | 2010-05-22 18:59:09 +0200 (sam., 22 mai 2010) | 20 lines

  Merged revisions 81471-81472 via svnmerge from
  svn+ssh://pythondev@svn.python.org/python/trunk

  ........
    r81471 | victor.stinner | 2010-05-22 15:37:56 +0200 (sam., 22 mai 2010) | 7 lines

    Issue #6268: More bugfixes about BOM, UTF-16 and UTF-32

     * Fix seek() method of codecs.open(), don't write the BOM twice after seek(0)
     * Fix reset() method of codecs, UTF-16, UTF-32 and StreamWriter classes
     * test_codecs: use "w+" mode instead of "wt+". "t" mode is not supported by
       Solaris or Windows, but does it really exist? I found it the in the issue.
  ........
    r81472 | victor.stinner | 2010-05-22 15:44:25 +0200 (sam., 22 mai 2010) | 4 lines

    Fix my last commit (r81471) about codecs

    Rememder: don't touch the code just before a commit
  ........
................
2010-05-22 17:01:13 +00:00
Victor Stinner a92ad7ee2c Merged revisions 81471-81472 via svnmerge from
svn+ssh://pythondev@svn.python.org/python/trunk

........
  r81471 | victor.stinner | 2010-05-22 15:37:56 +0200 (sam., 22 mai 2010) | 7 lines

  Issue #6268: More bugfixes about BOM, UTF-16 and UTF-32

   * Fix seek() method of codecs.open(), don't write the BOM twice after seek(0)
   * Fix reset() method of codecs, UTF-16, UTF-32 and StreamWriter classes
   * test_codecs: use "w+" mode instead of "wt+". "t" mode is not supported by
     Solaris or Windows, but does it really exist? I found it the in the issue.
........
  r81472 | victor.stinner | 2010-05-22 15:44:25 +0200 (sam., 22 mai 2010) | 4 lines

  Fix my last commit (r81471) about codecs

  Rememder: don't touch the code just before a commit
........
2010-05-22 16:59:09 +00:00
Victor Stinner 37b8200608 Merged revisions 81461 via svnmerge from
svn+ssh://pythondev@svn.python.org/python/branches/py3k

................
  r81461 | victor.stinner | 2010-05-22 04:16:27 +0200 (sam., 22 mai 2010) | 10 lines

  Merged revisions 81459 via svnmerge from
  svn+ssh://pythondev@svn.python.org/python/trunk

  ........
    r81459 | victor.stinner | 2010-05-22 04:11:07 +0200 (sam., 22 mai 2010) | 3 lines

    Issue #6268: Fix seek() method of codecs.open(), don't read the BOM twice
    after seek(0)
  ........
................
2010-05-22 02:17:42 +00:00
Victor Stinner 3fed0870a6 Merged revisions 81459 via svnmerge from
svn+ssh://pythondev@svn.python.org/python/trunk

........
  r81459 | victor.stinner | 2010-05-22 04:11:07 +0200 (sam., 22 mai 2010) | 3 lines

  Issue #6268: Fix seek() method of codecs.open(), don't read the BOM twice
  after seek(0)
........
2010-05-22 02:16:27 +00:00
Victor Stinner 158701d886 Merged revisions 80382 via svnmerge from
svn+ssh://pythondev@svn.python.org/python/branches/py3k

........
  r80382 | victor.stinner | 2010-04-22 21:38:16 +0200 (jeu., 22 avril 2010) | 3 lines

  Issue #8092: Fix PyUnicode_EncodeUTF8() to support error handler producing
  unicode string (eg. backslashreplace)
........
2010-04-22 19:41:01 +00:00
Victor Stinner 31be90b0c7 Issue #8092: Fix PyUnicode_EncodeUTF8() to support error handler producing
unicode string (eg. backslashreplace)
2010-04-22 19:38:16 +00:00
Philip Jenvey 66a1bd5568 Merged revisions 79779 via svnmerge from
svn+ssh://pythondev@svn.python.org/python/trunk

........
  r79779 | philip.jenvey | 2010-04-04 19:51:51 -0700 (Sun, 04 Apr 2010) | 2 lines

  fix escape_encode to return the correct consumed size
........
2010-04-05 03:05:24 +00:00
Florent Xicluna e36b2c693c Recorded merge of revisions 78462,78484 via svnmerge from
svn+ssh://pythondev@svn.python.org/python/branches/py3k

................
  r78462 | florent.xicluna | 2010-02-26 12:12:33 +0100 (ven, 26 fév 2010) | 9 lines

  Merged revisions 78461 via svnmerge from
  svn+ssh://pythondev@svn.python.org/python/trunk

  ........
    r78461 | florent.xicluna | 2010-02-26 11:40:58 +0100 (ven, 26 fév 2010) | 2 lines

    #691291: codecs.open() should not convert end of lines on reading and writing.
  ........
................
  r78484 | florent.xicluna | 2010-02-27 12:31:21 +0100 (sam, 27 fév 2010) | 9 lines

  Merged revisions 78482 via svnmerge from
  svn+ssh://pythondev@svn.python.org/python/trunk

  ........
    r78482 | florent.xicluna | 2010-02-27 12:19:18 +0100 (sam, 27 fév 2010) | 2 lines

    Add entry for issue #691291.
  ........
................
2010-02-27 11:38:27 +00:00
Florent Xicluna c1c415f304 Merged revisions 78461 via svnmerge from
svn+ssh://pythondev@svn.python.org/python/trunk

........
  r78461 | florent.xicluna | 2010-02-26 11:40:58 +0100 (ven, 26 fév 2010) | 2 lines

  #691291: codecs.open() should not convert end of lines on reading and writing.
........
2010-02-26 11:12:33 +00:00
Ezio Melotti e96159335f Merged revisions 77727 via svnmerge from
svn+ssh://pythondev@svn.python.org/python/trunk

........
  r77727 | ezio.melotti | 2010-01-24 18:58:36 +0200 (Sun, 24 Jan 2010) | 1 line

  use assert[Not]IsInstance where appropriate
........
2010-01-24 19:26:24 +00:00
Georg Brandl 7b10c9f301 Merged revisions 74871 via svnmerge from
svn+ssh://svn.python.org/python/branches/py3k

................
  r74871 | georg.brandl | 2009-09-17 13:41:24 +0200 (Do, 17 Sep 2009) | 12 lines

  Merged revisions 74869 via svnmerge from svn+ssh://pythondev@svn.python.org/python/trunk

  (Only the new tests, the code had already been corrected due to an API change
  in unicode_decode_call_errorhandler.)

  ........
    r74869 | georg.brandl | 2009-09-17 13:28:09 +0200 (Do, 17 Sep 2009) | 4 lines

    Issue #6922: Fix an infinite loop when trying to decode an invalid
    UTF-32 stream with a non-raising error handler like "replace" or "ignore".
  ........
................
2009-09-17 11:46:23 +00:00
Georg Brandl 791f4e15db Merged revisions 74869 via svnmerge from svn+ssh://pythondev@svn.python.org/python/trunk
(Only the new tests, the code had already been corrected due to an API change
in unicode_decode_call_errorhandler.)

........
  r74869 | georg.brandl | 2009-09-17 13:28:09 +0200 (Do, 17 Sep 2009) | 4 lines

  Issue #6922: Fix an infinite loop when trying to decode an invalid
  UTF-32 stream with a non-raising error handler like "replace" or "ignore".
........
2009-09-17 11:41:24 +00:00
Georg Brandl ab91fdef1f Merged revisions 73715 via svnmerge from
svn+ssh://svn.python.org/python/branches/py3k

........
  r73715 | benjamin.peterson | 2009-07-01 01:06:06 +0200 (Mi, 01 Jul 2009) | 1 line

  convert old fail* assertions to assert*
........
2009-08-13 08:51:18 +00:00
Benjamin Peterson c9c0f201fe convert old fail* assertions to assert* 2009-06-30 23:06:06 +00:00
Amaury Forgeot d'Arc e5344d6c45 Merged revisions 73698 via svnmerge from
svn+ssh://pythondev@svn.python.org/python/branches/py3k

........
  r73698 | amaury.forgeotdarc | 2009-06-30 00:36:49 +0200 (mar., 30 juin 2009) | 7 lines

  #6373: SystemError in str.encode('latin1', 'surrogateescape')
  if the string contains unpaired surrogates.
  (In debug build, crash in assert())

  This can happen with normal processing, if python starts with utf-8,
  then calls sys.setfilesystemencoding('latin-1')
........
2009-06-29 22:38:54 +00:00
Amaury Forgeot d'Arc 84ec8d9314 #6373: SystemError in str.encode('latin1', 'surrogateescape')
if the string contains unpaired surrogates.
(In debug build, crash in assert())

This can happen with normal processing, if python starts with utf-8,
then calls sys.setfilesystemencoding('latin-1')
2009-06-29 22:36:49 +00:00
Martin v. Löwis 43c57785d3 Rename utf8b error handler to surrogateescape. 2009-05-10 08:15:24 +00:00
Martin v. Löwis e0a2b72e61 Rename the surrogates error handler to surrogatepass. 2009-05-10 08:08:56 +00:00
Walter Dörwald 8dc33d56f5 Merged revisions 72404-72406 via svnmerge from
svn+ssh://pythondev@svn.python.org/python/trunk

........
  r72404 | walter.doerwald | 2009-05-06 16:28:24 +0200 (Mi, 06 Mai 2009) | 3 lines

  Issue 3739: The unicode-internal encoder now reports the number of *characters*
  consumed like any other encoder (instead of the number of bytes).
........
  r72406 | walter.doerwald | 2009-05-06 16:32:35 +0200 (Mi, 06 Mai 2009) | 2 lines

  Add NEWS entry about issue #3739.
........
2009-05-06 14:41:26 +00:00
Martin v. Löwis 011e842033 Issue #5915: Implement PEP 383, Non-decodable Bytes in
System Character Interfaces.
2009-05-05 04:43:17 +00:00
Martin v. Löwis db12d454e6 Issue #3672: Reject surrogates in utf-8 codec; add surrogates error
handler.
2009-05-02 18:52:14 +00:00
Antoine Pitrou 81fabdb437 Issue #4874: Most builtin decoders now reject unicode input. 2009-01-22 10:11:36 +00:00
Antoine Pitrou 616d28566b Issue #2394: implement more of the memoryview API. 2008-08-19 22:09:34 +00:00
Benjamin Peterson ee8712cda4 #2621 rename test.test_support to test.support 2008-05-20 21:35:26 +00:00