Commit Graph

31891 Commits

Author SHA1 Message Date
Martin Storsjö ca7e97bdcf g722: Fix the QMF scaling
This fixes clipping if the encoder input used the full 16 bit
input range (samples with a magnitude below 16383 worked fine).
The filtered subband samples should be 15 bit maximum, while
the code earlier produced them scaled to 16 bit.

This makes the decoder output have double the magnitude
compared to before.

The spec reference samples doesn't test the QMF at all, which
was why this part slipped past initially.

(cherry picked from commit b087ce2bee)

Signed-off-by: Martin Storsjö <martin@martin.st>
2012-03-06 15:45:30 +02:00
Justin Ruggles 4ae138cb12 ac3dsp: do not use pshufb in ac3_extract_exponents_ssse3()
We need to do unsigned saturation in order to cover the corner case when the
absolute coefficient value is 16777215 (the maximum value).

Fixes Bug #216
(cherry picked from commit d483bb58c3)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2012-03-06 13:55:35 +01:00
Fabian Greffrath 003f7e3dd0 Fix format string vulnerability detected by -Wformat-security.
Signed-off-by: Diego Biurrun <diego@biurrun.de>
(cherry picked from commit c9dbac36ad)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2012-03-05 18:01:37 +01:00
Ronald S. Bultje 85eb76a23f h264: fix mmxext chroma deblock to use correct TC values.
(cherry picked from commit b0c4f04338)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2012-03-05 18:00:43 +01:00
Ronald S. Bultje 5186984ee9 h264: change underread for 10bit QPEL to overread.
This prevents us from reading before the start of the buffer, and thus
prevents crashes resulting from this behaviour. Fixes bug 237.
(cherry picked from commit 291c9b6285)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2012-03-05 18:00:31 +01:00
Ronald S. Bultje b5331b979b cscd: use negative error values to indicate decode_init() failures.
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit 8a9faf33f2)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2012-03-05 14:48:35 +01:00
Vitor Sessak 11f3173e1b amrnbdec: check frame size before decoding.
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
Signed-off-by: Ronald S. Bultje <rsbultje@gmail.com>
(cherry picked from commit 882abda5a2)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2012-03-05 14:48:35 +01:00
Ronald S. Bultje cd17195d1c h264: prevent overreads in intra PCM decoding.
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit d1604b3de9)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2012-03-05 14:48:35 +01:00
Justin Ruggles 1128b10247 wmaenc: fix m/s stereo encoding for the first frame
We need to set ms_stereo in encode_init() in order to avoid incorrectly
encoding the first frame as non-m/s while flagging it as m/s. Fixes an
uncomfortable pop in the left channel at the start of playback.

CC:libav-stable@libav.org
(cherry picked from commit 51ddf35c90)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2012-03-04 21:26:29 +01:00
Justin Ruggles 6a073aa7a7 wmaenc: limit allowed sample rate to 48kHz
ff_wma_init() allows up to 50kHz, but this generates an exponent band
size table that requires 65 bands. The code assumes 25 bands in many
places, and using sample rates higher than 48kHz will lead to buffer
overwrites.

CC:libav-stable@libav.org
(cherry picked from commit 1ec075cfec)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2012-03-04 21:26:29 +01:00
Justin Ruggles 073891e875 wmaenc: limit block_align to MAX_CODED_SUPERFRAME_SIZE
This is near the theoretical limit for wma frame size and is the most that
our decoder can handle. Allowing higher bit rates will just end up padding
each frame with empty bytes.

Fixes invalid writes for avconv when using very high bit rates.

CC:libav-stable@libav.org
(cherry picked from commit c2b8dea182)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2012-03-04 21:26:29 +01:00
Justin Ruggles 2e341bc99a wmaenc: require a large enough output buffer to prevent overwrites
The maximum theoretical frame size is around 17000 bytes. Although in
practice it will generally be much smaller, we require a larger buffer
just to be safe.

CC: libav-stable@libav.org
(cherry picked from commit dfc4fdedf8)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2012-03-04 21:26:29 +01:00
Alex Converse b7c8fff803 mpegts: Do not call read_sl_header() when no bytes remain in the buffer.
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit 4df369692e)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2012-03-04 21:26:29 +01:00
Alex Converse 3f7e90cf0c mpegts: Pad the packet buffer in handle_packet().
This allows it to be used with get_bits without the thread of overreads.

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit 1aa708988a)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2012-03-04 21:26:28 +01:00
Ronald S. Bultje 78d4f8cc56 amrwb: remove duplicate arguments from extrapolate_isf().
Prevents warnings because the dst and src overlap (are the same) in the
memcpy() inside the function.

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit 9d87374ec0)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2012-03-04 21:26:28 +01:00
Ronald S. Bultje de2656ec25 amrwb: error out early if mode is invalid.
Prevents using the invalid mode as an index in a static array, which
would generate invalid reads.

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit 154b8bb800)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2012-03-04 21:26:28 +01:00
Ronald S. Bultje 9686a2c2cf matroska: check buffer size for RM-style byte reordering.
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit 9c239f6026)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2012-03-04 21:26:28 +01:00
Ronald S. Bultje b863979c0f wma: fix invalid buffer size assumptions causing random overreads.
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit 349b7977e4)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2012-03-04 12:26:06 +01:00
Alex Converse fecd7468fc wmadec: Verify bitstream size makes sense before calling init_get_bits.
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
(cherry picked from commit 48f1e5212c)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2012-03-04 12:26:06 +01:00
Alex Converse 19da1a39e8 rv10/20: Fix a buffer overread caused by losing track of the remaining buffer size.
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit 2f6528537f)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2012-03-04 12:26:06 +01:00
Ronald S. Bultje 7e88df99e1 lcl: return negative error codes on decode_init() errors.
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit bd17a40a7e)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2012-03-04 12:26:06 +01:00
Justin Ruggles 7f3f85544c avutil: add AVERROR_UNKNOWN
Useful to return instead of -1 when the cause of the error is unknown,
typically from an external library.
(cherry picked from commit c9bca80132)

Conflicts:

	doc/APIchanges
	libavutil/avutil.h

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2012-03-04 12:26:06 +01:00
Ronald S. Bultje 750f5baf30 h264: error out on invalid bitdepth.
Fixes invalid reads while initializing the dequant tables, which uses
the bit depth to determine the QP table size.

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit 0ce4fe482c)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2012-03-04 12:26:06 +01:00
Ronald S. Bultje a63f3f714c huffyuv: do not abort on unknown pix_fmt; instead, return an error.
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit 63c9de6469)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2012-03-04 12:26:06 +01:00
Ronald S. Bultje 1dd1ee00d5 vmnc: return error on decode_init() failure.
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit 07a180972f)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2012-02-29 21:43:20 +01:00
Ronald S. Bultje 4493af756b rpza: error out on buffer overreads.
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit 78e9852a2e)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2012-02-29 21:43:20 +01:00
Ronald S. Bultje e904e9b720 qtrle: return error on decode_init() failure.
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit e54ae60e46)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2012-02-29 21:43:20 +01:00
Ronald S. Bultje 5f896773e0 swscale: fix another integer overflow.
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit 791de61bbb)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2012-02-29 21:43:20 +01:00
Ronald S. Bultje b2dcac7141 vp56: error out on invalid stream dimensions.
Prevents crashes when playing corrupt vp5/6 streams.

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit 8bc396fc0e)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2012-02-29 14:23:11 +01:00
Ronald S. Bultje 40ccc81146 asf: don't seek back on EOF.
Seeking back on EOF will reset the EOF flag, causing us to re-enter
the loop to find the next marker in the ASF file, thus potentially
causing an infinite loop.

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit bb6d5411e1)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2012-02-29 14:22:35 +01:00
Ronald S. Bultje 1c63d61372 asf: error out on ridiculously large minpktsize values.
They cause various issues further down in demuxing.

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit 6e57a02b9f)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2012-02-29 14:21:57 +01:00
Anton Khirnov 2ad77c60ef lavf: add functions for accessing the fourcc<->CodecID mapping tables.
Fixes bug 212.
(cherry picked from commit dd6d3b0e02)

Conflicts:

	doc/APIchanges

Signed-off-by: Anton Khirnov <anton@khirnov.net>
2012-02-29 10:44:37 +01:00
Paul B Mahol a1556d37b8 avutil: make intfloat api public
The functions are already av_ prefixed and intfloat header is already provided.
Install libavutil/intfloat.h

Signed-off-by: Paul B Mahol <onemda@gmail.com>
Signed-off-by: Anton Khirnov <anton@khirnov.net>
(cherry picked from commit 8b933129b9)

Conflicts:

	doc/APIchanges

Signed-off-by: Anton Khirnov <anton@khirnov.net>
2012-02-29 10:44:37 +01:00
Alex Converse 083a8a0037 mjpegbdec: Fix overflow in SOS.
Based in part by a fix from Michael Niedermayer <michaelni@gmx.at>

Fixes CVE-2011-3947

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
(cherry picked from commit b57d262412)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2012-02-28 22:20:45 +01:00
Ronald S. Bultje 71a939fee4 oma: don't read beyond end of leaf_table.
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit 934cd18a43)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2012-02-28 22:10:55 +01:00
Ronald S. Bultje 9dbd437da2 Indeo3: fix crashes on corrupt bitstreams.
Splits at borders of cells are invalid, since it leaves one of the
cells with a width/height of zero. Also, propagate errors on buffer
allocation failures, so we don't continue decoding (which crashes).

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit fc9bc08dca)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2012-02-28 20:57:44 +01:00
Ronald S. Bultje 2510e1476e vorbis: fix overflows in floor1[] vector and inverse db table index.
(cherry picked from commit 24947d4988)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2012-02-26 18:11:15 +01:00
Reinhard Tartler 0f839cff6b Fix parser not to clobber has_b_frames when extradata is set.
Because in contrast to the decoder, the parser does not setup low_delay.
The code in parse_nal_units would always end up setting has_b_frames
to "1", except when stream is explicitly marked as low delay.
Since the parser itself would create 'extradata', simply reopening
the parser would cause this.

This happens for instance in estimate_timings_from_pts(), which causes the
parser to be reopened on the same stream.

This fixes Libav #22 and FFmpeg (trac) #360

CC: libav-stable@libav.org

Based on a patch by Reimar Döffinger <Reimar.Doeffinger@gmx.de>
(commit 31ac0ac29b)

Comments and description adapted by Reinhard Tartler.

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
(cherry picked from commit 790a367d9e)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2012-02-26 15:56:55 +01:00
Ronald S. Bultje abe3572878 rm: prevent infinite loops for index parsing.
Specifically, prevent jumping back in the file for the next index, since
this can lead to infinite loops where we jump between indexes referring
to each other, and don't read indexes that don't fit in the file.

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit aac07a7a4c)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2012-02-26 10:04:04 +01:00
Ronald S. Bultje 0d30e2c6f2 fraps: release reference buffer on pix_fmt change.
Prevents crash when trying to copy from a non-existing plane in e.g.
a RGB32 reference image to a YUV420P target image

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit 830f70442a)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
2012-02-26 10:03:16 +01:00
Ronald S. Bultje a0473085f3 kgv1: release reference picture on size change.
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit 6c4c27adb6)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
2012-02-26 10:03:16 +01:00
Ronald S. Bultje e537dc230b kgv1: use avctx->get/release_buffer().
Also fixes crashes on corrupt bitstreams.

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit 33cd32b389)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
2012-02-26 10:03:16 +01:00
Ronald S. Bultje 19f4943d12 lcl: error out if uncompressed input buffer is smaller than framesize.
This prevents crashes when trying to read beyond the end of the buffer
while decoding frame data.

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit be129271ea)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
2012-02-26 10:03:16 +01:00
Ronald S. Bultje bf6d1a1ca7 mjpeg: abort decoding if packet is too large.
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit ab492ca2ab)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
2012-02-26 10:03:16 +01:00
Alex Converse 424b6edd19 tiff: Prevent overreads in the type_sizes array.
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit 447363870f)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
2012-02-26 10:03:16 +01:00
Ronald S. Bultje 4f48417fe7 swf: check return values for av_get/new_packet().
Prevents crashers when using the packet if allocation failed.

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit 31632e73f4)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
2012-02-26 10:03:16 +01:00
Ronald S. Bultje 8e3dc37bc0 truemotion2: error out if the huffman tree has no nodes.
This prevents crashers and errors further down when reading nodes in the
empty tree.

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit 2b83e8b700)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
2012-02-26 10:03:16 +01:00
Ronald S. Bultje 0312969b9e rmdec: when using INT4 deinterleaving, error out if sub_packet_h <= 1.
We read sub_packet_h / 2 packets per line of data (during deinterleaving),
which equals zero if sub_packet_h <= 1, thus causing us to not read any
data, leading to an infinite loop.

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit e30b3e59a4)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
2012-02-26 10:03:16 +01:00
Janne Grunau 62beae313a avplay: fix -threads option
The AVOptions based default to threads auto in 2473a45c8
works only if avplay does not use custom option handling
for -threads.

CC: <libav-stable@libav.org>
(cherry picked from commit e48a70e6da)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
2012-02-26 10:03:16 +01:00
Ronald S. Bultje 8011a29fa8 vc1parse: call vc1_init_common().
The parser uses VLC tables initialized in vc1_common_init(), therefore
we should call this function on parser init also.

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit c742ab4e81)

Conflicts:

	libavcodec/vc1.h

Signed-off-by: Anton Khirnov <anton@khirnov.net>
2012-02-26 10:03:16 +01:00