Commit Graph

18019 Commits

Author SHA1 Message Date
Michael Niedermayer cd0cfdc0a7 pcm-mpeg: Check for valid bps.
The code only supports 16 and 24 bps currently, 20bps causes
out of array reads.

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2012-03-03 23:55:16 +01:00
Michael Niedermayer d629f3edaa cook: check that category is smaller than 8
This fixes some out of global array accesses of dither_tab.

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Reviewed-by: Benjamin Larsson <benjamin@southpole.se>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2012-03-03 18:35:46 +01:00
Nicolas George b2792a436b lavc: unify similar code merged from both branches. 2012-03-03 15:56:45 +01:00
Michael Niedermayer c266eb1928 arm: Fix 10l typo
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2012-03-03 04:35:15 +01:00
Michael Niedermayer 70b5583baa kvmc: Fix out of reference frame reads.
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2012-03-03 04:14:37 +01:00
Michael Niedermayer 81d4b3af81 qpeg: fix overreads.
qpeg should probably be changed to use the checked bytestream reader.
But for now this fixes it and is significantly less work.

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2012-03-03 03:50:05 +01:00
Michael Niedermayer 4299dfa5de qpeg: Fix out of array writes.
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2012-03-03 03:37:52 +01:00
Fabian Greffrath aaa1173de7 srtdec: fix a format string vulnerability.
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2012-03-03 02:35:27 +01:00
Michael Niedermayer 268098d8b2 Merge remote-tracking branch 'qatar/master'
* qatar/master: (29 commits)
  amrwb: remove duplicate arguments from extrapolate_isf().
  amrwb: error out early if mode is invalid.
  h264: change underread for 10bit QPEL to overread.
  matroska: check buffer size for RM-style byte reordering.
  vp8: disable mmx functions with sse/sse2 counterparts on x86-64.
  vp8: change int stride to ptrdiff_t stride.
  wma: fix invalid buffer size assumptions causing random overreads.
  Windows Media Audio Lossless decoder
  rv10/20: Fix slice overflow with checked bitstream reader.
  h263dec: Disallow width/height changing with frame threads.
  rv10/20: Fix a buffer overread caused by losing track of the remaining buffer size.
  rmdec: Honor .RMF tag size rather than assuming 18.
  g722: Fix the QMF scaling
  r3d: don't set codec timebase.
  electronicarts: set timebase for tgv video.
  electronicarts: parse the framerate for cmv video.
  ogg: don't set codec timebase
  electronicarts: don't set codec timebase
  avs: don't set codec timebase
  wavpack: Fix an integer overflow
  ...

Conflicts:
	libavcodec/arm/vp8dsp_init_arm.c
	libavcodec/fraps.c
	libavcodec/h264.c
	libavcodec/mpeg4videodec.c
	libavcodec/mpegvideo.c
	libavcodec/msmpeg4.c
	libavcodec/pnmdec.c
	libavcodec/qpeg.c
	libavcodec/rawenc.c
	libavcodec/ulti.c
	libavcodec/vcr1.c
	libavcodec/version.h
	libavcodec/wmalosslessdec.c
	libavformat/electronicarts.c
	libswscale/ppc/yuv2rgb_altivec.c
	tests/ref/acodec/g722
	tests/ref/fate/ea-cmv

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2012-03-03 00:23:10 +01:00
Michael Niedermayer 689f65126b simple_idct: idct_4col_put: Fix out of array reads.
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2012-03-02 22:09:44 +01:00
Michael Niedermayer 422e3a74b9 rawdec: fix input overread.
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2012-03-02 22:04:00 +01:00
Ronald S. Bultje 9d87374ec0 amrwb: remove duplicate arguments from extrapolate_isf().
Prevents warnings because the dst and src overlap (are the same) in the
memcpy() inside the function.

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
2012-03-02 12:50:00 -08:00
Michael Niedermayer e7b43e8e84 truemotion1: Check input buffer size against header size.
Fixes overread.

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2012-03-02 21:36:42 +01:00
Michael Niedermayer a3f5ee297a mjpeg: Check for interlaced progressive frames
Fixes null pointer dereference.

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2012-03-02 21:35:58 +01:00
Michael Niedermayer 0af48e29f5 snow: check reference frame indices.
Fixes NULL ptr dereference

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2012-03-02 20:53:00 +01:00
Ronald S. Bultje 154b8bb800 amrwb: error out early if mode is invalid.
Prevents using the invalid mode as an index in a static array, which
would generate invalid reads.

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
2012-03-02 10:34:50 -08:00
Ronald S. Bultje 291c9b6285 h264: change underread for 10bit QPEL to overread.
This prevents us from reading before the start of the buffer, and thus
prevents crashes resulting from this behaviour. Fixes bug 237.
2012-03-02 10:33:05 -08:00
Ronald S. Bultje 45549339bc vp8: disable mmx functions with sse/sse2 counterparts on x86-64.
x86-64 is guaranteed to have at least SSE2, therefore the MMX/MMX2
functions will never be used in practice.
2012-03-02 10:32:05 -08:00
Ronald S. Bultje bd66f073fe vp8: change int stride to ptrdiff_t stride.
On 64bit platforms with 32bit int, this means we won't have to sign-
extend the integer anymore.
2012-03-02 10:31:50 -08:00
Ronald S. Bultje 349b7977e4 wma: fix invalid buffer size assumptions causing random overreads.
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
2012-03-02 10:31:28 -08:00
Mashiat Sarker Shakkhar 9d25f1f619 Windows Media Audio Lossless decoder
Decodes 16-bit WMA Lossless encoded files. 24-bit is not supported yet.

Bitstream parser written by Andreas Öman with contributions from
Baptiste Coudurier and Ulion.

Includes a number of bug-fixes from Benjamin Larsson, Michael Niedermayer and
Konstantin Shishkov, shine and polish by Diego Biurrun.

Signed-off-by: Diego Biurrun <diego@biurrun.de>
2012-03-02 19:10:29 +01:00
Michael Niedermayer 8fdd93eaad huffyuv: pad classic huffman tables so as to avoid bitreader overread.
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2012-03-02 18:49:39 +01:00
Michael Niedermayer 64c58f1436 vc1: mquant is not allowed to be 0
Fixes out of bounds read.
Checked against SMPTE 421M-2006

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2012-03-02 18:40:42 +01:00
Alex Converse 9243ec4a50 rv10/20: Fix slice overflow with checked bitstream reader. 2012-03-02 09:31:32 -08:00
Michael Niedermayer 71db86d53b h263dec: Disallow width/height changing with frame threads.
Fixes CVE-2011-3937

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2012-03-02 09:31:32 -08:00
Alex Converse 2f6528537f rv10/20: Fix a buffer overread caused by losing track of the remaining buffer size.
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
2012-03-02 09:31:32 -08:00
Martin Storsjö b087ce2bee g722: Fix the QMF scaling
This fixes clipping if the encoder input used the full 16 bit
input range (samples with a magnitude below 16383 worked fine).
The filtered subband samples should be 15 bit maximum, while
the code earlier produced them scaled to 16 bit.

This makes the decoder output have double the magnitude
compared to before.

The spec reference samples doesn't test the QMF at all, which
was why this part slipped past initially.

Signed-off-by: Martin Storsjö <martin@martin.st>
2012-03-02 18:58:19 +02:00
Michael Niedermayer e93d911e48 h263: fix zygo debug printing overreading.
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2012-03-02 16:53:37 +01:00
Michael Niedermayer d964db5742 wmadec: fix off by 1 error on the pow_tab index check.
Fixes global out of array read.

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2012-03-02 16:44:49 +01:00
Michael Niedermayer ec3cd74f2d h261: check mtype.
Fixes out of array read

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2012-03-02 15:58:14 +01:00
Derek Buitenhuis f604eab30a wavpack: Fix an integer overflow
Integer Overflow Checker detected an integer
overflow while FATE was running.

See: http://fate.libav.org/x86_64-linux-ioc/

Signed-off-by: Derek Buitenhuis <derek.buitenhuis@gmail.com>
Signed-off-by: Kostya Shishkov <kostya.shishkov@gmail.com>
2012-03-02 08:26:36 +01:00
Derek Buitenhuis 83c418e68e wavpack: Fix an integer overflow
Integer Overflow Checker detected an integer
overflow while FATE was running.

Signed-off-by: Derek Buitenhuis <derek.buitenhuis@gmail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2012-03-02 02:40:24 +01:00
Michael Niedermayer 7a7b1f5c4d roqvideodec: improve end of input buffer check
This fixes a out of array read.

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2012-03-02 02:32:14 +01:00
Michael Niedermayer 1eb7f39c7b Merge remote-tracking branch 'qatar/master'
* qatar/master:
  h264: error out on invalid bitdepth.
  aacsbr: use a swap index for the Y matrix rather than copy buffers.
  huffyuv: do not abort on unknown pix_fmt; instead, return an error.
  lcl: return negative error codes on decode_init() errors.
  rtpenc: Use MB info side data for splitting H263 packets for RFC 2190
  h263enc: Add an option for outputting info about MBs as side data
  avpacket: Add a function for shrinking already allocated side data
  nellymoserdec: Saner and faster IMDCT windowing

Conflicts:
	doc/APIchanges
	libavcodec/avpacket.c
	libavcodec/version.h

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2012-03-01 23:11:31 +01:00
Diego Biurrun 1c4717be4f mjpegdec: use correct variable in av_log invocation
libavcodec/mjpegdec.c:1463: warning: format ‘%x’ expects type ‘unsigned int’, but argument 5 has type ‘const uint8_t *’
2012-03-01 23:11:15 +01:00
Diego Biurrun 324deaa268 Replace AVFrame pointer type punning by proper struct member assignments. 2012-03-01 23:11:13 +01:00
Diego Biurrun 47c0ac96aa Replace AVFrame pointer casts by proper struct member accesses. 2012-03-01 23:11:11 +01:00
Diego Biurrun 562b6c744a Remove unnecessary AVFrame pointer casts. 2012-03-01 23:11:10 +01:00
Diego Biurrun 2f4b476e04 msmpeg4: Split encoding backend code off from general backend code. 2012-03-01 23:11:09 +01:00
Anton Khirnov eb727387fd lavc: shrink encoded video packet size after encoding.
Based on a patch by Nicolas George <nicolas.george <at> normalesup.org>
2012-03-01 22:21:35 +01:00
Michael Niedermayer 75d11b55d7 vc1: avoid reading beyond the last line in vc1_draw_sprites()
Fixes overread

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2012-03-01 22:20:07 +01:00
Ronald S. Bultje 0ce4fe482c h264: error out on invalid bitdepth.
Fixes invalid reads while initializing the dequant tables, which uses
the bit depth to determine the QP table size.

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
2012-03-01 12:15:51 -08:00
Christophe Gisquet cc412b7104 aacsbr: use a swap index for the Y matrix rather than copy buffers.
Signed-off-by: Alex Converse <alex.converse@gmail.com>
2012-03-01 11:03:00 -08:00
Michael Niedermayer dc945b1fa8 eatgq: Pass error code from tgq_decode_mb() and let the caller fail.
This fixes a over read.

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2012-03-01 19:57:55 +01:00
Michael Niedermayer 32f0c65828 vc1: fix out of array reads in vc1_inv_trans_4x4_c()
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2012-03-01 19:35:12 +01:00
Michael Niedermayer 80c702efeb vc1: fix out of array reads in vc1_inv_trans_4x8_c()
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2012-03-01 19:35:12 +01:00
Michael Niedermayer af796ba4b8 vc1: fix out of array reads in vc1_inv_trans_8x4_c()
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2012-03-01 19:35:12 +01:00
Ronald S. Bultje 63c9de6469 huffyuv: do not abort on unknown pix_fmt; instead, return an error.
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
2012-03-01 10:23:36 -08:00
Michael Niedermayer b2a7c01733 mpc: Fix mpc_CC table and use.
This is based on the reference implementation and fixes
a global out of array read.

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2012-03-01 16:16:13 +01:00
Michael Niedermayer b84211ac71 mpc: Fix mpc_SCF use and content.
This fixes a out of global array read.
This change is based on the reference mpc imlementation.

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2012-03-01 16:15:33 +01:00