Commit Graph

187 Commits

Author SHA1 Message Date
Andreas Rheinhardt 01dbcbb37a avcodec/smacker: Use unsigned for shift
Given that the code currently accepts only 27 bits long Huffman codes,
the shift 1 << (length - 1) with length in 1..28 that is performed when
parsing the tree is safe. Yet if this limit were ever expanded to the
full 32 bits, this shift would be potentially undefined. So simply use
unsigned.

Reviewed-by: Paul B Mahol <onemda@gmail.com>
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
2020-09-18 01:47:57 +02:00
Andreas Rheinhardt 056a8fc071 avcodec/smacker: Forward error codes
Reviewed-by: Paul B Mahol <onemda@gmail.com>
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
2020-09-18 01:46:44 +02:00
Andreas Rheinhardt e028e8aa39 avcodec/smacker: Use same variable for return values and errors
smacker_decode_header_tree() uses different variables for return values
(res) and for errors (err) leading to code like
res = foo(bar);
if (res < 0) {
    err = res;
    goto error;
}
Given that no positive return value is ever used at all one can simplify
the above by removing the intermediate res.

Reviewed-by: Paul B Mahol <onemda@gmail.com>
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
2020-09-18 01:44:09 +02:00
Andreas Rheinhardt 4656c1771b avcodec/smacker: Directly goto error in case of error
The earlier version did not error out directly in case an error happens,
because it would lead to a leak: An allocated array is only reachable
via a local variable at that time; it is only attached to more permanent
storage at the end. While it would be possible to add custom code for
freeing on error (instead of reusing the ordinary code for doing so),
this commit takes the opposite approach and attaches the newly allocated
array to its permanent place immediately after its allocation.

Reviewed-by: Paul B Mahol <onemda@gmail.com>
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
2020-09-18 01:35:43 +02:00
Andreas Rheinhardt bd076cacc3 avcodec/smacker: Improve header table error checks
The extradata for Smacker video contains Huffman trees as well as a
field containing the size (in bytes) of said Huffman tree when stored
as a table. Due to three special values the decoder allocates more than
the size field indicates; yet when it parses the table it only errors
out if the number of elements exceeds the number of allocated elements
and not the number of elements as indicated by the size field. As a
consequence, there might be less than three elements available at the
end, so that another check for this is necessary.

This commit changes this: It is always made sure that the three elements
reserved to (potentially) use them to store the special values are not
used to store ordinary tree entries. This allows to remove the extra
check at the end.

Reviewed-by: Paul B Mahol <onemda@gmail.com>
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
2020-09-18 01:32:46 +02:00
Andreas Rheinhardt 191b48e315 avcodec/smacker: Remove code duplication when decoding header trees
Reviewed-by: Paul B Mahol <onemda@gmail.com>
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
2020-09-18 01:27:39 +02:00
Andreas Rheinhardt 2cef031674 avcodec/smacker: Don't warn for Huffmann tables with one element
The Huffmann tables used by Smacker can consist of exactly one leaf only
in which case the length of the corresponding code is zero; there is
then exactly one value encoded. Our VLC can't handle this and therefore
this case needs to be treated separately; it has been implemented in
commit 48cbdaea15. Yet said commit also
made the decoder emit an error message (despite not erroring out) in this
case, although it seems that this is rather a limitation of our VLC API.

Reviewed-by: Paul B Mahol <onemda@gmail.com>
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
2020-09-18 01:09:25 +02:00
Andreas Rheinhardt 2cb5e3cff9 avcodec/smacker: Remove write-only and unused variables
Reviewed-by: Paul B Mahol <onemda@gmail.com>
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
2020-09-18 00:55:48 +02:00
Andreas Rheinhardt b146a61c22 avcodec/smacker: Add FF_CODEC_CAP_INIT_CLEANUP
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
2020-06-30 11:59:54 +02:00
Michael Niedermayer 6f5c18da59 avcodec/smacker: Check space before decoding type
Fixes: Timeout (232sec -> 280ms)
Fixes: 19682/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_SMACKER_fuzzer-5654129649385472

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2020-01-30 19:57:34 +01:00
Michael Niedermayer a76897e19c avcodec/smacker: Fix integer overflows in pred[] in smka_decode_frame()
Fixes: signed integer overflow: -2147481503 + -32732 cannot be represented in type 'int'
Fixes: 17782/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_SMACKAUD_fuzzer-5769672225456128

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2019-10-20 19:57:51 +02:00
Michael Niedermayer 5ce3c9eadc avcodec/smacker: cleanup on errors in smka_decode_frame()
Fixes: multiple memleaks
Fixes: 17660/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_SMACKAUD_fuzzer-5689769928949760
Fixes: 18064/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_SMACKAUD_fuzzer-5631086809317376

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2019-10-16 19:17:57 +02:00
Michael Niedermayer 033d2c4884 avcodec/smacker: Fix integer overflow in signed int multiply in SMK_BLK_FILL
Fixes: signed integer overflow: 238 * 16843009 cannot be represented in type 'int'
Fixes: 16958/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_SMACKER_fuzzer-5193905355620352

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2019-09-24 17:53:39 +02:00
James Almer 9ea6d2149e avcodec/decode: add a flags parameter to ff_reget_buffer()
Some decoders may not need a writable buffer in some specific cases, but only
a reference to the existing buffer with updated frame properties instead, for
the purpose of returning duplicate frames. For this, the
FF_REGET_BUFFER_FLAG_READONLY flag is added, which will prevent potential
allocations and buffer copies when they are not needed.

Signed-off-by: James Almer <jamrial@gmail.com>
2019-09-04 10:07:12 -03:00
Michael Niedermayer 1c9a56b139 avcodec/smacker: Check that not all tables are skiped
If all tables are skipped it would be impossible to encode any
"non black" video.

Fixes: Timeout (78sec -> 1ms)
Fixes: 15821/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_SMACKER_fuzzer-5652598838788096

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2019-08-23 22:26:50 +02:00
James Almer d2ad6f1192 Merge commit '0ccddbad200c1d9439c5a836501917d515cddf76'
* commit '0ccddbad200c1d9439c5a836501917d515cddf76':
  smacker: limit recursion depth of smacker_decode_bigtree

See 946ecd19ea

Merged-by: James Almer <jamrial@gmail.com>
2017-11-12 01:13:07 -03:00
James Almer b3e5899e47 Merge commit 'cd4663dc80323ba64989d0c103d51ad3ee0e9c2f'
* commit 'cd4663dc80323ba64989d0c103d51ad3ee0e9c2f':
  smacker: add sanity check for length in smacker_decode_tree()

See b829da3639

Merged-by: James Almer <jamrial@gmail.com>
2017-11-12 01:12:44 -03:00
Andreas Cadhalpun 0ccddbad20 smacker: limit recursion depth of smacker_decode_bigtree
This fixes segmentation faults due to stack-overflow caused by too deep
recursion.

Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
Signed-off-by: Sean McGovern <gseanmcg@gmail.com>
2017-11-10 20:41:32 -05:00
Michael Niedermayer cd4663dc80 smacker: add sanity check for length in smacker_decode_tree()
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>

Bug-Id: 1098
Cc: libav-stable@libav.org
Signed-off-by: Sean McGovern <gseanmcg@gmail.com>
2017-11-10 20:38:44 -05:00
James Almer bc98788dd2 Merge commit '5edded9df31bc4712a023f89941b4c278f1bd6f5'
* commit '5edded9df31bc4712a023f89941b4c278f1bd6f5':
  smacker: Improve error handling

See c1947015b2

Merged-by: James Almer <jamrial@gmail.com>
2017-11-01 14:13:04 -03:00
James Almer a33a15751e Merge commit 'b98f082d8ddc0a0d8317114d8414ab51de60ef02'
* commit 'b98f082d8ddc0a0d8317114d8414ab51de60ef02':
  smacker: Check that the data size is a multiple of a sample vector

See 4a9af07a49

Merged-by: James Almer <jamrial@gmail.com>
2017-11-01 14:07:48 -03:00
Diego Biurrun 5edded9df3 smacker: Improve error handling
Return sensible error values and forward error codes.
2017-06-01 14:22:01 +02:00
Michael Niedermayer b98f082d8d smacker: Check that the data size is a multiple of a sample vector
Fixes out of array access
Fixes: ce19e41f0ef1e52a23edc488faecdb58/asan_heap-oob_2504e97_4202_ffa0df1baed14022b9bfd4f8ac23d0cb.smk

Bug-Id: CVE-2015-8365
CC: libav-stable@libav.org

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 4a9af07a49)
Signed-off-by: Diego Biurrun <diego@biurrun.de>
2017-06-01 14:22:01 +02:00
Luca Barbato 73fc82f343 vlc: Add header #include when the types are used
Do not rely on indirectly including it from bitstream.h.

Signed-off-by: Diego Biurrun <diego@biurrun.de>
2017-05-02 18:38:26 +02:00
Diego Biurrun a25dac976a Use bitstream_init8() where appropriate 2017-02-07 18:27:21 +01:00
Alexandra Hájková 6efbc88a5c smacker: Convert to the new bitstream reader 2016-11-24 11:22:11 +01:00
Andreas Cadhalpun 946ecd19ea smacker: limit recursion depth of smacker_decode_bigtree
This fixes segmentation faults due to stack-overflow caused by too deep
recursion.

Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
2016-11-23 00:57:10 +01:00
Clément Bœsch ae753dbd0d Merge commit 'b668662939de3a02454cfc9ba3e6d10b87527a40'
* commit 'b668662939de3a02454cfc9ba3e6d10b87527a40':
  get_bits: Move BITSTREAM_READER_LE definition before all relevant #includes

The merge commit also includes changes for libavcodec/interplayacm.c and
libavcodec/truemotion2rt.c

Merged-by: Clément Bœsch <clement@stupeflix.com>
2016-06-29 11:35:10 +02:00
Diego Biurrun b668662939 get_bits: Move BITSTREAM_READER_LE definition before all relevant #includes
This avoids the danger that get_bits.h might get indirectly #included before
BITSTREAM_READER_LE is defined.

Also sort headers into canonical order where appropriate.
2016-06-07 13:09:57 +02:00
Derek Buitenhuis 5eb4073781 Merge commit '6b96d2dcdaa60d7919d710432c6ca204b7fab0ab'
* commit '6b96d2dcdaa60d7919d710432c6ca204b7fab0ab':
  cosmetics: Drop particularly redundant silly comments

Merged-by: Derek Buitenhuis <derek.buitenhuis@gmail.com>
2016-02-24 17:35:47 +00:00
Diego Biurrun 6b96d2dcda cosmetics: Drop particularly redundant silly comments 2016-02-18 15:35:43 +01:00
Michael Niedermayer 4a9af07a49 avcodec/smacker: Check that the data size is a multiple of a sample vector
Fixes out of array access
Fixes: ce19e41f0ef1e52a23edc488faecdb58/asan_heap-oob_2504e97_4202_ffa0df1baed14022b9bfd4f8ac23d0cb.smk

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2015-11-15 15:25:51 +01:00
Michael Niedermayer 21d8c6612f avcodec/smacker: Check init_get_bits8() for failure
Fixes: CID1322314

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2015-09-04 02:15:50 +02:00
Michael Niedermayer 444e9874a7 Merge commit 'def97856de6021965db86c25a732d78689bd6bb0'
* commit 'def97856de6021965db86c25a732d78689bd6bb0':
  lavc: AV-prefix all codec capabilities

Conflicts:
	cmdutils.c
	ffmpeg.c
	ffplay.c
	libavcodec/8svx.c
	libavcodec/aacenc.c
	libavcodec/ac3dec.c
	libavcodec/adpcm.c
	libavcodec/alac.c
	libavcodec/atrac3plusdec.c
	libavcodec/bink.c
	libavcodec/dnxhddec.c
	libavcodec/dvdec.c
	libavcodec/dvenc.c
	libavcodec/ffv1dec.c
	libavcodec/ffv1enc.c
	libavcodec/fic.c
	libavcodec/flacdec.c
	libavcodec/flacenc.c
	libavcodec/flvdec.c
	libavcodec/fraps.c
	libavcodec/frwu.c
	libavcodec/gifdec.c
	libavcodec/h261dec.c
	libavcodec/hevc.c
	libavcodec/iff.c
	libavcodec/imc.c
	libavcodec/libopenjpegdec.c
	libavcodec/libvo-aacenc.c
	libavcodec/libvorbisenc.c
	libavcodec/libvpxdec.c
	libavcodec/libvpxenc.c
	libavcodec/libx264.c
	libavcodec/mjpegbdec.c
	libavcodec/mjpegdec.c
	libavcodec/mpegaudiodec_float.c
	libavcodec/msmpeg4dec.c
	libavcodec/mxpegdec.c
	libavcodec/nvenc_h264.c
	libavcodec/nvenc_hevc.c
	libavcodec/pngdec.c
	libavcodec/qpeg.c
	libavcodec/ra288.c
	libavcodec/rv10.c
	libavcodec/s302m.c
	libavcodec/sp5xdec.c
	libavcodec/takdec.c
	libavcodec/tiff.c
	libavcodec/tta.c
	libavcodec/utils.c
	libavcodec/v210dec.c
	libavcodec/vp6.c
	libavcodec/vp9.c
	libavcodec/wavpack.c
	libavcodec/yop.c

Merged-by: Michael Niedermayer <michael@niedermayer.cc>
2015-07-27 22:50:18 +02:00
Vittorio Giovara def97856de lavc: AV-prefix all codec capabilities
Express bitfields more simply.

Signed-off-by: Vittorio Giovara <vittorio.giovara@gmail.com>
2015-07-27 15:24:58 +01:00
Lukasz Marek 969382162f lavc/smacker: fix mem leak in case of init failure
Signed-off-by: Lukasz Marek <lukasz.m.luki2@gmail.com>
2014-11-23 03:25:34 +01:00
Timothy Gu 0035783488 smacker: remove dead code
Signed-off-by: Timothy Gu <timothygu99@gmail.com>
Reviewed-by: Paul B Mahol <onemda@gmail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-07-18 13:31:19 +02:00
Michael Niedermayer 683b6e3183 avcodec/smacker: use av_mallocz_array()
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-04-18 20:43:02 +02:00
Michael Niedermayer b07a5e9b6b avformat/smacker: check for format mismatch more completely
Fixes use of uninitialized memory
Fixes: msan_uninit-mem_7f6e83322950_9769_wetlogo.smk
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2013-12-20 18:31:42 +01:00
Michael Niedermayer fe3808edde Merge commit 'a837c4f2df96a30bf9aa4115b426d608487c7101'
* commit 'a837c4f2df96a30bf9aa4115b426d608487c7101':
  zmbvenc: use the AVFrame API properly.
  flicvideo: use the AVFrame API properly.
  smacker: use the AVFrame API properly.
  mmvideo: use the AVFrame API properly.

Conflicts:
	libavcodec/flicvideo.c
	libavcodec/mmvideo.c
	libavcodec/smacker.c
	libavcodec/zmbvenc.c

See: 76e27b1d05
See: 099e57bc38
Merged-by: Michael Niedermayer <michaelni@gmx.at>
2013-11-17 01:55:38 +01:00
Anton Khirnov 84099f51f3 smacker: use the AVFrame API properly. 2013-11-16 13:23:38 +01:00
Michael Niedermayer ee77140afa Merge commit 'b2bed9325dbd6be0da1d91ffed3f513c40274fd2'
* commit 'b2bed9325dbd6be0da1d91ffed3f513c40274fd2':
  cosmetics: Group .name and .long_name together in codec/format declarations

Conflicts:
	libavcodec/8svx.c
	libavcodec/alac.c
	libavcodec/cljr.c
	libavcodec/dnxhddec.c
	libavcodec/dnxhdenc.c
	libavcodec/dpxenc.c
	libavcodec/dvdec.c
	libavcodec/dvdsubdec.c
	libavcodec/dvdsubenc.c
	libavcodec/ffv1dec.c
	libavcodec/flacdec.c
	libavcodec/flvdec.c
	libavcodec/fraps.c
	libavcodec/frwu.c
	libavcodec/g726.c
	libavcodec/gif.c
	libavcodec/gifdec.c
	libavcodec/h261dec.c
	libavcodec/h263dec.c
	libavcodec/iff.c
	libavcodec/imc.c
	libavcodec/libopencore-amr.c
	libavcodec/libopenjpegdec.c
	libavcodec/libopenjpegenc.c
	libavcodec/libspeexenc.c
	libavcodec/libvo-amrwbenc.c
	libavcodec/libvorbisenc.c
	libavcodec/libvpxenc.c
	libavcodec/libx264.c
	libavcodec/libxavs.c
	libavcodec/libxvid.c
	libavcodec/ljpegenc.c
	libavcodec/mjpegbdec.c
	libavcodec/mjpegdec.c
	libavcodec/mpeg12dec.c
	libavcodec/mpeg4videodec.c
	libavcodec/msmpeg4dec.c
	libavcodec/pgssubdec.c
	libavcodec/pngdec.c
	libavcodec/pngenc.c
	libavcodec/proresdec_lgpl.c
	libavcodec/proresenc_kostya.c
	libavcodec/ra144enc.c
	libavcodec/rawdec.c
	libavcodec/rv10.c
	libavcodec/sp5xdec.c
	libavcodec/takdec.c
	libavcodec/tta.c
	libavcodec/v210dec.c
	libavcodec/vp6.c
	libavcodec/wavpack.c
	libavcodec/xbmenc.c
	libavcodec/yop.c

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2013-10-04 12:34:23 +02:00
Diego Biurrun b2bed9325d cosmetics: Group .name and .long_name together in codec/format declarations 2013-10-03 23:32:01 +02:00
Paul B Mahol 1b99514dfd avcodec/smacker: use init_get_bits8()
Signed-off-by: Paul B Mahol <onemda@gmail.com>
2013-09-12 16:34:54 +00:00
Michael Niedermayer f144168d9b avcodec/smacker: fix some memleaks
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2013-09-12 16:17:32 +02:00
Michael Niedermayer fbab9a7c22 avcodec/smacker: remove check that has become redundant
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2013-09-12 16:17:32 +02:00
Michael Niedermayer 59a08d1939 avcodec/smacker: use the correct field instead of a hardcoded value for the index check in smacker_decode_tree()
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2013-09-12 16:17:32 +02:00
Michael Niedermayer af539319af Merge commit 'f3d57dc69145f1b7acb4870da9ce60378190a1fd'
* commit 'f3d57dc69145f1b7acb4870da9ce60378190a1fd':
  smacker: Free memory properly if the init function fails

Conflicts:
	libavcodec/smacker.c

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2013-09-12 12:57:58 +02:00
Michael Niedermayer f8932d9902 Merge commit '0679cec6e8802643bbe6d5f68ca1110a7d3171da'
* commit '0679cec6e8802643bbe6d5f68ca1110a7d3171da':
  smacker: Make sure we don't fill in huffman codes out of range

Conflicts:
	libavcodec/smacker.c

See: 1285baaab5
the added test is redundant but will be used after future refactoring

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2013-09-12 12:51:07 +02:00
Michael Niedermayer d510d70df0 Merge commit 'd002fce24a7206e95f9492031f45adc35711e183'
* commit 'd002fce24a7206e95f9492031f45adc35711e183':
  smacker: Check malloc return values

Conflicts:
	libavcodec/smacker.c

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2013-09-12 11:55:08 +02:00