The initialization of the uint32_t plane32_lut matrix uses left shifts
of the form 1 << plane; plane can be as big as 31 which means that this
is undefined behaviour as 1 will be simply an int. So make it unsigned
to avoid this.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
Reviewed-by: Paul B Mahol <onemda@gmail.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Fixes: signed integer overflow: 1603085316 + 1238786562 cannot be represented in type 'int'
Fixes: 16203/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_RALF_fuzzer-5086088934195200
Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
It appears no valid file uses this, so this is not testable with
a valid file.
Fixes: assertion failure
Fixes: 16187/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_VORBIS_fuzzer-5638880618872832
Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Fixes: out of array write
Fixes: 16105/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_CFHD_fuzzer-5690817309573120
Fixes: 16119/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_CFHD_fuzzer-5099050675732480
Fixes: 16135/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_CFHD_fuzzer-5705501601431552
Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Fixes: index -1 out of bounds for type 'AV1ReferenceFrameState [8]'
Fixes: 16079/clusterfuzz-testcase-minimized-ffmpeg_DEMUXER_fuzzer-5758807440883712
Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Reviewed-by: James Almer <jamrial@gmail.com>
See: [FFmpeg-devel] [PATCH 05/13] avcodec/cbs_av1_syntax_template: Check ref_frame_idx before use
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Fixes: left shift of 1073741824 by 1 places cannot be represented in type 'int'
Fixes: signed integer overflow: 2049431315 + 262759074 cannot be represented in type 'int'
Fixes: 16012/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_APE_fuzzer-5719016003338240
Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Alternatively the array could be made bigger but the extra values
would not be read without other changes.
Fixes: Out of array access
Fixes: 15658/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_ATRAC9_fuzzer-5738260074070016
Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Reviewed-by: Lynne <dev@lynne.ee>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Recently we encountered an issue when decoding a h.263 file:
FFmpeg will freeze when decoding h.263 video with NVDEC. Turns out this is not directly related to NVDEC but is a problem that shows with several other HW decoders like VDPAU, though the exact kind of error is different (either error messages or freezing[1]). The root cause is that ff_thread_finish_setup() is called twice per frame from ff_h263_decode_frame(). This is not supported by ff_thread_finish_setup() and specifically checked for and warned against in the functions code. The issue is also specific to hw accelerated decoding only as the second call to ff_thread_finish_setup() is only issued when hw acceleration is on. The fix is simple: add a check that the first call is only send when hw acceleration is off, and the second call only when hw acceleration is on (see attached patch). This works fine as far as I was able to test with vdpau and nvdec/nvcuvid hw decoding. The patch also adds NVDEC to the hw config list if available.
I also noticed a secondary issue when browsing through the code which is that, according to documentation, ff_thread_finish_setup() should only be called if the codec implements update_thread_context(), which h263dec does not. The patch does not address this and I'm not sure any action needs to be taken here at all.
[1] This is depending on whether or not the hw decoder sets the HWACCEL_CAPS_ASYNC_SAFE flag
Signed-off-by: Timo Rothenpieler <timo@rothenpieler.org>
Theora is forced to be non zero if it is zero and a sample
is asked for, as suggested by reimar
Fixes: Timeout (2min -> 600ms)
Fixes: 15366/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_THEORA_fuzzer-5737849938247680
Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Fixes: signed integer overflow: 32796 * 65536 cannot be represented in type 'int'
Fixes: 15430/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_VC1IMAGE_fuzzer-5735424087031808
Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Fixes: 15524/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_TRUEMOTION2_fuzzer-5173148372172800
Fixes: signed integer overflow: 13701388 - -2134868270 cannot be represented in type 'int'
Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Changing the type to an unsigned one to avoid many casts was suggested
This may be inadequate for fixing the UB on ILP64
Fixes: signed integer overflow: -1418162611 * 383 cannot be represented in type 'int'
Fixes: 15547/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_APE_fuzzer-5691384901664768
Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Fixes: signed integer overflow: 755176387 + 1515360583 cannot be represented in type 'int'
Fixes: 15506/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_APE_fuzzer-5706859232624640
Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Fixes: left shift of negative value -4
Fixes: signed integer overflow: -15091694 * 167 cannot be represented in type 'int'
Fixes: signed integer overflow: 1898547155 + 453967445 cannot be represented in type 'int'
Fixes: 15258/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_APE_fuzzer-5759095564402688
Fixes: signed integer overflow: 962196438 * 31 cannot be represented in type 'int'
Fixes: 15364/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_APE_fuzzer-5718799845687296
Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
There is one way to find out if avpriv_find_start_code has found a start
code or not: One has to check whether the state variable contains a
start code, i.e. whether the three most significant bytes are 0x00 00 01.
Checking for whether the return value is the end of the designated
buffer is not enough: If the last four bytes constitute a start code,
the return value is also the end of the buffer. This happens with
sequence_end_codes which have been ignored for exactly this reason,
although e.g. all three files used for fate tests of cbs_mpeg2 contain
sequence_end_codes.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
1. Currently, cbs_mpeg2_split_fragment uses essentially three variables
to hold the start code values found by avpriv_find_start_code. By
rearranging the code, one of them can be omitted.
2. The return value of avpriv_find_start_code points to the byte after
the byte containing the start code identifier (or to the byte after the
last byte of the fragment's data if no start code was found), but
cbs_mpeg2_split_fragment needs to work with the pointer to the byte
containing the start code identifier; it already did this, but in a
clumsy way. This has been changed.
3. Also use the correct type for the variable holding the
CodedBitstreamUnitType.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
Sequence End units (or actually, sequence_end_codes) have up until now
not been decomposed; in fact due to a bug in cbs_mpeg2_split_fragment they
have mostly been treated as part of the preceding unit. So implement
decomposing them as preparation for fixing said bug.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
Up until now, if an SEI messages was to be added to a fragment, it was
tried to add said SEI message to the first SEI NAL unit of the fragment
and if this SEI NAL unit already contained H264_NAL_SEI SEI messages (an
arbitrary limit imposed by cbs_h264), adding failed; if there was no SEI
NAL unit, a new one has been added.
With this commit, the fragment is searched for further NAL units to add
the SEI messages to. If all of them are full, a new SEI NAL unit is added.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
cbs is currently inconsistent regarding the opaque field that can be
used as a special argument to av_buffer_create in order to be used
during freeing the buffer: ff_cbs_alloc_unit_content and all the free
functions used name this parameter as if it should contain a pointer to
the unit whose content is about to be created; but both
ff_cbs_alloc_unit_content as well as ff_cbs_h264_add_sei_message
actually use a pointer to the CodedBitstreamContext as opaque. It should
actually be neither, because it is unneeded (as is evidenced by the fact
that none of the free functions use this pointer at all) and because it
ties the unit's content to the lifetime of other objects, although a
refcounted buffer is supposed to have its own lifetime that only ends
when its reference count reaches zero. This problem manifests itself in
the pointer becoming dangling.
The pointer to the unit can become dangling if another unit is added to
the fragment later as happens in the bitstream filters; in this case,
the pointer can point to the wrong unit (if the fragment's unit array
needn't be relocated) or it can point to where the array was earlier.
It can also become dangling if the unit's content is meant to survive
the resetting of the fragment it was originally read with. This applies
to the extradata of H.264 and HEVC.
The pointer to the context can become dangling if the context is closed
before the content is freed. Although this doesn't seem to happen right
now, it could happen, in particular if one uses different
CodedBitstreamContexts for in- and output.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
The minimum header is 8 bytes, the smallest bitstream that is passed to
the MB decode code is 4 bytes
Fixes: Timeout (35sec -> 18sec)
Fixes: 15800/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_EATQI_fuzzer-5684154517159936
Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Reviewed-by: Paul B Mahol <onemda@gmail.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
The minimal size which unpack() will not fail on is 5 bytes
Fixes: Timeout (14sec -> 77ms) (testcase 15508)
Fixes: 15508/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_EATGV_fuzzer-5700053513011200
Fixes: 15996/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_EATGV_fuzzer-5751353223151616
Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
In function ff_h264_add_pixels4_8_mmi, there is no need to reset '%[ftmp0]'
to 0, because it's value has never changed since the start of the asm block.
This patch remove the redundant 'xor' and set src to zero once it was loaded.
In function ff_h264_idct_add_8_mmi, 'block' is seted to zero twice.
This patch removed the first setting zero operation and move the second one
after the load operation of block.
In function ff_h264_idct8_add_8_mmi, 'block' is seted to zero twice too.
This patch just removed the second setting zero operation.
This patch mainly simplifies the implementation of functions above,
the effect on the performance of whole h264 decoding process is not obvious.
According to the perf data, proportion of ff_h264_idct_add_8_mmi decreased from
0.29% to 0.26% and ff_h264_idct8_add_8_mmi decreased from 0.62% to 0.59% when decoding
H264 format on loongson 3A3000(For reference only , not very stable.).
Reviewed-by: Reimar Döffinger <Reimar.Doeffinger@gmx.de>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
The earlier version had three deficits:
1. It allowed to set the stream to RGB although this is not allowed when
the profile is 0 or 2.
2. If it set the stream to RGB, then it did not automatically set the
range to full range; the result was that one got a warning every time a
frame with color_config element was processed if the frame originally
had TV range and the user didn't explicitly choose PC range. Now one
gets only one warning in such a situation.
3. Intra-only frames in profile 0 are automatically BT.601, but if the
user wished another color space, he was not informed about his wishes
being unfulfillable.
The commit also improves the documentation about this.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
Michael Niedermayer