If a static VLC table gets initialized a second time (or concurrently by
two threads) and if said VLC table uses symbols that have the sign bit
of VLC_TYPE (a typedef for int16_t) set, initializing the VLC fails. The
reason is that the type of the symbol in the temporary array is an
uint16_t and so comparing it to the symbol read from the VLC table will
fail, because only the lower 16bits coincide. Said failure triggers an
assert.
Reviewed-by: Lynne <dev@lynne.ee>
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
The longest code of any of the VLC tables used is eight bits long, so
using nine bits long VLC tables is wasteful. Furthermore, there are only
seven VLC tables used, yet the code up until now made it look like there
should be eight. This has been corrected, too.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
Even though the length of these codes is > 8, only the lowest seven bits
are ever set (because the long codes are on the left of the tree), so
one can use an uint8_t for them, saving space.
Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
The longest motion vector VLC for mobiclip is six bits long, so using
eight bits for the VLC table is wasteful. Furthermore, the length can be
inlined.
Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
For both RealVideo 3.0 as well as RealVideo 4.0 the VLC table to use
depends upon the slice's quantization parameter; these are coded on five
bits in the bitstream and are therefore in the range of 0..31; yet the
last element here is not valid and therefore the quantizer is clipped to
the range 0..30 to get the index. But this is unnecessary: One can just
add one element more to the relevant array to avoid the clipping.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
Most of the VLCs used by RealVideo 3 and 4 obey three simple rules:
Shorter codes are on the left of the tree, for each length, the symbols
are ascending from left to right and the symbols either form a
permutation of 1..size or 0..(size - 1). For the latter case, one just
needs to store the length of each symbol and create the codes according
to the other rules; no explicit code or symbol array must be stored.
The former case is also treated in much the same way by artificially
assigning a length of zero to the symbol 0; when a length of zero was
encountered, the element was ignored except that the symbol counter was
still incremented. If the length was nonzero, the symbol would be
assigned via the symbol counter and the length copied over into a new
array.
Yet this is unnecessary, as ff_init_vlc_sparse() follows exactly the
same pattern: If a length of zero is encountered, the element is ignored
and only the symbol counter incremented. So one can directly forward the
length array and also need not create a symbol table oneself, because
ff_init_vlc_sparse() will infer the same symbol table in this case.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
Theora allows to use custom Huffman tables which are coded in the
bitstream as a tree: Whether the next node is a leaf or not is coded
in a bit; each node itself contains a five bit token. Each tree can
contain at most 32 leafs; typically they contain exactly 32 with the 32
symbols forming a permutation of 0..31. Yet the standard does not impose
either of these requirements. It explicitly allows less than 32 leafs
and multiple codes with the same token.
But our decoder used an algorithm that required the codes->token mapping
to be injective and that also presumed that there be at least two leafs:
Instead of using an array for codes, tokens and code lengths, the
decoder only had arrays for codes and code lengths. The code and length
for a given token were stored in entry[token]. As no symbols table was
used when initializing the VLC, the default one applied and therefore
the entry[token] got the symbol token (if the length of said entry is >0).
Yet if multiple codes had the same token, the codes and lengths from the
later token would overwrite the earlier codes and lengths.
Furthermore, less than 32 leafs could also lead to problems: Namely if
this was not the first time Huffman tables have been parsed in which
case the array is not zeroed initially so that old entries could make
the new table invalid.
libtheora seems to always use 32 leafs and no duplicate tokens; I am not
aware of any existing valid files that do not.
This is fixed by using a codes, symbols and lengths array when
initializing the VLC. In order to reduce the amount of stuff kept in the
context only the symbols and lengths (which both fit into an uint8_t)
are kept in the context; the codes are derived from the lengths
immediately before creating the tables.
There is now only one thing left which is not spec-compliant: Trees with
only one node (which has length zero) are not supported by
ff_init_vlc_sparse() yet.
Reviewed-by: Peter Ross <pross@xvid.org>
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
The longest codes of any VLC codebooks are 18 bits long and the VLC
tables itself use 9 bits; therefore it is sufficient to read twice from
the table, yet this has been done thrice.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
Up until now, quad VLCs are initialized with codes of type uint32_t,
pair VLCs with codes of type uint16_t. There were two separate loops in
the decoder's init function for each type of VLC. This commit unifies
this: The type of the codes are now passed in as void * and the actual
size of the codes is obtained from a table. This approach also allows to
use the smallest type for each VLC code table: some quad tables actually
fitted in uint16_t. This allows to remove about 7KB from the binary.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
Up until now, there was no cleanup in case initializing the Theora VLC
tables failed, leading to memleaks. This commit gets rid of them by
setting the FF_CODEC_CAP_INIT_CLEANUP flag for all decoders in vp3.c;
this also allows to remove some (now redundant) cleanup code.
Reviewed-by: Peter Ross <pross@xvid.org>
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
Fixes: division by zero
Fixes: 26293/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_ADPCM_PSX_fuzzer-5176665237618688
Fixes: 26331/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_ADPCM_PSX_fuzzer-5632330364092416
Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Fixes: signed integer overflow: 536870912 + 1610612736 cannot be represented in type 'int'
Fixes: 26288/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_MOBICLIP_fuzzer-6194364759670784
Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Fixes: signed integer overflow: 553590816 - -2145378049 cannot be represented in type 'int'
Fixes: 26315/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_EXR_fuzzer-5938755121446912
Fixes: 26340/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_EXR_fuzzer-5644316208529408
Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Fixes: signed integer overflow: 241173056 + 1953511200 cannot be represented in type 'int'
Fixes: 26086/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_H264_fuzzer-5068366420901888
Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
It makes no sense to call the functions to write styl, hlit or hclr boxes
with a different box name than "styl", "hlit" or "hclr". Therefore this
commit inlines these values in the functions, removes the function
parameter containing the box's name and removes the (non obsolete) box
names from the list of boxes.
Reviewed-by: Philip Langdale <philipl@overt.org>
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
Andreas Rheinhardt