I went through all codecs and put them into five basic categories:
1. JPEG range only
2. MPEG range only
3. Explicitly tagged
4. Broken (codec supports both but encoder ignores tags)
5. N/A (headerless or pseudo-formats)
Filters in category 5 remain untouched. The rest gain an explicit
assignment of their supported color ranges, with codecs in category
4 being set to MPEG-only for safety.
It might be considered redundant to distinguish between 0 (category 5)
and MPEG+JPEG (category 3), but in doing so we effectively communicate
that we can guarantee that these tags will be encoded, which is distinct
from the situation where there are some codecs that simply don't have
tagging or implied semantics (e.g. rawvideo).
A full list of codecs follows:
JPEG range only:
- amv
- roqvideo
MPEG range only:
- asv1, asv2
- avui
- cfhd
- cljr
- dnxhd
- dvvideo
- ffv1
- flv
- h261, h263, h263p
- {h263,vp8}_v4l2m2m
- huffyuv, ffvhuff
- jpeg2000
- libopenjpeg
- libtheora
- libwebp, libwebp_anim
- libx262
- libxavs, libxavs2
- libxvid
- mpeg1video, mpeg2video
- mpeg2_qsv
- mpeg2_vaapi
- mpeg4, msmpeg4, msmpeg4v2, wmv1, wmv2
- mpeg4_omx
- prores, prores_aw, prores_ks
- rv10, rv20
- snow
- speedhq
- svq1
- tiff
- utvideo
Explicitly tagged (MPEG/JPEG):
- {av1,h264,hevc}_nvenc
- {av1,h264,hevc}_vaapi
- {av1,h264,hevc,vp8,vp9,mpeg4}_mediacodec
- {av1,h264,hevc,vp9}_qsv
- h264_amf
- {h264,hevc,prores}_videotoolbox
- libaom-av1
- libkvazaar
- libopenh264
- librav1e
- libsvtav1
- libvpx, libvpx-vp9
- libx264
- libx265
- ljpeg
- mjpeg
- vc2
Broken (encoder ignores tags):
- {av1,hevc}_amf
- {h264,hevc,mpeg4}_v4l2m2m
- h264_omx
- libxeve
- magicyuv
- {vp8,vp9,mjpeg}_vaapi
N/A:
- ayuv, yuv4, y41p, v308, v210, v410, v408 (headerless)
- pgmyuv (headerless)
- rawvideo, bitpacked (headerless)
- vnull, wrapped_avframe (pseudocodecs)
This avoids using MpegEncContext.mecc; it already allows
to avoid touching the latter for snowenc and svq1enc.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@outlook.com>
Several of the potential choices of comparison functions
need an initialized MpegEncContext (initialized for encoding,
not only ff_mpv_common_init()) or they crash when called.
Modify ff_set_cmp() to check for this.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@outlook.com>
The majority of the stuff performed in it needs to be done only
once; so factor it out into a function of its own to be called
in the user's init function.
Also avoid using MpegEncContext in it, to separate the two
a bit more.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@outlook.com>
Only set the ScratchpadContext and let the users that need it
(i.e. encoders) set the MotionEstContext stuff themselves.
Also add an explicit pointer to ScratchpadContext to point
to the allocated buffer so that none of the other scratchpad
pointers is singled out as being used for the allocations.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@outlook.com>
Up until now, an initialized MpegEncContext had an array of
MPVPictures (way more than were ever needed) and the MPVPicture*
contained in the MPVWorkPictures as well as the input_picture
and reordered_input_picture arrays (for the encoder) pointed
into this array. Several of the pointers could point to the
same slot and because there was no reference counting involved,
one had to check for aliasing before unreferencing.
Furthermore, given that these pointers were not ownership pointers
the pointers were often simply reset without unreferencing
the slot (happened e.g. for the RV30 and RV40 decoders) or
there were moved without resetting the src pointer (happened
for the encoders where the entries in the input_picture
and reordered_input_picture arrays were not reset).
Instead actually releasing these pictures was performed by looping
over the whole array and checking which one of the entries needed
to be kept. Given that the array had way too many slots (36),
this meant that more than 30 MPVPictures have been unnecessarily
unreferenced in every ff_mpv_frame_start(); something similar
happened for the encoder.
This commit changes this by making the MPVPictures refcounted
via the RefStruct API. The MPVPictures itself are part of a pool
so that this does not entail constant allocations; instead,
the amount of allocations actually goes down, because the
earlier code used such a large array of MPVPictures (36 entries) and
allocated an AVFrame for every one of these on every
ff_mpv_common_init(). In fact, the pool is only freed when closing
the codec, so that reinitializations don't lead to new allocations
(this avoids having to sync the pool in update_thread_context).
Making MPVPictures refcounted also has another key benefit:
It makes it possible to directly share them across threads
(when using frame-threaded decoding), eliminating ugly code
with underlying av_frame_ref()'s; sharing these pictures
can't fail any more.
The pool is allocated in ff_mpv_decode_init() for decoders,
which therefore can fail now. This and the fact that the pool
is not unreferenced in ff_mpv_common_end() also necessitated
to mark several mpegvideo-decoders with the FF_CODEC_CAP_INIT_CLEANUP
flag.
*: This also means that there is no good reason any more for
ff_mpv_common_frame_size_change() to exist.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@outlook.com>
There are two types of MPVPictures: Three (cur_pic, last_pic, next_pic)
that are directly part of MpegEncContext and an array of MPVPictures
that are separately allocated and are mostly accessed via pointers
(cur|last|next)_pic_ptr; they are also used to store AVFrames in the
encoder (necessary due to B-frames). As the name implies, each of the
former is directly associated with one of the _ptr pointers:
They actually share the same underlying buffers, but the ones
that are part of the context can have their data pointers offset
and their linesize doubled for field pictures.
Up until now, each of these had their own references; in particular,
there was an underlying av_frame_ref() to sync cur_pic and cur_pic_ptr
etc. This is wasteful.
This commit changes this relationship: cur_pic, last_pic and next_pic
now become MPVWorkPictures; this structure does not have an AVFrame
at all any more, but only the cached values of data and linesize.
It also contains a pointer to the corresponding MPVPicture, establishing
a more natural relationsship between the two.
This already means that creating the context-pictures from the pointers
can no longer fail.
What has not been changed is the fact that the MPVPicture* pointers
are not ownership pointers and that the MPVPictures are part of an
array of MPVPictures that is owned by a single AVCodecContext.
Doing so will be done in a latter commit.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@outlook.com>
This avoids an indirection and is in preparation for removing
the AVFrame from MpegEncContext.(cur|last|next)_pic altogether.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@outlook.com>
current_picture->cur_pic, last_picture->last_pic, similarly
for new_picture and next_picture.
Also rename the corresponding *_ptr fields.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@outlook.com>
There are lots of files that don't need it: The number of object
files that actually need it went down from 2011 to 884 here.
Keep it for external users in order to not cause breakages.
Also improve the other headers a bit while just at it.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@outlook.com>
Otherwise svq1enc.o gets pulled in by the svq1encdsp checkasm
test and it in turn pulls the rest of lavc in.
Besides being bad size-wise this also has the downside that
it pulls in avpriv_(cga|vga16)_font from libavutil which are
marked as being imported from another library when building
libavcodec as a DLL and this breaks checkasm because it links
both lavc and lavu statically.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@outlook.com>
It used to be used with preallocated packet buffers with
the old encode API, but said API is no more and therefore
there is no reason for this to be public any more.
So deprecate it and use an internal replacement
for the encoders using it as an upper bound for the
size of their headers.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@outlook.com>
Makes it robust against adding fields before it, which will be useful in
following commits.
Majority of the patch generated by the following Coccinelle script:
@@
typedef AVOption;
identifier arr_name;
initializer list il;
initializer list[8] il1;
expression tail;
@@
AVOption arr_name[] = { il, { il1,
- tail
+ .unit = tail
}, ... };
with some manual changes, as the script:
* has trouble with options defined inside macros
* sometimes does not handle options under an #else branch
* sometimes swallows whitespace
Unnecessary since acf63d5350adeae551d412db699f8ca03f7e76b9;
also avoids relocations.
Reviewed-by: Anton Khirnov <anton@khirnov.net>
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@outlook.com>
It is only used by encoders; this unfortunately necessitated
to add separate allocations to the SVQ1 encoder which uses
motion estimation without being a full member of mpegvideo.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@outlook.com>
Frame counters can overflow relatively easily (INT_MAX number of frames is
slightly more than 1 year for 60 fps content), so make sure we use 64 bit
values for them.
Also deprecate the old 32 bit frame_number attribute.
Signed-off-by: Marton Balint <cus@passwd.hu>
GCC 11 has a bug: When it creates clones of recursive functions
(to inline some parameters), it clones a recursive function
eight times by default, even when this exceeds the recursion
depth. This happens with encode_block() in libavcodec/svq1enc.c
where a parameter level is always in the range 0..5;
but GCC 11 also creates functions corresponding to level UINT_MAX
and UINT_MAX - 1 (on -O3; -O2 is fine).
Using such levels would produce undefined behaviour and because
of this GCC emits bogus -Warray-bounds warnings for these clones.
Since commit d08b2900a9, certain
symbols that are accessed like ff_svq1_inter_multistage_vlc[level]
are declared with hidden visibility, which allows compilers
to bake the offset implied by level into the instructions
if level is a compile-time constant as it is in the clones.
Yet this leads to insane offsets for level == UINT_MAX which
can be incompatible with the supported offset ranges of relocations.
This happens in the small code model (the default code model for
AArch64).
This commit therefore works around this bug by disabling cloning
recursive functions for GCC 10 and 11. GCC 10 is affected by the
underlying bug (see
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=102513), so the workaround
also targets it, although it only produces three versions of
encode_block(), so it does not seem to trigger the actual issue here.
The issue has been mitigated in GCC 12.1 (it no longer creates clones
for impossible values; see also commit
1cb7fd317c), so the workaround
does not target it.
Reported-by: J. Dekker <jdek@itanimul.li>
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@outlook.com>
Signed-off-by: J. Dekker <jdek@itanimul.li>
This is more natural, because said context is only used
for the duration of one call to svq1_encode_frame().
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@outlook.com>
Currently, SVQ1EncContext is defined in a header that is also
included by the arch-specific code that initializes the one
and only dsp function that this encoder uses directly.
But the arch-specific functions to set this dsp function
do not need anything from SVQ1EncContext. This commit therefore
adds a small SVQ1EncDSPContext whose only member is said
function pointer and renames svq1enc.h to svq1encdsp.h
to avoid exposing unnecessary internals to these init
functions (and the whole mpegvideo with it).
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@outlook.com>
It reduces typing: Before this patch, there were 105 codecs
whose long_name-definition exceeded the 80 char line length
limit. Now there are only nine of them.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@outlook.com>
Up until now, these encoders received non-refcounted packets
(whose data was owned by the corresponding AVCodecContext)
from ff_alloc_packet(); these packets were made refcounted lateron
by av_packet_make_refcounted() generically.
This commit makes these encoders accept user-supplied buffers by
replacing av_packet_make_refcounted() with an equivalent function
that is based upon get_encode_buffer().
(I am pretty certain that one can also set the flag for mpegvideo-
based encoders, but I want to double-check this later. What is certain
is that it reallocates the buffer owned by the AVCodecContext
which should maybe be moved to encode.c, so that proresenc_kostya.c
and ttaenc.c can make use of it, too.)
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@outlook.com>
These tables are only used by encoders and only for the current picture;
ergo they need not be put into the picture at all, but rather into
the encoder's context. They also don't need to be refcounted,
because there is only one owner.
In contrast to this, the earlier code refcounts them which
incurs unnecessary overhead. These references are not unreferenced
in ff_mpeg_unref_picture() (they are kept in order to have something
like a buffer pool), so that several buffers are kept at the same
time, although only one is needed, thereby wasting memory.
The code also propagates references to other pictures not part of
the pictures array (namely the copy of the current/next/last picture
in the MpegEncContext which get references of their own). These
references are not unreferenced in ff_mpeg_unref_picture() (the
buffers are probably kept in order to have something like a pool),
yet if the current picture is a B-frame, it gets unreferenced
at the end of ff_mpv_encode_picture() and its slot in the picture
array will therefore be reused the next time; but the copy of the
current picture also still has its references and therefore
these buffers will be made duplicated in order to make them writable
in the next call to ff_mpv_encode_picture(). This is of course
unnecessary.
Finally, ff_find_unused_picture() is supposed to just return
any unused picture and the code is supposed to work with it;
yet for the vsynth*-mpeg4-adap tests the result depends upon
the content of these buffers; given that this patchset
changes the content of these buffers (the initial content is now
the state of these buffers after encoding the last frame;
before this patch the buffers used came from the last picture
that occupied the same slot in the picture array) their ref-files
needed to be changed. This points to a bug somewhere (if one removes
the initialization, one gets uninitialized reads in
adaptive_quantization in ratecontrol.c).
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@outlook.com>
encode_block() in svq1enc.c looks like the following:
static int encode_block(int block[7][256], int level)
{
int best_score = 0;
for (unsigned x = 0; x < level; x++) {
int v = block[1][x];
block[level][x] = 0;
best_score += v * v;
}
if (level > 0 && best_score > 64) {
int score = 0;
score += encode_block(block, level - 1);
score += encode_block(block, level - 1);
if (score < best_score) {
best_score = score;
}
}
return best_score;
}
When called from outside of encode_block(), it is always called with
level == 5.
This triggers a bug [1] in GCC: On -O3, it creates eight clones of
encode_block with different values of level inlined into it. The clones
with negative values are of course useless*, but they also lead to
-Warray-bounds warnings, because they access block[-1].
This has been mitigated in GCC 12: It no longer creates clones
for parameters that it knows are impossible. Somehow switching levels
to unsigned makes GCC know this. Therefore this commit does this.
(For GCC 11, this changes the warning to "array subscript 4294967295 is
above array bounds" from "array subscript -1 is below array bounds".)
[1]: https://gcc.gnu.org/bugzilla/show_bug.cgi?id=102513
*: These clones can actually be discarded when compiling with
-ffunction-sections.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@outlook.com>
and remove FF_CODEC_CAP_INIT_THREADSAFE
All our native codecs are already init-threadsafe
(only wrappers for external libraries and hwaccels
are typically not marked as init-threadsafe yet),
so it is only natural for this to also be the default state.
Reviewed-by: Anton Khirnov <anton@khirnov.net>
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@outlook.com>
This is more spec-compliant because it does not rely
on dead-code elimination by the compiler. Especially
MSVC has problems with this, as can be seen in
https://ffmpeg.org/pipermail/ffmpeg-devel/2022-May/296373.html
or
https://ffmpeg.org/pipermail/ffmpeg-devel/2022-May/297022.html
This commit does not eliminate every instance where we rely
on dead code elimination: It only tackles branching to
the initialization of arch-specific dsp code, not e.g. all
uses of CONFIG_ and HAVE_ checks. But maybe it is already
enough to compile FFmpeg with MSVC with whole-programm-optimizations
enabled (if one does not disable too many components).
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@outlook.com>
Several encoders (roqvideo, svq1, snow, and the mpegvideo family)
currently call ff_get_buffer(). However this function is written
assuming it is called by a decoder. Though nothing has been obviously
broken by this until now, that may change in the future.
To avoid potential future issues, introduce a simple encode-specific
wrapper around avcodec_default_get_buffer2() and enforce its use in
encoders.
This is possible, because every given FFCodec has to implement
exactly one of these. Doing so decreases sizeof(FFCodec) and
therefore decreases the size of the binary.
Notice that in case of position-independent code the decrease
is in .data.rel.ro, so that this translates to decreased
memory consumption.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@outlook.com>
It is currently a "Picture", an mpegvideo-specific type
that has a lot of baggage, all of which is unnecessary
for new_picture, because only its embedded AVFrame
is ever used. So just use an ordinary AVFrame.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@outlook.com>
Up until now, codec.h contains both public and private parts
of AVCodec. This exposes the internals of AVCodec to users
and leads them into the temptation of actually using them
and forces us to forward-declare structures and types that
users can't use at all.
This commit changes this by adding a new structure FFCodec to
codec_internal.h that extends AVCodec, i.e. contains the public
AVCodec as first member; the private fields of AVCodec are moved
to this structure, leaving codec.h clean.
Reviewed-by: Anton Khirnov <anton@khirnov.net>
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@outlook.com>
Also move FF_CODEC_TAGS_END as well as struct AVCodecDefault.
This reduces the amount of files that have to include internal.h
(which comes with quite a lot of indirect inclusions), as e.g.
most encoders don't need it. It is furthemore in preparation
for moving the private part of AVCodec out of the public codec.h.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@outlook.com>
Up until now, ff_alloc_packet2() has a min_size parameter:
It is supposed to be a lower bound on the final size of the packet
to allocate. If it is not too far from the upper bound (namely,
if it is at least half the upper bound), then ff_alloc_packet2()
already allocates the final, already refcounted packet; if it is
not, then the packet is not refcounted and its data only points to
a buffer owned by the AVCodecContext (in this case, the packet will
be made refcounted in encode_simple_internal() in libavcodec/encode.c).
The goal of this was to avoid data copies and intermediate buffers
if one has a precise lower bound.
Yet those encoders for which precise lower bounds exist have recently
been switched to ff_get_encode_buffer() (which automatically allocates
final buffers), leaving only two encoders to actually set the min_size
to something else than zero (namely aliaspixenc and hapenc). Both of
these encoders use a very low lower bound that is not helpful in any
nontrivial case.
This commit therefore removes the min_size parameter as well as the
codepath in ff_alloc_packet2() for the allocation of final buffers.
Furthermore, the function has been renamed to ff_alloc_packet() and
moved to encode.h alongside ff_get_encode_buffer().
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@outlook.com>
Given that the AVCodec.next pointer has now been removed, most of the
AVCodecs are not modified at all any more and can therefore be made
const (as this patch does); the only exceptions are the very few codecs
for external libraries that have a init_static_data callback.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
Signed-off-by: James Almer <jamrial@gmail.com>
Deprecated in 40cf1bbacc.
(The currently disabled filter vf_mcdeint and vf_uspp were users of
this field; they have not been changed, so that whoever wants to fix
them can see the state of these filters when they were disabled.)
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@outlook.com>
Signed-off-by: James Almer <jamrial@gmail.com>
This already makes several encoders (namely FLV, H.263, H.263+ and
RealVideo 1.0 and 2.0 and SVQ1) that use this init-threadsafe.
It also makes the Snow encoder init-threadsafe; it was already marked
as such since commit d49210788b, because
it was thought to be harmless if one and the same object was
initialized by multiple threads at the same time.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
Several encoders used code like the following to check for the amount of
bytes left in a PutBitContext:
pb->buf_end - pb->buf - (put_bits_count(pb) >> 3)
Besides the fact that using the pointers directly might pose
a maintainence burden in the future this also leads to suboptimal code:
The above code reads all three pointers (buf, buf_ptr and buf_end), but
touching buf is unnecessary and switching to put_bytes_left()
automatically fixes this.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
Niklas Haas