Commit Graph

120956 Commits

Author SHA1 Message Date
Zhao Zhili 6ce02bcc3a avcodec/aarch64/vvc: Optimize apply_bdof
Before this patch, prof_grad_filter calculate
gh[0], gh[1], gv[0], gv[1] and save them to stack.

derive_bdof_vx_vy load them from stack and calculate
gh[0] + gh[1], gv[0] + gv[1].

apply_bdof_min_block load them from stack and calculate
gh[0] - gh[1], gv[0] - gv[1]

This patch add bdof_grad_filter, which calculate gh[0] + gh[1],
gh[0] - gh[1], gv[0] + gv[1], gv[0] - gv[1], and save them to
stack, so derive_bdof_vx_vy and apply_bdof_min_block can use the
results directly.

prof_grad_filter is kept for reuse by other functions in the future.

Benchmark on rpi5 with gcc 12
                               Before               After
--------------------------------------------------------------------
apply_bdof_8_8x16_c:       |   7431.4 ( 1.00x)   |   7371.7 ( 1.00x)
apply_bdof_8_8x16_neon:    |   1175.4 ( 6.32x)   |   1036.3 ( 7.11x)
apply_bdof_8_16x8_c:       |   7182.2 ( 1.00x)   |   7201.1 ( 1.00x)
apply_bdof_8_16x8_neon:    |   1021.7 ( 7.03x)   |    879.9 ( 8.18x)
apply_bdof_8_16x16_c:      |  14577.1 ( 1.00x)   |  14589.3 ( 1.00x)
apply_bdof_8_16x16_neon:   |   2012.8 ( 7.24x)   |   1743.3 ( 8.37x)
apply_bdof_10_8x16_c:      |   7292.4 ( 1.00x)   |   7308.5 ( 1.00x)
apply_bdof_10_8x16_neon:   |   1156.3 ( 6.31x)   |   1045.3 ( 6.99x)
apply_bdof_10_16x8_c:      |   7112.4 ( 1.00x)   |   7214.4 ( 1.00x)
apply_bdof_10_16x8_neon:   |   1007.6 ( 7.06x)   |    904.8 ( 7.97x)
apply_bdof_10_16x16_c:     |  14363.3 ( 1.00x)   |  14476.4 ( 1.00x)
apply_bdof_10_16x16_neon:  |   1986.9 ( 7.23x)   |   1783.1 ( 8.12x)
apply_bdof_12_8x16_c:      |   7433.3 ( 1.00x)   |   7374.7 ( 1.00x)
apply_bdof_12_8x16_neon:   |   1155.9 ( 6.43x)   |   1040.8 ( 7.09x)
apply_bdof_12_16x8_c:      |   7171.1 ( 1.00x)   |   7376.3 ( 1.00x)
apply_bdof_12_16x8_neon:   |   1010.8 ( 7.09x)   |    899.4 ( 8.20x)
apply_bdof_12_16x16_c:     |  14515.5 ( 1.00x)   |  14731.5 ( 1.00x)
apply_bdof_12_16x16_neon:  |   1988.4 ( 7.30x)   |   1785.2 ( 8.25x)
2025-09-03 06:55:37 +00:00
Zhao Zhili 2e92417603 avcodec/aarch64/vvc: Optimize derive_bdof_vx_vy
Implement line tricks and pixel tricks. See comments in inter.S
for details.

Benchmark on rpi5 with gcc 12
                               Before             After
-----------------------------------------------------------------
apply_bdof_8_8x16_c:       |   7375.5 ( 1.00x) |  7473.8 ( 1.00x)
apply_bdof_8_8x16_neon:    |   1875.1 ( 3.93x) |  1135.8 ( 6.58x)
apply_bdof_8_16x8_c:       |   7273.9 ( 1.00x) |  7204.0 ( 1.00x)
apply_bdof_8_16x8_neon:    |   1738.2 ( 4.18x) |  1013.0 ( 7.11x)
apply_bdof_8_16x16_c:      |  14744.9 ( 1.00x) | 14712.6 ( 1.00x)
apply_bdof_8_16x16_neon:   |   3446.7 ( 4.28x) |  1997.7 ( 7.36x)
apply_bdof_10_8x16_c:      |   7352.4 ( 1.00x) |  7485.7 ( 1.00x)
apply_bdof_10_8x16_neon:   |   1861.0 ( 3.95x) |  1134.1 ( 6.60x)
apply_bdof_10_16x8_c:      |   7330.5 ( 1.00x) |  7232.8 ( 1.00x)
apply_bdof_10_16x8_neon:   |   1747.2 ( 4.20x) |  1002.6 ( 7.21x)
apply_bdof_10_16x16_c:     |  14522.4 ( 1.00x) | 14664.8 ( 1.00x)
apply_bdof_10_16x16_neon:  |   3490.5 ( 4.16x) |  1978.4 ( 7.41x)
apply_bdof_12_8x16_c:      |   7389.0 ( 1.00x) |  7380.1 ( 1.00x)
apply_bdof_12_8x16_neon:   |   1861.3 ( 3.97x) |  1134.0 ( 6.51x)
apply_bdof_12_16x8_c:      |   7283.1 ( 1.00x) |  7336.9 ( 1.00x)
apply_bdof_12_16x8_neon:   |   1749.1 ( 4.16x) |  1002.3 ( 7.32x)
apply_bdof_12_16x16_c:     |  14580.7 ( 1.00x) | 14502.7 ( 1.00x)
apply_bdof_12_16x16_neon:  |   3472.9 ( 4.20x) |  1978.3 ( 7.33x)

Signed-off-by: Zhao Zhili <zhilizhao@tencent.com>
2025-09-03 06:55:37 +00:00
damitha 3800421e7c add clamping for escpase coeff 2025-09-03 06:48:02 +00:00
Niklas Haas a700f0f72d avfilter/vf_zscale: simplify and fix alpha handling
There's no reason to use a completely separate graph just to process the
alpha plane in isolation - zimg supports native alpha handling as part of the
main image.

Fixes several issues with this filter when adding or removing alpha planes,
and also adds support for scaling premultiplied alpha (which reduces artefacts
near the image borders).
2025-09-02 17:06:25 +02:00
Niklas Haas c7adeb75ac avfilter/vf_scale: require straight alpha for alpha blending
Fortunately, we only care if this flag is set - otherwise, this filter is
alpha mode agnostic (since it is purely scaling, etc).

That said, there is an argument to be made that we should prefer premul
alpha when scaling, because scaling in straight alpha will leak garbage
pixels; but I think that would be a too backwards-incompatible change to
be worth thinking about at this time.
2025-09-02 17:06:25 +02:00
Niklas Haas 446a25b70f avfilter: add alpha_mode consistency assertion 2025-09-02 17:06:25 +02:00
Niklas Haas 39631c99d4 avfilter/drawutils: add alpha mode support
This allows drawing colors onto images with premultiplied alpha, by first
premultiplying the specified RGBA values.
2025-09-02 17:06:25 +02:00
Niklas Haas a90e8f1c1d avfilter: use ff_draw_init_from_link() where possible 2025-09-02 17:06:25 +02:00
Niklas Haas eddbdc8b7c avfilter/drawutils: add ff_draw_init_from_link()
The overwhelming majority of references to ff_draw_init2() just set the
colorspace properties from a filter link. This wrapper allows us to update
all such referencen automatically.
2025-09-02 17:06:25 +02:00
Niklas Haas d05484757a avfilter/vf_fade: set correct alpha mode requirement
Fading to black in non-alpha mode works for either straight or premultiplied
alpha, but fading to a color does not.
2025-09-02 17:06:25 +02:00
Niklas Haas ba8aa0e7b3 avfilter/x86/vf_overlay: simplify function signature
No reason to pass all the variables again, if we're already passing the
context.
2025-09-02 17:06:25 +02:00
Niklas Haas 59bb51a8f2 avfilter/vf_overlay: support premultiplied base layers
Conceptually, these are pretty simple to handle, since they are basically
equivalent to the case of alpha being absent, since the only thing the
destination alpha plane is used for is unpremultiplyng straight alpha.

For planar formats, the total number of cases doesn't change in principle,
since before this patch we have:

- alpha present, overlay straight
- alpha present, overlay premultiplied
- alpha absent, overlay straight
- alpha absent, overlay premultiplied

And now we have:

- main straight, overlay straight
- main straight, overlay premultiplied
- main premultiplied, overlay straight
- main premultiplied, overlay premultiplied

That said, we do gain some cases now that we used to (incorrectly) ignore,
like premultiplied yuva420p10.

Notably, we can skip defining separate functions for the case of main alpha
being absent, since that can be a single cheap branch inside the function
itself, on whether or not to also process the alpha plane. Otherwise, as long
as we treat "alpha absent" as "main premultiplied", the per-plane logic will
skip touching the nonexistent alpha plane.

The only format that actually needs extra cases is packed rgb, but that's only
two additional cases in total.

Also arguably simplifies the assignment logic massively.
2025-09-02 17:06:25 +02:00
Niklas Haas 6d6bbdaab0 avfilter/vf_overlay: rename variables for clarity
`is_straight`, `alpha_mode` etc. are more consistently named to refer to
either the main image, or the overlay.
2025-09-02 17:06:25 +02:00
Niklas Haas 6f3eddbedd avfilter/vf_overlay: configure alpha mode on the link
And use the link-tagged value instead of the hard-coded parameter.
2025-09-02 17:06:25 +02:00
Niklas Haas 1c8ca56a88 avfilter/vf_libplacebo: add an alpha_mode setting
Chooses the desired output alpha mode. Note that this depends on
an upstream version of libplacebo new enough to respect the corresponding
AVFrame field in pl_map_avframe_ex.
2025-09-02 17:06:25 +02:00
Niklas Haas dc02514f6b avfilter/vf_setparams: add alpha_mode parameter 2025-09-02 17:06:25 +02:00
Niklas Haas e6ab7fe582 avfilter/vf_scale: set correct alpha mode after format change
While vf_scale cannot directly convert between premultiplied and straight
alpha, the effective tagging can still change as a result of a change in
the pixel format (i.e. adding or removing an alpha channel).
2025-09-02 17:06:25 +02:00
Niklas Haas a3d518a57c avfilter/vf_extractplanes: require straight alpha input 2025-09-02 17:06:25 +02:00
Niklas Haas ab1bc440f7 avfilter/vf_alphamerge: configure correct alpha mode 2025-09-02 17:06:25 +02:00
Niklas Haas 2032820e90 fftools/ffprobe: add AVFrame.alpha_mode support 2025-09-02 17:06:25 +02:00
Niklas Haas 4f99026c9f fftools/ffplay: add alpha_mode support
SDL seems to hard-code straight alpha compositing.
2025-09-02 17:06:25 +02:00
Niklas Haas f07573f496 fftools/ffmpeg: add alpha_mode support
The implementation exactly mirrors the existing plumbing for color_ranges
and color_spaces.
2025-09-02 17:06:25 +02:00
Niklas Haas 2e2ca4e740 avfilter/vf_showinfo: print alpha mode when relevant 2025-09-02 17:06:25 +02:00
Niklas Haas 2b1ef029eb avfilter/vf_format: add alpha_modes parameter 2025-09-02 17:06:25 +02:00
Niklas Haas de297ae2fc avfilter/buffersrc: add alpha_mode parameter 2025-09-02 17:06:25 +02:00
Niklas Haas 0fbd90d78f avfilter/buffersink: add support for alpha modes 2025-09-02 17:06:25 +02:00
Niklas Haas 65580592c9 avcodec/jpegxl: parse and signal correct alpha mode
This header bit ("alpha_associated") was incorrectly ignored.
2025-09-02 17:06:25 +02:00
Niklas Haas 51572ff08a avcodec/libjxlenc: also attach extra channel info
Works around a bug where older versions of libjxl don't correctly forward
the alpha channel information to the extra channel info.
2025-09-02 17:06:25 +02:00
Niklas Haas a88cc0a53e avcodec/libjxl: set correct alpha mode
JPEG XL supports both premultiplied and straight alpha, and the basic info
struct contains signalling for this. Forward the correct tagging on decode
and encode.
2025-09-02 17:06:25 +02:00
Niklas Haas 702a5ac29b avcodec/exr: set correct alpha mode
OpenEXR always uses premultiplied alpha, as per the spec.

cf. https://openexr.com/en/latest/TechnicalIntroduction.html

> By convention, all color channels are premultiplied by alpha, so that
> `foreground + (1-alpha) x background` performs a correct “over” operation.
> (See Premultiplied vs. Un-Premultiplied Color Channels)
>
> In the visual effects industry premultiplied color channels are the norm,
> and application software packages typically use internal image
> representations that are also premultiplied.
2025-09-02 17:06:25 +02:00
Niklas Haas d6ce720765 avcodec/png: set correct alpha mode
PNG always uses straight alpha.

cf. https://www.w3.org/TR/PNG-Rationale.html

> Although each form of alpha storage has its advantages, we did not want to
> require all PNG viewers to handle both forms. We standardized on non-
> premultiplied alpha as being the lossless and more general case.
2025-09-02 17:06:25 +02:00
Niklas Haas 02d531ca05 avcodec/encode: enforce alpha mode compatibility at encode time
Error out if trying to encode frames with an incompatible alpha mode.
2025-09-02 17:06:25 +02:00
Niklas Haas ecebf9c693 avcodec/avcodec: add AVCodecContext.alpha_mode
Following in the footsteps of the previous commit, this commit adds the
new fields to AVCodecContext so we can start properly setting it on codecs,
as well as limiting the list of supported options to detect a format mismatch
during encode.

This commit also sets up the necessary infrastructure to start using the
newly added field in all codecs.
2025-09-02 17:06:24 +02:00
Niklas Haas fb3a4f6180 avfilter/formats: use vf_premultiply_dynamic for alpha mode autoconversion 2025-09-02 17:06:09 +02:00
Niklas Haas c39a67e64f avfilter/vf_premultiply: add dynamic variant of this filter
We need a filter that can premultiply and unpremultiply the alpha channel
dynamically, on demand, in response to the negotiated alpha mode, analogous
to how vf_scale operates. Introduce a new filter "vf_premultiply_dynamic"
that accomplishes this.
2025-09-02 17:06:09 +02:00
Niklas Haas 6bac13d20a avfilter/vf_premultiply: signal correct alpha mode
Set the output link according to the chosen direction, and the input link
to its opposite (unless the input is not in-place, in which case there is no
alpha channel).

Also update the alpha mode on the output frame, since it may differ.
2025-09-02 17:06:09 +02:00
Niklas Haas eea99a77ed avfilter: add link negotiation for AVAlphaMode 2025-09-02 17:06:08 +02:00
Niklas Haas 8b375b2ffd avfilter/avfiltergraph: allow different conversion filters per merger
Instead of hard-coding the assumption that all video properties are handled
by vf_scale, generalize the struct slightly to allow different conversion
filters for each property being merged. The avfiltergraph code creates a list
of conversion filters needed and inserts all of them at once.

Because a conversion filter might itself e.g. not support all formats,
it's possible that we need to go through the whole process of negotiating
formats multiple times, and keep adding conversion filters until all of them
are settled. Do this by simply jumping back to the beginning of the loop
to ensure that the `convertor_count` field remains contiguous.
2025-09-02 17:05:51 +02:00
Niklas Haas dbcdcaede8 avfilter/formats: fix param name on ff_set_common_color_spaces*
Purely cosmetic.
2025-09-02 17:05:51 +02:00
Niklas Haas 22929bd044 avutil/frame: add AVFrame.alpha_mode
FFmpeg currently handles alpha in a quasi-arbitrary way. Some filters/codecs
assume alpha is premultiplied, others assume it is independent. If there is
to be any hope for order in this chaos, we need to start by defining an enum
for the possible range of values.
2025-09-02 17:05:50 +02:00
Nicolas Gaullier f24474dcfc avcodec/aac_ac3_parser: do not override the profile set by the decoder
Parsing the ADTS header is not enough to detect HE-AAC v1/v2.

Regression since 64bb91fd3b.

Fixes #11600

Signed-off-by: Nicolas Gaullier <nicolas.gaullier@cji.paris>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2025-09-02 16:05:46 +02:00
Nicolas Gaullier 593b426902 avcodec/aac_ac3_parser: simplify
Use ff_adts_header_parse_buf() wrapper to simplify as GetBitContext is
no longer needed (it was introduced for USAC).

Partially reverts 64bb91fd3b.

Signed-off-by: Nicolas Gaullier <nicolas.gaullier@cji.paris>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2025-09-02 16:05:46 +02:00
Nicolas Gaullier 9bd10fa32c avcodec/aac_ac3_parser: remove unused USAC/ADTS code
ff_adts_header_parse() parse the object_type from a 2 bits field.
See also 696ea1c223.

Partially reverts 64bb91fd3b.

Signed-off-by: Nicolas Gaullier <nicolas.gaullier@cji.paris>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2025-09-02 16:05:46 +02:00
Maryla Ustarroz-Calonge 5e210f0552 avcodec/libaom: Add test for HDR10+ metadata support
Signed-off-by: Maryla Ustarroz-Calonge <maryla@google.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2025-09-02 16:05:45 +02:00
Maryla Ustarroz-Calonge a282500087 avcodec/libaom: Add HDR10+ metadata support
Signed-off-by: Maryla Ustarroz-Calonge <maryla@google.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2025-09-02 15:56:56 +02:00
Patrick Wang d7173e982e fftools/ffmpeg: fix gracefully shutdown
d119ae2fd8 removed the loop-breaking condition
received_sigterm.
Thus, signals no longer gracefully shutdown ffmpeg.

Fixes: #10834

Signed-off-by: Patrick Wang <mail6543210@gmail.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2025-09-02 15:56:27 +02:00
Zsolt Vadasz 173988aa0c libavfilter/vf_minterpolate: Use local variable in array
Signed-off-by: Zsolt Vadasz <zsolt_vadasz@protonmail.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2025-09-02 15:56:23 +02:00
Zsolt Vadasz 5d7e4b7496 libavfilter/vf_minterpolate: Hoist hot loop in set_frame_data
Signed-off-by: Zsolt Vadasz <zsolt_vadasz@protonmail.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2025-09-02 15:56:02 +02:00
Niklas Haas 5d3538b8dc .forgejo/CODEOWNERS: add myself for various files 2025-09-02 13:17:02 +00:00
Martin Storsjö 5a893c1806 checkasm: sw_ops: Avoid division by zero
If we're invoked with range == UINT_MAX, we end up doing
"rnd() % (UINT_MAX + 1)", which is equal to "rnd() % 0". On
arm (on all platforms) and on MSVC i386, this ends up crashing
at runtime.

This fixes the crash.
2025-09-02 14:28:56 +03:00