Commit Graph

97 Commits

Author SHA1 Message Date
Diego de Souza 30e6effff9 avcodec/nvdec: add 4:2:2 decoding and 10-bit support
This commit adds support for 4:2:2 decoding for HEVC and H.264 on
NVIDIA Blackwell GPUs. Additionally, it supports 10-bit decoding
for H.264 on Blackwell GPUs.

Signed-off-by: Diego de Souza <ddesouza@nvidia.com>
Signed-off-by: Timo Rothenpieler <timo@rothenpieler.org>
2025-02-02 20:01:56 +01:00
Vitaly Buka c5287178b4 avcodec/hevc/hevcdec: Don't add to null pointer
`POS(1,` and `POS(2,` may trigger UBSAN report:
"runtime error: applying non-zero offset 304 to null pointer"

Looks like values are not used without `chroma_format_idc`,
so maybe there is no other issues than the UB.

Can't reproduce with "fate".

Signed-off-by: Vitaly Buka <vitalybuka@google.com>
Signed-off-by: James Almer <jamrial@gmail.com>
2025-01-30 11:35:10 -03:00
James Almer 5cd49e1bfd avcodec/hevc/hevcdec: use av_frame_side_data_add() where useful
Signed-off-by: James Almer <jamrial@gmail.com>
2025-01-10 15:02:31 -03:00
Marton Balint 19c95ecbff avcodec: deprecate AVCodecContext properties
These properties are unreliable because they depend on the frames decoded so
far, users should check directly the presence of the decoded AVFrame side data
or AVFrame flags.

Signed-off-by: Marton Balint <cus@passwd.hu>
2025-01-05 22:25:29 +01:00
Anton Khirnov 2e956d9c0f lavc/container_fifo: move to lavu and make public
This can be useful in other places, e.g. it can replace objpool in
fftools.

The API is modified in the following nontrivial ways:
* opaque pointers can be passed through to all user callbacks
* read and write were previously separate callbacks in order to
  accomodate the caller wishing to write a new reference to the FIFO and
  keep the original one; the two callbacks are now merged into one, and
  a flags argument is added that allows to request such behaviour on a
  per-call basis
* new peek and drain functions
2024-12-15 14:05:15 +01:00
Anton Khirnov 56ba57b672 lavc/refstruct: move to lavu and make public
It is highly versatile and generally useful.
2024-12-15 14:03:47 +01:00
Michael Niedermayer 4fc43e45f9 avcodec/hevc/hevcdec: initialize qp_y_tab
This does not replicate on my setup, thus this is a blind fix based on ossfuzz trace

Fixes: use of uninitialized value
Fixes: 71747/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_HEVC_fuzzer-5427736120721408

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2024-12-02 03:14:49 +01:00
James Almer 66014c79ab avcodec/h2645_sei: move some common SEI syncing code to ff_h2645_sei_ctx_replace()
Instead of duplicating it across all supported decoders.

Signed-off-by: James Almer <jamrial@gmail.com>
2024-11-13 12:49:52 -03:00
James Almer e33b162c7d avcodec/h2645_sei: use the RefStruct API for film_grain_characteristics
And ensure the buffer is synced between threads.
Based on a patch by Dale Curtis <dalecurtis@chromium.org>

Signed-off-by: James Almer <jamrial@gmail.com>
2024-11-11 16:10:15 -03:00
James Almer fd4a2c9b02 avcodec/aom_film_grain: allocate film grain metadata dynamically
This removes the ABI breaking use of sizeof(AVFilmGrainParams), and achieves the
same size reduction to decoder structs as 08b1bffa49.

Signed-off-by: James Almer <jamrial@gmail.com>
2024-11-11 16:10:15 -03:00
James Almer 91b6ba653e Revert "avcodec/h2645: allocate film grain metadata dynamically"
AVFilmGrainAFGS1Params, the offending struct, is using sizeof(AVFilmGrainParams)
when it should not. This change also forgot to make the necessary changes to the
frame threading sync code.
Both of these will be fixed by the following commit.

H274FilmGrainDatabase will be handled later.

This reverts commit 08b1bffa49.

Signed-off-by: James Almer <jamrial@gmail.com>
2024-11-11 15:26:21 -03:00
Dale Curtis 08b1bffa49 avcodec/h2645: allocate film grain metadata dynamically
Film grain support adds a huge amount of overhead to the H264Context
structure for a feature that is rarely used. On low end devices or
pages that have lots of media this bloats memory usage rapidly.

This changes the static film grain metadata allocations to be dynamic
which reduces the H264Context size from 851808 bytes to 53444 bytes.

Bug: https://crbug.com/359358875
Signed-off-by: Dale Curtis <dalecurtis@chromium.org>
Signed-off-by: Niklas Haas <git@haasn.dev>
2024-10-25 01:02:35 +02:00
Anton Khirnov 79c47dfd25 lavc/hevcdec: unbreak WPP/progress2 code
The "progress2" API in pthread_slice.c currently associates a progress
value with a thread rather than a job, relying on the broken assumption
that a job's thread number is equal to its job number modulo thread
count.

This removes this API entirely, and changes hevcdec to use a
ThreadProgress-based implementation that associates a
mutex/cond/progress value with every job.

Fixes races and deadlocks in hevdec with slice threading, e.g. some of
those mentioned in #11221.
2024-10-15 10:37:44 +02:00
James Almer aef221b22a avcodec/hevc/refs: export Stereo 3D side data
Use the 3D Reference Displays Info SEI message to link a view_id with
an eye.

Signed-off-by: James Almer <jamrial@gmail.com>
2024-09-23 17:15:02 +02:00
Anton Khirnov 14746871e1 lavc/hevcdec: implement decoding MV-HEVC
At most two layers are supported.

Aspects of this work were sponsored by Vimeo and Meta.
2024-09-23 17:15:02 +02:00
Anton Khirnov 75914b5822 lavc/hevc/hevcdec: implement MV-HEVC inter-layer prediction
The per-frame reference picture set contains two more lists -
INTER_LAYER[01]. Assuming at most two layers, INTER_LAYER1 is always
empty, but is added anyway for completeness.

When inter-layer prediction is enabled, INTER_LAYER0 for the
second-layer frame will contain the base-layer frame from the same
access unit, if it exists.

The new lists are then used in per-slice reference picture set
construction as per F.8.3.4 "Decoding process for reference picture
lists construction".
2024-09-23 17:11:40 +02:00
Anton Khirnov 02a9435cb0 lavc/hevcdec: implement slice header parsing for nuh_layer_id>0
Cf. F.7.3.6.1 "General slice segment header syntax"
2024-09-23 17:11:40 +02:00
Anton Khirnov 7d245866b8 lavc/hevc/ps: implement SPS parsing for nuh_layer_id>0
Cf. F.7.3.2.2 "Sequence parameter set RBSP syntax", which extends normal
SPS parsing with special clauses depending on MultiLayerExtSpsFlag.
2024-09-23 17:11:40 +02:00
Anton Khirnov 536bb98888 lavc/hevcdec: set per-CTB filter parameters for WPP
Fixes #10887
2024-09-13 12:10:11 +02:00
Anton Khirnov 21ef80f3d9 lavc/hevcdec: remove a duplicate variable
In hls_decode_entry_wpp(), self_id is always identical to thread.
2024-09-13 12:10:11 +02:00
Anton Khirnov b53800691c lavc/hevcdec: set output frame pkt_dts
pkt_dts needs to be set manually when using the receive_frame() callback, so
it was unset after 2fdecbb239.

Fixes PTS guessing for certain files with broken timestamps. Cf.
https://github.com/mpv-player/mpv/issues/14806

Reported-by: llyyr <llyyr.public@gmail.com>
2024-09-13 11:56:53 +02:00
Anton Khirnov eec1a7a6bb lavc/hevc: check framerate num/den to be strictly positive
Rather than just != 0. These values are read as uint32 and can become
negative when cast to int.
2024-09-06 14:09:03 +02:00
Anton Khirnov fc8df81cb1 lavc/hevcdec: move active SPS from HEVCParamSets to HEVCLayerContext
Currently active SPS is a per-layer property.
2024-09-06 13:59:29 +02:00
Anton Khirnov 9bccc634af lavc/hevcdec: make a HEVCFrame hold a reference to its PPS
ff_hevc_get_ref_list() needs the PPS of a previously decoded frame,
which may be different from the currently active one.
2024-09-06 13:59:29 +02:00
Anton Khirnov 672713761b lavc/hevcdec: move HEVCContext.sao_pixel_buffer_[vh] to HEVCLayerContext
Handle them together with other sps-dependent arrays.

Note that current code only allocates these arrays when hwaccel is not
set, but this is wrong as the relevant code runs BEFORE get_format() is
called and hence before we know whether hwaccel is in use.
2024-09-06 13:59:29 +02:00
Anton Khirnov 6fcf0045cf lavc/hevcdec: move HEVCContext.{tab_mvf,rpl_tab}_pool to HEVCLayerContext
pic_arrays_{init,free}() no longer access HEVCContext
2024-09-06 13:59:29 +02:00
Anton Khirnov 4f87ff7666 lavc/hevcdec: move HEVCContext.{horizontal,vertical}_bs to HEVCLayerContext 2024-09-06 13:59:29 +02:00
Anton Khirnov e8baf2fb97 lavc/hevcdec: move HEVCContext.qp_y_tab to HEVCLayerContext 2024-09-06 13:59:29 +02:00
Anton Khirnov d5188adba8 lavc/hevcdec: move HEVCContext.tab_slice_address to HEVCLayerContext 2024-09-06 13:59:29 +02:00
Anton Khirnov eeb369d24f lavc/hevcdec: move HEVCContext.filter_slice_edges to HEVCLayerContext 2024-09-06 13:59:29 +02:00
Anton Khirnov 463a85c5a1 lavc/hevcdec: move HEVCContext.is_pcm to HEVCLayerContext 2024-09-06 13:59:29 +02:00
Anton Khirnov 0704559932 lavc/hevcdec: move HEVCContext.tab_ipm to HEVCLayerContext 2024-09-06 13:59:29 +02:00
Anton Khirnov 59de042cf6 lavc/hevcdec: move HEVCContext.cbf_luma to HEVCLayerContext 2024-09-06 13:59:29 +02:00
Anton Khirnov cf7add8d70 lavc/hevcdec: move HEVCContext.tab_ct_depth to HEVCLayerContext 2024-09-06 13:59:29 +02:00
Anton Khirnov 1ca4c2a96d lavc/hevcdec: move HEVCContext.skip_flag to HEVCLayerContext 2024-09-06 13:59:29 +02:00
Anton Khirnov ed2d25e7f0 lavc/hevcdec: move HEVCContext.deblock to HEVCLayerContext 2024-09-06 13:59:29 +02:00
Anton Khirnov 3d4294e344 lavc/hevcdec: move HEVCContext.sao to HEVCLayerContext 2024-09-06 13:59:29 +02:00
Anton Khirnov 747609e967 lavc/hevcdec: move HEVCContext.bs_{width,height} to HEVCLayerContext 2024-09-06 13:59:29 +02:00
Anton Khirnov c7f0753a6b lavc/hevcdec: move active VPS from HEVCParamSets to HEVCContext
Active VPS is a property of the decoding process, not of the list of
parameter sets.

Check that the VPS can only change in a base layer - while this can
never happen currently (as no other layers can exist in the decoder), it
will become useful when multilayer decoding is supported.
2024-09-06 13:59:29 +02:00
Anton Khirnov 4c0d669b20 lavc/hevcdec: add a per-layer context
Start by moving the DPB to it.

Only one context exists for now, so decoder behaviour should not change
with this commit, but that will change in the future.
2024-09-06 13:59:29 +02:00
James Almer a754ee0844 avcodec/h2645_parse: replace three bool arguments in ff_h2645_packet_split with a single flags one
Signed-off-by: James Almer <jamrial@gmail.com>
2024-08-19 20:23:20 -03:00
Anton Khirnov 631a725670 lavc/hevcdec: call ff_thread_finish_setup() even if hwaccel is in use
Serializing frame threading for non-threadsafe hwaccels is handled at the
generic level, the decoder does not need to care about it.
2024-08-19 21:37:22 +02:00
Anton Khirnov 4b9adb35b6 lavc/hevcdec: simplify output logic
Current code is written around the "simple" decode API's limitation that
a single input packet (AU/coded frame) triggers the output of at most
one output frame. However the spec contains two cases where a coded
frame may cause multiple frames to be output (cf. C.5.2.2.2):
* start of a new sequence
* overflowing sps_max_dec_pic_buffering

The decoder currently contains rather convoluted logic to handle these
cases:
* decode/output/per-frame sequence counters,
* HEVC_FRAME_FLAG_BUMPING
* ff_hevc_bump_frame()
* special clauses in ff_hevc_output_frame()

However, with the receive_frame() API none of that is necessary, as we
can just output multiple frames at once. Previously added ContainerFifo
allows that to be done in a straightforward and efficient manner.
2024-08-19 21:37:22 +02:00
Anton Khirnov 79afc45c03 lavc/hevcdec: use a ContainerFifo to hold frames scheduled for output
Instead of a single AVFrame.

Will be useful in future commits, where we will want to produce multiple
output frames for a single coded frame.
2024-08-19 21:37:22 +02:00
Anton Khirnov 2fdecbb239 lavc/hevcdec: switch to receive_frame()
Required by following commits, where we will want to output multiple
frames per packet.
2024-08-19 21:37:22 +02:00
James Almer afb06aef7e avcodec/decode: remove unused argument from ff_frame_new_side_data_from_buf()
Signed-off-by: James Almer <jamrial@gmail.com>
2024-07-29 14:00:48 -03:00
Michael Niedermayer d52fabaec0 avcodec/hevc/hevcdec: avoid signed shifts with lt_idx_sps
Fixes: left shift of 1 by 31 places cannot be represented in type 'int'
Fixes: 70122/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_HEVC_fuzzer-5172200613675008

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2024-07-23 23:21:14 +02:00
Michael Niedermayer 5d9544cfb0 avcodec/hevc/hevcdec: Do not allow slices to depend on failed slices
An alternative would be to leave the context unchanged on failure of hls_slice_header()

Fixes: out of array access
Fixes: NULL pointer dereference
Fixes: 69584/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_HEVC_fuzzer-5931086299856896
Fixes: 69724/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_HEVC_fuzzer-5104066422702080
Fixes: 70422/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_HEVC_fuzzer-5908731129298944

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2024-07-23 23:21:13 +02:00
Fei Wang 71f802cdc9 lavc/hevcdec: Update slice index before hwaccel decode slice
Otherwise, slice index will never update for hwaccel decode, and slice
RPL will be always overlap into first one which use slice index to construct.

Fixes hwaccel decoding after 47d34ba7fb

Signed-off-by: Fei Wang <fei.w.wang@intel.com>
2024-07-12 16:27:34 +08:00
Fei Wang e741cf665d lavc/hevcdec: Put slice address checking after hwaccel decode slice
Slice address tab only been updated in software decode slice data.

Fixes hwaccel decoding after d725c737fe.

Signed-off-by: Fei Wang <fei.w.wang@intel.com>
2024-07-12 16:27:34 +08:00