Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[ffmpeg][vaapi][ARGB/BGRA] mismatch of ARGB/BRGA #690

Closed
fulinjie opened this issue Jul 23, 2019 · 13 comments · May be fixed by #692
Closed

[ffmpeg][vaapi][ARGB/BGRA] mismatch of ARGB/BRGA #690

fulinjie opened this issue Jul 23, 2019 · 13 comments · May be fixed by #692
Assignees
@Jister @wangyan-intel @LhGu
Labels
Bug Issue: identify as a bug Debugging Issue: debugging and update status if any change P2 Medium priority VP Video Processing

Comments

@fulinjie
Copy link
Contributor

cmdline:
ffmpeg -hwaccel vaapi -vaapi_device /dev/dri/renderD128 -hwaccel_output_format vaapi -i Nature.h264 -vf scale_vaapi=format=rgb32,hwdownload,format=rgb32 -f rawvideo /dev/null

https://github.com/FFmpeg/FFmpeg/blob/3883c9d1479d3e653bb66c9960fdd2ebd6ba9594/libavutil/hwcontext_vaapi.c#L607
if (expected_format->fourcc == test_image.format.fourcc) {

expected_format->fourcc is different from test_image.format.fourcc.

There is mismatch between ffmpeg and media driver for ARGB/BGRA, thus vaDeriveImage can't be used, and it causes the performance drop.
@wangyan-intel
Copy link

@fulinjie Could you share your video clip for debugging? Thanks.

@fulinjie
Copy link
Contributor Author

@wangyan-intel Random clips should be ok, since the cmdline simply decoded a clips and do color space convert.

@wangyan-intel
Copy link

I tried it by one MP4 h264 clip but cannot find this issue. the commit of iHD driver is c2aae3e. Could you please double check it? Thanks.

@fulinjie
Copy link
Contributor Author

Hi Yan, update to latest version of iHD driver, and this issue still exists.
This happens when vaapi_frames_init is called for VPP (csc, called the second time.)

vaCreateImage is used instead of vaDeriveImage, thus makes the performance drop.

wangyan42164 added a commit to wangyan42164/media-driver that referenced this issue Jul 24, 2019
@wangyan42164
Split BGRA/ARGB FOURCC mapping for the surface.
0ea451d 
Fixes intel#690.

Signed-off-by: Yan Wang <[email protected]>
@wangyan42164 wangyan42164 mentioned this issue Jul 24, 2019
Open
@wangyan42164
Copy link
Contributor

@fulinjie Could you please try #692? It works fine on my side. Thanks.

@FurongZhang FurongZhang added the Debugging Issue: debugging and update status if any change label Jul 26, 2019
@fulinjie
Copy link
Contributor Author

Hi @wangyan42164, how is this going?

fulinjie added a commit to fulinjie/ffmpeg that referenced this issue Oct 24, 2019
@fulinjie
lavu/hwcontext_vaapi: skip checking for hwmap as a work aound
e573407 
Since issue for ARGB/BGRA mismatch in driver is not addressed:
intel/media-driver#690

Provided a work around in ffmpeg to test the hwmap:
ffmpeg -hwaccel qsv -qsv_device /dev/dri/renderD128 -v verbose
    -c:v h264_qsv -i Nature.h264
    -vf scale_qsv=format=rgb32,hwdownload,format=rgb32 -f null -

Signed-off-by: Linjie Fu <[email protected]>
fulinjie added a commit to fulinjie/ffmpeg that referenced this issue Oct 24, 2019
@fulinjie
lavu/hwcontext_vaapi: skip checking for hwmap as a work aound
9ae3b08 
Since issue for ARGB/BGRA mismatch in driver is not addressed:
intel/media-driver#690

Provided a work around in ffmpeg to test the hwmap:

ffmpeg -hwaccel qsv -qsv_device /dev/dri/renderD128 -v verbose -c:v
h264_qsv -i Nature.h264 -vf
scale_qsv=format=rgb32,hwmap=mode=direct,format=rgb32 -f rawvideo
/dev/null

Signed-off-by: Linjie Fu <[email protected]>
@wangyan-intel
Copy link

@XinfengZhang As I know, @Jister is working on this?

@XinfengZhang XinfengZhang added Bug Issue: identify as a bug P2 Medium priority VP Video Processing labels Jan 31, 2021
@MicroYY
Copy link
Contributor

MicroYY commented Jan 18, 2022

We're trying @wangyan42164 's fix. As it's an old PR and contains confilct, @wangyan42164 could you pls rebase the patch?

LhGu added a commit to LhGu/media-driver that referenced this issue Jan 24, 2022
@LhGu
fix: intel#690
ed5426a 
Signed-off-by: Gu, Lihao <[email protected]>
@LhGu LhGu mentioned this issue Jan 24, 2022
Closed
LhGu added a commit to LhGu/media-driver that referenced this issue Jan 24, 2022
@LhGu
Split BGRA media_format for the surface
c2a9751 
fix: intel#690
Signed-off-by: Gu, Lihao <[email protected]>
LhGu added a commit to LhGu/media-driver that referenced this issue Jan 24, 2022
@LhGu
Split BGRA format for the surface
6d95b5a 
fix: intel#690
Signed-off-by: Gu, Lihao <[email protected]>
LhGu added a commit to LhGu/media-driver that referenced this issue Jan 24, 2022
@LhGu
Fix mismatch in bgra format flow
e6f301b 
fix: intel#690
Signed-off-by: Gu, Lihao <[email protected]>
@LhGu LhGu mentioned this issue Jan 24, 2022
Closed
LhGu added a commit to LhGu/media-driver that referenced this issue Jan 24, 2022
@LhGu
Split BGRA/ARGB media_format for the surface
2f009e5 
fix: intel#690
Signed-off-by: [email protected]
@LhGu LhGu mentioned this issue Jan 24, 2022
Closed
LhGu added a commit to LhGu/media-driver that referenced this issue Jan 24, 2022
@LhGu
Change VA_FOURCC format in the mapping relation to support bgra
8a6dbc4 
Fix: intel#690
Signed-off-by: Gu, Lihao <[email protected]>
@LhGu LhGu mentioned this issue Jan 24, 2022
Closed
@MicroYY MicroYY added P3 Low priority no customer usage, no business requirements, not from communities, just from internal and removed P2 Medium priority labels Mar 15, 2022
@MicroYY
Copy link
Contributor

MicroYY commented Mar 15, 2022

@LhGu What's the status of your PR?

@LhGu
Copy link
Member

LhGu commented Mar 15, 2022

@LhGu What's the status of your PR?

It is under review.

@LhGu
Copy link
Member

LhGu commented Mar 18, 2022 •
edited
Loading

There is a limitation in the driver stack. It only supports the hex order is B-G-R-A, and our driver treats the hex order as VA_FOURCC_ARGB format. I already have two PRs（#1334， #1335 ）to work around it.

However, any work around code may be confused and affect our internal test cases. Just for this issue, you can use this FFmpeg command to get the same output:
ffmpeg -hwaccel vaapi -vaapi_device /dev/dri/renderD128 -hwaccel_output_format vaapi -i Nature.h264 -vf scale_vaapi=format=argb,hwdownload,format=argb -f rawvideo /dev/null

This command output is the same as original command (B-G-R-A) , and its VA_FOURCC will be consistent with FFmpeg.

This case requests the read access, and prefers not to be given direct-mapped memory. The reason has been explained in the below link: https://github.com/FFmpeg/FFmpeg/blob/3883c9d1479d3e653bb66c9960fdd2ebd6ba9594/libavutil/hwcontext_vaapi.c#L780
If you want to test the HW map, you need to change this judgement code in FFmpeg, besides the consistent VA_FOURCC.

I think that we can close this issue with no driver code changed.

@LhGu
Copy link
Member

LhGu commented May 16, 2022

This issue needs feedback, but no feedback from submitter yet. I will close this issue if no more information provided from submitter in 1 month. Please feel free to re-open this issue if it still exist.

@LhGu LhGu added P2 Medium priority and removed P3 Low priority no customer usage, no business requirements, not from communities, just from internal labels May 16, 2022
@LhGu LhGu self-assigned this May 16, 2022
@LhGu
Copy link
Member

LhGu commented May 31, 2022

Close it now, and please feel free to re-open this issue if you want to discuss more.

@LhGu LhGu closed this as completed May 31, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@Jister Jister

@wangyan-intel wangyan-intel

@LhGu LhGu

Labels
Bug Issue: identify as a bug Debugging Issue: debugging and update status if any change P2 Medium priority VP Video Processing
Projects
None yet
Milestone
No milestone
8 participants
@wangyan42164 @Jister @fulinjie @MicroYY @XinfengZhang @FurongZhang @wangyan-intel @LhGu