Skip to content

stream_http: add libcurl-based HTTP/HTTPS backend#17879

Open
kasper93 wants to merge 14 commits into
mpv-player:masterfrom
kasper93:add-curl
Open

stream_http: add libcurl-based HTTP/HTTPS backend#17879
kasper93 wants to merge 14 commits into
mpv-player:masterfrom
kasper93:add-curl

Conversation

@kasper93
Copy link
Copy Markdown
Member

@kasper93 kasper93 commented May 6, 2026

No description provided.

@kasper93 kasper93 force-pushed the add-curl branch 3 times, most recently from 026fd5a to 12dee02 Compare May 6, 2026 23:50
Comment thread stream/stream_curl.c
Comment thread common/global.h
Comment thread stream/stream_curl.c
Comment thread stream/stream_curl.c
Comment thread stream/stream_http.c Outdated
Comment thread stream/stream_http.c Outdated
Comment thread stream/stream_http.c Outdated
Comment thread DOCS/man/options.rst
Comment thread stream/network.c Outdated
@Andarwinux
Copy link
Copy Markdown
Contributor

I tested this PR and confirmed it works, including HTTP3👍

mingw test build https://github.com/Andarwinux/mpv-winbuild/releases/download/2026-05-07-cfd818bcae/mpv-x86_64-v3-20260507-git-27cd42ff65.7z

@kasper93 kasper93 force-pushed the add-curl branch 4 times, most recently from b98cf95 to 9ad5a36 Compare May 7, 2026 10:25
Comment thread stream/stream_http.c Outdated
Comment thread ci/build-win32.ps1 Outdated
Comment thread stream/stream_curl.c Outdated
Comment thread demux/demux_lavf.c Outdated
void *curl_data = NULL;
int r = AVERROR(ENOSYS);
if (priv->opts->prefer_curl)
r = mp_curl_avio_open(demuxer, pb, &curl_data, url, flags);
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There should be some logging for which backend was used, would be useful for debugging purposes. It would also be nice to have more logging in the curl backend itself

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It would also be nice to have more logging in the curl backend itself

What logging do you need? headers can be enabled with msg-level=curl=trace and the reset is just few errors, opening logging is handled by common code.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There should be some logging for which backend was used

It's already logged, you will see either lavf or curl, depending on the logging parent.

[  2.748475]                   curl: Opening ...
[  2.794586]                   curl: Mime-type: 'video/mp4'

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

headers can be enabled with msg-level=curl=trace

There are no logs for me with the "curl" module on your branch.

It's already logged, you will see either lavf or curl, depending on the logging parent.

[http] status=206 compressed=0 size=372656696 seekable=1
[http] resize stream to 131072 bytes, drop 0 bytes
[http] Mime-type: 'video/mp4'
[http] Stream opened successfully.

This is all I see. the only way I can tell it's using the curl backend is because of the first line

Copy link
Copy Markdown
Member Author

@kasper93 kasper93 May 8, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've renamed it to curl (in your version it would be msg-level=http=trace), but you can know it was using it, because http tag was exclusively used by this stream.

Comment thread stream/stream_curl.c
Comment thread demux/demux_lavf.c Outdated
Comment thread stream/stream_http.c Outdated
@kasper93
Copy link
Copy Markdown
Member Author

kasper93 commented May 8, 2026

Renamed to stream_curl. Resolved review comments. I have some local changes still, to few areas, will update later.

@kasper93 kasper93 force-pushed the add-curl branch 3 times, most recently from b29d9d8 to d8e2461 Compare May 8, 2026 08:27
@kasper93
Copy link
Copy Markdown
Member Author

kasper93 commented May 8, 2026

Added ftp/ftps support. This is significantly more stable than Lavf one.

@kasper93 kasper93 force-pushed the add-curl branch 3 times, most recently from cbda43e to 00da271 Compare May 9, 2026 08:15
Comment thread demux/avio_crypto.c Outdated
@kasper93 kasper93 force-pushed the add-curl branch 2 times, most recently from b2f3fa8 to 026c479 Compare May 9, 2026 09:35
Comment thread meson.options
@na-na-hi
Copy link
Copy Markdown
Contributor

Play HLS livestream - press q to quit:

==7474==ERROR: AddressSanitizer: attempting free on address which was not malloc()-ed: 0x6080002263f0 in thread T0
    #0 0x7f8472c15fb0 in __interceptor_free (/usr/lib/x86_64-linux-gnu/libasan.so.5+0xe8fb0)
    #1 0x564888593723 in ffurl_closep (./mpv/build/mpv+0x15d3723)
    #2 0x564888593922 in ffurl_close (./mpv/build/mpv+0x15d3922)
    #3 0x5648885939b6 in avio_close (./mpv/build/mpv+0x15d39b6)
    #4 0x5648873ec5ac in demux_close_lavf ../demux/demux_lavf.c:1742
    #5 0x5648873a862c in demux_shutdown ../demux/demux.c:1089
    #6 0x5648873a8a72 in demux_free ../demux/demux.c:1124
    #7 0x56488742664f in close_lazy_segments ../demux/demux_timeline.c:203
    #8 0x56488742d52f in d_close ../demux/demux_timeline.c:821
    #9 0x5648873a862c in demux_shutdown ../demux/demux.c:1089
    #10 0x5648873a8a72 in demux_free ../demux/demux.c:1124
    #11 0x5648873a8dfc in demux_cancel_and_free ../demux/demux.c:1191
    #12 0x564887560347 in kill_demuxers_reentrant ../player/loadfile.c:153
    #13 0x564887560d11 in uninit_demuxer ../player/loadfile.c:234
    #14 0x564887573ef6 in play_current_file ../player/loadfile.c:2041
    #15 0x564887575748 in mp_play_files ../player/loadfile.c:2201
    #16 0x5648875783cf in mpv_main ../player/main.c:464
    #17 0x5648877e89ad in main ../osdep/main-fn-unix.c:5
    #18 0x7f84709fd09a in __libc_start_main ../csu/libc-start.c:308
    #19 0x564887339799 in _start (./mpv/build/mpv+0x379799)

0x6080002263f0 is located 80 bytes inside of 96-byte region [0x6080002263a0,0x608000226400)
allocated by thread T26 (demux) here:
    #0 0x7f8472c16518 in calloc (/usr/lib/x86_64-linux-gnu/libasan.so.5+0xe9518)
    #1 0x5648877197b2 in ta_zalloc_size ../ta/ta.c:156
    #2 0x564887734b90 in mp_curl_avio_open ../stream/stream_curl.c:1047
    #3 0x5648873e3c2f in nested_io_open ../demux/demux_lavf.c:954
    #4 0x5648885ad2ce in avformat_open_input (./mpv/build/mpv+0x15ed2ce)

Thread T26 (demux) created by T0 here:
    #0 0x7f8472b7ddb0 in __interceptor_pthread_create (/usr/lib/x86_64-linux-gnu/libasan.so.5+0x50db0)
    #1 0x5648873a8f14 in demux_start_thread ../demux/demux.c:1202
    #2 0x564887562ff3 in enable_demux_thread ../player/loadfile.c:440
    #3 0x5648875723ea in play_current_file ../player/loadfile.c:1859
    #4 0x564887575748 in mp_play_files ../player/loadfile.c:2201
    #5 0x5648875783cf in mpv_main ../player/main.c:464
    #6 0x5648877e89ad in main ../osdep/main-fn-unix.c:5
    #7 0x7f84709fd09a in __libc_start_main ../csu/libc-start.c:308

SUMMARY: AddressSanitizer: bad-free (/usr/lib/x86_64-linux-gnu/libasan.so.5+0xe8fb0) in __interceptor_free
==7474==ABORTING

@kasper93
Copy link
Copy Markdown
Member Author

kasper93 commented May 12, 2026

Play HLS livestream - press q to quit:

Thanks for report! This is also reproducible with stream_lavf. I can take a look into fixing that when I have some time.

EDIT: Fixed by https://code.ffmpeg.org/FFmpeg/FFmpeg/pulls/23082

Comment thread stream/stream_curl.h Outdated
kasper93 and others added 14 commits May 12, 2026 18:36
stream-lavf-o[cookies] only feeds the lavf stream backend. Instead of
side-loading it, use core `--cookies-file` option.
Introduces stream_http, an internal libcurl-driven backend for http://
and https:// URLs. Runs all transfers on a dedicated curl multi thread
with a producer-side ring buffer, and exposes HTTP/2 multiplexing,
QUIC/HTTP/3 (when libcurl supports it), HSTS and TCP keep-alive.

Should generally be more stable and faster than FFmpeg HTTP/1.1 impl,
additionally connections are kept-alive between files open, so if you
open playlist of network files and navigate through them, it will re-use
the same connection.

Build is gated on the new 'libcurl' meson option (auto). When disabled
or unavailable, mpv silently falls back to FFmpeg's HTTP implementation.
Hook AVFormatContext.io_open so HLS/DASH segment fetches and other
nested FFmpeg I/O go through stream_http when libcurl is available.
This lets segments inherit the connection reuse, HTTP/2 multiplexing,
mid-stream resume and conditional compression of the top-level stream.

Falls back to FFmpeg's default I/O for non-HTTP URLs and when the curl
backend is not built in.
Allow only subset of supported protocols by libcurl.
At the initial file open, the buffer may be populated with more data,
than later requested by decoder, but don't drop this data. This fixes
initial open to perform multiple reads of the area, especially for
images that are likelly fully read for probbing already.
It's significantly more stable than Lavf. Fixes seeking out of cached
ranges. Additionally correctly marks demuxer_is_network for ftp too.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

7 participants