aka "--load-info".
Don't save a Response object to info JSON. Just create a new WebSocket
connection during the download. Due to Niconico's logic, the manifest
m3u8 url will be unusable soon if there is no active WebSocket
connection, so the reconnection will give us a valid manifest m3u8,
unless the WebSocket url has already expired.
- Use "downloader_options" to pass options used by the downloader.
- Combine the two downloaders into one.
- Don't inherit from "HlsFD".
Co-authored-by: pukkandan <pukkandan.ytdlp@gmail.com>
Major changes:
- Make a downloader for live timeshift videos. Time-based download rate
limit applies. RetryManager-based error recovery applies.
- Fix the incorrect url for WebSocket reconnection.
- Correctly close the WebSocket connection.
- [!] Apply "FFmpegFixupM3u8PP" for both non-timeshift and timeshift
MPEG-TS files by adding "m3u8_*" prefixes and inheriting from "HlsFD".
- [!] Change the protocol from "hls+fmp4" to "hls" in "startWatching"
WebSocket requests because I didn't see it in my test.
Minor changes:
- Support metadata extraction when no formats.
- Set "live_status" instead of "is_live".
- Clean up "info_dict": Change WebSocket configs to private to hide them
from users; extract common fields and remove unused ones.
- Update a download test.
- Fix bug in `axel` Cookie header arg
- Pass cookies to `curl` as strings
- Write session cookies for `aria2c` and `wget`
Closes#7539
Authored by: bashonly
- Remove `Cookie` header from `http_headers` immediately after loading into cookiejar
- Restore compat for `--load-info-json` cookies
- Add more tests
- Fix improper passing of Cookie header by `MailRu` extractor
Closes#7558
Authored by: bashonly, pukkandan
New networking interface consists of a `RequestDirector` that directs
each `Request` to appropriate `RequestHandler` and returns the
`Response` or raises `RequestError`. The handlers define adapters to
transform its internal Request/Response/Errors to our interfaces.
User-facing changes:
- Fix issues with per request proxies on redirects for urllib
- Support for `ALL_PROXY` environment variable for proxy setting
- Support for `socks5h` proxy
- Closes https://github.com/yt-dlp/yt-dlp/issues/6325, https://github.com/ytdl-org/youtube-dl/issues/22618, https://github.com/ytdl-org/youtube-dl/pull/28093
- Raise error when using `https` proxy instead of silently converting it to `http`
Authored by: coletdjnz
- ffmpeg: Calculate cookies from cookiejar and pass with `-cookies` arg instead of `-headers`
- aria2c, curl, wget: Write cookiejar to file and use external FD built-in cookiejar support
- httpie: Calculate cookies from cookiejar instead of `http_headers`
- axel: Calculate cookies from cookiejar and disable http redirection if cookies are passed
- May break redirects, but axel simply don't have proper cookie support
Ref: https://github.com/yt-dlp/yt-dlp/security/advisories/GHSA-v8mc-9377-rwjj
Authored by: bashonly, coletdjnz
This is useful for HLS where detecting whether the format is
actually DRM requires the child manifest to be downloaded.
Makes the error message when using `--test` inconsistent,
but doesn't really matter.
[YouTube] [core] Improve platform debug log, based on yt-dlp
d1c6c5c4d6
Except:
* 6ed34338285f722d0da312ce0af3a15a077a3e2a [jsinterp] Add short-cut evaluation for common expression
* There was no performance improvement when tested with https://github.com/ytdl-org/youtube-dl/issues/30641
* e8de54bce50f6f77a4d7e8e80675f7003d5bf630 [core] Handle `/../` sequences in HTTP URLs
* We plan to implement this differently
* `fragment_query`: passthrough any query in generic mpd/m3u8 manifest URLs to their fragments
* Add support for `extra_param_to_segment_url` to DASH downloader
Authored by: bashonly, pukkandan
