MarcoBuster/yt-dlp
1
0
Fork 0
mirror of https://github.com/yt-dlp/yt-dlp.git synced 2024-12-05 02:53:00 +01:00
Code Issues Projects Releases Wiki Activity
22,776 Commits 3 Branches 101 Tags 194 MiB
fbc66e3ab3
Commit Graph

459 Commits

Author SHA1 Message Date
coletdjnz
c26f9b991a
[extractor/youtube] Support changing extraction language (#4470) 
Adds `--extractor-args youtube:lang=<supported lang code>` extractor arg to prefer translated fields (e.g. title and description) of that language, if available, for all YouTube extractors. See README or error message for list of supported language codes.

Closes https://github.com/yt-dlp/yt-dlp/issues/387

Authored by: coletdjnz
 2022-09-09 05:16:46 +00:00
coletdjnz
3ffb2f5bea
[extractor/youtube] Fix video like count extraction 
Support new combined button layout
Authored by: coletdjnz
 2022-09-09 12:34:39 +12:00
pukkandan
17ffed1842
[docs] Improvements 
* Move detailed installation instructions to https://github.com/yt-dlp/yt-dlp/wiki/Installation
* Link to wiki where applicable
* Fix some mistakes. Closes #4853, Closes #4855, Closes #4852
* Improve some error messages
 2022-09-07 17:38:05 +05:30
pukkandan
7c6eb424d3
[extractor/youtube] Detect lazy-load-for-videos embeds 
Closes #4812
 2022-09-02 02:01:57 +05:30
pukkandan
05deb747bb
[jsinterp] Fix escape in regex 2022-09-01 16:46:32 +05:30
pukkandan
b505e8517a
[extractor/youtube] Fallback regex for nsig code extraction 2022-09-01 16:46:32 +05:30
coletdjnz
1ff88b7aec
[extractor/youtube] Add no-youtube-prefer-utc-upload-date compat option (#4771) 
This option reverts 992f9a730b and 17322130a9 to prefer the non-UTC upload date in microformats.

Authored by: coletdjnz, pukkandan
 2022-09-01 10:02:28 +00:00
pukkandan
da4db748fa
[utils] Add deprecation_warning 
See https://github.com/yt-dlp/yt-dlp/pull/2173#issuecomment-1097021515
 2022-08-30 21:03:07 +05:30
pukkandan
d81ba7d491
[jsinterp, extractor/youtube] Minor fixes 2022-08-30 18:13:37 +05:30
pukkandan
c4b2df872d
[jsinterp] Fix _separate 
Ref: https://github.com/yt-dlp/yt-dlp/issues/4635#issuecomment-1231126941
 2022-08-30 16:06:40 +05:30
Samantaz Fox
224b5a35f7
[extractor/youtube] Update iOS Innertube clients (#4792) 
Authored by: SamantazFox
 2022-08-29 03:36:55 +00:00
coletdjnz
50ac0e5416
[extractor/youtube] Use device-specific user agent (#4770) 
Thwart latest fingerprinting attempt (see https://github.com/iv-org/invidious/issues/3230#issuecomment-1226887639)

Authored by: coletdjnz
 2022-08-28 22:59:54 +00:00
pukkandan
5e01315aa1
[cache, extractor/youtube] Invalidate old cache 2022-08-27 07:25:14 +05:30
pukkandan
992dc6b486
[jsinterp] Implement timeout 
Workaround for #4716
 2022-08-22 06:19:06 +05:30
pukkandan
b25cac650f
[extractor/youtube] Fix bug in format sorting 2022-08-21 00:56:27 +05:30
pukkandan
90a1df305b
[test] Fix test_youtube_signature 2022-08-21 00:51:03 +05:30
pukkandan
a831c2ea90
[cleanup] Misc 2022-08-19 05:08:21 +05:30
pukkandan
25836db6be
[extractor/youtube] Add fallback to phantomjs 
Related #4635
 2022-08-18 21:35:18 +05:30
pukkandan
580ce00782
[youtube] Improve signature caching 
and refactor related functions
 2022-08-18 21:33:30 +05:30
pukkandan
f6ca640b12
[jsinterp] Fix for youtube player 1f7d5369 
Closes #4635 again
 2022-08-18 16:38:35 +05:30
pukkandan
3ce2933693
[youtube] Fix error reporting of "Incomplete data" 
Related: #4669
 2022-08-16 22:01:48 +05:30
pukkandan
5c6d2ef9d1
[youtube] Improve format sorting for IOS formats 
When no itag/resolution is available for reference, use the closest resolution
 2022-08-15 14:04:05 +05:30
Lesmiscore
62b58c0936
[docs] Consistent use of e.g. (#4643) 
Authored by: Lesmiscore
 2022-08-14 17:34:13 +05:30
pukkandan
8f53dc44a0
[jsinterp] Handle new youtube signature functions 
Closes #4635
 2022-08-14 05:12:32 +05:30
pukkandan
7e798d725e
[extractor] Fix format sorting of channels 2022-08-11 07:23:46 +05:30
coletdjnz
c7dcf0b31e
[extractor/youtube] Add androidSdkVersion parameter to Android Innertube clients 
Required to prevent YouTube returning a bad player response in some cases.

See: https://github.com/yt-dlp/yt-dlp/pull/4593, https://github.com/TeamNewPipe/NewPipe/issues/8713, https://github.com/iv-org/invidious/issues/3230, https://github.com/Tyrrrz/YoutubeExplode/issues/647

Authored by: coletdjnz
 2022-08-08 12:03:10 +12:00
pukkandan
a416623436
[extractor/youtube] Extract more format info 2022-08-08 01:47:07 +05:30
coletdjnz
a3e9642116
[extractor/youtube] Prevent redirect to unwanted videos (#4593) 
Example: https://www.youtube.com/watch?v=aQvGIIdgFDM

Authored by: coletdjnz
 2022-08-07 19:13:20 +05:30
coletdjnz
a0c830f488
[extractor/youtube] Bump Innertube client versions 
YouTube may be requiring new versions soon. See https://github.com/iv-org/invidious/issues/3230, https://github.com/TeamNewPipe/NewPipe/issues/8713

Authored by: coletdjnz
 2022-08-02 19:02:05 +12:00
pukkandan
be5c1ae862
Standardize retry mechanism (#1649) 
* [utils] Create `RetryManager`
* Migrate all retries to use the manager
* [extractor] Add wrapper methods for convenience
* Standardize console messages for retries
* Add `--retry-sleep` for extractors
 2022-08-02 01:43:18 +05:30
pukkandan
bfd973ece3 [extractors] Use new framework for existing embeds (#4307) 
`Brightcove` is difficult to migrate because it's subclasses may depend
on the signature of the current functions. So it is left as-is for now

Note: Tests have not been migrated
 2022-08-02 01:08:16 +05:30
pukkandan
f2e8dbcc00 [extractor, test] Basic framework for embed tests (#4307) 
and split download tests so they can be more easily run in CI

Authored by: coletdjnz
 2022-08-02 01:08:16 +05:30
lazypete365
e325a21a1f
[extractor/youtube] Add live_status=post_live (#4495) 
Related: https://github.com/yt-dlp/yt-dlp/issues/1564
Authored by: lazypete365
 2022-08-01 01:42:04 +05:30
pukkandan
31b532a1f2
[cleanup] Misc 2022-07-31 03:35:16 +05:30
pukkandan
c646d76f67
[webvtt, extractor/youtube] Extract auto-subs from livestream VODs 
Closes #4130

Authored by: pukkandan, fstirlitz
 2022-07-31 02:20:11 +05:30
pukkandan
07b47084ba
[extractor/youtube] Parse translated subtitles only when requested 
Closes #4274
 2022-07-31 02:20:11 +05:30
pukkandan
693f060040
[youtube,twitch] Allow waiting for channels to become live 
Closes #2597
 2022-07-26 09:33:19 +05:30
ftk
45e8a04e48
[extractor/youtube] More metadata for storyboards (#4334) 
Authored by: ftk
 2022-07-12 20:46:45 +05:30
pukkandan
a3fb1ca5ab
[extractor/youtube] Fix duration check for post-live manifestless mode 2022-07-10 09:59:32 +05:30
Andrew
12a1b2254d
[extractor/youtube, cleanup] Fix tests (#4293) 
Authored by: sheerluck
 2022-07-08 02:50:02 +05:30
pukkandan
a3976e0760
Improve chapter sanitization 2022-07-07 11:28:56 +05:30
pukkandan
ae61d108dd
[cleanup] Misc cleanup 2022-06-29 06:43:27 +05:30
crazymoose77756
962ffcf89c
[cleanup] Fix some typos (#4194) 
Authored by: crazymoose77756
 2022-06-26 17:50:06 -07:00
pukkandan
14f25df2b6
[compat] Remove deprecated functions from core code 2022-06-25 00:14:12 +05:30
pukkandan
ac66811112
[compat] Remove more functions 
Removing any more will require changes to a large number of extractors
 2022-06-25 00:08:55 +05:30
Brett824
06cc8f103b
[extractor/youtube] Mark videos as fully watched (#4146) 
* Also fixes videos appearing as shorts in watch history

Closes #2555
Authored by: Brett824
 2022-06-23 16:30:17 -07:00
pukkandan
9809740ba5
[extractor, cleanup] Reduce direct use of _downloader 2022-06-23 09:57:26 +05:30
pukkandan
471d0367c7
[youtube:clips] Support downloading clips 
Closes #2543
 2022-06-22 02:50:55 +05:30
pukkandan
a70635b8a1
[cleanup, utils] Don't use kwargs for format_field 2022-06-18 08:13:22 +05:30
pukkandan
0df111a371
[youtube] Extract comment_count from webpage 
Closes #4091
 2022-06-17 12:00:55 +05:30
coletdjnz
4ce05f5759
[extractor/youtube] Fix live chat for videos with content warning 
Fixes #4051
Authored by: coletdjnz
 2022-06-12 17:56:50 +12:00
pukkandan
56ba69e4c9
[cleanup] Misc fixes 
Closes #4027
 2022-06-11 05:00:12 +05:30
coletdev
2ae778b8fc
[extractor/youtube] Add innertube_host and innertube_key extractor args (#3916) 
Allows user to override Innertube API host or key for all requests
Authored by: coletdjnz
 2022-06-08 22:18:01 +00:00
MMM
36195c4461
[dash] Show fragment count with --live-from-start (#3493) 
Authored by: flashdagger
 2022-06-07 05:44:08 -07:00
coletdjnz
65141660ab
[extractor/youtube] Fix bug in b7c47b7438 
Closes #3997

Authored by: coletdjnz
 2022-06-07 12:26:36 +12:00
pukkandan
b7c47b7438
[extractor] Add _search_json 
All fetching of JSON objects should eventually be done with this function
but only `youtube` is being refactored for now
 2022-06-06 19:46:45 +05:30
pukkandan
1890fc6389
[cleanup] Misc fixes 
Cherry-picks from: #3498, #3947
Related: #3949, https://github.com/yt-dlp/yt-dlp/issues/1839#issuecomment-1140313836
Authored by: pukkandan, flashdagger, gamer191
 2022-06-03 21:45:35 +05:30
coletdev
c7a7baaa13
[extractor/youtube] Fix :ytnotifications extractor (#3775) 
Still some issues, see https://github.com/yt-dlp/yt-dlp/pull/3775

Authored by: coletdjnz
 2022-06-03 07:04:39 +00:00
pukkandan
6b0b0a289a
[extractor/youtube:tab] Detect videoRenderer in _post_thread_continuation_entries 2022-06-01 02:03:24 +05:30
coletdjnz
ee27297f82
[extractor/youtube] Fix initial player response extraction 
Authored by: pukkandan, coletdjnz
 2022-05-29 19:54:22 +12:00
pukkandan
0fe51254cb
[extractor/youtube] Bring back _extract_chapters_from_description 
Closes #3886
 2022-05-29 01:00:41 +05:30
pukkandan
52023f1291
[extractor/youtube] Make signature extraction non-fatal 
and reduce verbosity of it's warning

Closes #3882
 2022-05-29 00:00:24 +05:30
pukkandan
829bbd1d05
[youtube] Add warning for PostLiveDvr 
Closes #3746, Related #1564
 2022-05-27 05:07:00 +05:30
pukkandan
2762dbb17e
[compat] Add functools.cached_property 2022-05-20 21:06:37 +05:30
pukkandan
2414649192
[cleanup] Misc cleanup 2022-05-17 00:07:43 +05:30
Jordan Weatherby
d1c4f6d4da
[youtube] Added piped instance urls (#3714) 
Closes #3661
Authored by: JordanWeatherby
 2022-05-12 13:52:13 -07:00
pukkandan
82d020804d
[extractor] Use classmethod/property where possible 
and refactor lazy extractors accordingly.

This reduces the need to create extractor instances
 2022-05-13 00:23:26 +05:30
pukkandan
0f06bcd759
[cleanup] Minor fixes (See desc) 
* [youtube] Fix `--youtube-skip-dash-manifest`
* [build] Use `$()` in `Makefile`. Closes #3684
* Fix bug in 385ffb467b
* Fix bug in 43d7f5a5d0
* [cleanup] Remove unnecessary `utf-8` from `str.encode`/`bytes.decode`
* [utils] LazyList: Expose unnecessarily "protected" attributes
and other minor cleanup
 2022-05-09 17:59:26 +05:30
pukkandan
91e5e839d3
[youtube] Deprioritize format 22 
Reduces chance of encountering #3372
 2022-05-07 08:05:32 +05:30
coletdev
6e634cbe42
[youtube] Add YoutubeStoriesIE (#3362) 
Get channel stories with `ytstories:<channel UCID>`

Authored-by: coletdjnz
 2022-05-01 06:46:28 +00:00
pukkandan
bfec31bec8
[youtube] De-prioritize auto-generated thumbnails 
Closes #3112
 2022-04-29 07:41:39 +05:30
pukkandan
1d485a1a79
[cleanup] Misc fixes 
Closes #3565, https://github.com/yt-dlp/yt-dlp/issues/3514#issuecomment-1105944364
 2022-04-29 07:39:33 +05:30
pukkandan
0a41f331cc
[doc] Minor improvements 
Closes #3518, Closes #3560
 2022-04-29 06:53:36 +05:30
pukkandan
19a0394044
[cleanup] Misc cleanup and refactor (#2173) 2022-04-18 02:28:28 +05:30
coletdjnz
affc4fefea
[youtube] Fix episode metadata extraction 2022-04-15 16:22:03 +12:00
pukkandan
f82711587c
[cleanup] Sort imports 
Using https://github.com/PyCQA/isort

    isort -m VERTICAL_HANGING_INDENT --py 36 -l 80 --rr -n --tc .
 2022-04-12 05:32:52 +05:30
pukkandan
86e5f3ed2e
[cleanup] Upgrade syntax 
Using https://github.com/asottile/pyupgrade

1. `__future__` imports and `coding: utf-8` were removed
2. Files were rewritten with `pyupgrade --py36-plus --keep-percent-format`
3. f-strings were cherry-picked from `pyupgrade --py36-plus`

Extractors are left untouched (except removing header) to avoid unnecessary merge conflicts
 2022-04-12 05:32:51 +05:30
krichbanana
ca5300c7ed
[youtube] Add :ytnotifications extractor (#3347) 
Authored by: krichbanana
 2022-04-09 12:55:24 -07:00
pukkandan
a25bca9f89
[youtube, cleanup] Minor refactoring 
Authored by: coletdjnz, pukkandan
 2022-04-10 01:21:34 +05:30
coletdev
61d3665d9d
[youtube] Fix uploader for collaborative playlists (#3332) 
Authored by: coletdjnz
 2022-04-07 01:11:16 -07:00
pukkandan
a44ca5a470
[cleanup] Misc fixes 
Closes https://github.com/yt-dlp/yt-dlp/pull/3213, Closes https://github.com/yt-dlp/yt-dlp/pull/3117

Related: https://github.com/yt-dlp/yt-dlp/issues/3146#issuecomment-1077323114, https://github.com/yt-dlp/yt-dlp/pull/3277#discussion_r841019671, a825ffbffa (commitcomment-68538986), https://github.com/yt-dlp/yt-dlp/issues/2360, 5fa3c9a88f (r70393519), 5fa3c9a88f (r70393254)
 2022-04-05 18:12:18 +05:30
pukkandan
a17526e427
[youtube:tab] Minor improvements (See desc) 
* Support shorts on channel homepage
* Extract thumbnail of OLAK playlists
 2022-04-03 19:01:03 +05:30
coletdev
ad210f4fd4
[youtube:search] Support hashtag entries (#3265) 
Authored-by: coletdjnz
 2022-04-02 06:11:14 +00:00
David
e7870111e8
[YouTube] Add new age-gate bypass (#3233) 
Closes #3182
Authored by: zerodytrash, pukkandan
 2022-03-29 03:05:31 -07:00
coletdev
1c1b2f96ae
[youtube:tab] Fix duration extraction for shorts (#3171) 
Related: https://github.com/TeamNewPipe/NewPipe/issues/8034
Authored-by: coletdjnz
 2022-03-28 00:49:42 +00:00
pukkandan
1235d333ab
[youtube] Fix auto-translated automatic captions 
d49669acad only covered ASR

Closes #2956
 2022-03-27 14:06:26 +05:30
pukkandan
18e4940825
[youtube] Add extractor-arg to skip auto-translated subs 2022-03-27 14:04:20 +05:30
shirt
727029c508
[youtube] Detect DRM better 
Authored by: shirt-dev
 2022-03-27 11:27:27 +05:30
coletdev
fd2ad7cb24
[youtube:tab] Return shorts url if video is a short (#3168) 
Allows filtering out shorts from feeds with `--match-filter`
Closes #3165
Authored-by: coletdjnz
 2022-03-27 05:20:25 +00:00
pukkandan
ae72962643
[youtube] Try embedded client variants before agegate 
agegate variants appears to be broken, but don't remove them for the time-being
 2022-03-25 05:00:41 +05:30
pukkandan
52efa4b312
[extractor] Add _perform_login function (#2943) 
* Adds new functions `_initialize_pre_login` and `_perform_login` as part of the extractor API
* Adds `ie.supports_login` to the public API
 2022-03-18 13:53:33 -07:00
pukkandan
e4b98809cf
[youtube] Fix pagination of membership tab 2022-03-18 05:23:51 +05:30
pukkandan
a1b2d84360
[youtube] Avoid false positives when detecting damaged formats 
Closes #3083
 2022-03-16 19:46:29 +05:30
coletdjnz
5dbc77df26
[youtube:api] Prefer minified JSON response 
Authored-by: coletdjnz
 2022-03-16 09:29:15 +13:00
coletdev
b3edc8068e
[downloader/mhtml] Fix fragments with absolute urls (#3044) 
Authored-by: coletdjnz
 2022-03-13 22:03:40 +00:00
coletdev
17322130a9
[youtube] Improve video upload date handling (#3029) 
* Don't prefer UTC upload date for past live streams/premieres
* Improve regex (fixes a regression)

Authored-by: coletdjnz
 2022-03-13 22:02:44 +00:00
pukkandan
ff91cf7483
[utils] Add get_first 2022-03-09 02:26:52 +05:30
pukkandan
a405b38f20
[youtube] Further de-prioritize 3gp format 2022-03-08 23:02:38 +05:30
pukkandan
08d30158ec
[cleanup, docs] Misc cleanup 
Closes #2828, closes #2734, closes #2802, closes #2937
 2022-03-08 22:38:06 +05:30
coletdev
992f9a730b
[youtube] Prefer UTC upload date for videos (#2223) 
Except for live/scheduled streams/premieres. 
Closes #1881
Related: #2402 
Authored-by: coletdjnz
 2022-03-08 12:58:19 +05:30
pukkandan
d49669acad
[youtube] Fix automatic captions 
Closes #2956
 2022-03-05 09:42:12 +05:30
pukkandan
e48b3875ec
Revert 2e4cacd038 
Closes #2923
 2022-03-04 20:18:14 +05:30
coletdev
7c219ea601
[youtube:tab] Follow redirect to regional channel (#2884) 
Closes #2694
Authored by: coletdjnz
 2022-02-28 21:08:19 -08:00
Lesmiscore (Naoya Ozaki)
a539f06570
[downloader/fragment] Improve --live-from-start for YouTube livestreams (#2870) 2022-02-25 02:00:46 +09:00
pukkandan
1108613f02
[youtube:tab] Reject webpage data if redirected to home page 
Closes #2660
 2022-02-23 04:25:55 +05:30
pukkandan
a30a6ed3e4
[youtube:tab] Add approximate_date extractor-arg 2022-02-23 04:25:55 +05:30
Lesmiscore
a7d4acc018
[youtube] Escape possible $ in _extract_n_function_name regex 2022-02-20 17:33:58 +09:00
pukkandan
0ad92dfb18
[youtube] De-prioritize potentially damaged formats 
Closes #2823
 2022-02-18 19:41:37 +05:30
pukkandan
2e4cacd038
[youtube] Fix intermittent failure of embed-based age-gate bypass 2022-02-18 18:03:13 +05:30
MinePlayersPE
c571b3a6ab
[youtube] Fix n-sig extraction for phone player JS (#2816) 
Authored by: MinePlayersPE
 2022-02-18 00:12:20 -08:00
coletdev
60f393e48b
[youtube] Ensure subtitle urls are absolute (#2765) 
Closes #2755

Authored by: coletdjnz
 2022-02-13 13:36:01 -08:00
pukkandan
ab6df717d1
[youtube] Differentiate descriptive audio by language code 
Related: #2677
 2022-02-12 02:13:17 +05:30
pukkandan
0c8d9e5fec
[youtube] Label original auto-subs 
Closes #2655
 2022-02-12 01:50:49 +05:30
pukkandan
84bbc54599
[youtube:search] Add tests 2022-02-04 12:02:01 +05:30
coletdjnz
af5c1c553e
[youtube] Fix search extractor 
Regression introduced in 16aa9ea41d. Closes #2628
Authored-by: coletdjnz
 2022-02-04 10:32:56 +13:00
pukkandan
455a15e2dc
[cleanup,docs] Minor fixes 
Closes #2541, #2484
 2022-02-03 21:00:39 +05:30
pukkandan
182bda88e8
[youtube, cleanup] Misc fixes and cleanup 2022-02-02 02:15:53 +05:30
pukkandan
16aa9ea41d [youtube] Add extractor YoutubeMusicSearchURLIE 
Closes #2568
 2022-02-02 00:11:38 +05:30
pukkandan
48416bc4a8
[youtube] Fix n-sig for player e06dea74 2022-02-01 08:10:41 +05:30
coletdjnz
87e8e8a7d0
[youtube:api] Do not use seek when reading HTTPError response 
Authored-by: coletdjnz
 2022-01-23 19:11:32 +13:00
pukkandan
6970b6005e
[cleanup] Minor fixes 
Closes #2334
 2022-01-21 13:27:44 +05:30
pukkandan
e0ddbd02bd
[cleanup] Use format_field where applicable 2022-01-21 13:27:40 +05:30
coletdjnz
6c73052c0a
[youtube] Extract channel subscriber count (#2399) 
Closes #2350
* Adds `channel_follower_count` field
Authored-by: coletdjnz
 2022-01-21 06:04:36 +00:00
coletdjnz
396a76f7bf
[youtube] Enforce UTC (#2402) 
and [utils] use `utcnow` in `datetime_from_str`

Related: #2223 
Authored by: coletdjnz
 2022-01-20 20:32:01 +05:30
coletdjnz
301d07fc4b
[youtube:tab] Extract channel banner (#2400) 
Closes #2237
Authored by: coletdjnz
 2022-01-20 20:29:09 +05:30
krichbanana
64f36541c9
[youtube:tab] Raise error on tab redirect (#2318) 
Closes #2306
Authored by: krichbanana, coletdjnz
 2022-01-20 03:01:57 +05:30
pukkandan
66f7c6a3e0
[youtube] Do not return upload_date for playlists 
Closes #2349
Partially reverts #1018
Re-opens #1883
 2022-01-19 19:07:40 +05:30
pukkandan
596379e260
[youtube] Make invalid storyboard URL non-fatal 
Closes #2382
 2022-01-19 18:57:29 +05:30
pukkandan
b6ce9bb038
[youtube] Detect live-stream embeds 
Closes #2380
 2022-01-19 18:56:55 +05:30
Unit 193
4c968755fc
[PornHub,YouTube] Refresh onion addresses (#2272) 
Authored by: unit193
 2022-01-09 20:08:34 +05:30
pukkandan
976ae3eabb
[youtube] Update tests 2022-01-07 17:25:58 +05:30
coletdjnz
f0d785d3ed
[youtube:tab] Extract more playlist metadata (#2069) 
* Add fields modified_date, modified_timestamp
* Add field playlist_count
* [youtube:tab] Extract view_count, playlist_count, modified_date

Authored by: coletdjnz, pukkandan
 2022-01-07 16:33:02 +05:30
coletdjnz
18c7683d27
[youtube:api] Update Innertube clients (#2163) 
* Updated iOS clients to support 60fps formats (see: https://github.com/TeamNewPipe/NewPipeExtractor/issues/680#issuecomment-1002724558)
* General update of versions and keys of other clients
 Authored-by: coletdjnz
 2022-01-02 05:22:31 +00:00
pukkandan
185bf31070
[youtube] End live-from-start properly when stream ends with 403 
Closes #2089
 2021-12-26 16:14:00 +05:30
coletdjnz
a709d87335
[youtube:tab] Extract video thumbnails from playlist (#2096) 
closes #1184
Co-Authored-by: coletdjnz, pukkandan
 2021-12-24 03:42:02 +00:00
The Hatsune Daishi
adbc4ec4bb
[dash,youtube] Download live from start to end (#888) 
* Add option `--live-from-start` to enable downloading live videos from start
* Add key `is_from_start` in formats to identify formats (of live videos) that downloads from start
* [dash] Create protocol `http_dash_segments_generator` that allows a function to be passed instead of fragments
* [fragment] Allow multiple live dash formats to download simultaneously
* [youtube] Implement fragment re-fetching for the live dash formats
* [youtube] Re-extract dash manifest every 5 hours (manifest expires in 6hrs)
* [postprocessor/ffmpeg] Add `FFmpegFixupDuplicateMoovPP` to fixup duplicated moov atoms

Known issue: Ctrl+C doesn't work on Windows when downloading multiple formats

Closes #1521
Authored by: nao20010128nao, pukkandan
 2021-12-20 11:36:46 +05:30
coletdjnz
f3aa3c3f98
[youtube:tab] Extract more metadata from feeds/channels/playlists (#1018) 
Parse relative time text, extract live, upcoming status, availability and channel id from feeds/channels/playlists (where applicable). 
Closes #1883
Authored-by: coletdjnz
 2021-12-20 04:47:53 +00:00
pukkandan
37e57a9fd4
[youtube:tab] Ignore query when redirecting channel to playlist 
and cleanup of related code
Closes #2046
 2021-12-19 09:05:59 +05:30
pukkandan
39ca3b5c7f
[extractor] Standardize _live_title 2021-12-15 22:09:07 +05:30
coletdjnz
46383212b3
[youtube:comments] Add more options for limiting number of comments extracted (#1626) 
Extends `max_comments` extractor arg to support `max-parents,max-replies,max-replies-per-thread`.
Authored-by: coletdjnz
 2021-12-15 04:29:48 +00:00
pukkandan
ff9f925b63
[test/download] Add more fields 2021-12-15 03:38:12 +05:30
pukkandan
5f7cb91ae9
[youtube] Fix ytsearchdate 
Related: #1851
 2021-12-01 23:23:45 +05:30
pukkandan
b29165267f
[youtube] Decrypt n-sig for URLs with ratebypass 
Closes #1796
 2021-11-29 22:51:18 +05:30
pukkandan
a61fd4cf6f
[youtube:search_url] Add playlist/channel support 
Closes #1213, #1214
 2021-11-24 09:33:15 +05:30
pukkandan
a6213a4925
[cleanup,youtube] Reorganize Tab and Search extractor inheritances 2021-11-24 09:28:59 +05:30
pukkandan
dd2a987d3f
[tests] Fix tests 2021-11-19 06:30:25 +05:30
pukkandan
9222c38182
[cleanup] Minor cleanup 
Closes #1696, Closes #1673
 2021-11-19 05:36:28 +05:30
pukkandan
720c309932
[youtube] Add storyboard formats 
Closes: #1553, https://github.com/ytdl-org/youtube-dl/issues/9868
Related: https://github.com/ytdl-org/youtube-dl/pull/14951
 2021-11-17 01:29:34 +05:30
coletdjnz
d0d012d4e7
[youtube] Add default player client (#1685) 
Authored-by: coletdjnz
 2021-11-16 01:22:01 +00:00
pukkandan
e339d25a0d
[youtube] Minor improvement to format sorting 2021-11-13 15:15:23 +05:30
pukkandan
c586f9e8de
[cleanup] minor fixes 2021-11-10 04:19:54 +05:30
pukkandan
aa9369a2d8
[cleanup] Minor improvements to error and debug messages 2021-11-10 04:19:33 +05:30
Ryan Hendrickson
d9190e4467
[youtube] Add Invidious list for playlists/channels (#1567) 
Authored by: rhendric
 2021-11-06 08:37:34 +05:30
pukkandan
34921b4345
[utils] Add join_nonempty 2021-11-06 07:53:55 +05:30
pukkandan
c18d4482b1
[youtube] Fix sorting for some videos 2021-11-05 00:12:11 +05:30
pukkandan
a4211baff5
[cleanup] Minor cleanup 2021-11-04 03:53:15 +05:30
pukkandan
d89257f398
[youtube] Remove unnecessary no-playlist warning 2021-11-03 16:35:09 +05:30
pukkandan
a0bb6ce58d
[youtube] refactor itag processing 2021-10-31 13:26:44 +05:30
pukkandan
404f611f1c
[youtube] Fix throttling by decrypting n-sig (#1437) 2021-10-31 09:53:58 +05:30
pukkandan
fccf502118
[youtube] Populate thumbnail with the best "known" thumbnail 
Closes #402, Related: https://github.com/yt-dlp/yt-dlp/issues/340#issuecomment-950290624
 2021-10-24 15:00:18 +05:30
pukkandan
96565c7e55
[cleanup] Add keyword automatically to SearchIE descriptions 
and some minor cleanup of docs
 2021-10-23 21:20:19 +05:30
pukkandan
49a57e70a9
[cleanup] misc 2021-10-23 02:09:10 +05:30
pukkandan
176f1866cb
Add HDR information to formats 2021-10-18 18:35:02 +05:30
pukkandan
e820fbaa6f
Do not verify thumbnail URLs by default 
Partially reverts cca80fe611 and 0ba692acc8

Unless `--check-formats` is specified, this causes yt-dlp to return incorrect thumbnail urls.
See https://github.com/yt-dlp/yt-dlp/issues/340#issuecomment-877909966, #402

But the overhead in general use is not worth it

Closes #694, #725
 2021-10-18 15:44:47 +05:30
pukkandan
24b0a72b30
[cleanup] Remove broken youtube login code 2021-10-18 09:25:51 +05:30
coletdjnz
aae16f6ed9
[youtube:comments] Fix comment section not being extracted in new layouts (#1324) 
Co-authored-by: coletdjnz, pukkandan
 2021-10-18 02:58:42 +00:00
pukkandan
580d3274e5 [youtube] Expose different formats with same itag 2021-10-16 20:28:17 +05:30
pukkandan
ecdc9049c0
[YouTube] Add auto-translated subtitles 
Closes #1245
 2021-10-12 15:21:32 +05:30
pukkandan
a2160aa45f
[extractor] Generalize getcomments implementation 2021-10-12 15:21:30 +05:30
pukkandan
cc16383ff3
[extractor] Simplify search extractors 2021-10-12 15:21:30 +05:30
coletdjnz
ac56cf38a4
[youtube:tab] Fallback to API when webpage fails to download (#1122) 
and add some extractor_args to force this mode
Authored by: coletdjnz
 2021-10-09 02:49:25 +05:30
pukkandan
1276a43a77
[youtube] Fix non-fatal errors in fetching player 2021-10-06 05:45:19 +05:30
pukkandan
e6f21b3d92
[docs,cleanup] Some minor refactoring and improve docs 2021-09-30 03:32:52 +05:30
coletdjnz
99e9e001de
[youtube] Cleanup authentication code (#786) 
Authored by: coletdjnz
 2021-09-24 06:22:17 +05:30
pukkandan
5e3f2f8fc4
[youtube] Return full URL instead of just ID 2021-09-22 05:37:41 +05:30
pukkandan
3cd786dbd7
[youtube] Warn when trying to download clips 2021-09-19 19:41:10 +05:30
coletdjnz
b6de707d13
[youtube] Improvements to JS player extraction (See desc) (#860) 
* fallback player url extraction when it fails to be extracted from the webpage
* don't download js player unnecessarily for clients that don't require it
* try to extract js player url from any additional client configs
* ability to skip the js player usage/download using `player_skip=js`
* ability to skip the initial webpage download using `player_skip=webpage`

known issue:
* authentication for multi-channel accounts and multi-account cookies may not work correctly if the webpage or client configs are skipped
*  formats from the web client requiring signature decryption will be skipped if player js extraction is skipped

Authored by: coletdjnz
 2021-09-06 12:56:41 +05:30
pukkandan
526d74ec5a
[cleanup] Misc 2021-09-05 11:16:23 +05:30
coletdjnz
c0ac49bcca
[youtube] Retry on 'Unknown Error' (#854) 
and do not repeat unimportant alerts

Closes #839
Authored by: coletdjnz
 2021-09-04 08:03:42 +05:30
pukkandan
a7429aa9fa
[youtube] Fix subtitle names 2021-09-02 02:26:27 +05:30
pukkandan
c311988d19
[youtube] Improve 26e8e04454 
The streams of the same itag may have slightly different size/bitrate
 2021-08-26 08:27:29 +05:30
pukkandan
26e8e04454
[youtube] Prefer audio stream that YouTube considers default 
Fixes: https://github.com/ytdl-org/youtube-dl/issues/29864
Related: https://github.com/clsid2/mpc-hc/issues/1268
 2021-08-26 08:08:34 +05:30
coletdjnz
0bb1bc1b10
[youtube] Remove annotations and deprecate --write-annotations (#765) 
Closes #692 
Authored by: coletdjnz
 2021-08-24 09:22:40 +05:30
pukkandan
88acdbc269
[extractor] Better error message for DRM (#729) 
Closes #636
 2021-08-23 01:38:38 +05:30
Tom-Oliver Heidel
9b5fa9ee7c
[youtube] Add av01 itags to known formats list (#747) 
Authored by: blackjack4494
 2021-08-23 01:29:43 +05:30
pukkandan
3fb4e21b38
[lazy_extractors] Fix suitable and add flake8 test 2021-08-23 01:04:29 +05:30
pukkandan
4dfbf8696b
[utils] Add parse_qs 2021-08-23 00:50:43 +05:30
pukkandan
8fc54b1230
[youtube] Add shorts to _VALID_URL 
Normally the generic extractor will redirect the URL,
but the cookies consent screen may sometimes appear instead

Closes #752
 2021-08-23 00:50:42 +05:30
pukkandan
5ad28e7ffd
[extractor] Common function _match_valid_url 2021-08-23 00:50:40 +05:30
coletdjnz
641ad5d813
[youtube] Extract error messages from HTTPError response (#644) 
Authored by: coletdjnz
 2021-08-13 11:48:26 +05:30
PSlava
858a65ecc1
[youtube] Improve signature function detection (#641) 
Authored by: PSlava (Slava <slash@i-slash.com>)
 2021-08-08 09:24:37 +05:30
pukkandan
379e44ed3c [youtube] Raise appropriate error when API pages can't be downloaded 2021-08-07 21:17:06 +05:30
pukkandan
57015a4a3f
[youtube] extractor-arg to show live dash formats 
If replay is enabled, these formats can be used to download the last 4 hours
 2021-08-07 12:47:54 +05:30
pukkandan
cf87314d4e
[youtube] Extract SAPISID only once 2021-08-02 08:00:08 +05:30
David
e7e94f2a5c
[youtube] Add age-gate bypass for unverified accounts (#600) 
Adds `_creator` variants for each client

Authored by: zerodytrash, colethedj, pukkandan
 2021-08-02 02:43:46 +05:30
coletdjnz
eca330cb88
[youtube] Fix default global API key 
bug introduced in 000c15a4ca
 2021-08-01 06:12:26 +00:00
pukkandan
3619f78d2c
[youtube] Misc cleanup (#577) 
Authored by: pukkandan, colethedj
 2021-07-31 03:01:49 +05:30
pukkandan
65c2fde23f
[youtube] Add thirdParty to agegate clients (#577) 
* This allows more videos like `tf2U5Vyj0oU` to become embeddable
    See https://github.com/yt-dlp/yt-dlp/pull/575#issuecomment-888837000
* Also added tests for all types of age-gate

Closes #581
 2021-07-31 02:20:21 +05:30
pukkandan
000c15a4ca
[youtube] simplify and de-duplicate client definitions (#577) 2021-07-31 02:14:15 +05:30
colethedj
9275f62cf8
[youtube] Improve age-gate detection (#577) 
Authored by: colethedj
 2021-07-31 02:13:55 +05:30