mirror of
https://github.com/yt-dlp/yt-dlp.git
synced 2024-09-01 12:05:26 +02:00
Merge branch 'master' into deezer
This commit is contained in:
commit
d1f9f08963
18
.github/ISSUE_TEMPLATE/1_broken_site.yml
vendored
18
.github/ISSUE_TEMPLATE/1_broken_site.yml
vendored
@ -1,5 +1,5 @@
|
||||
name: Broken site
|
||||
description: Report broken or misfunctioning site
|
||||
description: Report error in a supported site
|
||||
labels: [triage, site-bug]
|
||||
body:
|
||||
- type: checkboxes
|
||||
@ -7,7 +7,7 @@ body:
|
||||
label: DO NOT REMOVE OR SKIP THE ISSUE TEMPLATE
|
||||
description: Fill all fields even if you think it is irrelevant for the issue
|
||||
options:
|
||||
- label: I understand that I will be **blocked** if I remove or skip any mandatory\* field
|
||||
- label: I understand that I will be **blocked** if I *intentionally* remove or skip any mandatory\* field
|
||||
required: true
|
||||
- type: checkboxes
|
||||
id: checklist
|
||||
@ -16,15 +16,15 @@ body:
|
||||
description: |
|
||||
Carefully read and work through this check list in order to prevent the most common mistakes and misuse of yt-dlp:
|
||||
options:
|
||||
- label: I'm reporting a broken site
|
||||
- label: I'm reporting that a **supported** site is broken
|
||||
required: true
|
||||
- label: I've verified that I'm running yt-dlp version **2022.09.01** ([update instructions](https://github.com/yt-dlp/yt-dlp#update)) or later (specify commit)
|
||||
- label: I've verified that I'm running yt-dlp version **2023.03.04** ([update instructions](https://github.com/yt-dlp/yt-dlp#update)) or later (specify commit)
|
||||
required: true
|
||||
- label: I've checked that all provided URLs are playable in a browser with the same IP and same login details
|
||||
required: true
|
||||
- label: I've checked that all URLs and arguments with special characters are [properly quoted or escaped](https://github.com/yt-dlp/yt-dlp/wiki/FAQ#video-url-contains-an-ampersand--and-im-getting-some-strange-output-1-2839-or-v-is-not-recognized-as-an-internal-or-external-command)
|
||||
required: true
|
||||
- label: I've searched the [bugtracker](https://github.com/yt-dlp/yt-dlp/issues?q=) for similar issues **including closed ones**. DO NOT post duplicates
|
||||
- label: I've searched [known issues](https://github.com/yt-dlp/yt-dlp/issues/3766) and the [bugtracker](https://github.com/yt-dlp/yt-dlp/issues?q=) for similar issues **including closed ones**. DO NOT post duplicates
|
||||
required: true
|
||||
- label: I've read the [guidelines for opening an issue](https://github.com/yt-dlp/yt-dlp/blob/master/CONTRIBUTING.md#opening-an-issue)
|
||||
required: true
|
||||
@ -50,6 +50,8 @@ body:
|
||||
options:
|
||||
- label: Run **your** yt-dlp command with **-vU** flag added (`yt-dlp -vU <your command line>`)
|
||||
required: true
|
||||
- label: "If using API, add `'verbose': True` to `YoutubeDL` params instead"
|
||||
required: false
|
||||
- label: Copy the WHOLE output (starting with `[debug] Command-line config`) and insert it below
|
||||
required: true
|
||||
- type: textarea
|
||||
@ -62,7 +64,7 @@ body:
|
||||
[debug] Command-line config: ['-vU', 'test:youtube']
|
||||
[debug] Portable config "yt-dlp.conf": ['-i']
|
||||
[debug] Encodings: locale cp65001, fs utf-8, pref cp65001, out utf-8, error utf-8, screen utf-8
|
||||
[debug] yt-dlp version 2022.09.01 [9d339c4] (win32_exe)
|
||||
[debug] yt-dlp version 2023.03.04 [9d339c4] (win32_exe)
|
||||
[debug] Python 3.8.10 (CPython 64bit) - Windows-10-10.0.22000-SP0
|
||||
[debug] Checking exe version: ffmpeg -bsfs
|
||||
[debug] Checking exe version: ffprobe -bsfs
|
||||
@ -70,8 +72,8 @@ body:
|
||||
[debug] Optional libraries: Cryptodome-3.15.0, brotli-1.0.9, certifi-2022.06.15, mutagen-1.45.1, sqlite3-2.6.0, websockets-10.3
|
||||
[debug] Proxy map: {}
|
||||
[debug] Fetching release info: https://api.github.com/repos/yt-dlp/yt-dlp/releases/latest
|
||||
Latest version: 2022.09.01, Current version: 2022.09.01
|
||||
yt-dlp is up to date (2022.09.01)
|
||||
Latest version: 2023.03.04, Current version: 2023.03.04
|
||||
yt-dlp is up to date (2023.03.04)
|
||||
<more lines>
|
||||
render: shell
|
||||
validations:
|
||||
|
@ -7,7 +7,7 @@ body:
|
||||
label: DO NOT REMOVE OR SKIP THE ISSUE TEMPLATE
|
||||
description: Fill all fields even if you think it is irrelevant for the issue
|
||||
options:
|
||||
- label: I understand that I will be **blocked** if I remove or skip any mandatory\* field
|
||||
- label: I understand that I will be **blocked** if I *intentionally* remove or skip any mandatory\* field
|
||||
required: true
|
||||
- type: checkboxes
|
||||
id: checklist
|
||||
@ -18,13 +18,13 @@ body:
|
||||
options:
|
||||
- label: I'm reporting a new site support request
|
||||
required: true
|
||||
- label: I've verified that I'm running yt-dlp version **2022.09.01** ([update instructions](https://github.com/yt-dlp/yt-dlp#update)) or later (specify commit)
|
||||
- label: I've verified that I'm running yt-dlp version **2023.03.04** ([update instructions](https://github.com/yt-dlp/yt-dlp#update)) or later (specify commit)
|
||||
required: true
|
||||
- label: I've checked that all provided URLs are playable in a browser with the same IP and same login details
|
||||
required: true
|
||||
- label: I've checked that none of provided URLs [violate any copyrights](https://github.com/yt-dlp/yt-dlp/blob/master/CONTRIBUTING.md#is-the-website-primarily-used-for-piracy) or contain any [DRM](https://en.wikipedia.org/wiki/Digital_rights_management) to the best of my knowledge
|
||||
required: true
|
||||
- label: I've searched the [bugtracker](https://github.com/yt-dlp/yt-dlp/issues?q=) for similar issues **including closed ones**. DO NOT post duplicates
|
||||
- label: I've searched [known issues](https://github.com/yt-dlp/yt-dlp/issues/3766) and the [bugtracker](https://github.com/yt-dlp/yt-dlp/issues?q=) for similar issues **including closed ones**. DO NOT post duplicates
|
||||
required: true
|
||||
- label: I've read the [guidelines for opening an issue](https://github.com/yt-dlp/yt-dlp/blob/master/CONTRIBUTING.md#opening-an-issue)
|
||||
required: true
|
||||
@ -62,6 +62,8 @@ body:
|
||||
options:
|
||||
- label: Run **your** yt-dlp command with **-vU** flag added (`yt-dlp -vU <your command line>`)
|
||||
required: true
|
||||
- label: "If using API, add `'verbose': True` to `YoutubeDL` params instead"
|
||||
required: false
|
||||
- label: Copy the WHOLE output (starting with `[debug] Command-line config`) and insert it below
|
||||
required: true
|
||||
- type: textarea
|
||||
@ -74,7 +76,7 @@ body:
|
||||
[debug] Command-line config: ['-vU', 'test:youtube']
|
||||
[debug] Portable config "yt-dlp.conf": ['-i']
|
||||
[debug] Encodings: locale cp65001, fs utf-8, pref cp65001, out utf-8, error utf-8, screen utf-8
|
||||
[debug] yt-dlp version 2022.09.01 [9d339c4] (win32_exe)
|
||||
[debug] yt-dlp version 2023.03.04 [9d339c4] (win32_exe)
|
||||
[debug] Python 3.8.10 (CPython 64bit) - Windows-10-10.0.22000-SP0
|
||||
[debug] Checking exe version: ffmpeg -bsfs
|
||||
[debug] Checking exe version: ffprobe -bsfs
|
||||
@ -82,8 +84,8 @@ body:
|
||||
[debug] Optional libraries: Cryptodome-3.15.0, brotli-1.0.9, certifi-2022.06.15, mutagen-1.45.1, sqlite3-2.6.0, websockets-10.3
|
||||
[debug] Proxy map: {}
|
||||
[debug] Fetching release info: https://api.github.com/repos/yt-dlp/yt-dlp/releases/latest
|
||||
Latest version: 2022.09.01, Current version: 2022.09.01
|
||||
yt-dlp is up to date (2022.09.01)
|
||||
Latest version: 2023.03.04, Current version: 2023.03.04
|
||||
yt-dlp is up to date (2023.03.04)
|
||||
<more lines>
|
||||
render: shell
|
||||
validations:
|
||||
|
@ -7,7 +7,7 @@ body:
|
||||
label: DO NOT REMOVE OR SKIP THE ISSUE TEMPLATE
|
||||
description: Fill all fields even if you think it is irrelevant for the issue
|
||||
options:
|
||||
- label: I understand that I will be **blocked** if I remove or skip any mandatory\* field
|
||||
- label: I understand that I will be **blocked** if I *intentionally* remove or skip any mandatory\* field
|
||||
required: true
|
||||
- type: checkboxes
|
||||
id: checklist
|
||||
@ -18,11 +18,11 @@ body:
|
||||
options:
|
||||
- label: I'm requesting a site-specific feature
|
||||
required: true
|
||||
- label: I've verified that I'm running yt-dlp version **2022.09.01** ([update instructions](https://github.com/yt-dlp/yt-dlp#update)) or later (specify commit)
|
||||
- label: I've verified that I'm running yt-dlp version **2023.03.04** ([update instructions](https://github.com/yt-dlp/yt-dlp#update)) or later (specify commit)
|
||||
required: true
|
||||
- label: I've checked that all provided URLs are playable in a browser with the same IP and same login details
|
||||
required: true
|
||||
- label: I've searched the [bugtracker](https://github.com/yt-dlp/yt-dlp/issues?q=) for similar issues **including closed ones**. DO NOT post duplicates
|
||||
- label: I've searched [known issues](https://github.com/yt-dlp/yt-dlp/issues/3766) and the [bugtracker](https://github.com/yt-dlp/yt-dlp/issues?q=) for similar issues **including closed ones**. DO NOT post duplicates
|
||||
required: true
|
||||
- label: I've read the [guidelines for opening an issue](https://github.com/yt-dlp/yt-dlp/blob/master/CONTRIBUTING.md#opening-an-issue)
|
||||
required: true
|
||||
@ -58,6 +58,8 @@ body:
|
||||
options:
|
||||
- label: Run **your** yt-dlp command with **-vU** flag added (`yt-dlp -vU <your command line>`)
|
||||
required: true
|
||||
- label: "If using API, add `'verbose': True` to `YoutubeDL` params instead"
|
||||
required: false
|
||||
- label: Copy the WHOLE output (starting with `[debug] Command-line config`) and insert it below
|
||||
required: true
|
||||
- type: textarea
|
||||
@ -70,7 +72,7 @@ body:
|
||||
[debug] Command-line config: ['-vU', 'test:youtube']
|
||||
[debug] Portable config "yt-dlp.conf": ['-i']
|
||||
[debug] Encodings: locale cp65001, fs utf-8, pref cp65001, out utf-8, error utf-8, screen utf-8
|
||||
[debug] yt-dlp version 2022.09.01 [9d339c4] (win32_exe)
|
||||
[debug] yt-dlp version 2023.03.04 [9d339c4] (win32_exe)
|
||||
[debug] Python 3.8.10 (CPython 64bit) - Windows-10-10.0.22000-SP0
|
||||
[debug] Checking exe version: ffmpeg -bsfs
|
||||
[debug] Checking exe version: ffprobe -bsfs
|
||||
@ -78,8 +80,8 @@ body:
|
||||
[debug] Optional libraries: Cryptodome-3.15.0, brotli-1.0.9, certifi-2022.06.15, mutagen-1.45.1, sqlite3-2.6.0, websockets-10.3
|
||||
[debug] Proxy map: {}
|
||||
[debug] Fetching release info: https://api.github.com/repos/yt-dlp/yt-dlp/releases/latest
|
||||
Latest version: 2022.09.01, Current version: 2022.09.01
|
||||
yt-dlp is up to date (2022.09.01)
|
||||
Latest version: 2023.03.04, Current version: 2023.03.04
|
||||
yt-dlp is up to date (2023.03.04)
|
||||
<more lines>
|
||||
render: shell
|
||||
validations:
|
||||
|
14
.github/ISSUE_TEMPLATE/4_bug_report.yml
vendored
14
.github/ISSUE_TEMPLATE/4_bug_report.yml
vendored
@ -7,7 +7,7 @@ body:
|
||||
label: DO NOT REMOVE OR SKIP THE ISSUE TEMPLATE
|
||||
description: Fill all fields even if you think it is irrelevant for the issue
|
||||
options:
|
||||
- label: I understand that I will be **blocked** if I remove or skip any mandatory\* field
|
||||
- label: I understand that I will be **blocked** if I *intentionally* remove or skip any mandatory\* field
|
||||
required: true
|
||||
- type: checkboxes
|
||||
id: checklist
|
||||
@ -18,13 +18,13 @@ body:
|
||||
options:
|
||||
- label: I'm reporting a bug unrelated to a specific site
|
||||
required: true
|
||||
- label: I've verified that I'm running yt-dlp version **2022.09.01** ([update instructions](https://github.com/yt-dlp/yt-dlp#update)) or later (specify commit)
|
||||
- label: I've verified that I'm running yt-dlp version **2023.03.04** ([update instructions](https://github.com/yt-dlp/yt-dlp#update)) or later (specify commit)
|
||||
required: true
|
||||
- label: I've checked that all provided URLs are playable in a browser with the same IP and same login details
|
||||
required: true
|
||||
- label: I've checked that all URLs and arguments with special characters are [properly quoted or escaped](https://github.com/yt-dlp/yt-dlp/wiki/FAQ#video-url-contains-an-ampersand--and-im-getting-some-strange-output-1-2839-or-v-is-not-recognized-as-an-internal-or-external-command)
|
||||
required: true
|
||||
- label: I've searched the [bugtracker](https://github.com/yt-dlp/yt-dlp/issues?q=) for similar issues **including closed ones**. DO NOT post duplicates
|
||||
- label: I've searched [known issues](https://github.com/yt-dlp/yt-dlp/issues/3766) and the [bugtracker](https://github.com/yt-dlp/yt-dlp/issues?q=) for similar issues **including closed ones**. DO NOT post duplicates
|
||||
required: true
|
||||
- label: I've read the [guidelines for opening an issue](https://github.com/yt-dlp/yt-dlp/blob/master/CONTRIBUTING.md#opening-an-issue)
|
||||
required: true
|
||||
@ -43,6 +43,8 @@ body:
|
||||
options:
|
||||
- label: Run **your** yt-dlp command with **-vU** flag added (`yt-dlp -vU <your command line>`)
|
||||
required: true
|
||||
- label: "If using API, add `'verbose': True` to `YoutubeDL` params instead"
|
||||
required: false
|
||||
- label: Copy the WHOLE output (starting with `[debug] Command-line config`) and insert it below
|
||||
required: true
|
||||
- type: textarea
|
||||
@ -55,7 +57,7 @@ body:
|
||||
[debug] Command-line config: ['-vU', 'test:youtube']
|
||||
[debug] Portable config "yt-dlp.conf": ['-i']
|
||||
[debug] Encodings: locale cp65001, fs utf-8, pref cp65001, out utf-8, error utf-8, screen utf-8
|
||||
[debug] yt-dlp version 2022.09.01 [9d339c4] (win32_exe)
|
||||
[debug] yt-dlp version 2023.03.04 [9d339c4] (win32_exe)
|
||||
[debug] Python 3.8.10 (CPython 64bit) - Windows-10-10.0.22000-SP0
|
||||
[debug] Checking exe version: ffmpeg -bsfs
|
||||
[debug] Checking exe version: ffprobe -bsfs
|
||||
@ -63,8 +65,8 @@ body:
|
||||
[debug] Optional libraries: Cryptodome-3.15.0, brotli-1.0.9, certifi-2022.06.15, mutagen-1.45.1, sqlite3-2.6.0, websockets-10.3
|
||||
[debug] Proxy map: {}
|
||||
[debug] Fetching release info: https://api.github.com/repos/yt-dlp/yt-dlp/releases/latest
|
||||
Latest version: 2022.09.01, Current version: 2022.09.01
|
||||
yt-dlp is up to date (2022.09.01)
|
||||
Latest version: 2023.03.04, Current version: 2023.03.04
|
||||
yt-dlp is up to date (2023.03.04)
|
||||
<more lines>
|
||||
render: shell
|
||||
validations:
|
||||
|
14
.github/ISSUE_TEMPLATE/5_feature_request.yml
vendored
14
.github/ISSUE_TEMPLATE/5_feature_request.yml
vendored
@ -7,7 +7,7 @@ body:
|
||||
label: DO NOT REMOVE OR SKIP THE ISSUE TEMPLATE
|
||||
description: Fill all fields even if you think it is irrelevant for the issue
|
||||
options:
|
||||
- label: I understand that I will be **blocked** if I remove or skip any mandatory\* field
|
||||
- label: I understand that I will be **blocked** if I *intentionally* remove or skip any mandatory\* field
|
||||
required: true
|
||||
- type: checkboxes
|
||||
id: checklist
|
||||
@ -20,9 +20,9 @@ body:
|
||||
required: true
|
||||
- label: I've looked through the [README](https://github.com/yt-dlp/yt-dlp#readme)
|
||||
required: true
|
||||
- label: I've verified that I'm running yt-dlp version **2022.09.01** ([update instructions](https://github.com/yt-dlp/yt-dlp#update)) or later (specify commit)
|
||||
- label: I've verified that I'm running yt-dlp version **2023.03.04** ([update instructions](https://github.com/yt-dlp/yt-dlp#update)) or later (specify commit)
|
||||
required: true
|
||||
- label: I've searched the [bugtracker](https://github.com/yt-dlp/yt-dlp/issues?q=) for similar issues **including closed ones**. DO NOT post duplicates
|
||||
- label: I've searched [known issues](https://github.com/yt-dlp/yt-dlp/issues/3766) and the [bugtracker](https://github.com/yt-dlp/yt-dlp/issues?q=) for similar issues **including closed ones**. DO NOT post duplicates
|
||||
required: true
|
||||
- label: I've read the [guidelines for opening an issue](https://github.com/yt-dlp/yt-dlp/blob/master/CONTRIBUTING.md#opening-an-issue)
|
||||
required: true
|
||||
@ -40,6 +40,8 @@ body:
|
||||
label: Provide verbose output that clearly demonstrates the problem
|
||||
options:
|
||||
- label: Run **your** yt-dlp command with **-vU** flag added (`yt-dlp -vU <your command line>`)
|
||||
- label: "If using API, add `'verbose': True` to `YoutubeDL` params instead"
|
||||
required: false
|
||||
- label: Copy the WHOLE output (starting with `[debug] Command-line config`) and insert it below
|
||||
- type: textarea
|
||||
id: log
|
||||
@ -51,7 +53,7 @@ body:
|
||||
[debug] Command-line config: ['-vU', 'test:youtube']
|
||||
[debug] Portable config "yt-dlp.conf": ['-i']
|
||||
[debug] Encodings: locale cp65001, fs utf-8, pref cp65001, out utf-8, error utf-8, screen utf-8
|
||||
[debug] yt-dlp version 2022.09.01 [9d339c4] (win32_exe)
|
||||
[debug] yt-dlp version 2023.03.04 [9d339c4] (win32_exe)
|
||||
[debug] Python 3.8.10 (CPython 64bit) - Windows-10-10.0.22000-SP0
|
||||
[debug] Checking exe version: ffmpeg -bsfs
|
||||
[debug] Checking exe version: ffprobe -bsfs
|
||||
@ -59,7 +61,7 @@ body:
|
||||
[debug] Optional libraries: Cryptodome-3.15.0, brotli-1.0.9, certifi-2022.06.15, mutagen-1.45.1, sqlite3-2.6.0, websockets-10.3
|
||||
[debug] Proxy map: {}
|
||||
[debug] Fetching release info: https://api.github.com/repos/yt-dlp/yt-dlp/releases/latest
|
||||
Latest version: 2022.09.01, Current version: 2022.09.01
|
||||
yt-dlp is up to date (2022.09.01)
|
||||
Latest version: 2023.03.04, Current version: 2023.03.04
|
||||
yt-dlp is up to date (2023.03.04)
|
||||
<more lines>
|
||||
render: shell
|
||||
|
14
.github/ISSUE_TEMPLATE/6_question.yml
vendored
14
.github/ISSUE_TEMPLATE/6_question.yml
vendored
@ -7,7 +7,7 @@ body:
|
||||
label: DO NOT REMOVE OR SKIP THE ISSUE TEMPLATE
|
||||
description: Fill all fields even if you think it is irrelevant for the issue
|
||||
options:
|
||||
- label: I understand that I will be **blocked** if I remove or skip any mandatory\* field
|
||||
- label: I understand that I will be **blocked** if I *intentionally* remove or skip any mandatory\* field
|
||||
required: true
|
||||
- type: markdown
|
||||
attributes:
|
||||
@ -26,9 +26,9 @@ body:
|
||||
required: true
|
||||
- label: I've looked through the [README](https://github.com/yt-dlp/yt-dlp#readme)
|
||||
required: true
|
||||
- label: I've verified that I'm running yt-dlp version **2022.09.01** ([update instructions](https://github.com/yt-dlp/yt-dlp#update)) or later (specify commit)
|
||||
- label: I've verified that I'm running yt-dlp version **2023.03.04** ([update instructions](https://github.com/yt-dlp/yt-dlp#update)) or later (specify commit)
|
||||
required: true
|
||||
- label: I've searched the [bugtracker](https://github.com/yt-dlp/yt-dlp/issues?q=) for similar questions **including closed ones**. DO NOT post duplicates
|
||||
- label: I've searched [known issues](https://github.com/yt-dlp/yt-dlp/issues/3766) and the [bugtracker](https://github.com/yt-dlp/yt-dlp/issues?q=) for similar questions **including closed ones**. DO NOT post duplicates
|
||||
required: true
|
||||
- label: I've read the [guidelines for opening an issue](https://github.com/yt-dlp/yt-dlp/blob/master/CONTRIBUTING.md#opening-an-issue)
|
||||
required: true
|
||||
@ -46,6 +46,8 @@ body:
|
||||
label: Provide verbose output that clearly demonstrates the problem
|
||||
options:
|
||||
- label: Run **your** yt-dlp command with **-vU** flag added (`yt-dlp -vU <your command line>`)
|
||||
- label: "If using API, add `'verbose': True` to `YoutubeDL` params instead"
|
||||
required: false
|
||||
- label: Copy the WHOLE output (starting with `[debug] Command-line config`) and insert it below
|
||||
- type: textarea
|
||||
id: log
|
||||
@ -57,7 +59,7 @@ body:
|
||||
[debug] Command-line config: ['-vU', 'test:youtube']
|
||||
[debug] Portable config "yt-dlp.conf": ['-i']
|
||||
[debug] Encodings: locale cp65001, fs utf-8, pref cp65001, out utf-8, error utf-8, screen utf-8
|
||||
[debug] yt-dlp version 2022.09.01 [9d339c4] (win32_exe)
|
||||
[debug] yt-dlp version 2023.03.04 [9d339c4] (win32_exe)
|
||||
[debug] Python 3.8.10 (CPython 64bit) - Windows-10-10.0.22000-SP0
|
||||
[debug] Checking exe version: ffmpeg -bsfs
|
||||
[debug] Checking exe version: ffprobe -bsfs
|
||||
@ -65,7 +67,7 @@ body:
|
||||
[debug] Optional libraries: Cryptodome-3.15.0, brotli-1.0.9, certifi-2022.06.15, mutagen-1.45.1, sqlite3-2.6.0, websockets-10.3
|
||||
[debug] Proxy map: {}
|
||||
[debug] Fetching release info: https://api.github.com/repos/yt-dlp/yt-dlp/releases/latest
|
||||
Latest version: 2022.09.01, Current version: 2022.09.01
|
||||
yt-dlp is up to date (2022.09.01)
|
||||
Latest version: 2023.03.04, Current version: 2023.03.04
|
||||
yt-dlp is up to date (2023.03.04)
|
||||
<more lines>
|
||||
render: shell
|
||||
|
@ -1,5 +1,5 @@
|
||||
name: Broken site
|
||||
description: Report broken or misfunctioning site
|
||||
description: Report error in a supported site
|
||||
labels: [triage, site-bug]
|
||||
body:
|
||||
%(no_skip)s
|
||||
@ -10,7 +10,7 @@ body:
|
||||
description: |
|
||||
Carefully read and work through this check list in order to prevent the most common mistakes and misuse of yt-dlp:
|
||||
options:
|
||||
- label: I'm reporting a broken site
|
||||
- label: I'm reporting that a **supported** site is broken
|
||||
required: true
|
||||
- label: I've verified that I'm running yt-dlp version **%(version)s** ([update instructions](https://github.com/yt-dlp/yt-dlp#update)) or later (specify commit)
|
||||
required: true
|
||||
@ -18,7 +18,7 @@ body:
|
||||
required: true
|
||||
- label: I've checked that all URLs and arguments with special characters are [properly quoted or escaped](https://github.com/yt-dlp/yt-dlp/wiki/FAQ#video-url-contains-an-ampersand--and-im-getting-some-strange-output-1-2839-or-v-is-not-recognized-as-an-internal-or-external-command)
|
||||
required: true
|
||||
- label: I've searched the [bugtracker](https://github.com/yt-dlp/yt-dlp/issues?q=) for similar issues **including closed ones**. DO NOT post duplicates
|
||||
- label: I've searched [known issues](https://github.com/yt-dlp/yt-dlp/issues/3766) and the [bugtracker](https://github.com/yt-dlp/yt-dlp/issues?q=) for similar issues **including closed ones**. DO NOT post duplicates
|
||||
required: true
|
||||
- label: I've read the [guidelines for opening an issue](https://github.com/yt-dlp/yt-dlp/blob/master/CONTRIBUTING.md#opening-an-issue)
|
||||
required: true
|
||||
|
@ -18,7 +18,7 @@ body:
|
||||
required: true
|
||||
- label: I've checked that none of provided URLs [violate any copyrights](https://github.com/yt-dlp/yt-dlp/blob/master/CONTRIBUTING.md#is-the-website-primarily-used-for-piracy) or contain any [DRM](https://en.wikipedia.org/wiki/Digital_rights_management) to the best of my knowledge
|
||||
required: true
|
||||
- label: I've searched the [bugtracker](https://github.com/yt-dlp/yt-dlp/issues?q=) for similar issues **including closed ones**. DO NOT post duplicates
|
||||
- label: I've searched [known issues](https://github.com/yt-dlp/yt-dlp/issues/3766) and the [bugtracker](https://github.com/yt-dlp/yt-dlp/issues?q=) for similar issues **including closed ones**. DO NOT post duplicates
|
||||
required: true
|
||||
- label: I've read the [guidelines for opening an issue](https://github.com/yt-dlp/yt-dlp/blob/master/CONTRIBUTING.md#opening-an-issue)
|
||||
required: true
|
||||
|
@ -16,7 +16,7 @@ body:
|
||||
required: true
|
||||
- label: I've checked that all provided URLs are playable in a browser with the same IP and same login details
|
||||
required: true
|
||||
- label: I've searched the [bugtracker](https://github.com/yt-dlp/yt-dlp/issues?q=) for similar issues **including closed ones**. DO NOT post duplicates
|
||||
- label: I've searched [known issues](https://github.com/yt-dlp/yt-dlp/issues/3766) and the [bugtracker](https://github.com/yt-dlp/yt-dlp/issues?q=) for similar issues **including closed ones**. DO NOT post duplicates
|
||||
required: true
|
||||
- label: I've read the [guidelines for opening an issue](https://github.com/yt-dlp/yt-dlp/blob/master/CONTRIBUTING.md#opening-an-issue)
|
||||
required: true
|
||||
|
2
.github/ISSUE_TEMPLATE_tmpl/4_bug_report.yml
vendored
2
.github/ISSUE_TEMPLATE_tmpl/4_bug_report.yml
vendored
@ -18,7 +18,7 @@ body:
|
||||
required: true
|
||||
- label: I've checked that all URLs and arguments with special characters are [properly quoted or escaped](https://github.com/yt-dlp/yt-dlp/wiki/FAQ#video-url-contains-an-ampersand--and-im-getting-some-strange-output-1-2839-or-v-is-not-recognized-as-an-internal-or-external-command)
|
||||
required: true
|
||||
- label: I've searched the [bugtracker](https://github.com/yt-dlp/yt-dlp/issues?q=) for similar issues **including closed ones**. DO NOT post duplicates
|
||||
- label: I've searched [known issues](https://github.com/yt-dlp/yt-dlp/issues/3766) and the [bugtracker](https://github.com/yt-dlp/yt-dlp/issues?q=) for similar issues **including closed ones**. DO NOT post duplicates
|
||||
required: true
|
||||
- label: I've read the [guidelines for opening an issue](https://github.com/yt-dlp/yt-dlp/blob/master/CONTRIBUTING.md#opening-an-issue)
|
||||
required: true
|
||||
|
@ -16,7 +16,7 @@ body:
|
||||
required: true
|
||||
- label: I've verified that I'm running yt-dlp version **%(version)s** ([update instructions](https://github.com/yt-dlp/yt-dlp#update)) or later (specify commit)
|
||||
required: true
|
||||
- label: I've searched the [bugtracker](https://github.com/yt-dlp/yt-dlp/issues?q=) for similar issues **including closed ones**. DO NOT post duplicates
|
||||
- label: I've searched [known issues](https://github.com/yt-dlp/yt-dlp/issues/3766) and the [bugtracker](https://github.com/yt-dlp/yt-dlp/issues?q=) for similar issues **including closed ones**. DO NOT post duplicates
|
||||
required: true
|
||||
- label: I've read the [guidelines for opening an issue](https://github.com/yt-dlp/yt-dlp/blob/master/CONTRIBUTING.md#opening-an-issue)
|
||||
required: true
|
||||
|
2
.github/ISSUE_TEMPLATE_tmpl/6_question.yml
vendored
2
.github/ISSUE_TEMPLATE_tmpl/6_question.yml
vendored
@ -22,7 +22,7 @@ body:
|
||||
required: true
|
||||
- label: I've verified that I'm running yt-dlp version **%(version)s** ([update instructions](https://github.com/yt-dlp/yt-dlp#update)) or later (specify commit)
|
||||
required: true
|
||||
- label: I've searched the [bugtracker](https://github.com/yt-dlp/yt-dlp/issues?q=) for similar questions **including closed ones**. DO NOT post duplicates
|
||||
- label: I've searched [known issues](https://github.com/yt-dlp/yt-dlp/issues/3766) and the [bugtracker](https://github.com/yt-dlp/yt-dlp/issues?q=) for similar questions **including closed ones**. DO NOT post duplicates
|
||||
required: true
|
||||
- label: I've read the [guidelines for opening an issue](https://github.com/yt-dlp/yt-dlp/blob/master/CONTRIBUTING.md#opening-an-issue)
|
||||
required: true
|
||||
|
6
.github/PULL_REQUEST_TEMPLATE.md
vendored
6
.github/PULL_REQUEST_TEMPLATE.md
vendored
@ -2,8 +2,6 @@
|
||||
|
||||
### Description of your *pull request* and other information
|
||||
|
||||
</details>
|
||||
|
||||
<!--
|
||||
|
||||
Explanation of your *pull request* in arbitrary form goes here. Please **make sure the description explains the purpose and effect** of your *pull request* and is worded well enough to be understood. Provide as much **context and examples** as possible
|
||||
@ -32,7 +30,7 @@ ### Before submitting a *pull request* make sure you have:
|
||||
- [ ] [Searched](https://github.com/yt-dlp/yt-dlp/search?q=is%3Apr&type=Issues) the bugtracker for similar pull requests
|
||||
- [ ] Checked the code with [flake8](https://pypi.python.org/pypi/flake8) and [ran relevant tests](https://github.com/yt-dlp/yt-dlp/blob/master/CONTRIBUTING.md#developer-instructions)
|
||||
|
||||
### In order to be accepted and merged into yt-dlp each piece of code must be in public domain or released under [Unlicense](http://unlicense.org/). Check one of the following options:
|
||||
### In order to be accepted and merged into yt-dlp each piece of code must be in public domain or released under [Unlicense](http://unlicense.org/). Check all of the following options that apply:
|
||||
- [ ] I am the original author of this code and I am willing to release it under [Unlicense](http://unlicense.org/)
|
||||
- [ ] I am not the original author of this code but it is in public domain or released under [Unlicense](http://unlicense.org/) (provide reliable evidence)
|
||||
|
||||
@ -41,3 +39,5 @@ ### What is the purpose of your *pull request*?
|
||||
- [ ] New extractor ([Piracy websites will not be accepted](https://github.com/yt-dlp/yt-dlp/blob/master/CONTRIBUTING.md#is-the-website-primarily-used-for-piracy))
|
||||
- [ ] Core bug fix/improvement
|
||||
- [ ] New feature (It is strongly [recommended to open an issue first](https://github.com/yt-dlp/yt-dlp/blob/master/CONTRIBUTING.md#adding-new-feature-or-making-overarching-changes))
|
||||
|
||||
</details>
|
||||
|
548
.github/workflows/build.yml
vendored
548
.github/workflows/build.yml
vendored
@ -1,324 +1,356 @@
|
||||
name: Build
|
||||
on: workflow_dispatch
|
||||
name: Build Artifacts
|
||||
on:
|
||||
workflow_call:
|
||||
inputs:
|
||||
version:
|
||||
required: true
|
||||
type: string
|
||||
channel:
|
||||
required: false
|
||||
default: stable
|
||||
type: string
|
||||
unix:
|
||||
default: true
|
||||
type: boolean
|
||||
linux_arm:
|
||||
default: true
|
||||
type: boolean
|
||||
macos:
|
||||
default: true
|
||||
type: boolean
|
||||
macos_legacy:
|
||||
default: true
|
||||
type: boolean
|
||||
windows:
|
||||
default: true
|
||||
type: boolean
|
||||
windows32:
|
||||
default: true
|
||||
type: boolean
|
||||
meta_files:
|
||||
default: true
|
||||
type: boolean
|
||||
secrets:
|
||||
GPG_SIGNING_KEY:
|
||||
required: false
|
||||
|
||||
workflow_dispatch:
|
||||
inputs:
|
||||
version:
|
||||
description: Version tag (YYYY.MM.DD[.REV])
|
||||
required: true
|
||||
type: string
|
||||
channel:
|
||||
description: Update channel (stable/nightly)
|
||||
required: true
|
||||
default: stable
|
||||
type: string
|
||||
unix:
|
||||
description: yt-dlp, yt-dlp.tar.gz, yt-dlp_linux, yt-dlp_linux.zip
|
||||
default: true
|
||||
type: boolean
|
||||
linux_arm:
|
||||
description: yt-dlp_linux_aarch64, yt-dlp_linux_armv7l
|
||||
default: true
|
||||
type: boolean
|
||||
macos:
|
||||
description: yt-dlp_macos, yt-dlp_macos.zip
|
||||
default: true
|
||||
type: boolean
|
||||
macos_legacy:
|
||||
description: yt-dlp_macos_legacy
|
||||
default: true
|
||||
type: boolean
|
||||
windows:
|
||||
description: yt-dlp.exe, yt-dlp_min.exe, yt-dlp_win.zip
|
||||
default: true
|
||||
type: boolean
|
||||
windows32:
|
||||
description: yt-dlp_x86.exe
|
||||
default: true
|
||||
type: boolean
|
||||
meta_files:
|
||||
description: SHA2-256SUMS, SHA2-512SUMS, _update_spec
|
||||
default: true
|
||||
type: boolean
|
||||
|
||||
permissions:
|
||||
contents: read
|
||||
|
||||
jobs:
|
||||
prepare:
|
||||
unix:
|
||||
if: inputs.unix
|
||||
runs-on: ubuntu-latest
|
||||
outputs:
|
||||
version_suffix: ${{ steps.version_suffix.outputs.version_suffix }}
|
||||
ytdlp_version: ${{ steps.bump_version.outputs.ytdlp_version }}
|
||||
head_sha: ${{ steps.push_release.outputs.head_sha }}
|
||||
steps:
|
||||
- uses: actions/checkout@v3
|
||||
with:
|
||||
fetch-depth: 0
|
||||
- uses: actions/setup-python@v4
|
||||
with:
|
||||
python-version: '3.10'
|
||||
- uses: actions/checkout@v3
|
||||
- uses: actions/setup-python@v4
|
||||
with:
|
||||
python-version: "3.10"
|
||||
- uses: conda-incubator/setup-miniconda@v2
|
||||
with:
|
||||
miniforge-variant: Mambaforge
|
||||
use-mamba: true
|
||||
channels: conda-forge
|
||||
auto-update-conda: true
|
||||
activate-environment: ""
|
||||
auto-activate-base: false
|
||||
- name: Install Requirements
|
||||
run: |
|
||||
sudo apt-get -y install zip pandoc man sed
|
||||
python -m pip install -U pip setuptools wheel
|
||||
python -m pip install -U Pyinstaller -r requirements.txt
|
||||
reqs=$(mktemp)
|
||||
cat > $reqs << EOF
|
||||
python=3.10.*
|
||||
pyinstaller
|
||||
cffi
|
||||
brotli-python
|
||||
EOF
|
||||
sed '/^brotli.*/d' requirements.txt >> $reqs
|
||||
mamba create -n build --file $reqs
|
||||
|
||||
- name: Set version suffix
|
||||
id: version_suffix
|
||||
env:
|
||||
PUSH_VERSION_COMMIT: ${{ secrets.PUSH_VERSION_COMMIT }}
|
||||
if: "env.PUSH_VERSION_COMMIT == ''"
|
||||
run: echo ::set-output name=version_suffix::$(date -u +"%H%M%S")
|
||||
- name: Bump version
|
||||
id: bump_version
|
||||
run: |
|
||||
python devscripts/update-version.py ${{ steps.version_suffix.outputs.version_suffix }}
|
||||
make issuetemplates
|
||||
|
||||
- name: Push to release
|
||||
id: push_release
|
||||
run: |
|
||||
git config --global user.name github-actions
|
||||
git config --global user.email github-actions@example.com
|
||||
git add -u
|
||||
git commit -m "[version] update" -m "Created by: ${{ github.event.sender.login }}" -m ":ci skip all :ci run dl"
|
||||
git push origin --force ${{ github.event.ref }}:release
|
||||
echo ::set-output name=head_sha::$(git rev-parse HEAD)
|
||||
- name: Update master
|
||||
env:
|
||||
PUSH_VERSION_COMMIT: ${{ secrets.PUSH_VERSION_COMMIT }}
|
||||
if: "env.PUSH_VERSION_COMMIT != ''"
|
||||
run: git push origin ${{ github.event.ref }}
|
||||
|
||||
|
||||
build_unix:
|
||||
needs: prepare
|
||||
runs-on: ubuntu-18.04 # Standalone executable should be built on minimum supported OS
|
||||
|
||||
steps:
|
||||
- uses: actions/checkout@v3
|
||||
- uses: actions/setup-python@v4
|
||||
with:
|
||||
python-version: '3.10'
|
||||
- name: Install Requirements
|
||||
run: |
|
||||
sudo apt-get -y install zip pandoc man
|
||||
python -m pip install --upgrade pip setuptools wheel twine
|
||||
python -m pip install Pyinstaller -r requirements.txt
|
||||
|
||||
- name: Prepare
|
||||
run: |
|
||||
python devscripts/update-version.py ${{ needs.prepare.outputs.version_suffix }}
|
||||
- name: Prepare
|
||||
run: |
|
||||
python devscripts/update-version.py -c ${{ inputs.channel }} ${{ inputs.version }}
|
||||
python devscripts/make_lazy_extractors.py
|
||||
- name: Build Unix executables
|
||||
run: |
|
||||
- name: Build Unix platform-independent binary
|
||||
run: |
|
||||
make all tar
|
||||
- name: Build Unix standalone binary
|
||||
shell: bash -l {0}
|
||||
run: |
|
||||
unset LD_LIBRARY_PATH # Harmful; set by setup-python
|
||||
conda activate build
|
||||
python pyinst.py --onedir
|
||||
(cd ./dist/yt-dlp_linux && zip -r ../yt-dlp_linux.zip .)
|
||||
python pyinst.py
|
||||
- name: Get SHA2-SUMS
|
||||
id: get_sha
|
||||
run: |
|
||||
mv ./dist/yt-dlp_linux ./yt-dlp_linux
|
||||
mv ./dist/yt-dlp_linux.zip ./yt-dlp_linux.zip
|
||||
|
||||
- name: Upload artifacts
|
||||
uses: actions/upload-artifact@v3
|
||||
with:
|
||||
path: |
|
||||
yt-dlp
|
||||
yt-dlp.tar.gz
|
||||
dist/yt-dlp_linux
|
||||
dist/yt-dlp_linux.zip
|
||||
- name: Upload artifacts
|
||||
uses: actions/upload-artifact@v3
|
||||
with:
|
||||
path: |
|
||||
yt-dlp
|
||||
yt-dlp.tar.gz
|
||||
yt-dlp_linux
|
||||
yt-dlp_linux.zip
|
||||
|
||||
- name: Build and publish on PyPi
|
||||
env:
|
||||
TWINE_USERNAME: __token__
|
||||
TWINE_PASSWORD: ${{ secrets.PYPI_TOKEN }}
|
||||
if: "env.TWINE_PASSWORD != ''"
|
||||
run: |
|
||||
rm -rf dist/*
|
||||
python devscripts/set-variant.py pip -M "You installed yt-dlp with pip or using the wheel from PyPi; Use that to update"
|
||||
python setup.py sdist bdist_wheel
|
||||
twine upload dist/*
|
||||
|
||||
- name: Install SSH private key for Homebrew
|
||||
env:
|
||||
BREW_TOKEN: ${{ secrets.BREW_TOKEN }}
|
||||
if: "env.BREW_TOKEN != ''"
|
||||
uses: yt-dlp/ssh-agent@v0.5.3
|
||||
with:
|
||||
ssh-private-key: ${{ env.BREW_TOKEN }}
|
||||
- name: Update Homebrew Formulae
|
||||
env:
|
||||
BREW_TOKEN: ${{ secrets.BREW_TOKEN }}
|
||||
if: "env.BREW_TOKEN != ''"
|
||||
run: |
|
||||
git clone git@github.com:yt-dlp/homebrew-taps taps/
|
||||
python devscripts/update-formulae.py taps/Formula/yt-dlp.rb "${{ needs.prepare.outputs.ytdlp_version }}"
|
||||
git -C taps/ config user.name github-actions
|
||||
git -C taps/ config user.email github-actions@example.com
|
||||
git -C taps/ commit -am 'yt-dlp: ${{ needs.prepare.outputs.ytdlp_version }}'
|
||||
git -C taps/ push
|
||||
|
||||
|
||||
build_macos:
|
||||
runs-on: macos-11
|
||||
needs: prepare
|
||||
linux_arm:
|
||||
if: inputs.linux_arm
|
||||
permissions:
|
||||
contents: read
|
||||
packages: write # for creating cache
|
||||
runs-on: ubuntu-latest
|
||||
strategy:
|
||||
matrix:
|
||||
architecture:
|
||||
- armv7
|
||||
- aarch64
|
||||
|
||||
steps:
|
||||
- uses: actions/checkout@v3
|
||||
# NB: In order to create a universal2 application, the version of python3 in /usr/bin has to be used
|
||||
- name: Install Requirements
|
||||
run: |
|
||||
- uses: actions/checkout@v3
|
||||
with:
|
||||
path: ./repo
|
||||
- name: Virtualized Install, Prepare & Build
|
||||
uses: yt-dlp/run-on-arch-action@v2
|
||||
with:
|
||||
# Ref: https://github.com/uraimo/run-on-arch-action/issues/55
|
||||
env: |
|
||||
GITHUB_WORKFLOW: build
|
||||
githubToken: ${{ github.token }} # To cache image
|
||||
arch: ${{ matrix.architecture }}
|
||||
distro: ubuntu18.04 # Standalone executable should be built on minimum supported OS
|
||||
dockerRunArgs: --volume "${PWD}/repo:/repo"
|
||||
install: | # Installing Python 3.10 from the Deadsnakes repo raises errors
|
||||
apt update
|
||||
apt -y install zlib1g-dev python3.8 python3.8-dev python3.8-distutils python3-pip
|
||||
python3.8 -m pip install -U pip setuptools wheel
|
||||
# Cannot access requirements.txt from the repo directory at this stage
|
||||
python3.8 -m pip install -U Pyinstaller mutagen pycryptodomex websockets brotli certifi
|
||||
|
||||
run: |
|
||||
cd repo
|
||||
python3.8 -m pip install -U Pyinstaller -r requirements.txt # Cached version may be out of date
|
||||
python3.8 devscripts/update-version.py -c ${{ inputs.channel }} ${{ inputs.version }}
|
||||
python3.8 devscripts/make_lazy_extractors.py
|
||||
python3.8 pyinst.py
|
||||
|
||||
- name: Upload artifacts
|
||||
uses: actions/upload-artifact@v3
|
||||
with:
|
||||
path: | # run-on-arch-action designates armv7l as armv7
|
||||
repo/dist/yt-dlp_linux_${{ (matrix.architecture == 'armv7' && 'armv7l') || matrix.architecture }}
|
||||
|
||||
macos:
|
||||
if: inputs.macos
|
||||
runs-on: macos-11
|
||||
|
||||
steps:
|
||||
- uses: actions/checkout@v3
|
||||
# NB: In order to create a universal2 application, the version of python3 in /usr/bin has to be used
|
||||
- name: Install Requirements
|
||||
run: |
|
||||
brew install coreutils
|
||||
/usr/bin/python3 -m pip install -U --user pip Pyinstaller -r requirements.txt
|
||||
|
||||
- name: Prepare
|
||||
run: |
|
||||
/usr/bin/python3 devscripts/update-version.py ${{ needs.prepare.outputs.version_suffix }}
|
||||
- name: Prepare
|
||||
run: |
|
||||
/usr/bin/python3 devscripts/update-version.py -c ${{ inputs.channel }} ${{ inputs.version }}
|
||||
/usr/bin/python3 devscripts/make_lazy_extractors.py
|
||||
- name: Build
|
||||
run: |
|
||||
- name: Build
|
||||
run: |
|
||||
/usr/bin/python3 pyinst.py --target-architecture universal2 --onedir
|
||||
(cd ./dist/yt-dlp_macos && zip -r ../yt-dlp_macos.zip .)
|
||||
/usr/bin/python3 pyinst.py --target-architecture universal2
|
||||
|
||||
- name: Upload artifacts
|
||||
uses: actions/upload-artifact@v3
|
||||
with:
|
||||
path: |
|
||||
dist/yt-dlp_macos
|
||||
dist/yt-dlp_macos.zip
|
||||
- name: Upload artifacts
|
||||
uses: actions/upload-artifact@v3
|
||||
with:
|
||||
path: |
|
||||
dist/yt-dlp_macos
|
||||
dist/yt-dlp_macos.zip
|
||||
|
||||
|
||||
build_macos_legacy:
|
||||
macos_legacy:
|
||||
if: inputs.macos_legacy
|
||||
runs-on: macos-latest
|
||||
needs: prepare
|
||||
|
||||
steps:
|
||||
- uses: actions/checkout@v3
|
||||
- name: Install Python
|
||||
# We need the official Python, because the GA ones only support newer macOS versions
|
||||
env:
|
||||
PYTHON_VERSION: 3.10.5
|
||||
MACOSX_DEPLOYMENT_TARGET: 10.9 # Used up by the Python build tools
|
||||
run: |
|
||||
- uses: actions/checkout@v3
|
||||
- name: Install Python
|
||||
# We need the official Python, because the GA ones only support newer macOS versions
|
||||
env:
|
||||
PYTHON_VERSION: 3.10.5
|
||||
MACOSX_DEPLOYMENT_TARGET: 10.9 # Used up by the Python build tools
|
||||
run: |
|
||||
# Hack to get the latest patch version. Uncomment if needed
|
||||
#brew install python@3.10
|
||||
#export PYTHON_VERSION=$( $(brew --prefix)/opt/python@3.10/bin/python3 --version | cut -d ' ' -f 2 )
|
||||
curl https://www.python.org/ftp/python/${PYTHON_VERSION}/python-${PYTHON_VERSION}-macos11.pkg -o "python.pkg"
|
||||
sudo installer -pkg python.pkg -target /
|
||||
python3 --version
|
||||
- name: Install Requirements
|
||||
run: |
|
||||
- name: Install Requirements
|
||||
run: |
|
||||
brew install coreutils
|
||||
python3 -m pip install -U --user pip Pyinstaller -r requirements.txt
|
||||
|
||||
- name: Prepare
|
||||
run: |
|
||||
python3 devscripts/update-version.py ${{ needs.prepare.outputs.version_suffix }}
|
||||
- name: Prepare
|
||||
run: |
|
||||
python3 devscripts/update-version.py -c ${{ inputs.channel }} ${{ inputs.version }}
|
||||
python3 devscripts/make_lazy_extractors.py
|
||||
- name: Build
|
||||
run: |
|
||||
- name: Build
|
||||
run: |
|
||||
python3 pyinst.py
|
||||
mv dist/yt-dlp_macos dist/yt-dlp_macos_legacy
|
||||
|
||||
- name: Upload artifacts
|
||||
uses: actions/upload-artifact@v3
|
||||
with:
|
||||
path: |
|
||||
dist/yt-dlp_macos_legacy
|
||||
- name: Upload artifacts
|
||||
uses: actions/upload-artifact@v3
|
||||
with:
|
||||
path: |
|
||||
dist/yt-dlp_macos_legacy
|
||||
|
||||
|
||||
build_windows:
|
||||
windows:
|
||||
if: inputs.windows
|
||||
runs-on: windows-latest
|
||||
needs: prepare
|
||||
|
||||
steps:
|
||||
- uses: actions/checkout@v3
|
||||
- uses: actions/setup-python@v4
|
||||
with: # 3.8 is used for Win7 support
|
||||
python-version: '3.8'
|
||||
- name: Install Requirements
|
||||
run: | # Custom pyinstaller built with https://github.com/yt-dlp/pyinstaller-builds
|
||||
python -m pip install --upgrade pip setuptools wheel py2exe
|
||||
pip install "https://yt-dlp.github.io/Pyinstaller-Builds/x86_64/pyinstaller-5.3-py3-none-any.whl" -r requirements.txt
|
||||
- uses: actions/checkout@v3
|
||||
- uses: actions/setup-python@v4
|
||||
with: # 3.8 is used for Win7 support
|
||||
python-version: "3.8"
|
||||
- name: Install Requirements
|
||||
run: | # Custom pyinstaller built with https://github.com/yt-dlp/pyinstaller-builds
|
||||
python -m pip install -U pip setuptools wheel py2exe
|
||||
pip install -U "https://yt-dlp.github.io/Pyinstaller-Builds/x86_64/pyinstaller-5.8.0-py3-none-any.whl" -r requirements.txt
|
||||
|
||||
- name: Prepare
|
||||
run: |
|
||||
python devscripts/update-version.py ${{ needs.prepare.outputs.version_suffix }}
|
||||
- name: Prepare
|
||||
run: |
|
||||
python devscripts/update-version.py -c ${{ inputs.channel }} ${{ inputs.version }}
|
||||
python devscripts/make_lazy_extractors.py
|
||||
- name: Build
|
||||
run: |
|
||||
- name: Build
|
||||
run: |
|
||||
python setup.py py2exe
|
||||
Move-Item ./dist/yt-dlp.exe ./dist/yt-dlp_min.exe
|
||||
python pyinst.py
|
||||
python pyinst.py --onedir
|
||||
Compress-Archive -Path ./dist/yt-dlp/* -DestinationPath ./dist/yt-dlp_win.zip
|
||||
|
||||
- name: Upload artifacts
|
||||
uses: actions/upload-artifact@v3
|
||||
with:
|
||||
path: |
|
||||
dist/yt-dlp.exe
|
||||
dist/yt-dlp_min.exe
|
||||
dist/yt-dlp_win.zip
|
||||
- name: Upload artifacts
|
||||
uses: actions/upload-artifact@v3
|
||||
with:
|
||||
path: |
|
||||
dist/yt-dlp.exe
|
||||
dist/yt-dlp_min.exe
|
||||
dist/yt-dlp_win.zip
|
||||
|
||||
|
||||
build_windows32:
|
||||
windows32:
|
||||
if: inputs.windows32
|
||||
runs-on: windows-latest
|
||||
needs: prepare
|
||||
|
||||
steps:
|
||||
- uses: actions/checkout@v3
|
||||
- uses: actions/setup-python@v4
|
||||
with: # 3.7 is used for Vista support. See https://github.com/yt-dlp/yt-dlp/issues/390
|
||||
python-version: '3.7'
|
||||
architecture: 'x86'
|
||||
- name: Install Requirements
|
||||
run: |
|
||||
python -m pip install --upgrade pip setuptools wheel
|
||||
pip install "https://yt-dlp.github.io/Pyinstaller-Builds/i686/pyinstaller-5.3-py3-none-any.whl" -r requirements.txt
|
||||
- uses: actions/checkout@v3
|
||||
- uses: actions/setup-python@v4
|
||||
with: # 3.7 is used for Vista support. See https://github.com/yt-dlp/yt-dlp/issues/390
|
||||
python-version: "3.7"
|
||||
architecture: "x86"
|
||||
- name: Install Requirements
|
||||
run: |
|
||||
python -m pip install -U pip setuptools wheel
|
||||
pip install -U "https://yt-dlp.github.io/Pyinstaller-Builds/i686/pyinstaller-5.8.0-py3-none-any.whl" -r requirements.txt
|
||||
|
||||
- name: Prepare
|
||||
run: |
|
||||
python devscripts/update-version.py ${{ needs.prepare.outputs.version_suffix }}
|
||||
- name: Prepare
|
||||
run: |
|
||||
python devscripts/update-version.py -c ${{ inputs.channel }} ${{ inputs.version }}
|
||||
python devscripts/make_lazy_extractors.py
|
||||
- name: Build
|
||||
run: |
|
||||
- name: Build
|
||||
run: |
|
||||
python pyinst.py
|
||||
|
||||
- name: Upload artifacts
|
||||
uses: actions/upload-artifact@v3
|
||||
with:
|
||||
path: |
|
||||
dist/yt-dlp_x86.exe
|
||||
- name: Upload artifacts
|
||||
uses: actions/upload-artifact@v3
|
||||
with:
|
||||
path: |
|
||||
dist/yt-dlp_x86.exe
|
||||
|
||||
|
||||
publish_release:
|
||||
meta_files:
|
||||
if: inputs.meta_files && always()
|
||||
needs:
|
||||
- unix
|
||||
- linux_arm
|
||||
- macos
|
||||
- macos_legacy
|
||||
- windows
|
||||
- windows32
|
||||
runs-on: ubuntu-latest
|
||||
needs: [prepare, build_unix, build_windows, build_windows32, build_macos, build_macos_legacy]
|
||||
|
||||
steps:
|
||||
- uses: actions/checkout@v3
|
||||
- uses: actions/download-artifact@v3
|
||||
- uses: actions/download-artifact@v3
|
||||
|
||||
- name: Get Changelog
|
||||
run: |
|
||||
changelog=$(grep -oPz '(?s)(?<=### ${{ needs.prepare.outputs.ytdlp_version }}\n{2}).+?(?=\n{2,3}###)' Changelog.md) || true
|
||||
echo "changelog<<EOF" >> $GITHUB_ENV
|
||||
echo "$changelog" >> $GITHUB_ENV
|
||||
echo "EOF" >> $GITHUB_ENV
|
||||
- name: Make Update spec
|
||||
run: |
|
||||
echo "# This file is used for regulating self-update" >> _update_spec
|
||||
echo "lock 2022.07.18 .+ Python 3.6" >> _update_spec
|
||||
- name: Make SHA2-SUMS files
|
||||
run: |
|
||||
sha256sum artifact/yt-dlp | awk '{print $1 " yt-dlp"}' >> SHA2-256SUMS
|
||||
sha256sum artifact/yt-dlp.tar.gz | awk '{print $1 " yt-dlp.tar.gz"}' >> SHA2-256SUMS
|
||||
sha256sum artifact/yt-dlp.exe | awk '{print $1 " yt-dlp.exe"}' >> SHA2-256SUMS
|
||||
sha256sum artifact/yt-dlp_win.zip | awk '{print $1 " yt-dlp_win.zip"}' >> SHA2-256SUMS
|
||||
sha256sum artifact/yt-dlp_min.exe | awk '{print $1 " yt-dlp_min.exe"}' >> SHA2-256SUMS
|
||||
sha256sum artifact/yt-dlp_x86.exe | awk '{print $1 " yt-dlp_x86.exe"}' >> SHA2-256SUMS
|
||||
sha256sum artifact/yt-dlp_macos | awk '{print $1 " yt-dlp_macos"}' >> SHA2-256SUMS
|
||||
sha256sum artifact/yt-dlp_macos.zip | awk '{print $1 " yt-dlp_macos.zip"}' >> SHA2-256SUMS
|
||||
sha256sum artifact/yt-dlp_macos_legacy | awk '{print $1 " yt-dlp_macos_legacy"}' >> SHA2-256SUMS
|
||||
sha256sum artifact/dist/yt-dlp_linux | awk '{print $1 " yt-dlp_linux"}' >> SHA2-256SUMS
|
||||
sha256sum artifact/dist/yt-dlp_linux.zip | awk '{print $1 " yt-dlp_linux.zip"}' >> SHA2-256SUMS
|
||||
sha512sum artifact/yt-dlp | awk '{print $1 " yt-dlp"}' >> SHA2-512SUMS
|
||||
sha512sum artifact/yt-dlp.tar.gz | awk '{print $1 " yt-dlp.tar.gz"}' >> SHA2-512SUMS
|
||||
sha512sum artifact/yt-dlp.exe | awk '{print $1 " yt-dlp.exe"}' >> SHA2-512SUMS
|
||||
sha512sum artifact/yt-dlp_win.zip | awk '{print $1 " yt-dlp_win.zip"}' >> SHA2-512SUMS
|
||||
sha512sum artifact/yt-dlp_min.exe | awk '{print $1 " yt-dlp_min.exe"}' >> SHA2-512SUMS
|
||||
sha512sum artifact/yt-dlp_x86.exe | awk '{print $1 " yt-dlp_x86.exe"}' >> SHA2-512SUMS
|
||||
sha512sum artifact/yt-dlp_macos | awk '{print $1 " yt-dlp_macos"}' >> SHA2-512SUMS
|
||||
sha512sum artifact/yt-dlp_macos.zip | awk '{print $1 " yt-dlp_macos.zip"}' >> SHA2-512SUMS
|
||||
sha512sum artifact/yt-dlp_macos_legacy | awk '{print $1 " yt-dlp_macos_legacy"}' >> SHA2-512SUMS
|
||||
sha512sum artifact/dist/yt-dlp_linux | awk '{print $1 " yt-dlp_linux"}' >> SHA2-512SUMS
|
||||
sha512sum artifact/dist/yt-dlp_linux.zip | awk '{print $1 " yt-dlp_linux.zip"}' >> SHA2-512SUMS
|
||||
- name: Make SHA2-SUMS files
|
||||
run: |
|
||||
cd ./artifact/
|
||||
sha256sum * > ../SHA2-256SUMS
|
||||
sha512sum * > ../SHA2-512SUMS
|
||||
|
||||
- name: Publish Release
|
||||
uses: yt-dlp/action-gh-release@v1
|
||||
with:
|
||||
tag_name: ${{ needs.prepare.outputs.ytdlp_version }}
|
||||
name: yt-dlp ${{ needs.prepare.outputs.ytdlp_version }}
|
||||
target_commitish: ${{ needs.prepare.outputs.head_sha }}
|
||||
body: |
|
||||
#### [A description of the various files]((https://github.com/yt-dlp/yt-dlp#release-files)) are in the README
|
||||
- name: Make Update spec
|
||||
run: |
|
||||
cat >> _update_spec << EOF
|
||||
# This file is used for regulating self-update
|
||||
lock 2022.08.18.36 .+ Python 3.6
|
||||
EOF
|
||||
|
||||
---
|
||||
<details open><summary><h3>Changelog</summary>
|
||||
<p>
|
||||
- name: Sign checksum files
|
||||
env:
|
||||
GPG_SIGNING_KEY: ${{ secrets.GPG_SIGNING_KEY }}
|
||||
if: env.GPG_SIGNING_KEY != ''
|
||||
run: |
|
||||
gpg --batch --import <<< "${{ secrets.GPG_SIGNING_KEY }}"
|
||||
for signfile in ./SHA*SUMS; do
|
||||
gpg --batch --detach-sign "$signfile"
|
||||
done
|
||||
|
||||
${{ env.changelog }}
|
||||
|
||||
</p>
|
||||
</details>
|
||||
files: |
|
||||
SHA2-256SUMS
|
||||
SHA2-512SUMS
|
||||
artifact/yt-dlp
|
||||
artifact/yt-dlp.tar.gz
|
||||
artifact/yt-dlp.exe
|
||||
artifact/yt-dlp_win.zip
|
||||
artifact/yt-dlp_min.exe
|
||||
artifact/yt-dlp_x86.exe
|
||||
artifact/yt-dlp_macos
|
||||
artifact/yt-dlp_macos.zip
|
||||
artifact/yt-dlp_macos_legacy
|
||||
artifact/dist/yt-dlp_linux
|
||||
artifact/dist/yt-dlp_linux.zip
|
||||
_update_spec
|
||||
- name: Upload artifacts
|
||||
uses: actions/upload-artifact@v3
|
||||
with:
|
||||
path: |
|
||||
SHA*SUMS*
|
||||
_update_spec
|
||||
|
14
.github/workflows/core.yml
vendored
14
.github/workflows/core.yml
vendored
@ -1,5 +1,8 @@
|
||||
name: Core Tests
|
||||
on: [push, pull_request]
|
||||
permissions:
|
||||
contents: read
|
||||
|
||||
jobs:
|
||||
tests:
|
||||
name: Core Tests
|
||||
@ -9,13 +12,13 @@ jobs:
|
||||
fail-fast: false
|
||||
matrix:
|
||||
os: [ubuntu-latest]
|
||||
# CPython 3.9 is in quick-test
|
||||
python-version: ['3.7', '3.10', 3.11-dev, pypy-3.7, pypy-3.8]
|
||||
# CPython 3.11 is in quick-test
|
||||
python-version: ['3.8', '3.9', '3.10', pypy-3.7, pypy-3.8]
|
||||
run-tests-ext: [sh]
|
||||
include:
|
||||
# atleast one of each CPython/PyPy tests must be in windows
|
||||
- os: windows-latest
|
||||
python-version: '3.8'
|
||||
python-version: '3.7'
|
||||
run-tests-ext: bat
|
||||
- os: windows-latest
|
||||
python-version: pypy-3.9
|
||||
@ -30,5 +33,6 @@ jobs:
|
||||
run: pip install pytest
|
||||
- name: Run tests
|
||||
continue-on-error: False
|
||||
run: ./devscripts/run_tests.${{ matrix.run-tests-ext }} core
|
||||
# Linter is in quick-test
|
||||
run: |
|
||||
python3 -m yt_dlp -v || true # Print debug head
|
||||
./devscripts/run_tests.${{ matrix.run-tests-ext }} core
|
||||
|
3
.github/workflows/download.yml
vendored
3
.github/workflows/download.yml
vendored
@ -1,5 +1,8 @@
|
||||
name: Download Tests
|
||||
on: [push, pull_request]
|
||||
permissions:
|
||||
contents: read
|
||||
|
||||
jobs:
|
||||
quick:
|
||||
name: Quick Download Tests
|
||||
|
81
.github/workflows/publish.yml
vendored
Normal file
81
.github/workflows/publish.yml
vendored
Normal file
@ -0,0 +1,81 @@
|
||||
name: Publish
|
||||
on:
|
||||
workflow_call:
|
||||
inputs:
|
||||
nightly:
|
||||
default: false
|
||||
required: false
|
||||
type: boolean
|
||||
version:
|
||||
required: true
|
||||
type: string
|
||||
target_commitish:
|
||||
required: true
|
||||
type: string
|
||||
secrets:
|
||||
ARCHIVE_REPO_TOKEN:
|
||||
required: false
|
||||
|
||||
permissions:
|
||||
contents: write
|
||||
|
||||
jobs:
|
||||
publish:
|
||||
runs-on: ubuntu-latest
|
||||
|
||||
steps:
|
||||
- uses: actions/checkout@v3
|
||||
with:
|
||||
fetch-depth: 0
|
||||
- uses: actions/download-artifact@v3
|
||||
- uses: actions/setup-python@v4
|
||||
with:
|
||||
python-version: "3.10"
|
||||
|
||||
- name: Generate release notes
|
||||
run: |
|
||||
cat >> ./RELEASE_NOTES << EOF
|
||||
#### A description of the various files are in the [README](https://github.com/yt-dlp/yt-dlp#release-files)
|
||||
---
|
||||
<details><summary><h3>Changelog</h3></summary>
|
||||
$(python ./devscripts/make_changelog.py -vv)
|
||||
</details>
|
||||
EOF
|
||||
echo "**This is an automated nightly pre-release build**" >> ./PRERELEASE_NOTES
|
||||
cat ./RELEASE_NOTES >> ./PRERELEASE_NOTES
|
||||
echo "Generated from: https://github.com/${{ github.repository }}/commit/${{ inputs.target_commitish }}" >> ./ARCHIVE_NOTES
|
||||
cat ./RELEASE_NOTES >> ./ARCHIVE_NOTES
|
||||
|
||||
- name: Archive nightly release
|
||||
env:
|
||||
GH_TOKEN: ${{ secrets.ARCHIVE_REPO_TOKEN }}
|
||||
GH_REPO: ${{ vars.ARCHIVE_REPO }}
|
||||
if: |
|
||||
inputs.nightly && env.GH_TOKEN != '' && env.GH_REPO != ''
|
||||
run: |
|
||||
gh release create \
|
||||
--notes-file ARCHIVE_NOTES \
|
||||
--title "yt-dlp nightly ${{ inputs.version }}" \
|
||||
${{ inputs.version }} \
|
||||
artifact/*
|
||||
|
||||
- name: Prune old nightly release
|
||||
if: inputs.nightly && !vars.ARCHIVE_REPO
|
||||
env:
|
||||
GH_TOKEN: ${{ github.token }}
|
||||
run: |
|
||||
gh release delete --yes --cleanup-tag "nightly" || true
|
||||
git tag --delete "nightly" || true
|
||||
sleep 5 # Enough time to cover deletion race condition
|
||||
|
||||
- name: Publish release${{ inputs.nightly && ' (nightly)' || '' }}
|
||||
env:
|
||||
GH_TOKEN: ${{ github.token }}
|
||||
if: (inputs.nightly && !vars.ARCHIVE_REPO) || !inputs.nightly
|
||||
run: |
|
||||
gh release create \
|
||||
--notes-file ${{ inputs.nightly && 'PRE' || '' }}RELEASE_NOTES \
|
||||
--target ${{ inputs.target_commitish }} \
|
||||
--title "yt-dlp ${{ inputs.nightly && 'nightly ' || '' }}${{ inputs.version }}" \
|
||||
${{ inputs.nightly && '--prerelease "nightly"' || inputs.version }} \
|
||||
artifact/*
|
16
.github/workflows/quick-test.yml
vendored
16
.github/workflows/quick-test.yml
vendored
@ -1,5 +1,8 @@
|
||||
name: Quick Test
|
||||
on: [push, pull_request]
|
||||
permissions:
|
||||
contents: read
|
||||
|
||||
jobs:
|
||||
tests:
|
||||
name: Core Test
|
||||
@ -7,24 +10,23 @@ jobs:
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- uses: actions/checkout@v3
|
||||
- name: Set up Python
|
||||
- name: Set up Python 3.11
|
||||
uses: actions/setup-python@v4
|
||||
with:
|
||||
python-version: 3.9
|
||||
python-version: '3.11'
|
||||
- name: Install test requirements
|
||||
run: pip install pytest pycryptodomex
|
||||
- name: Run tests
|
||||
run: ./devscripts/run_tests.sh core
|
||||
run: |
|
||||
python3 -m yt_dlp -v || true
|
||||
./devscripts/run_tests.sh core
|
||||
flake8:
|
||||
name: Linter
|
||||
if: "!contains(github.event.head_commit.message, 'ci skip all')"
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- uses: actions/checkout@v3
|
||||
- name: Set up Python
|
||||
uses: actions/setup-python@v4
|
||||
with:
|
||||
python-version: 3.9
|
||||
- uses: actions/setup-python@v4
|
||||
- name: Install flake8
|
||||
run: pip install flake8
|
||||
- name: Make lazy extractors
|
||||
|
51
.github/workflows/release-nightly.yml
vendored
Normal file
51
.github/workflows/release-nightly.yml
vendored
Normal file
@ -0,0 +1,51 @@
|
||||
name: Release (nightly)
|
||||
on:
|
||||
push:
|
||||
branches:
|
||||
- master
|
||||
paths:
|
||||
- "yt_dlp/**.py"
|
||||
- "!yt_dlp/version.py"
|
||||
concurrency:
|
||||
group: release-nightly
|
||||
cancel-in-progress: true
|
||||
permissions:
|
||||
contents: read
|
||||
|
||||
jobs:
|
||||
prepare:
|
||||
if: vars.BUILD_NIGHTLY != ''
|
||||
runs-on: ubuntu-latest
|
||||
outputs:
|
||||
version: ${{ steps.get_version.outputs.version }}
|
||||
|
||||
steps:
|
||||
- uses: actions/checkout@v3
|
||||
- name: Get version
|
||||
id: get_version
|
||||
run: |
|
||||
python devscripts/update-version.py "$(date -u +"%H%M%S")" | grep -Po "version=\d+(\.\d+){3}" >> "$GITHUB_OUTPUT"
|
||||
|
||||
build:
|
||||
needs: prepare
|
||||
uses: ./.github/workflows/build.yml
|
||||
with:
|
||||
version: ${{ needs.prepare.outputs.version }}
|
||||
channel: nightly
|
||||
permissions:
|
||||
contents: read
|
||||
packages: write # For package cache
|
||||
secrets:
|
||||
GPG_SIGNING_KEY: ${{ secrets.GPG_SIGNING_KEY }}
|
||||
|
||||
publish:
|
||||
needs: [prepare, build]
|
||||
uses: ./.github/workflows/publish.yml
|
||||
secrets:
|
||||
ARCHIVE_REPO_TOKEN: ${{ secrets.ARCHIVE_REPO_TOKEN }}
|
||||
permissions:
|
||||
contents: write
|
||||
with:
|
||||
nightly: true
|
||||
version: ${{ needs.prepare.outputs.version }}
|
||||
target_commitish: ${{ github.sha }}
|
129
.github/workflows/release.yml
vendored
Normal file
129
.github/workflows/release.yml
vendored
Normal file
@ -0,0 +1,129 @@
|
||||
name: Release
|
||||
on: workflow_dispatch
|
||||
permissions:
|
||||
contents: read
|
||||
|
||||
jobs:
|
||||
prepare:
|
||||
permissions:
|
||||
contents: write
|
||||
runs-on: ubuntu-latest
|
||||
outputs:
|
||||
version: ${{ steps.update_version.outputs.version }}
|
||||
head_sha: ${{ steps.push_release.outputs.head_sha }}
|
||||
|
||||
steps:
|
||||
- uses: actions/checkout@v3
|
||||
with:
|
||||
fetch-depth: 0
|
||||
|
||||
- uses: actions/setup-python@v4
|
||||
with:
|
||||
python-version: "3.10"
|
||||
|
||||
- name: Update version
|
||||
id: update_version
|
||||
run: |
|
||||
python devscripts/update-version.py ${{ vars.PUSH_VERSION_COMMIT == '' && '"$(date -u +"%H%M%S")"' || '' }} | \
|
||||
grep -Po "version=\d+\.\d+\.\d+(\.\d+)?" >> "$GITHUB_OUTPUT"
|
||||
|
||||
- name: Update documentation
|
||||
run: |
|
||||
make doc
|
||||
sed '/### /Q' Changelog.md >> ./CHANGELOG
|
||||
echo '### ${{ steps.update_version.outputs.version }}' >> ./CHANGELOG
|
||||
python ./devscripts/make_changelog.py -vv -c >> ./CHANGELOG
|
||||
echo >> ./CHANGELOG
|
||||
grep -Poz '(?s)### \d+\.\d+\.\d+.+' 'Changelog.md' | head -n -1 >> ./CHANGELOG
|
||||
cat ./CHANGELOG > Changelog.md
|
||||
|
||||
- name: Push to release
|
||||
id: push_release
|
||||
run: |
|
||||
git config --global user.name github-actions
|
||||
git config --global user.email github-actions@example.com
|
||||
git add -u
|
||||
git commit -m "Release ${{ steps.update_version.outputs.version }}" \
|
||||
-m "Created by: ${{ github.event.sender.login }}" -m ":ci skip all :ci run dl"
|
||||
git push origin --force ${{ github.event.ref }}:release
|
||||
echo "head_sha=$(git rev-parse HEAD)" >> "$GITHUB_OUTPUT"
|
||||
|
||||
- name: Update master
|
||||
if: vars.PUSH_VERSION_COMMIT != ''
|
||||
run: git push origin ${{ github.event.ref }}
|
||||
|
||||
publish_pypi_homebrew:
|
||||
needs: prepare
|
||||
runs-on: ubuntu-latest
|
||||
|
||||
steps:
|
||||
- uses: actions/checkout@v3
|
||||
- uses: actions/setup-python@v4
|
||||
with:
|
||||
python-version: "3.10"
|
||||
|
||||
- name: Install Requirements
|
||||
run: |
|
||||
sudo apt-get -y install pandoc man
|
||||
python -m pip install -U pip setuptools wheel twine
|
||||
python -m pip install -U -r requirements.txt
|
||||
|
||||
- name: Prepare
|
||||
run: |
|
||||
python devscripts/update-version.py ${{ needs.prepare.outputs.version }}
|
||||
python devscripts/make_lazy_extractors.py
|
||||
|
||||
- name: Build and publish on PyPI
|
||||
env:
|
||||
TWINE_USERNAME: __token__
|
||||
TWINE_PASSWORD: ${{ secrets.PYPI_TOKEN }}
|
||||
if: env.TWINE_PASSWORD != ''
|
||||
run: |
|
||||
rm -rf dist/*
|
||||
make pypi-files
|
||||
python devscripts/set-variant.py pip -M "You installed yt-dlp with pip or using the wheel from PyPi; Use that to update"
|
||||
python setup.py sdist bdist_wheel
|
||||
twine upload dist/*
|
||||
|
||||
- name: Checkout Homebrew repository
|
||||
env:
|
||||
BREW_TOKEN: ${{ secrets.BREW_TOKEN }}
|
||||
PYPI_TOKEN: ${{ secrets.PYPI_TOKEN }}
|
||||
if: env.BREW_TOKEN != '' && env.PYPI_TOKEN != ''
|
||||
uses: actions/checkout@v3
|
||||
with:
|
||||
repository: yt-dlp/homebrew-taps
|
||||
path: taps
|
||||
ssh-key: ${{ secrets.BREW_TOKEN }}
|
||||
|
||||
- name: Update Homebrew Formulae
|
||||
env:
|
||||
BREW_TOKEN: ${{ secrets.BREW_TOKEN }}
|
||||
PYPI_TOKEN: ${{ secrets.PYPI_TOKEN }}
|
||||
if: env.BREW_TOKEN != '' && env.PYPI_TOKEN != ''
|
||||
run: |
|
||||
python devscripts/update-formulae.py taps/Formula/yt-dlp.rb "${{ needs.prepare.outputs.version }}"
|
||||
git -C taps/ config user.name github-actions
|
||||
git -C taps/ config user.email github-actions@example.com
|
||||
git -C taps/ commit -am 'yt-dlp: ${{ needs.prepare.outputs.version }}'
|
||||
git -C taps/ push
|
||||
|
||||
build:
|
||||
needs: prepare
|
||||
uses: ./.github/workflows/build.yml
|
||||
with:
|
||||
version: ${{ needs.prepare.outputs.version }}
|
||||
permissions:
|
||||
contents: read
|
||||
packages: write # For package cache
|
||||
secrets:
|
||||
GPG_SIGNING_KEY: ${{ secrets.GPG_SIGNING_KEY }}
|
||||
|
||||
publish:
|
||||
needs: [prepare, build]
|
||||
uses: ./.github/workflows/publish.yml
|
||||
permissions:
|
||||
contents: write
|
||||
with:
|
||||
version: ${{ needs.prepare.outputs.version }}
|
||||
target_commitish: ${{ needs.prepare.outputs.head_sha }}
|
15
.gitignore
vendored
15
.gitignore
vendored
@ -30,16 +30,18 @@ cookies
|
||||
*.f4v
|
||||
*.flac
|
||||
*.flv
|
||||
*.gif
|
||||
*.jpeg
|
||||
*.jpg
|
||||
*.m4a
|
||||
*.mpga
|
||||
*.m4v
|
||||
*.mhtml
|
||||
*.mkv
|
||||
*.mov
|
||||
*.mp3
|
||||
*.mp4
|
||||
*.mpga
|
||||
*.oga
|
||||
*.ogg
|
||||
*.opus
|
||||
*.png
|
||||
@ -47,6 +49,7 @@ cookies
|
||||
*.srt
|
||||
*.swf
|
||||
*.swp
|
||||
*.tt
|
||||
*.ttml
|
||||
*.url
|
||||
*.vtt
|
||||
@ -69,6 +72,7 @@ dist/
|
||||
zip/
|
||||
tmp/
|
||||
venv/
|
||||
.venv/
|
||||
completions/
|
||||
|
||||
# Misc
|
||||
@ -85,6 +89,7 @@ updates_key.pem
|
||||
.tox
|
||||
*.class
|
||||
*.isorted
|
||||
*.stackdump
|
||||
|
||||
# Generated
|
||||
AUTHORS
|
||||
@ -116,9 +121,5 @@ yt-dlp.zip
|
||||
*/extractor/lazy_extractors.py
|
||||
|
||||
# Plugins
|
||||
ytdlp_plugins/extractor/*
|
||||
!ytdlp_plugins/extractor/__init__.py
|
||||
!ytdlp_plugins/extractor/sample.py
|
||||
ytdlp_plugins/postprocessor/*
|
||||
!ytdlp_plugins/postprocessor/__init__.py
|
||||
!ytdlp_plugins/postprocessor/sample.py
|
||||
ytdlp_plugins/
|
||||
yt-dlp-plugins
|
||||
|
@ -127,7 +127,7 @@ ### Are you willing to share account details if needed?
|
||||
|
||||
### Is the website primarily used for piracy?
|
||||
|
||||
We follow [youtube-dl's policy](https://github.com/ytdl-org/youtube-dl#can-you-add-support-for-this-anime-video-site-or-site-which-shows-current-movies-for-free) to not support services that is primarily used for infringing copyright. Additionally, it has been decided to not to support porn sites that specialize in deep fake. We also cannot support any service that serves only [DRM protected content](https://en.wikipedia.org/wiki/Digital_rights_management).
|
||||
We follow [youtube-dl's policy](https://github.com/ytdl-org/youtube-dl#can-you-add-support-for-this-anime-video-site-or-site-which-shows-current-movies-for-free) to not support services that is primarily used for infringing copyright. Additionally, it has been decided to not to support porn sites that specialize in fakes. We also cannot support any service that serves only [DRM protected content](https://en.wikipedia.org/wiki/Digital_rights_management).
|
||||
|
||||
|
||||
|
||||
@ -351,8 +351,9 @@ #### Example
|
||||
```python
|
||||
thumbnail_data = data.get('thumbnails') or []
|
||||
thumbnails = [{
|
||||
'url': item['url']
|
||||
} for item in thumbnail_data] # correct
|
||||
'url': item['url'],
|
||||
'height': item.get('h'),
|
||||
} for item in thumbnail_data if item.get('url')] # correct
|
||||
```
|
||||
|
||||
and not like:
|
||||
@ -360,12 +361,27 @@ #### Example
|
||||
```python
|
||||
thumbnail_data = data.get('thumbnails')
|
||||
thumbnails = [{
|
||||
'url': item['url']
|
||||
'url': item['url'],
|
||||
'height': item.get('h'),
|
||||
} for item in thumbnail_data] # incorrect
|
||||
```
|
||||
|
||||
In this case, `thumbnail_data` will be `None` if the field was not found and this will cause the loop `for item in thumbnail_data` to raise a fatal error. Using `or []` avoids this error and results in setting an empty list in `thumbnails` instead.
|
||||
|
||||
Alternately, this can be further simplified by using `traverse_obj`
|
||||
|
||||
```python
|
||||
thumbnails = [{
|
||||
'url': item['url'],
|
||||
'height': item.get('h'),
|
||||
} for item in traverse_obj(data, ('thumbnails', lambda _, v: v['url']))]
|
||||
```
|
||||
|
||||
or, even better,
|
||||
|
||||
```python
|
||||
thumbnails = traverse_obj(data, ('thumbnails', ..., {'url': 'url', 'height': 'h'}))
|
||||
```
|
||||
|
||||
### Provide fallbacks
|
||||
|
||||
|
102
CONTRIBUTORS
102
CONTRIBUTORS
@ -3,6 +3,8 @@ shirt-dev (collaborator)
|
||||
coletdjnz/colethedj (collaborator)
|
||||
Ashish0804 (collaborator)
|
||||
nao20010128nao/Lesmiscore (collaborator)
|
||||
bashonly (collaborator)
|
||||
Grub4K (collaborator)
|
||||
h-h-h-h
|
||||
pauldubois98
|
||||
nixxo
|
||||
@ -295,7 +297,6 @@ Mehavoid
|
||||
winterbird-code
|
||||
yashkc2025
|
||||
aldoridhoni
|
||||
bashonly
|
||||
jacobtruman
|
||||
masta79
|
||||
palewire
|
||||
@ -309,3 +310,102 @@ shreyasminocha
|
||||
tejasa97
|
||||
xenov
|
||||
satan1st
|
||||
0xGodspeed
|
||||
5736d79
|
||||
587021c
|
||||
basrieter
|
||||
Bobscorn
|
||||
CNugteren
|
||||
columndeeply
|
||||
DoubleCouponDay
|
||||
Fabi019
|
||||
GautamMKGarg
|
||||
itachi-19
|
||||
jeroenj
|
||||
josanabr
|
||||
LiviaMedeiros
|
||||
nikita-moor
|
||||
snapdgn
|
||||
SuperSonicHub1
|
||||
tannertechnology
|
||||
Timendum
|
||||
tobi1805
|
||||
TokyoBlackHole
|
||||
ajayyy
|
||||
Alienmaster
|
||||
bsun0000
|
||||
changren-wcr
|
||||
ClosedPort22
|
||||
CrankDatSouljaBoy
|
||||
cruel-efficiency
|
||||
endotronic
|
||||
Generator
|
||||
gibson042
|
||||
How-Bout-No
|
||||
invertico
|
||||
jahway603
|
||||
jwoglom
|
||||
lksj
|
||||
megapro17
|
||||
mlampe
|
||||
MrOctopus
|
||||
nosoop
|
||||
puc9
|
||||
sashashura
|
||||
schnusch
|
||||
SG5
|
||||
the-marenga
|
||||
tkgmomosheep
|
||||
vitkhab
|
||||
glensc
|
||||
synthpop123
|
||||
tntmod54321
|
||||
milkknife
|
||||
Bnyro
|
||||
CapacitorSet
|
||||
stelcodes
|
||||
skbeh
|
||||
muddi900
|
||||
digitall
|
||||
chengzhicn
|
||||
mexus
|
||||
JChris246
|
||||
redraskal
|
||||
Spicadox
|
||||
barsnick
|
||||
docbender
|
||||
KurtBestor
|
||||
Chrissi2812
|
||||
FrederikNS
|
||||
gschizas
|
||||
JC-Chung
|
||||
mzhou
|
||||
OndrejBakan
|
||||
ab4cbef
|
||||
aionescu
|
||||
amra
|
||||
ByteDream
|
||||
carusocr
|
||||
chexxor
|
||||
felixonmars
|
||||
FrankZ85
|
||||
FriedrichRehren
|
||||
gregsadetsky
|
||||
LeoniePhiline
|
||||
LowSuggestion912
|
||||
Matumo
|
||||
OIRNOIR
|
||||
OMEGARAZER
|
||||
oxamun
|
||||
pmitchell86
|
||||
qbnu
|
||||
qulaz
|
||||
rebane2001
|
||||
road-master
|
||||
rohieb
|
||||
sdht0
|
||||
seproDev
|
||||
Hill-98
|
||||
LXYan2333
|
||||
mushbite
|
||||
venkata-krishnas
|
||||
|
574
Changelog.md
574
Changelog.md
@ -1,15 +1,575 @@
|
||||
# Changelog
|
||||
|
||||
<!--
|
||||
# Instuctions for creating release
|
||||
|
||||
* Run `make doc`
|
||||
* Update Changelog.md and CONTRIBUTORS
|
||||
* Change "Based on ytdl" version in Readme.md if needed
|
||||
* Commit as `Release <version>` and push to master
|
||||
* Dispatch the workflow https://github.com/yt-dlp/yt-dlp/actions/workflows/build.yml on master
|
||||
# To create a release, dispatch the https://github.com/yt-dlp/yt-dlp/actions/workflows/release.yml workflow on master
|
||||
-->
|
||||
|
||||
### 2023.03.04
|
||||
|
||||
#### Extractor changes
|
||||
- bilibili
|
||||
- [Fix for downloading wrong subtitles](https://github.com/yt-dlp/yt-dlp/commit/8a83baaf218ab89e6e7faa76b7c7be3a2ec19e3a) ([#6358](https://github.com/yt-dlp/yt-dlp/issues/6358)) by [LXYan2333](https://github.com/LXYan2333)
|
||||
- ESPNcricinfo
|
||||
- [Handle new URL pattern](https://github.com/yt-dlp/yt-dlp/commit/640c934823fc2d1ec77ec932566078014058635f) ([#6321](https://github.com/yt-dlp/yt-dlp/issues/6321)) by [venkata-krishnas](https://github.com/venkata-krishnas)
|
||||
- lefigaro
|
||||
- [Add extractors](https://github.com/yt-dlp/yt-dlp/commit/eb8fd6d044e8926532772b72be0645c6b8ecb3aa) ([#6309](https://github.com/yt-dlp/yt-dlp/issues/6309)) by [elyse0](https://github.com/elyse0)
|
||||
- lumni
|
||||
- [Add extractor](https://github.com/yt-dlp/yt-dlp/commit/1f8489cccbdc6e96027ef527b88717458f0900e8) ([#6302](https://github.com/yt-dlp/yt-dlp/issues/6302)) by [carusocr](https://github.com/carusocr)
|
||||
- Prankcast
|
||||
- [Fix tags](https://github.com/yt-dlp/yt-dlp/commit/ed4cc4ea793314c50ae3f82e98248c1de1c25694) ([#6316](https://github.com/yt-dlp/yt-dlp/issues/6316)) by [columndeeply](https://github.com/columndeeply)
|
||||
- rutube
|
||||
- [Extract chapters from description](https://github.com/yt-dlp/yt-dlp/commit/22ccd5420b3eb0782776071f12cccd1fedaa1fd0) ([#6345](https://github.com/yt-dlp/yt-dlp/issues/6345)) by [mushbite](https://github.com/mushbite)
|
||||
- SportDeutschland
|
||||
- [Rewrite extractor](https://github.com/yt-dlp/yt-dlp/commit/45db357289b4e1eec09093c8bc5446520378f426) by [pukkandan](https://github.com/pukkandan)
|
||||
- telecaribe
|
||||
- [Add extractor](https://github.com/yt-dlp/yt-dlp/commit/b40471282286bd2b09c485bf79afd271d229272c) ([#6311](https://github.com/yt-dlp/yt-dlp/issues/6311)) by [elyse0](https://github.com/elyse0)
|
||||
- tubetugraz
|
||||
- [Support `--twofactor` (#6424)](https://github.com/yt-dlp/yt-dlp/commit/f44cb4e77bb9be8be291d02ab6f79dc0b4c0d4a1) ([#6427](https://github.com/yt-dlp/yt-dlp/issues/6427)) by [Ferdi265](https://github.com/Ferdi265)
|
||||
- tunein
|
||||
- [Fix extractors](https://github.com/yt-dlp/yt-dlp/commit/46580ced56c90b559885aded6aa8f46f20a9cdce) ([#6310](https://github.com/yt-dlp/yt-dlp/issues/6310)) by [elyse0](https://github.com/elyse0)
|
||||
- twitch
|
||||
- [Update for GraphQL API changes](https://github.com/yt-dlp/yt-dlp/commit/4a6272c6d1bff89969b67cd22b26ebe6d7e72279) ([#6318](https://github.com/yt-dlp/yt-dlp/issues/6318)) by [elyse0](https://github.com/elyse0)
|
||||
- twitter
|
||||
- [Fix retweet extraction](https://github.com/yt-dlp/yt-dlp/commit/cf605226521e99c89fc8dff26a319025810e63a0) ([#6422](https://github.com/yt-dlp/yt-dlp/issues/6422)) by [selfisekai](https://github.com/selfisekai)
|
||||
- xvideos
|
||||
- quickies: [Add extractor](https://github.com/yt-dlp/yt-dlp/commit/283a0b5bc511f3b350eead4488158f50c20ec526) ([#6414](https://github.com/yt-dlp/yt-dlp/issues/6414)) by [Yakabuff](https://github.com/Yakabuff)
|
||||
|
||||
#### Misc. changes
|
||||
- build
|
||||
- [Fix publishing to PyPI and homebrew](https://github.com/yt-dlp/yt-dlp/commit/55676fe498345a389a2539d8baaba958d6d61c3e) by [bashonly](https://github.com/bashonly)
|
||||
- [Only archive if `vars.ARCHIVE_REPO` is set](https://github.com/yt-dlp/yt-dlp/commit/08ff6d59f97b5f5f0128f6bf6fbef56fd836cc52) by [Grub4K](https://github.com/Grub4K)
|
||||
- cleanup
|
||||
- Miscellaneous: [392389b](https://github.com/yt-dlp/yt-dlp/commit/392389b7df7b818f794b231f14dc396d4875fbad) by [pukkandan](https://github.com/pukkandan)
|
||||
- devscripts
|
||||
- `make_changelog`: [Stop at `Release ...` commit](https://github.com/yt-dlp/yt-dlp/commit/7accdd9845fe7ce9d0aa5a9d16faaa489c1294eb) by [pukkandan](https://github.com/pukkandan)
|
||||
|
||||
### 2023.03.03
|
||||
|
||||
#### Important changes
|
||||
- **A new release type has been added!**
|
||||
* [`nightly`](https://github.com/yt-dlp/yt-dlp/releases/tag/nightly) builds will be made after each push, containing the latest fixes (but also possibly bugs).
|
||||
* When using `--update`/`-U`, a release binary will only update to its current channel (either `stable` or `nightly`).
|
||||
* The `--update-to` option has been added allowing the user more control over program upgrades (or downgrades).
|
||||
* `--update-to` can change the release channel (`stable`, `nightly`) and also upgrade or downgrade to specific tags.
|
||||
* **Usage**: `--update-to CHANNEL`, `--update-to TAG`, `--update-to CHANNEL@TAG`
|
||||
- **YouTube throttling fixes!**
|
||||
|
||||
#### Core changes
|
||||
- [Add option `--break-match-filters`](https://github.com/yt-dlp/yt-dlp/commit/fe2ce85aff0aa03735fc0152bb8cb9c3d4ef0753) by [pukkandan](https://github.com/pukkandan)
|
||||
- [Fix `--break-on-existing` with `--lazy-playlist`](https://github.com/yt-dlp/yt-dlp/commit/d21056f4cf0a1623daa107f9181074f5725ac436) by [pukkandan](https://github.com/pukkandan)
|
||||
- dependencies
|
||||
- [Simplify `Cryptodome`](https://github.com/yt-dlp/yt-dlp/commit/65f6e807804d2af5e00f2aecd72bfc43af19324a) by [pukkandan](https://github.com/pukkandan)
|
||||
- jsinterp
|
||||
- [Handle `Date` at epoch 0](https://github.com/yt-dlp/yt-dlp/commit/9acf1ee25f7ad3920ede574a9de95b8c18626af4) by [pukkandan](https://github.com/pukkandan)
|
||||
- plugins
|
||||
- [Don't look in `.egg` directories](https://github.com/yt-dlp/yt-dlp/commit/b059188383eee4fa336ef728dda3ff4bb7335625) by [pukkandan](https://github.com/pukkandan)
|
||||
- update
|
||||
- [Add option `--update-to`, including to nightly](https://github.com/yt-dlp/yt-dlp/commit/77df20f14cc9ed41dfe3a1fe2d77fd27f5365a94) ([#6220](https://github.com/yt-dlp/yt-dlp/issues/6220)) by [bashonly](https://github.com/bashonly), [Grub4K](https://github.com/Grub4K), [pukkandan](https://github.com/pukkandan)
|
||||
- utils
|
||||
- `LenientJSONDecoder`: [Parse unclosed objects](https://github.com/yt-dlp/yt-dlp/commit/cc09083636ce21e58ff74f45eac2dbda507462b0) by [pukkandan](https://github.com/pukkandan)
|
||||
- `Popen`: [Shim undocumented `text_mode` property](https://github.com/yt-dlp/yt-dlp/commit/da8e2912b165005f76779a115a071cd6132ceedf) by [Grub4K](https://github.com/Grub4K)
|
||||
|
||||
#### Extractor changes
|
||||
- [Fix DRM detection in m3u8](https://github.com/yt-dlp/yt-dlp/commit/43a3eaf96393b712d60cbcf5c6cb1e90ed7f42f5) by [pukkandan](https://github.com/pukkandan)
|
||||
- generic
|
||||
- [Detect manifest links via extension](https://github.com/yt-dlp/yt-dlp/commit/b38cae49e6f4849c8ee2a774bdc3c1c647ae5f0e) by [bashonly](https://github.com/bashonly)
|
||||
- [Handle basic-auth when checking redirects](https://github.com/yt-dlp/yt-dlp/commit/8e9fe43cd393e69fa49b3d842aa3180c1d105b8f) by [pukkandan](https://github.com/pukkandan)
|
||||
- GoogleDrive
|
||||
- [Fix some audio](https://github.com/yt-dlp/yt-dlp/commit/4d248e29d20d983ededab0b03d4fe69dff9eb4ed) by [pukkandan](https://github.com/pukkandan)
|
||||
- iprima
|
||||
- [Fix extractor](https://github.com/yt-dlp/yt-dlp/commit/9fddc12ab022a31754e0eaa358fc4e1dfa974587) ([#6291](https://github.com/yt-dlp/yt-dlp/issues/6291)) by [std-move](https://github.com/std-move)
|
||||
- mediastream
|
||||
- [Improve WinSports support](https://github.com/yt-dlp/yt-dlp/commit/2d5a8c5db2bd4ff1c2e45e00cd890a10f8ffca9e) ([#6401](https://github.com/yt-dlp/yt-dlp/issues/6401)) by [bashonly](https://github.com/bashonly)
|
||||
- ntvru
|
||||
- [Extract HLS and DASH formats](https://github.com/yt-dlp/yt-dlp/commit/77d6d136468d0c23c8e79bc937898747804f585a) ([#6403](https://github.com/yt-dlp/yt-dlp/issues/6403)) by [bashonly](https://github.com/bashonly)
|
||||
- tencent
|
||||
- [Add more formats and info](https://github.com/yt-dlp/yt-dlp/commit/18d295c9e0f95adc179eef345b7af64d6372db78) ([#5950](https://github.com/yt-dlp/yt-dlp/issues/5950)) by [Hill-98](https://github.com/Hill-98)
|
||||
- yle_areena
|
||||
- [Extract non-Kaltura videos](https://github.com/yt-dlp/yt-dlp/commit/40d77d89027cd0e0ce31d22aec81db3e1d433900) ([#6402](https://github.com/yt-dlp/yt-dlp/issues/6402)) by [bashonly](https://github.com/bashonly)
|
||||
- youtube
|
||||
- [Construct dash formats with `range` query](https://github.com/yt-dlp/yt-dlp/commit/5038f6d713303e0967d002216e7a88652401c22a) by [pukkandan](https://github.com/pukkandan) (With fixes in [f34804b](https://github.com/yt-dlp/yt-dlp/commit/f34804b2f920f62a6e893a14a9e2a2144b14dd23) by [bashonly](https://github.com/bashonly), [coletdjnz](https://github.com/coletdjnz))
|
||||
- [Detect and break on looping comments](https://github.com/yt-dlp/yt-dlp/commit/7f51861b1820c37b157a239b1fe30628d907c034) ([#6301](https://github.com/yt-dlp/yt-dlp/issues/6301)) by [coletdjnz](https://github.com/coletdjnz)
|
||||
- [Extract channel `view_count` when `/about` tab is passed](https://github.com/yt-dlp/yt-dlp/commit/31e183557fcd1b937582f9429f29207c1261f501) by [pukkandan](https://github.com/pukkandan)
|
||||
|
||||
#### Misc. changes
|
||||
- build
|
||||
- [Add `cffi` as a dependency for `yt_dlp_linux`](https://github.com/yt-dlp/yt-dlp/commit/776d1c3f0c9b00399896dd2e40e78e9a43218109) by [bashonly](https://github.com/bashonly)
|
||||
- [Automated builds and nightly releases](https://github.com/yt-dlp/yt-dlp/commit/29cb20bd563c02671b31dd840139e93dd37150a1) ([#6220](https://github.com/yt-dlp/yt-dlp/issues/6220)) by [bashonly](https://github.com/bashonly), [Grub4K](https://github.com/Grub4K) (With fixes in [bfc861a](https://github.com/yt-dlp/yt-dlp/commit/bfc861a91ee65c9b0ac169754f512e052c6827cf) by [pukkandan](https://github.com/pukkandan))
|
||||
- [Sign SHA files and release public key](https://github.com/yt-dlp/yt-dlp/commit/12647e03d417feaa9ea6a458bea5ebd747494a53) by [Grub4K](https://github.com/Grub4K)
|
||||
- cleanup
|
||||
- [Fix `Changelog`](https://github.com/yt-dlp/yt-dlp/commit/17ca19ab60a6a13eb8a629c51442b5248b0d8394) by [pukkandan](https://github.com/pukkandan)
|
||||
- jsinterp: [Give functions names to help debugging](https://github.com/yt-dlp/yt-dlp/commit/b2e0343ba0fc5d8702e90f6ba2b71358e2677e0b) by [pukkandan](https://github.com/pukkandan)
|
||||
- Miscellaneous: [4815bbf](https://github.com/yt-dlp/yt-dlp/commit/4815bbfc41cf641e4a0650289dbff968cb3bde76), [5b28cef](https://github.com/yt-dlp/yt-dlp/commit/5b28cef72db3b531680d89c121631c73ae05354f) by [pukkandan](https://github.com/pukkandan)
|
||||
- devscripts
|
||||
- [Script to generate changelog](https://github.com/yt-dlp/yt-dlp/commit/d400e261cf029a3f20d364113b14de973be75404) ([#6220](https://github.com/yt-dlp/yt-dlp/issues/6220)) by [Grub4K](https://github.com/Grub4K) (With fixes in [9344964](https://github.com/yt-dlp/yt-dlp/commit/93449642815a6973a4b09b289982ca7e1f961b5f))
|
||||
|
||||
### 2023.02.17
|
||||
|
||||
* Merge youtube-dl: Upto [commit/2dd6c6e](https://github.com/ytdl-org/youtube-dl/commit/2dd6c6e)
|
||||
* Fix `--concat-playlist`
|
||||
* Imply `--no-progress` when `--print`
|
||||
* Improve default subtitle language selection by [sdht0](https://github.com/sdht0)
|
||||
* Make `title` completely non-fatal
|
||||
* Sanitize formats before sorting by [pukkandan](https://github.com/pukkandan)
|
||||
* Support module level `__bool__` and `property`
|
||||
* [dependencies] Standardize `Cryptodome` imports
|
||||
* [hls] Allow extractors to provide AES key by [Grub4K](https://github.com/Grub4K), [bashonly](https://github.com/bashonly)
|
||||
* [ExtractAudio] Handle outtmpl without ext by [carusocr](https://github.com/carusocr)
|
||||
* [extractor/common] Fix `_search_nuxt_data` by [LowSuggestion912](https://github.com/LowSuggestion912)
|
||||
* [extractor/generic] Avoid catastrophic backtracking in KVS regex by [bashonly](https://github.com/bashonly)
|
||||
* [jsinterp] Support `if` statements
|
||||
* [plugins] Fix zip search paths
|
||||
* [utils] `traverse_obj`: Various improvements by [Grub4K](https://github.com/Grub4K)
|
||||
* [utils] `traverse_obj`: Fix more bugs
|
||||
* [utils] `traverse_obj`: Fix several behavioral problems by [Grub4K](https://github.com/Grub4K)
|
||||
* [utils] Don't use Content-length with encoding by [felixonmars](https://github.com/felixonmars)
|
||||
* [utils] Fix `time_seconds` to use the provided TZ by [Grub4K](https://github.com/Grub4K), [Lesmiscore](https://github.com/Lesmiscore)
|
||||
* [utils] Fix race condition in `make_dir` by [aionescu](https://github.com/aionescu)
|
||||
* [utils] Use local kernel32 for file locking on Windows by [Grub4K](https://github.com/Grub4K)
|
||||
* [compat_utils] Improve `passthrough_module`
|
||||
* [compat_utils] Simplify `EnhancedModule`
|
||||
* [build] Update pyinstaller
|
||||
* [pyinst] Fix for pyinstaller 5.8
|
||||
* [devscripts] Provide `pyinstaller` hooks
|
||||
* [devscripts/pyinstaller] Analyze sub-modules of `Cryptodome`
|
||||
* [cleanup] Misc fixes and cleanup
|
||||
* [extractor/anchorfm] Add episode extractor by [HobbyistDev](https://github.com/HobbyistDev), [bashonly](https://github.com/bashonly)
|
||||
* [extractor/boxcast] Add extractor by [HobbyistDev](https://github.com/HobbyistDev)
|
||||
* [extractor/ebay] Add extractor by [JChris246](https://github.com/JChris246)
|
||||
* [extractor/hypergryph] Add extractor by [HobbyistDev](https://github.com/HobbyistDev), [bashonly](https://github.com/bashonly)
|
||||
* [extractor/NZOnScreen] Add extractor by [gregsadetsky](https://github.com/gregsadetsky), [pukkandan](https://github.com/pukkandan)
|
||||
* [extractor/rozhlas] Add extractor RozhlasVltavaIE by [amra](https://github.com/amra)
|
||||
* [extractor/tempo] Add IVXPlayer extractor by [HobbyistDev](https://github.com/HobbyistDev)
|
||||
* [extractor/txxx] Add extractors by [chio0hai](https://github.com/chio0hai)
|
||||
* [extractor/vocaroo] Add extractor by [SuperSonicHub1](https://github.com/SuperSonicHub1), [qbnu](https://github.com/qbnu)
|
||||
* [extractor/wrestleuniverse] Add extractors by [Grub4K](https://github.com/Grub4K), [bashonly](https://github.com/bashonly)
|
||||
* [extractor/yappy] Add extractor by [HobbyistDev](https://github.com/HobbyistDev), [dirkf](https://github.com/dirkf)
|
||||
* [extractor/youtube] **Fix `uploader_id` extraction** by [bashonly](https://github.com/bashonly)
|
||||
* [extractor/youtube] Add hyperpipe instances by [Generator](https://github.com/Generator)
|
||||
* [extractor/youtube] Handle `consent.youtube`
|
||||
* [extractor/youtube] Support `/live/` URL
|
||||
* [extractor/youtube] Update invidious and piped instances by [rohieb](https://github.com/rohieb)
|
||||
* [extractor/91porn] Fix title and comment extraction by [pmitchell86](https://github.com/pmitchell86)
|
||||
* [extractor/AbemaTV] Cache user token whenever appropriate by [Lesmiscore](https://github.com/Lesmiscore)
|
||||
* [extractor/bfmtv] Support `rmc` prefix by [carusocr](https://github.com/carusocr)
|
||||
* [extractor/biliintl] Add intro and ending chapters by [HobbyistDev](https://github.com/HobbyistDev)
|
||||
* [extractor/clyp] Support `wav` by [qulaz](https://github.com/qulaz)
|
||||
* [extractor/crunchyroll] Add intro chapter by [ByteDream](https://github.com/ByteDream)
|
||||
* [extractor/crunchyroll] Better message for premium videos
|
||||
* [extractor/crunchyroll] Fix incorrect premium-only error by [Grub4K](https://github.com/Grub4K)
|
||||
* [extractor/DouyuTV] Use new API by [hatienl0i261299](https://github.com/hatienl0i261299)
|
||||
* [extractor/embedly] Embedded links may be for other extractors
|
||||
* [extractor/freesound] Workaround invalid URL in webpage by [rebane2001](https://github.com/rebane2001)
|
||||
* [extractor/GoPlay] Use new API by [jeroenj](https://github.com/jeroenj)
|
||||
* [extractor/Hidive] Fix subtitles and age-restriction by [chexxor](https://github.com/chexxor)
|
||||
* [extractor/huya] Support HD streams by [felixonmars](https://github.com/felixonmars)
|
||||
* [extractor/moviepilot] Fix extractor by [panatexxa](https://github.com/panatexxa)
|
||||
* [extractor/nbc] Fix `NBC` and `NBCStations` extractors by [bashonly](https://github.com/bashonly)
|
||||
* [extractor/nbc] Fix XML parsing by [bashonly](https://github.com/bashonly)
|
||||
* [extractor/nebula] Remove broken cookie support by [hheimbuerger](https://github.com/hheimbuerger)
|
||||
* [extractor/nfl] Add `NFLPlus` extractors by [bashonly](https://github.com/bashonly)
|
||||
* [extractor/niconico] Add support for like history by [Matumo](https://github.com/Matumo), [pukkandan](https://github.com/pukkandan)
|
||||
* [extractor/nitter] Update instance list by [OIRNOIR](https://github.com/OIRNOIR)
|
||||
* [extractor/npo] Fix extractor and add HD support by [seproDev](https://github.com/seproDev)
|
||||
* [extractor/odkmedia] Add `OnDemandChinaEpisodeIE` by [HobbyistDev](https://github.com/HobbyistDev), [pukkandan](https://github.com/pukkandan)
|
||||
* [extractor/pornez] Handle relative URLs in iframe by [JChris246](https://github.com/JChris246)
|
||||
* [extractor/radiko] Fix format sorting for Time Free by [road-master](https://github.com/road-master)
|
||||
* [extractor/rcs] Fix extractors by [nixxo](https://github.com/nixxo), [pukkandan](https://github.com/pukkandan)
|
||||
* [extractor/reddit] Support user posts by [OMEGARAZER](https://github.com/OMEGARAZER)
|
||||
* [extractor/rumble] Fix format sorting by [pukkandan](https://github.com/pukkandan)
|
||||
* [extractor/servus] Rewrite extractor by [Ashish0804](https://github.com/Ashish0804), [FrankZ85](https://github.com/FrankZ85), [StefanLobbenmeier](https://github.com/StefanLobbenmeier)
|
||||
* [extractor/slideslive] Fix slides and chapters/duration by [bashonly](https://github.com/bashonly)
|
||||
* [extractor/SportDeutschland] Fix extractor by [FriedrichRehren](https://github.com/FriedrichRehren)
|
||||
* [extractor/Stripchat] Fix extractor by [JChris246](https://github.com/JChris246), [bashonly](https://github.com/bashonly)
|
||||
* [extractor/tnaflix] Fix extractor by [bashonly](https://github.com/bashonly), [oxamun](https://github.com/oxamun)
|
||||
* [extractor/tvp] Support `stream.tvp.pl` by [selfisekai](https://github.com/selfisekai)
|
||||
* [extractor/twitter] Fix `--no-playlist` and add media `view_count` when using GraphQL by [Grub4K](https://github.com/Grub4K)
|
||||
* [extractor/twitter] Fix graphql extraction on some tweets by [selfisekai](https://github.com/selfisekai)
|
||||
* [extractor/vimeo] Fix `playerConfig` extraction by [LeoniePhiline](https://github.com/LeoniePhiline), [bashonly](https://github.com/bashonly)
|
||||
* [extractor/viu] Add `ViuOTTIndonesiaIE` extractor by [HobbyistDev](https://github.com/HobbyistDev)
|
||||
* [extractor/vk] Fix playlists for new API by [the-marenga](https://github.com/the-marenga)
|
||||
* [extractor/vlive] Replace with `VLiveWebArchiveIE` by [seproDev](https://github.com/seproDev)
|
||||
* [extractor/ximalaya] Update album `_VALID_URL` by [carusocr](https://github.com/carusocr)
|
||||
* [extractor/zdf] Use android API endpoint for UHD downloads by [seproDev](https://github.com/seproDev)
|
||||
* [extractor/drtv] Fix bug in [ab4cbef](https://github.com/yt-dlp/yt-dlp/commit/ab4cbef) by [bashonly](https://github.com/bashonly)
|
||||
|
||||
|
||||
### 2023.01.06
|
||||
|
||||
* Fix config locations by [Grub4K](https://github.com/Grub4K), [coletdjnz](https://github.com/coletdjnz), [pukkandan](https://github.com/pukkandan)
|
||||
* [downloader/aria2c] Disable native progress
|
||||
* [utils] `mimetype2ext`: `weba` is not standard
|
||||
* [utils] `windows_enable_vt_mode`: Better error handling
|
||||
* [build] Add minimal `pyproject.toml`
|
||||
* [update] Fix updater file removal on windows by [Grub4K](https://github.com/Grub4K)
|
||||
* [cleanup] Misc fixes and cleanup
|
||||
* [extractor/aitube] Add extractor by [HobbyistDev](https://github.com/HobbyistDev)
|
||||
* [extractor/drtv] Add series extractors by [FrederikNS](https://github.com/FrederikNS)
|
||||
* [extractor/volejtv] Add extractor by [HobbyistDev](https://github.com/HobbyistDev)
|
||||
* [extractor/xanimu] Add extractor by [JChris246](https://github.com/JChris246)
|
||||
* [extractor/youtube] Retry manifest refresh for live-from-start by [mzhou](https://github.com/mzhou)
|
||||
* [extractor/biliintl] Add `/media` to `VALID_URL` by [HobbyistDev](https://github.com/HobbyistDev)
|
||||
* [extractor/biliIntl] Add fallback to `video_data` by [HobbyistDev](https://github.com/HobbyistDev)
|
||||
* [extractor/crunchyroll:show] Add `language` to entries by [Chrissi2812](https://github.com/Chrissi2812)
|
||||
* [extractor/joj] Fix extractor by [OndrejBakan](https://github.com/OndrejBakan), [pukkandan](https://github.com/pukkandan)
|
||||
* [extractor/nbc] Update graphql query by [jacobtruman](https://github.com/jacobtruman)
|
||||
* [extractor/reddit] Add subreddit as `channel_id` by [gschizas](https://github.com/gschizas)
|
||||
* [extractor/tiktok] Add `TikTokLive` extractor by [JC-Chung](https://github.com/JC-Chung)
|
||||
|
||||
### 2023.01.02
|
||||
|
||||
* **Improve plugin architecture** by [Grub4K](https://github.com/Grub4K), [coletdjnz](https://github.com/coletdjnz), [flashdagger](https://github.com/flashdagger), [pukkandan](https://github.com/pukkandan)
|
||||
* Plugins can be loaded in any distribution of yt-dlp (binary, pip, source, etc.) and can be distributed and installed as packages. See [the readme](https://github.com/yt-dlp/yt-dlp/tree/05997b6e98e638d97d409c65bb5eb86da68f3b64#plugins) for more information
|
||||
* Add `--compat-options 2021,2022`
|
||||
* This allows devs to change defaults and make other potentially breaking changes more easily. If you need everything to work exactly as-is, put Use `--compat 2022` in your config to guard against future compat changes.
|
||||
* [downloader/aria2c] Native progress for aria2c via RPC by [Lesmiscore](https://github.com/Lesmiscore), [pukkandan](https://github.com/pukkandan)
|
||||
* Merge youtube-dl: Upto [commit/195f22f](https://github.com/ytdl-org/youtube-dl/commit/195f22f6) by [Grub4K](https://github.com/Grub4K), [pukkandan](https://github.com/pukkandan)
|
||||
* Add pre-processor stage `video`
|
||||
* Let `--parse/replace-in-metadata` run at any post-processing stage
|
||||
* Add `--enable-file-urls` by [coletdjnz](https://github.com/coletdjnz)
|
||||
* Add new field `aspect_ratio`
|
||||
* Add `ac4` to known codecs
|
||||
* Add `weba` to known extensions
|
||||
* [FFmpegVideoConvertor] Add `gif` to `--recode-video`
|
||||
* Add message when there are no subtitles/thumbnails
|
||||
* Deprioritize HEVC-over-FLV formats by [Lesmiscore](https://github.com/Lesmiscore)
|
||||
* Make early reject of `--match-filter` stricter
|
||||
* Fix `--cookies-from-browser` CLI parsing
|
||||
* Fix `original_url` in playlists
|
||||
* Fix bug in writing playlist info-json
|
||||
* Fix bugs in `PlaylistEntries`
|
||||
* [downloader/ffmpeg] Fix headers for video+audio formats by [Grub4K](https://github.com/Grub4K), [bashonly](https://github.com/bashonly)
|
||||
* [extractor] Add a way to distinguish IEs that returns only videos
|
||||
* [extractor] Implement universal format sorting and deprecate `_sort_formats`
|
||||
* [extractor] Let `_extract_format` functions obey `--ignore-no-formats`
|
||||
* [extractor/generic] Add `fragment_query` extractor arg for DASH and HLS by [bashonly](https://github.com/bashonly), [pukkandan](https://github.com/pukkandan)
|
||||
* [extractor/generic] Decode unicode-escaped embed URLs by [bashonly](https://github.com/bashonly)
|
||||
* [extractor/generic] Don't report redirect to https
|
||||
* [extractor/generic] Fix JSON LD manifest extraction by [bashonly](https://github.com/bashonly), [pukkandan](https://github.com/pukkandan)
|
||||
* [extractor/generic] Use `Accept-Encoding: identity` for initial request by [coletdjnz](https://github.com/coletdjnz)
|
||||
* [FormatSort] Add `mov` to `vext`
|
||||
* [jsinterp] Escape regex that looks like nested set
|
||||
* [webvtt] Handle premature EOF by [flashdagger](https://github.com/flashdagger)
|
||||
* [utils] `classproperty`: Add cache support
|
||||
* [utils] `get_exe_version`: Detect broken executables by [dirkf](https://github.com/dirkf), [pukkandan](https://github.com/pukkandan)
|
||||
* [utils] `js_to_json`: Fix bug in [f55523c](https://github.com/yt-dlp/yt-dlp/commit/f55523c) by [ChillingPepper](https://github.com/ChillingPepper), [pukkandan](https://github.com/pukkandan)
|
||||
* [utils] Make `ExtractorError` mutable
|
||||
* [utils] Move `FileDownloader.parse_bytes` into utils
|
||||
* [utils] Move format sorting code into `utils`
|
||||
* [utils] `windows_enable_vt_mode`: Proper implementation by [Grub4K](https://github.com/Grub4K)
|
||||
* [update] Workaround [#5632](https://github.com/yt-dlp/yt-dlp/issues/5632)
|
||||
* [docs] Improvements
|
||||
* [cleanup] Misc fixes and cleanup
|
||||
* [cleanup] Use `random.choices` by [freezboltz](https://github.com/freezboltz)
|
||||
* [extractor/airtv] Add extractor by [HobbyistDev](https://github.com/HobbyistDev)
|
||||
* [extractor/amazonminitv] Add extractors by [GautamMKGarg](https://github.com/GautamMKGarg), [nyuszika7h](https://github.com/nyuszika7h)
|
||||
* [extractor/beatbump] Add extractors by [Bobscorn](https://github.com/Bobscorn), [pukkandan](https://github.com/pukkandan)
|
||||
* [extractor/europarl] Add EuroParlWebstream extractor by [HobbyistDev](https://github.com/HobbyistDev)
|
||||
* [extractor/kanal2] Add extractor by [bashonly](https://github.com/bashonly), [glensc](https://github.com/glensc), [pukkandan](https://github.com/pukkandan)
|
||||
* [extractor/kankanews] Add extractor by [synthpop123](https://github.com/synthpop123)
|
||||
* [extractor/kick] Add extractor by [bashonly](https://github.com/bashonly)
|
||||
* [extractor/mediastream] Add extractor by [HobbyistDev](https://github.com/HobbyistDev), [elyse0](https://github.com/elyse0)
|
||||
* [extractor/noice] Add NoicePodcast extractor by [HobbyistDev](https://github.com/HobbyistDev)
|
||||
* [extractor/oneplace] Add OnePlacePodcast extractor by [HobbyistDev](https://github.com/HobbyistDev)
|
||||
* [extractor/rumble] Add RumbleIE extractor by [flashdagger](https://github.com/flashdagger)
|
||||
* [extractor/screencastify] Add extractor by [bashonly](https://github.com/bashonly)
|
||||
* [extractor/trtcocuk] Add extractor by [HobbyistDev](https://github.com/HobbyistDev)
|
||||
* [extractor/Veoh] Add user extractor by [tntmod54321](https://github.com/tntmod54321)
|
||||
* [extractor/videoken] Add extractors by [bashonly](https://github.com/bashonly)
|
||||
* [extractor/webcamerapl] Add extractor by [milkknife](https://github.com/milkknife)
|
||||
* [extractor/amazon] Add `AmazonReviews` extractor by [bashonly](https://github.com/bashonly)
|
||||
* [extractor/netverse] Add `NetverseSearch` extractor by [HobbyistDev](https://github.com/HobbyistDev)
|
||||
* [extractor/vimeo] Add `VimeoProIE` by [bashonly](https://github.com/bashonly), [pukkandan](https://github.com/pukkandan)
|
||||
* [extractor/xiami] Remove extractors by [synthpop123](https://github.com/synthpop123)
|
||||
* [extractor/youtube] Add `piped.video` by [Bnyro](https://github.com/Bnyro)
|
||||
* [extractor/youtube] Consider language in format de-duplication
|
||||
* [extractor/youtube] Extract DRC formats
|
||||
* [extractor/youtube] Fix `ytuser:`
|
||||
* [extractor/youtube] Fix bug in handling of music URLs
|
||||
* [extractor/youtube] Subtitles cannot be translated to `und`
|
||||
* [extractor/youtube:tab] Extract metadata from channel items by [coletdjnz](https://github.com/coletdjnz)
|
||||
* [extractor/ARD] Add vtt subtitles by [CapacitorSet](https://github.com/CapacitorSet)
|
||||
* [extractor/ArteTV] Extract chapters by [bashonly](https://github.com/bashonly), [iw0nderhow](https://github.com/iw0nderhow)
|
||||
* [extractor/bandcamp] Add `album_artist` by [stelcodes](https://github.com/stelcodes)
|
||||
* [extractor/bilibili] Fix `--no-playlist` for anthology
|
||||
* [extractor/bilibili] Improve `_VALID_URL` by [skbeh](https://github.com/skbeh)
|
||||
* [extractor/biliintl:series] Make partial download of series faster
|
||||
* [extractor/BiliLive] Fix extractor
|
||||
* [extractor/brightcove] Add `BrightcoveNewBaseIE` and fix embed extraction
|
||||
* [extractor/cda] Support premium and misc improvements by [selfisekai](https://github.com/selfisekai)
|
||||
* [extractor/ciscowebex] Support password-protected videos by [damianoamatruda](https://github.com/damianoamatruda)
|
||||
* [extractor/curiositystream] Fix auth by [mnn](https://github.com/mnn)
|
||||
* [extractor/embedly] Handle vimeo embeds
|
||||
* [extractor/fifa] Fix Preplay extraction by [dirkf](https://github.com/dirkf)
|
||||
* [extractor/foxsports] Fix extractor by [bashonly](https://github.com/bashonly)
|
||||
* [extractor/gronkh] Fix `_VALID_URL` by [muddi900](https://github.com/muddi900)
|
||||
* [extractor/hotstar] Improve format metadata
|
||||
* [extractor/iqiyi] Fix `Iq` JS regex by [bashonly](https://github.com/bashonly)
|
||||
* [extractor/la7] Improve extractor by [nixxo](https://github.com/nixxo)
|
||||
* [extractor/mediaset] Better embed detection and error messages by [nixxo](https://github.com/nixxo)
|
||||
* [extractor/mixch] Support `--wait-for-video`
|
||||
* [extractor/naver] Improve `_VALID_URL` for `NaverNowIE` by [bashonly](https://github.com/bashonly)
|
||||
* [extractor/naver] Treat fan subtitles as separate language
|
||||
* [extractor/netverse] Extract comments by [HobbyistDev](https://github.com/HobbyistDev)
|
||||
* [extractor/nosnl] Add support for /video by [HobbyistDev](https://github.com/HobbyistDev)
|
||||
* [extractor/odnoklassniki] Extract subtitles by [bashonly](https://github.com/bashonly)
|
||||
* [extractor/pinterest] Fix extractor by [bashonly](https://github.com/bashonly)
|
||||
* [extractor/plutotv] Fix videos with non-zero start by [digitall](https://github.com/digitall)
|
||||
* [extractor/polskieradio] Adapt to next.js redesigns by [selfisekai](https://github.com/selfisekai)
|
||||
* [extractor/reddit] Add vcodec to fallback format by [chengzhicn](https://github.com/chengzhicn)
|
||||
* [extractor/reddit] Extract crossposted media by [bashonly](https://github.com/bashonly)
|
||||
* [extractor/reddit] Extract video embeds in text posts by [bashonly](https://github.com/bashonly)
|
||||
* [extractor/rutube] Support private videos by [mexus](https://github.com/mexus)
|
||||
* [extractor/sibnet] Separate from VKIE
|
||||
* [extractor/slideslive] Fix extractor by [Grub4K](https://github.com/Grub4K), [bashonly](https://github.com/bashonly)
|
||||
* [extractor/slideslive] Support embeds and slides by [Grub4K](https://github.com/Grub4K), [bashonly](https://github.com/bashonly), [pukkandan](https://github.com/pukkandan)
|
||||
* [extractor/soundcloud] Support user permalink by [nosoop](https://github.com/nosoop)
|
||||
* [extractor/spankbang] Fix extractor by [JChris246](https://github.com/JChris246)
|
||||
* [extractor/stv] Detect DRM
|
||||
* [extractor/swearnet] Fix description bug
|
||||
* [extractor/tencent] Fix geo-restricted video by [elyse0](https://github.com/elyse0)
|
||||
* [extractor/tiktok] Fix subs, `DouyinIE`, improve `_VALID_URL` by [bashonly](https://github.com/bashonly)
|
||||
* [extractor/tiktok] Update `_VALID_URL`, add `api_hostname` arg by [bashonly](https://github.com/bashonly)
|
||||
* [extractor/tiktok] Update API hostname by [redraskal](https://github.com/redraskal)
|
||||
* [extractor/twitcasting] Fix videos with password by [Spicadox](https://github.com/Spicadox), [bashonly](https://github.com/bashonly)
|
||||
* [extractor/twitter] Heed `--no-playlist` for multi-video tweets by [Grub4K](https://github.com/Grub4K), [bashonly](https://github.com/bashonly)
|
||||
* [extractor/twitter] Refresh guest token when expired by [Grub4K](https://github.com/Grub4K), [bashonly](https://github.com/bashonly)
|
||||
* [extractor/twitter:spaces] Add `Referer` to m3u8 by [nixxo](https://github.com/nixxo)
|
||||
* [extractor/udemy] Fix lectures that have no URL and detect DRM
|
||||
* [extractor/unsupported] Add more URLs
|
||||
* [extractor/urplay] Support for audio-only formats by [barsnick](https://github.com/barsnick)
|
||||
* [extractor/wistia] Improve extension detection by [Grub4K](https://github.com/Grub4K), [bashonly](https://github.com/bashonly), [pukkandan](https://github.com/pukkandan)
|
||||
* [extractor/yle_areena] Support restricted videos by [docbender](https://github.com/docbender)
|
||||
* [extractor/youku] Fix extractor by [KurtBestor](https://github.com/KurtBestor)
|
||||
* [extractor/youporn] Fix metadata by [marieell](https://github.com/marieell)
|
||||
* [extractor/redgifs] Fix bug in [8c188d5](https://github.com/yt-dlp/yt-dlp/commit/8c188d5d09177ed213a05c900d3523867c5897fd)
|
||||
|
||||
|
||||
### 2022.11.11
|
||||
|
||||
* Merge youtube-dl: Upto [commit/de39d12](https://github.com/ytdl-org/youtube-dl/commit/de39d128)
|
||||
* Backport SSL configuration from Python 3.10 by [coletdjnz](https://github.com/coletdjnz)
|
||||
* Do more processing in `--flat-playlist`
|
||||
* Fix `--list` options not implying `-s` in some cases by [Grub4K](https://github.com/Grub4K), [bashonly](https://github.com/bashonly)
|
||||
* Fix end time of clips by [cruel-efficiency](https://github.com/cruel-efficiency)
|
||||
* Fix for `formats=None`
|
||||
* Write API params in debug head
|
||||
* [outtmpl] Ensure ASCII in json and add option for Unicode
|
||||
* [SponsorBlock] Add `type` field, obey `--retry-sleep extractor`, relax duration check for large segments
|
||||
* [SponsorBlock] **Support `chapter` category** by [ajayyy](https://github.com/ajayyy), [pukkandan](https://github.com/pukkandan)
|
||||
* [ThumbnailsConvertor] Fix filename escaping by [dirkf](https://github.com/dirkf), [pukkandan](https://github.com/pukkandan)
|
||||
* [ModifyChapters] Handle the entire video being marked for removal
|
||||
* [embedthumbnail] Fix thumbnail name in mp3 by [How-Bout-No](https://github.com/How-Bout-No)
|
||||
* [downloader/fragment] HLS download can continue without first fragment
|
||||
* [cookies] Improve `LenientSimpleCookie` by [Grub4K](https://github.com/Grub4K)
|
||||
* [jsinterp] Improve separating regex
|
||||
* [extractor/common] Fix `fatal=False` for `_search_nuxt_data`
|
||||
* [extractor/common] Improve `_generic_title`
|
||||
* [extractor/common] Fix `json_ld` type checks by [Grub4K](https://github.com/Grub4K)
|
||||
* [extractor/generic] Separate embed extraction into own function
|
||||
* [extractor/generic:quoted-html] Add extractor by [coletdjnz](https://github.com/coletdjnz), [pukkandan](https://github.com/pukkandan)
|
||||
* [extractor/unsupported] Raise error on known DRM-only sites by [coletdjnz](https://github.com/coletdjnz)
|
||||
* [utils] `js_to_json`: Improve escape handling by [Grub4K](https://github.com/Grub4K)
|
||||
* [utils] `strftime_or_none`: Workaround Python bug on Windows
|
||||
* [utils] `traverse_obj`: Always return list when branching, allow `re.Match` objects by [Grub4K](https://github.com/Grub4K)
|
||||
* [build, test] Harden workflows' security by [sashashura](https://github.com/sashashura)
|
||||
* [build] `py2exe`: Migrate to freeze API by [SG5](https://github.com/SG5), [pukkandan](https://github.com/pukkandan)
|
||||
* [build] Create `armv7l` and `aarch64` releases by [MrOctopus](https://github.com/MrOctopus), [pukkandan](https://github.com/pukkandan)
|
||||
* [build] Make linux binary truly standalone using `conda` by [mlampe](https://github.com/mlampe)
|
||||
* [build] Replace `set-output` with `GITHUB_OUTPUT` by [Lesmiscore](https://github.com/Lesmiscore)
|
||||
* [update] Use error code `100` for update errors
|
||||
* [compat] Fix `shutils.move` in restricted ACL mode on BSD by [ClosedPort22](https://github.com/ClosedPort22), [pukkandan](https://github.com/pukkandan)
|
||||
* [docs, devscripts] Document `pyinst`'s argument passthrough by [jahway603](https://github.com/jahway603)
|
||||
* [test] Allow `extract_flat` in download tests by [coletdjnz](https://github.com/coletdjnz), [pukkandan](https://github.com/pukkandan)
|
||||
* [cleanup] Misc fixes and cleanup by [pukkandan](https://github.com/pukkandan), [Alienmaster](https://github.com/Alienmaster)
|
||||
* [extractor/aeon] Add extractor by [DoubleCouponDay](https://github.com/DoubleCouponDay)
|
||||
* [extractor/agora] Add extractors by [selfisekai](https://github.com/selfisekai)
|
||||
* [extractor/camsoda] Add extractor by [zulaport](https://github.com/zulaport)
|
||||
* [extractor/cinetecamilano] Add extractor by [timendum](https://github.com/timendum)
|
||||
* [extractor/deuxm] Add extractors by [CrankDatSouljaBoy](https://github.com/CrankDatSouljaBoy)
|
||||
* [extractor/genius] Add extractors by [bashonly](https://github.com/bashonly)
|
||||
* [extractor/japandiet] Add extractors by [Lesmiscore](https://github.com/Lesmiscore)
|
||||
* [extractor/listennotes] Add extractor by [lksj](https://github.com/lksj), [pukkandan](https://github.com/pukkandan)
|
||||
* [extractor/nos.nl] Add extractor by [HobbyistDev](https://github.com/HobbyistDev)
|
||||
* [extractor/oftv] Add extractors by [DoubleCouponDay](https://github.com/DoubleCouponDay)
|
||||
* [extractor/podbayfm] Add extractor by [schnusch](https://github.com/schnusch)
|
||||
* [extractor/qingting] Add extractor by [bashonly](https://github.com/bashonly), [changren-wcr](https://github.com/changren-wcr)
|
||||
* [extractor/screen9] Add extractor by [tpikonen](https://github.com/tpikonen)
|
||||
* [extractor/swearnet] Add extractor by [HobbyistDev](https://github.com/HobbyistDev)
|
||||
* [extractor/YleAreena] Add extractor by [pukkandan](https://github.com/pukkandan), [vitkhab](https://github.com/vitkhab)
|
||||
* [extractor/zeenews] Add extractor by [m4tu4g](https://github.com/m4tu4g), [pukkandan](https://github.com/pukkandan)
|
||||
* [extractor/youtube:tab] **Update tab handling for redesign** by [coletdjnz](https://github.com/coletdjnz), [pukkandan](https://github.com/pukkandan)
|
||||
* Channel URLs download all uploads of the channel as multiple playlists, separated by tab
|
||||
* [extractor/youtube] Differentiate between no comments and disabled comments by [coletdjnz](https://github.com/coletdjnz)
|
||||
* [extractor/youtube] Extract `concurrent_view_count` for livestreams by [coletdjnz](https://github.com/coletdjnz)
|
||||
* [extractor/youtube] Fix `duration` for premieres by [nosoop](https://github.com/nosoop)
|
||||
* [extractor/youtube] Fix `live_status` by [coletdjnz](https://github.com/coletdjnz), [pukkandan](https://github.com/pukkandan)
|
||||
* [extractor/youtube] Ignore incomplete data error for comment replies by [coletdjnz](https://github.com/coletdjnz)
|
||||
* [extractor/youtube] Improve chapter parsing from description
|
||||
* [extractor/youtube] Mark videos as fully watched by [bsun0000](https://github.com/bsun0000)
|
||||
* [extractor/youtube] Update piped instances by [Generator](https://github.com/Generator)
|
||||
* [extractor/youtube] Update playlist metadata extraction for new layout by [coletdjnz](https://github.com/coletdjnz)
|
||||
* [extractor/youtube:tab] Fix video metadata from tabs by [coletdjnz](https://github.com/coletdjnz)
|
||||
* [extractor/youtube:tab] Let `approximate_date` return timestamp
|
||||
* [extractor/americastestkitchen] Fix extractor by [bashonly](https://github.com/bashonly)
|
||||
* [extractor/bbc] Support onion domains by [DoubleCouponDay](https://github.com/DoubleCouponDay)
|
||||
* [extractor/bilibili] Add chapters and misc cleanup by [lockmatrix](https://github.com/lockmatrix), [pukkandan](https://github.com/pukkandan)
|
||||
* [extractor/bilibili] Fix BilibiliIE and Bangumi extractors by [lockmatrix](https://github.com/lockmatrix), [pukkandan](https://github.com/pukkandan)
|
||||
* [extractor/bitchute] Better error for geo-restricted videos by [flashdagger](https://github.com/flashdagger)
|
||||
* [extractor/bitchute] Improve `BitChuteChannelIE` by [flashdagger](https://github.com/flashdagger), [pukkandan](https://github.com/pukkandan)
|
||||
* [extractor/bitchute] Simplify extractor by [flashdagger](https://github.com/flashdagger), [pukkandan](https://github.com/pukkandan)
|
||||
* [extractor/cda] Support login through API by [selfisekai](https://github.com/selfisekai)
|
||||
* [extractor/crunchyroll] Beta is now the only layout by [tejing1](https://github.com/tejing1)
|
||||
* [extractor/detik] Avoid unnecessary extraction
|
||||
* [extractor/doodstream] Remove extractor
|
||||
* [extractor/dplay] Add MotorTrendOnDemand extractor by [bashonly](https://github.com/bashonly)
|
||||
* [extractor/epoch] Support videos without data-trailer by [gibson042](https://github.com/gibson042), [pukkandan](https://github.com/pukkandan)
|
||||
* [extractor/fox] Extract thumbnail by [vitkhab](https://github.com/vitkhab)
|
||||
* [extractor/foxnews] Add `FoxNewsVideo` extractor
|
||||
* [extractor/hotstar] Add season support by [m4tu4g](https://github.com/m4tu4g)
|
||||
* [extractor/hotstar] Refactor v1 API calls
|
||||
* [extractor/iprima] Make json+ld non-fatal by [bashonly](https://github.com/bashonly)
|
||||
* [extractor/iq] Increase phantomjs timeout
|
||||
* [extractor/kaltura] Support playlists by [jwoglom](https://github.com/jwoglom), [pukkandan](https://github.com/pukkandan)
|
||||
* [extractor/lbry] Authenticate with cookies by [flashdagger](https://github.com/flashdagger)
|
||||
* [extractor/livestreamfails] Support posts by [invertico](https://github.com/invertico)
|
||||
* [extractor/mlb] Add `MLBArticle` extractor by [HobbyistDev](https://github.com/HobbyistDev)
|
||||
* [extractor/mxplayer] Improve extractor by [m4tu4g](https://github.com/m4tu4g)
|
||||
* [extractor/niconico] Always use HTTPS for requests
|
||||
* [extractor/nzherald] Support new video embed by [coletdjnz](https://github.com/coletdjnz)
|
||||
* [extractor/odnoklassniki] Support boosty.to embeds by [Lesmiscore](https://github.com/Lesmiscore), [megapro17](https://github.com/megapro17), [pukkandan](https://github.com/pukkandan)
|
||||
* [extractor/paramountplus] Update API token by [bashonly](https://github.com/bashonly)
|
||||
* [extractor/reddit] Add fallback format by [bashonly](https://github.com/bashonly)
|
||||
* [extractor/redgifs] Fix extractors by [bashonly](https://github.com/bashonly), [pukkandan](https://github.com/pukkandan)
|
||||
* [extractor/redgifs] Refresh auth token for 401 by [endotronic](https://github.com/endotronic), [pukkandan](https://github.com/pukkandan)
|
||||
* [extractor/rumble] Add HLS formats and extract more metadata by [flashdagger](https://github.com/flashdagger)
|
||||
* [extractor/sbs] Improve `_VALID_URL` by [bashonly](https://github.com/bashonly)
|
||||
* [extractor/skyit] Fix extractors by [nixxo](https://github.com/nixxo)
|
||||
* [extractor/stripchat] Fix hostname for HLS stream by [zulaport](https://github.com/zulaport)
|
||||
* [extractor/stripchat] Improve error message by [freezboltz](https://github.com/freezboltz)
|
||||
* [extractor/telegram] Add playlist support and more metadata by [bashonly](https://github.com/bashonly), [bsun0000](https://github.com/bsun0000)
|
||||
* [extractor/Tnaflix] Fix for HTTP 500 by [SG5](https://github.com/SG5), [pukkandan](https://github.com/pukkandan)
|
||||
* [extractor/tubitv] Better DRM detection by [bashonly](https://github.com/bashonly)
|
||||
* [extractor/tvp] Update extractors by [selfisekai](https://github.com/selfisekai)
|
||||
* [extractor/twitcasting] Fix `data-movie-playlist` extraction by [Lesmiscore](https://github.com/Lesmiscore)
|
||||
* [extractor/twitter] Add onion site to `_VALID_URL` by [DoubleCouponDay](https://github.com/DoubleCouponDay)
|
||||
* [extractor/twitter] Add Spaces extractor and GraphQL API by [Grub4K](https://github.com/Grub4K), [bashonly](https://github.com/bashonly), [nixxo](https://github.com/nixxo), [pukkandan](https://github.com/pukkandan)
|
||||
* [extractor/twitter] Support multi-video posts by [Grub4K](https://github.com/Grub4K)
|
||||
* [extractor/uktvplay] Fix `_VALID_URL`
|
||||
* [extractor/viu] Support subtitles of on-screen text by [tkgmomosheep](https://github.com/tkgmomosheep)
|
||||
* [extractor/VK] Fix playlist URLs by [the-marenga](https://github.com/the-marenga)
|
||||
* [extractor/vlive] Extract `release_timestamp`
|
||||
* [extractor/voot] Improve `_VALID_URL` by [freezboltz](https://github.com/freezboltz)
|
||||
* [extractor/wordpress:mb.miniAudioPlayer] Add embed extractor by [coletdjnz](https://github.com/coletdjnz)
|
||||
* [extractor/YoutubeWebArchive] Improve metadata extraction by [coletdjnz](https://github.com/coletdjnz)
|
||||
* [extractor/zee5] Improve `_VALID_URL` by [m4tu4g](https://github.com/m4tu4g)
|
||||
* [extractor/zenyandex] Fix extractors by [lksj](https://github.com/lksj), [puc9](https://github.com/puc9), [pukkandan](https://github.com/pukkandan)
|
||||
|
||||
|
||||
### 2022.10.04
|
||||
|
||||
* Allow a `set` to be passed as `download_archive` by [pukkandan](https://github.com/pukkandan), [bashonly](https://github.com/bashonly)
|
||||
* Allow open ranges for time ranges by [Lesmiscore](https://github.com/Lesmiscore)
|
||||
* Allow plugin extractors to replace the built-in ones
|
||||
* Don't download entire video when no matching `--download-sections`
|
||||
* Fix `--config-location -`
|
||||
* Improve [5736d79](https://github.com/yt-dlp/yt-dlp/pull/5044/commits/5736d79172c47ff84740d5720467370a560febad)
|
||||
* Fix for when playlists don't have `webpage_url`
|
||||
* Support environment variables in `--ffmpeg-location`
|
||||
* Workaround `libc_ver` not be available on Windows Store version of Python
|
||||
* [outtmpl] Curly braces to filter keys by [pukkandan](https://github.com/pukkandan)
|
||||
* [outtmpl] Make `%s` work in strfformat for all systems
|
||||
* [jsinterp] Workaround operator associativity issue
|
||||
* [cookies] Let `_get_mac_keyring_password` fail gracefully
|
||||
* [cookies] Parse cookies leniently by [Grub4K](https://github.com/Grub4K)
|
||||
* [phantomjs] Fix bug in [587021c](https://github.com/yt-dlp/yt-dlp/commit/587021cd9f717181b44e881941aca3f8d753758b) by [elyse0](https://github.com/elyse0)
|
||||
* [downloader/aria2c] Fix filename containing leading whitespace by [std-move](https://github.com/std-move)
|
||||
* [downloader/ism] Support ec-3 codec by [nixxo](https://github.com/nixxo)
|
||||
* [extractor] Fix `fatal=False` in `RetryManager`
|
||||
* [extractor] Improve json-ld extraction
|
||||
* [extractor] Make `_search_json` able to parse lists
|
||||
* [extractor] Escape `%` in `representation_id` of m3u8
|
||||
* [extractor/generic] Pass through referer from json-ld
|
||||
* [utils] `base_url`: URL paths can contain `&` by [elyse0](https://github.com/elyse0)
|
||||
* [utils] `js_to_json`: Improve
|
||||
* [utils] `Popen.run`: Fix default return in binary mode
|
||||
* [utils] `traverse_obj`: Rewrite, document and add tests by [Grub4K](https://github.com/Grub4K)
|
||||
* [devscripts] `make_lazy_extractors`: Fix for Docker by [josanabr](https://github.com/josanabr)
|
||||
* [docs] Misc Improvements
|
||||
* [cleanup] Misc fixes and cleanup by [pukkandan](https://github.com/pukkandan), [gamer191](https://github.com/gamer191)
|
||||
* [extractor/24tv.ua] Add extractors by [coletdjnz](https://github.com/coletdjnz)
|
||||
* [extractor/BerufeTV] Add extractor by [Fabi019](https://github.com/Fabi019)
|
||||
* [extractor/booyah] Add extractor by [HobbyistDev](https://github.com/HobbyistDev), [elyse0](https://github.com/elyse0)
|
||||
* [extractor/bundesliga] Add extractor by [Fabi019](https://github.com/Fabi019)
|
||||
* [extractor/GoPlay] Add extractor by [CNugteren](https://github.com/CNugteren), [basrieter](https://github.com/basrieter), [jeroenj](https://github.com/jeroenj)
|
||||
* [extractor/iltalehti] Add extractor by [tpikonen](https://github.com/tpikonen)
|
||||
* [extractor/IsraelNationalNews] Add extractor by [Bobscorn](https://github.com/Bobscorn)
|
||||
* [extractor/mediaworksnzvod] Add extractor by [coletdjnz](https://github.com/coletdjnz)
|
||||
* [extractor/MicrosoftEmbed] Add extractor by [DoubleCouponDay](https://github.com/DoubleCouponDay)
|
||||
* [extractor/nbc] Add NBCStations extractor by [bashonly](https://github.com/bashonly)
|
||||
* [extractor/onenewsnz] Add extractor by [coletdjnz](https://github.com/coletdjnz)
|
||||
* [extractor/prankcast] Add extractor by [HobbyistDev](https://github.com/HobbyistDev), [columndeeply](https://github.com/columndeeply)
|
||||
* [extractor/Smotrim] Add extractor by [Lesmiscore](https://github.com/Lesmiscore), [nikita-moor](https://github.com/nikita-moor)
|
||||
* [extractor/tencent] Add Iflix extractor by [elyse0](https://github.com/elyse0)
|
||||
* [extractor/unscripted] Add extractor by [HobbyistDev](https://github.com/HobbyistDev)
|
||||
* [extractor/adobepass] Add MSO AlticeOne (Optimum TV) by [CplPwnies](https://github.com/CplPwnies)
|
||||
* [extractor/youtube] **Download `post_live` videos from start** by [Lesmiscore](https://github.com/Lesmiscore), [pukkandan](https://github.com/pukkandan)
|
||||
* [extractor/youtube] Add support for Shorts audio pivot feed by [coletdjnz](https://github.com/coletdjnz), [pukkandan](https://github.com/pukkandan)
|
||||
* [extractor/youtube] Detect `lazy-load-for-videos` embeds
|
||||
* [extractor/youtube] Do not warn on duplicate chapters
|
||||
* [extractor/youtube] Fix video like count extraction by [coletdjnz](https://github.com/coletdjnz)
|
||||
* [extractor/youtube] Support changing extraction language by [coletdjnz](https://github.com/coletdjnz)
|
||||
* [extractor/youtube:tab] Improve continuation items extraction
|
||||
* [extractor/youtube:tab] Support `reporthistory` page
|
||||
* [extractor/amazonstore] Fix JSON extraction by [coletdjnz](https://github.com/coletdjnz), [pukkandan](https://github.com/pukkandan)
|
||||
* [extractor/amazonstore] Retry to avoid captcha page by [Lesmiscore](https://github.com/Lesmiscore)
|
||||
* [extractor/animeondemand] Remove extractor by [TokyoBlackHole](https://github.com/TokyoBlackHole)
|
||||
* [extractor/anvato] Fix extractor and refactor by [bashonly](https://github.com/bashonly)
|
||||
* [extractor/artetv] Remove duplicate stream urls by [Grub4K](https://github.com/Grub4K)
|
||||
* [extractor/audioboom] Support direct URLs and refactor by [pukkandan](https://github.com/pukkandan), [tpikonen](https://github.com/tpikonen)
|
||||
* [extractor/bandcamp] Extract `uploader_url`
|
||||
* [extractor/bilibili] Add space.bilibili extractors by [lockmatrix](https://github.com/lockmatrix)
|
||||
* [extractor/BilibiliSpace] Fix extractor and better error message by [lockmatrix](https://github.com/lockmatrix)
|
||||
* [extractor/BiliIntl] Support uppercase lang in `_VALID_URL` by [coletdjnz](https://github.com/coletdjnz)
|
||||
* [extractor/BiliIntlSeries] Fix `_VALID_URL`
|
||||
* [extractor/bongacams] Update `_VALID_URL` by [0xGodspeed](https://github.com/0xGodspeed)
|
||||
* [extractor/crunchyroll:beta] Improve handling of hardsubs by [Grub4K](https://github.com/Grub4K)
|
||||
* [extractor/detik] Generalize extractors by [HobbyistDev](https://github.com/HobbyistDev), [coletdjnz](https://github.com/coletdjnz)
|
||||
* [extractor/dplay:italy] Add default authentication by [Timendum](https://github.com/Timendum)
|
||||
* [extractor/heise] Fix extractor by [coletdjnz](https://github.com/coletdjnz)
|
||||
* [extractor/holodex] Fix `_VALID_URL` by [LiviaMedeiros](https://github.com/LiviaMedeiros)
|
||||
* [extractor/hrfensehen] Fix extractor by [snapdgn](https://github.com/snapdgn)
|
||||
* [extractor/hungama] Add subtitle by [GautamMKGarg](https://github.com/GautamMKGarg), [pukkandan](https://github.com/pukkandan)
|
||||
* [extractor/instagram] Extract more metadata by [pritam20ps05](https://github.com/pritam20ps05)
|
||||
* [extractor/JWPlatform] Fix extractor by [coletdjnz](https://github.com/coletdjnz)
|
||||
* [extractor/malltv] Fix video_id extraction by [HobbyistDev](https://github.com/HobbyistDev)
|
||||
* [extractor/MLBTV] Detect live streams
|
||||
* [extractor/motorsport] Support native embeds
|
||||
* [extractor/Mxplayer] Fix extractor by [itachi-19](https://github.com/itachi-19)
|
||||
* [extractor/nebula] Add nebula.tv by [tannertechnology](https://github.com/tannertechnology)
|
||||
* [extractor/nfl] Fix extractor by [bashonly](https://github.com/bashonly)
|
||||
* [extractor/ondemandkorea] Update `jw_config` regex by [julien-hadleyjack](https://github.com/julien-hadleyjack)
|
||||
* [extractor/paramountplus] Better DRM detection by [bashonly](https://github.com/bashonly)
|
||||
* [extractor/patreon] Sort formats
|
||||
* [extractor/rcs] Fix embed extraction by [coletdjnz](https://github.com/coletdjnz)
|
||||
* [extractor/redgifs] Fix extractor by [jhwgh1968](https://github.com/jhwgh1968)
|
||||
* [extractor/rutube] Fix `_EMBED_REGEX` by [coletdjnz](https://github.com/coletdjnz)
|
||||
* [extractor/RUTV] Fix warnings for livestreams by [Lesmiscore](https://github.com/Lesmiscore)
|
||||
* [extractor/soundcloud:search] More metadata in `--flat-playlist` by [SuperSonicHub1](https://github.com/SuperSonicHub1)
|
||||
* [extractor/telegraaf] Use mobile GraphQL API endpoint by [coletdjnz](https://github.com/coletdjnz)
|
||||
* [extractor/tennistv] Fix timestamp by [zenerdi0de](https://github.com/zenerdi0de)
|
||||
* [extractor/tiktok] Fix TikTokIE by [bashonly](https://github.com/bashonly)
|
||||
* [extractor/triller] Fix auth token by [bashonly](https://github.com/bashonly)
|
||||
* [extractor/trovo] Fix extractors by [Mehavoid](https://github.com/Mehavoid)
|
||||
* [extractor/tv2] Support new url format by [tobi1805](https://github.com/tobi1805)
|
||||
* [extractor/web.archive:youtube] Fix `_YT_INITIAL_PLAYER_RESPONSE_RE`
|
||||
* [extractor/wistia] Add support for channels by [coletdjnz](https://github.com/coletdjnz)
|
||||
* [extractor/wistia] Match IDs in embed URLs by [bashonly](https://github.com/bashonly)
|
||||
* [extractor/wordpress:playlist] Add generic embed extractor by [coletdjnz](https://github.com/coletdjnz)
|
||||
* [extractor/yandexvideopreview] Update `_VALID_URL` by [Grub4K](https://github.com/Grub4K)
|
||||
* [extractor/zee5] Fix `_VALID_URL` by [m4tu4g](https://github.com/m4tu4g)
|
||||
* [extractor/zee5] Generate device ids by [freezboltz](https://github.com/freezboltz)
|
||||
|
||||
|
||||
### 2022.09.01
|
||||
|
||||
|
@ -8,6 +8,7 @@ # Collaborators
|
||||
## [pukkandan](https://github.com/pukkandan)
|
||||
|
||||
[![ko-fi](https://img.shields.io/badge/_-Ko--fi-red.svg?logo=kofi&labelColor=555555&style=for-the-badge)](https://ko-fi.com/pukkandan)
|
||||
[![gh-sponsor](https://img.shields.io/badge/_-Github-red.svg?logo=github&labelColor=555555&style=for-the-badge)](https://github.com/sponsors/pukkandan)
|
||||
|
||||
* Owner of the fork
|
||||
|
||||
@ -25,8 +26,9 @@ ## [shirt](https://github.com/shirt-dev)
|
||||
|
||||
## [coletdjnz](https://github.com/coletdjnz)
|
||||
|
||||
[![gh-sponsor](https://img.shields.io/badge/_-Sponsor-red.svg?logo=githubsponsors&labelColor=555555&style=for-the-badge)](https://github.com/sponsors/coletdjnz)
|
||||
[![gh-sponsor](https://img.shields.io/badge/_-Github-red.svg?logo=github&labelColor=555555&style=for-the-badge)](https://github.com/sponsors/coletdjnz)
|
||||
|
||||
* Improved plugin architecture
|
||||
* YouTube improvements including: age-gate bypass, private playlists, multiple-clients (to avoid throttling) and a lot of under-the-hood improvements
|
||||
* Added support for new websites YoutubeWebArchive, MainStreaming, PRX, nzherald, Mediaklikk, StarTV etc
|
||||
* Improved/fixed support for Patreon, panopto, gfycat, itv, pbs, SouthParkDE etc
|
||||
@ -42,7 +44,7 @@ ## [Ashish0804](https://github.com/Ashish0804) <sub><sup>[Inactive]</sup></sub>
|
||||
* Improved/fixed support for HiDive, HotStar, Hungama, LBRY, LinkedInLearning, Mxplayer, SonyLiv, TV2, Vimeo, VLive etc
|
||||
|
||||
|
||||
## [Lesmiscore](https://github.com/Lesmiscore) (nao20010128nao)
|
||||
## [Lesmiscore](https://github.com/Lesmiscore) <sub><sup>(nao20010128nao)</sup></sub>
|
||||
|
||||
**Bitcoin**: bc1qfd02r007cutfdjwjmyy9w23rjvtls6ncve7r3s
|
||||
**Monacoin**: mona1q3tf7dzvshrhfe3md379xtvt2n22duhglv5dskr
|
||||
@ -50,3 +52,20 @@ ## [Lesmiscore](https://github.com/Lesmiscore) (nao20010128nao)
|
||||
* Download live from start to end for YouTube
|
||||
* Added support for new websites AbemaTV, mildom, PixivSketch, skeb, radiko, voicy, mirrativ, openrec, whowatch, damtomo, 17.live, mixch etc
|
||||
* Improved/fixed support for fc2, YahooJapanNews, tver, iwara etc
|
||||
|
||||
|
||||
## [bashonly](https://github.com/bashonly)
|
||||
|
||||
* `--update-to`, automated release, nightly builds
|
||||
* `--cookies-from-browser` support for Firefox containers
|
||||
* Added support for new websites Genius, Kick, NBCStations, Triller, VideoKen etc
|
||||
* Improved/fixed support for Anvato, Brightcove, Instagram, ParamountPlus, Reddit, SlidesLive, TikTok, Twitter, Vimeo etc
|
||||
|
||||
|
||||
## [Grub4K](https://github.com/Grub4K)
|
||||
|
||||
[![ko-fi](https://img.shields.io/badge/_-Ko--fi-red.svg?logo=kofi&labelColor=555555&style=for-the-badge)](https://ko-fi.com/Grub4K) [![gh-sponsor](https://img.shields.io/badge/_-Github-red.svg?logo=github&labelColor=555555&style=for-the-badge)](https://github.com/sponsors/Grub4K)
|
||||
|
||||
* `--update-to`, automated release, nightly builds
|
||||
* Rework internals like `traverse_obj`, various core refactors and bugs fixes
|
||||
* Helped fix crunchyroll, Twitter, wrestleuniverse, wistia, slideslive etc
|
||||
|
11
Makefile
11
Makefile
@ -17,8 +17,8 @@ pypi-files: AUTHORS Changelog.md LICENSE README.md README.txt supportedsites \
|
||||
clean-test:
|
||||
rm -rf test/testdata/sigs/player-*.js tmp/ *.annotations.xml *.aria2 *.description *.dump *.frag \
|
||||
*.frag.aria2 *.frag.urls *.info.json *.live_chat.json *.meta *.part* *.tmp *.temp *.unknown_video *.ytdl \
|
||||
*.3gp *.ape *.ass *.avi *.desktop *.f4v *.flac *.flv *.jpeg *.jpg *.m4a *.mpga *.m4v *.mhtml *.mkv *.mov \
|
||||
*.mp3 *.mp4 *.ogg *.opus *.png *.sbv *.srt *.swf *.swp *.ttml *.url *.vtt *.wav *.webloc *.webm *.webp
|
||||
*.3gp *.ape *.ass *.avi *.desktop *.f4v *.flac *.flv *.gif *.jpeg *.jpg *.m4a *.m4v *.mhtml *.mkv *.mov *.mp3 \
|
||||
*.mp4 *.mpga *.oga *.ogg *.opus *.png *.sbv *.srt *.swf *.swp *.tt *.ttml *.url *.vtt *.wav *.webloc *.webm *.webp
|
||||
clean-dist:
|
||||
rm -rf yt-dlp.1.temp.md yt-dlp.1 README.txt MANIFEST build/ dist/ .coverage cover/ yt-dlp.tar.gz completions/ \
|
||||
yt_dlp/extractor/lazy_extractors.py *.spec CONTRIBUTING.md.tmp yt-dlp yt-dlp.exe yt_dlp.egg-info/ AUTHORS .mailmap
|
||||
@ -74,17 +74,16 @@ offlinetest: codetest
|
||||
$(PYTHON) -m pytest -k "not download"
|
||||
|
||||
# XXX: This is hard to maintain
|
||||
CODE_FOLDERS = yt_dlp yt_dlp/downloader yt_dlp/extractor yt_dlp/postprocessor yt_dlp/compat \
|
||||
yt_dlp/extractor/anvato_token_generator
|
||||
CODE_FOLDERS = yt_dlp yt_dlp/downloader yt_dlp/extractor yt_dlp/postprocessor yt_dlp/compat yt_dlp/dependencies
|
||||
yt-dlp: yt_dlp/*.py yt_dlp/*/*.py
|
||||
mkdir -p zip
|
||||
for d in $(CODE_FOLDERS) ; do \
|
||||
mkdir -p zip/$$d ;\
|
||||
cp -pPR $$d/*.py zip/$$d/ ;\
|
||||
done
|
||||
touch -t 200001010101 zip/yt_dlp/*.py zip/yt_dlp/*/*.py zip/yt_dlp/*/*/*.py
|
||||
touch -t 200001010101 zip/yt_dlp/*.py zip/yt_dlp/*/*.py
|
||||
mv zip/yt_dlp/__main__.py zip/
|
||||
cd zip ; zip -q ../yt-dlp yt_dlp/*.py yt_dlp/*/*.py yt_dlp/*/*/*.py __main__.py
|
||||
cd zip ; zip -q ../yt-dlp yt_dlp/*.py yt_dlp/*/*.py __main__.py
|
||||
rm -rf zip
|
||||
echo '#!$(PYTHON)' > yt-dlp
|
||||
cat yt-dlp.zip >> yt-dlp
|
||||
|
402
README.md
402
README.md
@ -3,16 +3,16 @@
|
||||
|
||||
[![YT-DLP](https://raw.githubusercontent.com/yt-dlp/yt-dlp/master/.github/banner.svg)](#readme)
|
||||
|
||||
[![Release version](https://img.shields.io/github/v/release/yt-dlp/yt-dlp?color=brightgreen&label=Download&style=for-the-badge)](##installation "Installation")
|
||||
[![Release version](https://img.shields.io/github/v/release/yt-dlp/yt-dlp?color=brightgreen&label=Download&style=for-the-badge)](#installation "Installation")
|
||||
[![PyPi](https://img.shields.io/badge/-PyPi-blue.svg?logo=pypi&labelColor=555555&style=for-the-badge)](https://pypi.org/project/yt-dlp "PyPi")
|
||||
[![Donate](https://img.shields.io/badge/_-Donate-red.svg?logo=githubsponsors&labelColor=555555&style=for-the-badge)](Collaborators.md#collaborators "Donate")
|
||||
[![Matrix](https://img.shields.io/matrix/yt-dlp:matrix.org?color=brightgreen&labelColor=555555&label=&logo=element&style=for-the-badge)](https://matrix.to/#/#yt-dlp:matrix.org "Matrix")
|
||||
[![Discord](https://img.shields.io/discord/807245652072857610?color=blue&labelColor=555555&label=&logo=discord&style=for-the-badge)](https://discord.gg/H5MNcFW63r "Discord")
|
||||
[![Supported Sites](https://img.shields.io/badge/-Supported_Sites-brightgreen.svg?style=for-the-badge)](supportedsites.md "Supported Sites")
|
||||
[![License: Unlicense](https://img.shields.io/badge/-Unlicense-blue.svg?style=for-the-badge)](LICENSE "License")
|
||||
[![CI Status](https://img.shields.io/github/workflow/status/yt-dlp/yt-dlp/Core%20Tests/master?label=Tests&style=for-the-badge)](https://github.com/yt-dlp/yt-dlp/actions "CI Status")
|
||||
[![CI Status](https://img.shields.io/github/actions/workflow/status/yt-dlp/yt-dlp/core.yml?branch=master&label=Tests&style=for-the-badge)](https://github.com/yt-dlp/yt-dlp/actions "CI Status")
|
||||
[![Commits](https://img.shields.io/github/commit-activity/m/yt-dlp/yt-dlp?label=commits&style=for-the-badge)](https://github.com/yt-dlp/yt-dlp/commits "Commit History")
|
||||
[![Last Commit](https://img.shields.io/github/last-commit/yt-dlp/yt-dlp/master?label=&style=for-the-badge)](https://github.com/yt-dlp/yt-dlp/commits "Commit History")
|
||||
[![Last Commit](https://img.shields.io/github/last-commit/yt-dlp/yt-dlp/master?label=&style=for-the-badge&display_timestamp=committer)](https://github.com/yt-dlp/yt-dlp/commits "Commit History")
|
||||
|
||||
</div>
|
||||
<!-- MANPAGE: END EXCLUDED SECTION -->
|
||||
@ -48,7 +48,9 @@
|
||||
* [SponsorBlock Options](#sponsorblock-options)
|
||||
* [Extractor Options](#extractor-options)
|
||||
* [CONFIGURATION](#configuration)
|
||||
* [Configuration file encoding](#configuration-file-encoding)
|
||||
* [Authentication with .netrc file](#authentication-with-netrc-file)
|
||||
* [Notes about environment variables](#notes-about-environment-variables)
|
||||
* [OUTPUT TEMPLATE](#output-template)
|
||||
* [Output template examples](#output-template-examples)
|
||||
* [FORMAT SELECTION](#format-selection)
|
||||
@ -59,6 +61,8 @@
|
||||
* [Modifying metadata examples](#modifying-metadata-examples)
|
||||
* [EXTRACTOR ARGUMENTS](#extractor-arguments)
|
||||
* [PLUGINS](#plugins)
|
||||
* [Installing Plugins](#installing-plugins)
|
||||
* [Developing Plugins](#developing-plugins)
|
||||
* [EMBEDDING YT-DLP](#embedding-yt-dlp)
|
||||
* [Embedding examples](#embedding-examples)
|
||||
* [DEPRECATED OPTIONS](#deprecated-options)
|
||||
@ -72,13 +76,13 @@
|
||||
|
||||
# NEW FEATURES
|
||||
|
||||
* Merged with **youtube-dl v2021.12.17+ [commit/ed5c44e](https://github.com/ytdl-org/youtube-dl/commit/ed5c44e7b74ac77f87ca5ed6cb5e964a0c6a0678)**<!--([exceptions](https://github.com/yt-dlp/yt-dlp/issues/21))--> and **youtube-dlc v2020.11.11-3+ [commit/f9401f2](https://github.com/blackjack4494/yt-dlc/commit/f9401f2a91987068139c5f757b12fc711d4c0cee)**: You get all the features and patches of [youtube-dlc](https://github.com/blackjack4494/yt-dlc) in addition to the latest [youtube-dl](https://github.com/ytdl-org/youtube-dl)
|
||||
* Merged with **youtube-dl v2021.12.17+ [commit/2dd6c6e](https://github.com/ytdl-org/youtube-dl/commit/2dd6c6e)** ([exceptions](https://github.com/yt-dlp/yt-dlp/issues/21)) and **youtube-dlc v2020.11.11-3+ [commit/f9401f2](https://github.com/blackjack4494/yt-dlc/commit/f9401f2a91987068139c5f757b12fc711d4c0cee)**: You get all the features and patches of [youtube-dlc](https://github.com/blackjack4494/yt-dlc) in addition to the latest [youtube-dl](https://github.com/ytdl-org/youtube-dl)
|
||||
|
||||
* **[SponsorBlock Integration](#sponsorblock-options)**: You can mark/remove sponsor sections in youtube videos by utilizing the [SponsorBlock](https://sponsor.ajay.app) API
|
||||
* **[SponsorBlock Integration](#sponsorblock-options)**: You can mark/remove sponsor sections in YouTube videos by utilizing the [SponsorBlock](https://sponsor.ajay.app) API
|
||||
|
||||
* **[Format Sorting](#sorting-formats)**: The default format sorting options have been changed so that higher resolution and better codecs will be now preferred instead of simply using larger bitrate. Furthermore, you can now specify the sort order using `-S`. This allows for much easier format selection than what is possible by simply using `--format` ([examples](#format-selection-examples))
|
||||
|
||||
* **Merged with animelover1984/youtube-dl**: You get most of the features and improvements from [animelover1984/youtube-dl](https://github.com/animelover1984/youtube-dl) including `--write-comments`, `BiliBiliSearch`, `BilibiliChannel`, Embedding thumbnail in mp4/ogg/opus, playlist infojson etc. Note that the NicoNico livestreams are not available. See [#31](https://github.com/yt-dlp/yt-dlp/pull/31) for details.
|
||||
* **Merged with animelover1984/youtube-dl**: You get most of the features and improvements from [animelover1984/youtube-dl](https://github.com/animelover1984/youtube-dl) including `--write-comments`, `BiliBiliSearch`, `BilibiliChannel`, Embedding thumbnail in mp4/ogg/opus, playlist infojson etc. Note that NicoNico livestreams are not available. See [#31](https://github.com/yt-dlp/yt-dlp/pull/31) for details.
|
||||
|
||||
* **YouTube improvements**:
|
||||
* Supports Clips, Stories (`ytstories:<channel UCID>`), Search (including filters)**\***, YouTube Music Search, Channel-specific search, Search prefixes (`ytsearch:`, `ytsearchdate:`)**\***, Mixes, YouTube Music Albums/Channels ([except self-uploaded music](https://github.com/yt-dlp/yt-dlp/issues/723)), and Feeds (`:ytfav`, `:ytwatchlater`, `:ytsubs`, `:ythistory`, `:ytrec`, `:ytnotif`)
|
||||
@ -86,9 +90,9 @@ # NEW FEATURES
|
||||
* Supports some (but not all) age-gated content without cookies
|
||||
* Download livestreams from the start using `--live-from-start` (*experimental*)
|
||||
* `255kbps` audio is extracted (if available) from YouTube Music when premium cookies are given
|
||||
* Redirect channel's home URL automatically to `/video` to preserve the old behaviour
|
||||
* Channel URLs download all uploads of the channel, including shorts and live
|
||||
|
||||
* **Cookies from browser**: Cookies can be automatically extracted from all major web browsers using `--cookies-from-browser BROWSER[+KEYRING][:PROFILE]`
|
||||
* **Cookies from browser**: Cookies can be automatically extracted from all major web browsers using `--cookies-from-browser BROWSER[+KEYRING][:PROFILE][::CONTAINER]`
|
||||
|
||||
* **Download time range**: Videos can be downloaded partially based on either timestamps or chapters using `--download-sections`
|
||||
|
||||
@ -110,13 +114,15 @@ # NEW FEATURES
|
||||
|
||||
* **Output template improvements**: Output templates can now have date-time formatting, numeric offsets, object traversal etc. See [output template](#output-template) for details. Even more advanced operations can also be done with the help of `--parse-metadata` and `--replace-in-metadata`
|
||||
|
||||
* **Other new options**: Many new options have been added such as `--alias`, `--print`, `--concat-playlist`, `--wait-for-video`, `--retry-sleep`, `--sleep-requests`, `--convert-thumbnails`, `--force-download-archive`, `--force-overwrites`, `--break-on-reject` etc
|
||||
* **Other new options**: Many new options have been added such as `--alias`, `--print`, `--concat-playlist`, `--wait-for-video`, `--retry-sleep`, `--sleep-requests`, `--convert-thumbnails`, `--force-download-archive`, `--force-overwrites`, `--break-match-filter` etc
|
||||
|
||||
* **Improvements**: Regex and other operators in `--format`/`--match-filter`, multiple `--postprocessor-args` and `--downloader-args`, faster archive checking, more [format selection options](#format-selection), merge multi-video/audio, multiple `--config-locations`, `--exec` at different stages, etc
|
||||
|
||||
* **Plugins**: Extractors and PostProcessors can be loaded from an external file. See [plugins](#plugins) for details
|
||||
|
||||
* **Self-updater**: The releases can be updated using `yt-dlp -U`
|
||||
* **Self updater**: The releases can be updated using `yt-dlp -U`, and downgraded using `--update-to` if required
|
||||
|
||||
* **Nightly builds**: [Automated nightly builds](#update-channels) can be used with `--update-to nightly`
|
||||
|
||||
See [changelog](Changelog.md) or [commits](https://github.com/yt-dlp/yt-dlp/commits) for the full list of changes
|
||||
|
||||
@ -126,6 +132,7 @@ ### Differences in default behavior
|
||||
|
||||
Some of yt-dlp's default options are different from that of youtube-dl and youtube-dlc:
|
||||
|
||||
* yt-dlp supports only [Python 3.7+](## "Windows 7"), and *may* remove support for more versions as they [become EOL](https://devguide.python.org/versions/#python-release-cycle); while [youtube-dl still supports Python 2.6+ and 3.2+](https://github.com/ytdl-org/youtube-dl/issues/30568#issue-1118238743)
|
||||
* The options `--auto-number` (`-A`), `--title` (`-t`) and `--literal` (`-l`), no longer work. See [removed options](#Removed) for details
|
||||
* `avconv` is not supported as an alternative to `ffmpeg`
|
||||
* yt-dlp stores config files in slightly different locations to youtube-dl. See [CONFIGURATION](#configuration) for a list of correct locations
|
||||
@ -140,8 +147,8 @@ ### Differences in default behavior
|
||||
* `playlist_index` behaves differently when used with options like `--playlist-reverse` and `--playlist-items`. See [#302](https://github.com/yt-dlp/yt-dlp/issues/302) for details. You can use `--compat-options playlist-index` if you want to keep the earlier behavior
|
||||
* The output of `-F` is listed in a new format. Use `--compat-options list-formats` to revert this
|
||||
* Live chats (if available) are considered as subtitles. Use `--sub-langs all,-live_chat` to download all subtitles except live chat. You can also use `--compat-options no-live-chat` to prevent any live chat/danmaku from downloading
|
||||
* Youtube channel URLs are automatically redirected to `/video`. Append a `/featured` to the URL to download only the videos in the home page. If the channel does not have a videos tab, we try to download the equivalent `UU` playlist instead. For all other tabs, if the channel does not show the requested tab, an error will be raised. Also, `/live` URLs raise an error if there are no live videos instead of silently downloading the entire channel. You may use `--compat-options no-youtube-channel-redirect` to revert all these redirections
|
||||
* Unavailable videos are also listed for youtube playlists. Use `--compat-options no-youtube-unavailable-videos` to remove this
|
||||
* YouTube channel URLs download all uploads of the channel. To download only the videos in a specific tab, pass the tab's URL. If the channel does not show the requested tab, an error will be raised. Also, `/live` URLs raise an error if there are no live videos instead of silently downloading the entire channel. You may use `--compat-options no-youtube-channel-redirect` to revert all these redirections
|
||||
* Unavailable videos are also listed for YouTube playlists. Use `--compat-options no-youtube-unavailable-videos` to remove this
|
||||
* The upload dates extracted from YouTube are in UTC [when available](https://github.com/yt-dlp/yt-dlp/blob/89e4d86171c7b7c997c77d4714542e0383bf0db0/yt_dlp/extractor/youtube.py#L3898-L3900). Use `--compat-options no-youtube-prefer-utc-upload-date` to prefer the non-UTC upload date.
|
||||
* If `ffmpeg` is used as the downloader, the downloading and merging of formats happen in a single step when possible. Use `--compat-options no-direct-merge` to revert this
|
||||
* Thumbnail embedding in `mp4` is done with mutagen if possible. Use `--compat-options embed-thumbnail-atomicparsley` to force the use of AtomicParsley instead
|
||||
@ -149,12 +156,15 @@ ### Differences in default behavior
|
||||
* When `--embed-subs` and `--write-subs` are used together, the subtitles are written to disk and also embedded in the media file. You can use just `--embed-subs` to embed the subs and automatically delete the separate file. See [#630 (comment)](https://github.com/yt-dlp/yt-dlp/issues/630#issuecomment-893659460) for more info. `--compat-options no-keep-subs` can be used to revert this
|
||||
* `certifi` will be used for SSL root certificates, if installed. If you want to use system certificates (e.g. self-signed), use `--compat-options no-certifi`
|
||||
* yt-dlp's sanitization of invalid characters in filenames is different/smarter than in youtube-dl. You can use `--compat-options filename-sanitization` to revert to youtube-dl's behavior
|
||||
* yt-dlp tries to parse the external downloader outputs into the standard progress output if possible (Currently implemented: [~~aria2c~~](https://github.com/yt-dlp/yt-dlp/issues/5931)). You can use `--compat-options no-external-downloader-progress` to get the downloader output as-is
|
||||
|
||||
For ease of use, a few more compat options are available:
|
||||
|
||||
* `--compat-options all`: Use all compat options (Do NOT use)
|
||||
* `--compat-options youtube-dl`: Same as `--compat-options all,-multistreams`
|
||||
* `--compat-options youtube-dlc`: Same as `--compat-options all,-no-live-chat,-no-youtube-channel-redirect`
|
||||
* `--compat-options 2021`: Same as `--compat-options 2022,no-certifi,filename-sanitization,no-youtube-prefer-utc-upload-date`
|
||||
* `--compat-options 2022`: Same as `--compat-options no-external-downloader-progress`. Use this to enable all future compat options
|
||||
|
||||
|
||||
# INSTALLATION
|
||||
@ -173,12 +183,25 @@ # INSTALLATION
|
||||
|
||||
|
||||
## UPDATE
|
||||
You can use `yt-dlp -U` to update if you are [using the release binaries](#release-files)
|
||||
You can use `yt-dlp -U` to update if you are using the [release binaries](#release-files)
|
||||
|
||||
If you [installed with PIP](https://github.com/yt-dlp/yt-dlp/wiki/Installation#with-pip), simply re-run the same command that was used to install the program
|
||||
|
||||
For other third-party package managers, see [the wiki](https://github.com/yt-dlp/yt-dlp/wiki/Installation) or refer their documentation
|
||||
For other third-party package managers, see [the wiki](https://github.com/yt-dlp/yt-dlp/wiki/Installation#third-party-package-managers) or refer their documentation
|
||||
|
||||
<a id="update-channels"/>
|
||||
|
||||
There are currently two release channels for binaries, `stable` and `nightly`.
|
||||
`stable` is the default channel, and many of its changes have been tested by users of the nightly channel.
|
||||
The `nightly` channel has releases built after each push to the master branch, and will have the most recent fixes and additions, but also have more risk of regressions. They are available in [their own repo](https://github.com/yt-dlp/yt-dlp-nightly-builds/releases).
|
||||
|
||||
When using `--update`/`-U`, a release binary will only update to its current channel.
|
||||
This release channel can be changed by using the `--update-to` option. `--update-to` can also be used to upgrade or downgrade to specific tags from a channel.
|
||||
|
||||
Example usage:
|
||||
* `yt-dlp --update-to nightly` change to `nightly` channel and update to its latest release
|
||||
* `yt-dlp --update-to stable@2023.02.17` upgrade/downgrade to release to `stable` channel tag `2023.02.17`
|
||||
* `yt-dlp --update-to 2023.01.06` upgrade/downgrade to tag `2023.01.06` if it exists on the current channel
|
||||
|
||||
<!-- MANPAGE: BEGIN EXCLUDED SECTION -->
|
||||
## RELEASE FILES
|
||||
@ -199,6 +222,8 @@ #### Alternatives
|
||||
[yt-dlp_min.exe](https://github.com/yt-dlp/yt-dlp/releases/latest/download/yt-dlp_min.exe)|Windows (Win7 SP1+) standalone x64 binary built with `py2exe`<br/> ([Not recommended](#standalone-py2exe-builds-windows))
|
||||
[yt-dlp_linux](https://github.com/yt-dlp/yt-dlp/releases/latest/download/yt-dlp_linux)|Linux standalone x64 binary
|
||||
[yt-dlp_linux.zip](https://github.com/yt-dlp/yt-dlp/releases/latest/download/yt-dlp_linux.zip)|Unpackaged Linux executable (no auto-update)
|
||||
[yt-dlp_linux_armv7l](https://github.com/yt-dlp/yt-dlp/releases/latest/download/yt-dlp_linux_armv7l)|Linux standalone armv7l (32-bit) binary
|
||||
[yt-dlp_linux_aarch64](https://github.com/yt-dlp/yt-dlp/releases/latest/download/yt-dlp_linux_aarch64)|Linux standalone aarch64 (64-bit) binary
|
||||
[yt-dlp_win.zip](https://github.com/yt-dlp/yt-dlp/releases/latest/download/yt-dlp_win.zip)|Unpackaged Windows executable (no auto-update)
|
||||
[yt-dlp_macos.zip](https://github.com/yt-dlp/yt-dlp/releases/latest/download/yt-dlp_macos.zip)|Unpackaged MacOS (10.15+) executable (no auto-update)
|
||||
[yt-dlp_macos_legacy](https://github.com/yt-dlp/yt-dlp/releases/latest/download/yt-dlp_macos_legacy)|MacOS (10.9+) standalone x64 executable
|
||||
@ -209,11 +234,20 @@ #### Misc
|
||||
:---|:---
|
||||
[yt-dlp.tar.gz](https://github.com/yt-dlp/yt-dlp/releases/latest/download/yt-dlp.tar.gz)|Source tarball
|
||||
[SHA2-512SUMS](https://github.com/yt-dlp/yt-dlp/releases/latest/download/SHA2-512SUMS)|GNU-style SHA512 sums
|
||||
[SHA2-512SUMS.sig](https://github.com/yt-dlp/yt-dlp/releases/latest/download/SHA2-512SUMS.sig)|GPG signature file for SHA512 sums
|
||||
[SHA2-256SUMS](https://github.com/yt-dlp/yt-dlp/releases/latest/download/SHA2-256SUMS)|GNU-style SHA256 sums
|
||||
[SHA2-256SUMS.sig](https://github.com/yt-dlp/yt-dlp/releases/latest/download/SHA2-256SUMS.sig)|GPG signature file for SHA256 sums
|
||||
|
||||
The public key that can be used to verify the GPG signatures is [available here](https://github.com/yt-dlp/yt-dlp/blob/master/public.key)
|
||||
Example usage:
|
||||
```
|
||||
curl -L https://github.com/yt-dlp/yt-dlp/raw/master/public.key | gpg --import
|
||||
gpg --verify SHA2-256SUMS.sig SHA2-256SUMS
|
||||
gpg --verify SHA2-512SUMS.sig SHA2-512SUMS
|
||||
```
|
||||
<!-- MANPAGE: END EXCLUDED SECTION -->
|
||||
|
||||
|
||||
Note: The manpages, shell completion files etc. are available in the [source tarball](https://github.com/yt-dlp/yt-dlp/releases/latest/download/yt-dlp.tar.gz)
|
||||
**Note**: The manpages, shell completion files etc. are available inside the [source tarball](https://github.com/yt-dlp/yt-dlp/releases/latest/download/yt-dlp.tar.gz)
|
||||
|
||||
## DEPENDENCIES
|
||||
Python versions 3.7+ (CPython and PyPy) are supported. Other versions and implementations may or may not work correctly.
|
||||
@ -229,8 +263,9 @@ ### Strongly recommended
|
||||
|
||||
* [**ffmpeg** and **ffprobe**](https://www.ffmpeg.org) - Required for [merging separate video and audio files](#format-selection) as well as for various [post-processing](#post-processing-options) tasks. License [depends on the build](https://www.ffmpeg.org/legal.html)
|
||||
|
||||
<!-- TODO: ffmpeg has merged this patch. Remove this note once there is new release -->
|
||||
**Note**: There are some regressions in newer ffmpeg versions that causes various issues when used alongside yt-dlp. Since ffmpeg is such an important dependency, we provide [custom builds](https://github.com/yt-dlp/FFmpeg-Builds#ffmpeg-static-auto-builds) with patches for these issues at [yt-dlp/FFmpeg-Builds](https://github.com/yt-dlp/FFmpeg-Builds). See [the readme](https://github.com/yt-dlp/FFmpeg-Builds#patches-applied) for details on the specific issues solved by these builds
|
||||
There are bugs in ffmpeg that causes various issues when used alongside yt-dlp. Since ffmpeg is such an important dependency, we provide [custom builds](https://github.com/yt-dlp/FFmpeg-Builds#ffmpeg-static-auto-builds) with patches for some of these issues at [yt-dlp/FFmpeg-Builds](https://github.com/yt-dlp/FFmpeg-Builds). See [the readme](https://github.com/yt-dlp/FFmpeg-Builds#patches-applied) for details on the specific issues solved by these builds
|
||||
|
||||
**Important**: What you need is ffmpeg *binary*, **NOT** [the python package of the same name](https://pypi.org/project/ffmpeg)
|
||||
|
||||
### Networking
|
||||
* [**certifi**](https://github.com/certifi/python-certifi)\* - Provides Mozilla's root certificate bundle. Licensed under [MPLv2](https://github.com/certifi/python-certifi/blob/master/LICENSE)
|
||||
@ -275,7 +310,9 @@ ### Standalone PyInstaller Builds
|
||||
|
||||
On some systems, you may need to use `py` or `python` instead of `python3`.
|
||||
|
||||
Note that pyinstaller with versions below 4.4 [do not support](https://github.com/pyinstaller/pyinstaller#requirements-and-tested-platforms) Python installed from the Windows store without using a virtual environment.
|
||||
`pyinst.py` accepts any arguments that can be passed to `pyinstaller`, such as `--onefile/-F` or `--onedir/-D`, which is further [documented here](https://pyinstaller.org/en/stable/usage.html#what-to-generate).
|
||||
|
||||
**Note**: Pyinstaller versions below 4.4 [do not support](https://github.com/pyinstaller/pyinstaller#requirements-and-tested-platforms) Python installed from the Windows store without using a virtual environment.
|
||||
|
||||
**Important**: Running `pyinstaller` directly **without** using `pyinst.py` is **not** officially supported. This may or may not work correctly.
|
||||
|
||||
@ -298,11 +335,15 @@ ### Standalone Py2Exe Builds (Windows)
|
||||
|
||||
### Related scripts
|
||||
|
||||
* **`devscripts/update-version.py [revision]`** - Update the version number based on current date
|
||||
* **`devscripts/set-variant.py variant [-M update_message]`** - Set the build variant of the executable
|
||||
* **`devscripts/update-version.py`** - Update the version number based on current date.
|
||||
* **`devscripts/set-variant.py`** - Set the build variant of the executable.
|
||||
* **`devscripts/make_changelog.py`** - Create a markdown changelog using short commit messages and update `CONTRIBUTORS` file.
|
||||
* **`devscripts/make_lazy_extractors.py`** - Create lazy extractors. Running this before building the binaries (any variant) will improve their startup performance. Set the environment variable `YTDLP_NO_LAZY_EXTRACTORS=1` if you wish to forcefully disable lazy extractor loading.
|
||||
|
||||
You can also fork the project on github and run your fork's [build workflow](.github/workflows/build.yml) to automatically build a full release
|
||||
Note: See their `--help` for more info.
|
||||
|
||||
### Forking the project
|
||||
If you fork the project on GitHub, you can run your fork's [build workflow](.github/workflows/build.yml) to automatically build the selected version(s) as artifacts. Alternatively, you can run the [release workflow](.github/workflows/release.yml) or enable the [nightly workflow](.github/workflows/release-nightly.yml) to create full (pre-)releases.
|
||||
|
||||
# USAGE AND OPTIONS
|
||||
|
||||
@ -318,6 +359,11 @@ ## General Options:
|
||||
--version Print program version and exit
|
||||
-U, --update Update this program to the latest version
|
||||
--no-update Do not check for updates (default)
|
||||
--update-to [CHANNEL]@[TAG] Upgrade/downgrade to a specific version.
|
||||
CHANNEL and TAG defaults to "stable" and
|
||||
"latest" respectively if omitted; See
|
||||
"UPDATE" for details. Supported channels:
|
||||
stable, nightly
|
||||
-i, --ignore-errors Ignore download and postprocessing errors.
|
||||
The download will be considered successful
|
||||
even if the postprocessing fails
|
||||
@ -408,6 +454,8 @@ ## Network Options:
|
||||
--source-address IP Client-side IP address to bind to
|
||||
-4, --force-ipv4 Make all connections via IPv4
|
||||
-6, --force-ipv6 Make all connections via IPv6
|
||||
--enable-file-urls Enable file:// URLs. This is disabled by
|
||||
default for security reasons.
|
||||
|
||||
## Geo-restriction:
|
||||
--geo-verification-proxy URL Use this proxy to verify the IP address for
|
||||
@ -426,23 +474,24 @@ ## Geo-restriction:
|
||||
explicitly provided IP block in CIDR notation
|
||||
|
||||
## Video Selection:
|
||||
-I, --playlist-items ITEM_SPEC Comma separated playlist_index of the videos
|
||||
-I, --playlist-items ITEM_SPEC Comma separated playlist_index of the items
|
||||
to download. You can specify a range using
|
||||
"[START]:[STOP][:STEP]". For backward
|
||||
compatibility, START-STOP is also supported.
|
||||
Use negative indices to count from the right
|
||||
and negative STEP to download in reverse
|
||||
order. E.g. "-I 1:3,7,-5::2" used on a
|
||||
playlist of size 15 will download the videos
|
||||
playlist of size 15 will download the items
|
||||
at index 1,2,3,7,11,13,15
|
||||
--min-filesize SIZE Do not download any videos smaller than
|
||||
--min-filesize SIZE Abort download if filesize is smaller than
|
||||
SIZE, e.g. 50k or 44.6M
|
||||
--max-filesize SIZE Abort download if filesize is larger than
|
||||
SIZE, e.g. 50k or 44.6M
|
||||
--max-filesize SIZE Do not download any videos larger than SIZE,
|
||||
e.g. 50k or 44.6M
|
||||
--date DATE Download only videos uploaded on this date.
|
||||
The date can be "YYYYMMDD" or in the format
|
||||
[now|today|yesterday][-N[day|week|month|year]].
|
||||
E.g. --date today-2weeks
|
||||
E.g. "--date today-2weeks" downloads only
|
||||
videos uploaded on the same day two weeks ago
|
||||
--datebefore DATE Download only videos uploaded on or before
|
||||
this date. The date formats accepted is the
|
||||
same as --date
|
||||
@ -469,7 +518,10 @@ ## Video Selection:
|
||||
dogs" (caseless). Use "--match-filter -" to
|
||||
interactively ask whether to download each
|
||||
video
|
||||
--no-match-filter Do not use generic video filter (default)
|
||||
--no-match-filter Do not use any --match-filter (default)
|
||||
--break-match-filters FILTER Same as "--match-filters" but stops the
|
||||
download process when a video is rejected
|
||||
--no-break-match-filters Do not use any --break-match-filters (default)
|
||||
--no-playlist Download only the video, if the URL refers
|
||||
to a video and a playlist
|
||||
--yes-playlist Download the playlist, if the URL refers to
|
||||
@ -483,11 +535,9 @@ ## Video Selection:
|
||||
--max-downloads NUMBER Abort after downloading NUMBER files
|
||||
--break-on-existing Stop the download process when encountering
|
||||
a file that is in the archive
|
||||
--break-on-reject Stop the download process when encountering
|
||||
a file that has been filtered out
|
||||
--break-per-input --break-on-existing, --break-on-reject,
|
||||
--max-downloads, and autonumber resets per
|
||||
input URL
|
||||
--break-per-input Alters --max-downloads, --break-on-existing,
|
||||
--break-match-filter, and autonumber to
|
||||
reset per input URL
|
||||
--no-break-per-input --break-on-existing and similar options
|
||||
terminates the entire download queue
|
||||
--skip-playlist-after-errors N Number of allowed failures until the rest of
|
||||
@ -519,8 +569,8 @@ ## Download Options:
|
||||
linear=1::2 --retry-sleep fragment:exp=1:20
|
||||
--skip-unavailable-fragments Skip unavailable fragments for DASH,
|
||||
hlsnative and ISM downloads (default)
|
||||
(Alias: --no-abort-on-unavailable-fragment)
|
||||
--abort-on-unavailable-fragment
|
||||
(Alias: --no-abort-on-unavailable-fragments)
|
||||
--abort-on-unavailable-fragments
|
||||
Abort download if a fragment is unavailable
|
||||
(Alias: --no-skip-unavailable-fragments)
|
||||
--keep-fragments Keep downloaded fragments on disk after
|
||||
@ -562,7 +612,7 @@ ## Download Options:
|
||||
Needs ffmpeg. This option can be used
|
||||
multiple times to download multiple
|
||||
sections, e.g. --download-sections
|
||||
"*10:15-15:00" --download-sections "intro"
|
||||
"*10:15-inf" --download-sections "intro"
|
||||
--downloader [PROTO:]NAME Name or path of the external downloader to
|
||||
use (optionally) prefixed by the protocols
|
||||
(http, ftp, m3u8, dash, rstp, rtmp, mms) to
|
||||
@ -679,8 +729,7 @@ ## Filesystem Options:
|
||||
--cache-dir DIR Location in the filesystem where yt-dlp can
|
||||
store some downloaded information (such as
|
||||
client ids and signatures) permanently. By
|
||||
default $XDG_CACHE_HOME/yt-dlp or
|
||||
~/.cache/yt-dlp
|
||||
default ${XDG_CACHE_HOME}/yt-dlp
|
||||
--no-cache-dir Disable filesystem caching
|
||||
--rm-cache-dir Delete all filesystem cache files
|
||||
|
||||
@ -720,7 +769,7 @@ ## Verbosity and Simulation Options:
|
||||
screen, optionally prefixed with when to
|
||||
print it, separated by a ":". Supported
|
||||
values of "WHEN" are the same as that of
|
||||
--use-postprocessor, and "video" (default).
|
||||
--use-postprocessor (default: video).
|
||||
Implies --quiet. Implies --simulate unless
|
||||
--no-simulate or later stages of WHEN are
|
||||
used. This option can be used multiple times
|
||||
@ -773,7 +822,7 @@ ## Workarounds:
|
||||
--prefer-insecure Use an unencrypted connection to retrieve
|
||||
information about the video (Currently
|
||||
supported only for YouTube)
|
||||
--add-header FIELD:VALUE Specify a custom HTTP header and its value,
|
||||
--add-headers FIELD:VALUE Specify a custom HTTP header and its value,
|
||||
separated by a colon ":". You can use this
|
||||
option multiple times
|
||||
--bidi-workaround Work around terminals that lack
|
||||
@ -888,11 +937,11 @@ ## Post-Processing Options:
|
||||
specific bitrate like 128K (default 5)
|
||||
--remux-video FORMAT Remux the video into another container if
|
||||
necessary (currently supported: avi, flv,
|
||||
mkv, mov, mp4, webm, aac, aiff, alac, flac,
|
||||
m4a, mka, mp3, ogg, opus, vorbis, wav). If
|
||||
target container does not support the
|
||||
video/audio codec, remuxing will fail. You
|
||||
can specify multiple rules; e.g.
|
||||
gif, mkv, mov, mp4, webm, aac, aiff, alac,
|
||||
flac, m4a, mka, mp3, ogg, opus, vorbis,
|
||||
wav). If target container does not support
|
||||
the video/audio codec, remuxing will fail.
|
||||
You can specify multiple rules; e.g.
|
||||
"aac>m4a/mov>mp4/mkv" will remux aac to m4a,
|
||||
mov to mp4 and anything else to mkv
|
||||
--recode-video FORMAT Re-encode the video into another format if
|
||||
@ -947,13 +996,18 @@ ## Post-Processing Options:
|
||||
mkv/mka video files
|
||||
--no-embed-info-json Do not embed the infojson as an attachment
|
||||
to the video file
|
||||
--parse-metadata FROM:TO Parse additional metadata like title/artist
|
||||
--parse-metadata [WHEN:]FROM:TO
|
||||
Parse additional metadata like title/artist
|
||||
from other fields; see "MODIFYING METADATA"
|
||||
for details
|
||||
--replace-in-metadata FIELDS REGEX REPLACE
|
||||
for details. Supported values of "WHEN" are
|
||||
the same as that of --use-postprocessor
|
||||
(default: pre_process)
|
||||
--replace-in-metadata [WHEN:]FIELDS REGEX REPLACE
|
||||
Replace text in a metadata field using the
|
||||
given regex. This option can be used
|
||||
multiple times
|
||||
multiple times. Supported values of "WHEN"
|
||||
are the same as that of --use-postprocessor
|
||||
(default: pre_process)
|
||||
--xattrs Write metadata to the video file's xattrs
|
||||
(using dublin core and xdg standards)
|
||||
--concat-playlist POLICY Concatenate videos in a playlist. One of
|
||||
@ -974,18 +1028,18 @@ ## Post-Processing Options:
|
||||
--ffmpeg-location PATH Location of the ffmpeg binary; either the
|
||||
path to the binary or its containing directory
|
||||
--exec [WHEN:]CMD Execute a command, optionally prefixed with
|
||||
when to execute it (after_move if
|
||||
unspecified), separated by a ":". Supported
|
||||
values of "WHEN" are the same as that of
|
||||
--use-postprocessor. Same syntax as the
|
||||
output template can be used to pass any
|
||||
field as arguments to the command. After
|
||||
download, an additional field "filepath"
|
||||
that contains the final path of the
|
||||
downloaded file is also available, and if no
|
||||
fields are passed, %(filepath)q is appended
|
||||
to the end of the command. This option can
|
||||
be used multiple times
|
||||
when to execute it, separated by a ":".
|
||||
Supported values of "WHEN" are the same as
|
||||
that of --use-postprocessor (default:
|
||||
after_move). Same syntax as the output
|
||||
template can be used to pass any field as
|
||||
arguments to the command. After download, an
|
||||
additional field "filepath" that contains
|
||||
the final path of the downloaded file is
|
||||
also available, and if no fields are passed,
|
||||
%(filepath,_filename|)q is appended to the
|
||||
end of the command. This option can be used
|
||||
multiple times
|
||||
--no-exec Remove any previously defined --exec
|
||||
--convert-subs FORMAT Convert the subtitles to another format
|
||||
(currently supported: ass, lrc, srt, vtt)
|
||||
@ -1023,14 +1077,16 @@ ## Post-Processing Options:
|
||||
postprocessor is invoked. It can be one of
|
||||
"pre_process" (after video extraction),
|
||||
"after_filter" (after video passes filter),
|
||||
"before_dl" (before each video download),
|
||||
"post_process" (after each video download;
|
||||
default), "after_move" (after moving video
|
||||
file to it's final locations), "after_video"
|
||||
(after downloading and processing all
|
||||
formats of a video), or "playlist" (at end
|
||||
of playlist). This option can be used
|
||||
multiple times to add different postprocessors
|
||||
"video" (after --format; before
|
||||
--print/--output), "before_dl" (before each
|
||||
video download), "post_process" (after each
|
||||
video download; default), "after_move"
|
||||
(after moving video file to it's final
|
||||
locations), "after_video" (after downloading
|
||||
and processing all formats of a video), or
|
||||
"playlist" (at end of playlist). This option
|
||||
can be used multiple times to add different
|
||||
postprocessors
|
||||
|
||||
## SponsorBlock Options:
|
||||
Make chapter entries for, or remove various segments (sponsor,
|
||||
@ -1041,10 +1097,10 @@ ## SponsorBlock Options:
|
||||
for, separated by commas. Available
|
||||
categories are sponsor, intro, outro,
|
||||
selfpromo, preview, filler, interaction,
|
||||
music_offtopic, poi_highlight, all and
|
||||
default (=all). You can prefix the category
|
||||
with a "-" to exclude it. See [1] for
|
||||
description of the categories. E.g.
|
||||
music_offtopic, poi_highlight, chapter, all
|
||||
and default (=all). You can prefix the
|
||||
category with a "-" to exclude it. See [1]
|
||||
for description of the categories. E.g.
|
||||
--sponsorblock-mark all,-preview
|
||||
[1] https://wiki.sponsor.ajay.app/w/Segment_Categories
|
||||
--sponsorblock-remove CATS SponsorBlock categories to be removed from
|
||||
@ -1053,8 +1109,8 @@ ## SponsorBlock Options:
|
||||
remove takes precedence. The syntax and
|
||||
available categories are the same as for
|
||||
--sponsorblock-mark except that "default"
|
||||
refers to "all,-filler" and poi_highlight is
|
||||
not available
|
||||
refers to "all,-filler" and poi_highlight,
|
||||
chapter are not available
|
||||
--sponsorblock-chapter-title TEMPLATE
|
||||
An output template for the title of the
|
||||
SponsorBlock chapters created by
|
||||
@ -1079,29 +1135,40 @@ ## Extractor Options:
|
||||
--no-hls-split-discontinuity Do not split HLS playlists to different
|
||||
formats at discontinuities such as ad breaks
|
||||
(default)
|
||||
--extractor-args KEY:ARGS Pass these arguments to the extractor. See
|
||||
"EXTRACTOR ARGUMENTS" for details. You can
|
||||
use this option multiple times to give
|
||||
--extractor-args IE_KEY:ARGS Pass ARGS arguments to the IE_KEY extractor.
|
||||
See "EXTRACTOR ARGUMENTS" for details. You
|
||||
can use this option multiple times to give
|
||||
arguments for different extractors
|
||||
|
||||
# CONFIGURATION
|
||||
|
||||
You can configure yt-dlp by placing any supported command line option to a configuration file. The configuration is loaded from the following locations:
|
||||
|
||||
1. **Main Configuration**: The file given by `--config-location`
|
||||
1. **Portable Configuration**: `yt-dlp.conf` in the same directory as the bundled binary. If you are running from source-code (`<root dir>/yt_dlp/__main__.py`), the root directory is used instead.
|
||||
1. **Home Configuration**: `yt-dlp.conf` in the home path given by `-P`, or in the current directory if no such path is given
|
||||
1. **Main Configuration**:
|
||||
* The file given by `--config-location`
|
||||
1. **Portable Configuration**: (Recommended for portable installations)
|
||||
* If using a binary, `yt-dlp.conf` in the same directory as the binary
|
||||
* If running from source-code, `yt-dlp.conf` in the parent directory of `yt_dlp`
|
||||
1. **Home Configuration**:
|
||||
* `yt-dlp.conf` in the home path given by `-P`
|
||||
* If `-P` is not given, the current directory is searched
|
||||
1. **User Configuration**:
|
||||
* `%XDG_CONFIG_HOME%/yt-dlp/config` (recommended on Linux/macOS)
|
||||
* `%XDG_CONFIG_HOME%/yt-dlp.conf`
|
||||
* `%APPDATA%/yt-dlp/config` (recommended on Windows)
|
||||
* `%APPDATA%/yt-dlp/config.txt`
|
||||
* `${XDG_CONFIG_HOME}/yt-dlp.conf`
|
||||
* `${XDG_CONFIG_HOME}/yt-dlp/config` (recommended on Linux/macOS)
|
||||
* `${XDG_CONFIG_HOME}/yt-dlp/config.txt`
|
||||
* `${APPDATA}/yt-dlp.conf`
|
||||
* `${APPDATA}/yt-dlp/config` (recommended on Windows)
|
||||
* `${APPDATA}/yt-dlp/config.txt`
|
||||
* `~/yt-dlp.conf`
|
||||
* `~/yt-dlp.conf.txt`
|
||||
|
||||
`%XDG_CONFIG_HOME%` defaults to `~/.config` if undefined. On windows, `%APPDATA%` generally points to `C:\Users\<user name>\AppData\Roaming` and `~` points to `%HOME%` if present, `%USERPROFILE%` (generally `C:\Users\<user name>`), or `%HOMEDRIVE%%HOMEPATH%`
|
||||
* `~/.yt-dlp/config`
|
||||
* `~/.yt-dlp/config.txt`
|
||||
|
||||
1. **System Configuration**: `/etc/yt-dlp.conf`
|
||||
See also: [Notes about environment variables](#notes-about-environment-variables)
|
||||
1. **System Configuration**:
|
||||
* `/etc/yt-dlp.conf`
|
||||
* `/etc/yt-dlp/config`
|
||||
* `/etc/yt-dlp/config.txt`
|
||||
|
||||
E.g. with the following configuration file yt-dlp will always extract the audio, not copy the mtime, use a proxy and save all videos under `YouTube` directory in your home directory:
|
||||
```
|
||||
@ -1120,22 +1187,22 @@ # Save all videos under YouTube directory in your home directory
|
||||
-o ~/YouTube/%(title)s.%(ext)s
|
||||
```
|
||||
|
||||
Note that options in configuration file are just the same options aka switches used in regular command line calls; thus there **must be no whitespace** after `-` or `--`, e.g. `-o` or `--proxy` but not `- o` or `-- proxy`.
|
||||
**Note**: Options in configuration file are just the same options aka switches used in regular command line calls; thus there **must be no whitespace** after `-` or `--`, e.g. `-o` or `--proxy` but not `- o` or `-- proxy`. They must also be quoted when necessary as-if it were a UNIX shell.
|
||||
|
||||
You can use `--ignore-config` if you want to disable all configuration files for a particular yt-dlp run. If `--ignore-config` is found inside any configuration file, no further configuration will be loaded. For example, having the option in the portable configuration file prevents loading of home, user, and system configurations. Additionally, (for backward compatibility) if `--ignore-config` is found inside the system configuration file, the user configuration is not loaded.
|
||||
|
||||
### Config file encoding
|
||||
### Configuration file encoding
|
||||
|
||||
The config files are decoded according to the UTF BOM if present, and in the encoding from system locale otherwise.
|
||||
The configuration files are decoded according to the UTF BOM if present, and in the encoding from system locale otherwise.
|
||||
|
||||
If you want your file to be decoded differently, add `# coding: ENCODING` to the beginning of the file (e.g. `# coding: shift-jis`). There must be no characters before that, even spaces or BOM.
|
||||
|
||||
### Authentication with `.netrc` file
|
||||
|
||||
You may also want to configure automatic credentials storage for extractors that support authentication (by providing login and password with `--username` and `--password`) in order not to pass credentials as command line arguments on every yt-dlp execution and prevent tracking plain text passwords in the shell command history. You can achieve this using a [`.netrc` file](https://stackoverflow.com/tags/.netrc/info) on a per extractor basis. For that you will need to create a `.netrc` file in `--netrc-location` and restrict permissions to read/write by only you:
|
||||
You may also want to configure automatic credentials storage for extractors that support authentication (by providing login and password with `--username` and `--password`) in order not to pass credentials as command line arguments on every yt-dlp execution and prevent tracking plain text passwords in the shell command history. You can achieve this using a [`.netrc` file](https://stackoverflow.com/tags/.netrc/info) on a per-extractor basis. For that you will need to create a `.netrc` file in `--netrc-location` and restrict permissions to read/write by only you:
|
||||
```
|
||||
touch $HOME/.netrc
|
||||
chmod a-rwx,u+rw $HOME/.netrc
|
||||
touch ${HOME}/.netrc
|
||||
chmod a-rwx,u+rw ${HOME}/.netrc
|
||||
```
|
||||
After that you can add credentials for an extractor in the following format, where *extractor* is the name of the extractor in lowercase:
|
||||
```
|
||||
@ -1148,7 +1215,14 @@ ### Authentication with `.netrc` file
|
||||
```
|
||||
To activate authentication with the `.netrc` file you should pass `--netrc` to yt-dlp or place it in the [configuration file](#configuration).
|
||||
|
||||
The default location of the .netrc file is `$HOME` (`~`) in UNIX. On Windows, it is `%HOME%` if present, `%USERPROFILE%` (generally `C:\Users\<user name>`) or `%HOMEDRIVE%%HOMEPATH%`
|
||||
The default location of the .netrc file is `~` (see below).
|
||||
|
||||
### Notes about environment variables
|
||||
* Environment variables are normally specified as `${VARIABLE}`/`$VARIABLE` on UNIX and `%VARIABLE%` on Windows; but is always shown as `${VARIABLE}` in this documentation
|
||||
* yt-dlp also allow using UNIX-style variables on Windows for path-like options; e.g. `--output`, `--config-location`
|
||||
* If unset, `${XDG_CONFIG_HOME}` defaults to `~/.config` and `${XDG_CACHE_HOME}` to `~/.cache`
|
||||
* On Windows, `~` points to `${HOME}` if present; or, `${USERPROFILE}` or `${HOMEDRIVE}${HOMEPATH}` otherwise
|
||||
* On Windows, `${USERPROFILE}` generally points to `C:\Users\<user name>` and `${APPDATA}` to `${USERPROFILE}\AppData\Roaming`
|
||||
|
||||
# OUTPUT TEMPLATE
|
||||
|
||||
@ -1172,13 +1246,13 @@ # OUTPUT TEMPLATE
|
||||
|
||||
1. **Alternatives**: Alternate fields can be specified separated with a `,`. E.g. `%(release_date>%Y,upload_date>%Y|Unknown)s`
|
||||
|
||||
1. **Replacement**: A replacement value can specified using a `&` separator. If the field is *not* empty, this replacement value will be used instead of the actual field content. This is done after alternate fields are considered; thus the replacement is used if *any* of the alternative fields is *not* empty.
|
||||
1. **Replacement**: A replacement value can be specified using a `&` separator. If the field is *not* empty, this replacement value will be used instead of the actual field content. This is done after alternate fields are considered; thus the replacement is used if *any* of the alternative fields is *not* empty.
|
||||
|
||||
1. **Default**: A literal default value can be specified for when the field is empty using a `|` separator. This overrides `--output-na-placeholder`. E.g. `%(uploader|Unknown)s`
|
||||
|
||||
1. **More Conversions**: In addition to the normal format types `diouxXeEfFgGcrs`, yt-dlp additionally supports converting to `B` = **B**ytes, `j` = **j**son (flag `#` for pretty-printing), `h` = HTML escaping, `l` = a comma separated **l**ist (flag `#` for `\n` newline-separated), `q` = a string **q**uoted for the terminal (flag `#` to split a list into different arguments), `D` = add **D**ecimal suffixes (e.g. 10M) (flag `#` to use 1024 as factor), and `S` = **S**anitize as filename (flag `#` for restricted)
|
||||
1. **More Conversions**: In addition to the normal format types `diouxXeEfFgGcrs`, yt-dlp additionally supports converting to `B` = **B**ytes, `j` = **j**son (flag `#` for pretty-printing, `+` for Unicode), `h` = HTML escaping, `l` = a comma separated **l**ist (flag `#` for `\n` newline-separated), `q` = a string **q**uoted for the terminal (flag `#` to split a list into different arguments), `D` = add **D**ecimal suffixes (e.g. 10M) (flag `#` to use 1024 as factor), and `S` = **S**anitize as filename (flag `#` for restricted)
|
||||
|
||||
1. **Unicode normalization**: The format type `U` can be used for NFC [unicode normalization](https://docs.python.org/3/library/unicodedata.html#unicodedata.normalize). The alternate form flag (`#`) changes the normalization to NFD and the conversion flag `+` can be used for NFKC/NFKD compatibility equivalence normalization. E.g. `%(title)+.100U` is NFKC
|
||||
1. **Unicode normalization**: The format type `U` can be used for NFC [Unicode normalization](https://docs.python.org/3/library/unicodedata.html#unicodedata.normalize). The alternate form flag (`#`) changes the normalization to NFD and the conversion flag `+` can be used for NFKC/NFKD compatibility equivalence normalization. E.g. `%(title)+.100U` is NFKC
|
||||
|
||||
To summarize, the general syntax for a field is:
|
||||
```
|
||||
@ -1187,6 +1261,10 @@ # OUTPUT TEMPLATE
|
||||
|
||||
Additionally, you can set different output templates for the various metadata files separately from the general output template by specifying the type of file followed by the template separated by a colon `:`. The different file types supported are `subtitle`, `thumbnail`, `description`, `annotation` (deprecated), `infojson`, `link`, `pl_thumbnail`, `pl_description`, `pl_infojson`, `chapter`, `pl_video`. E.g. `-o "%(title)s.%(ext)s" -o "thumbnail:%(title)s\%(title)s.%(ext)s"` will put the thumbnails in a folder with the same name as the video. If any of the templates is empty, that type of file will not be written. E.g. `--write-thumbnail -o "thumbnail:"` will write thumbnails only for playlists and not for video.
|
||||
|
||||
<a id="outtmpl-postprocess-note"/>
|
||||
|
||||
**Note**: Due to post-processing (i.e. merging etc.), the actual output filename might differ. Use `--print after_move:filepath` to get the name after all post-processing is complete.
|
||||
|
||||
The available fields are:
|
||||
|
||||
- `id` (string): Video identifier
|
||||
@ -1213,6 +1291,7 @@ # OUTPUT TEMPLATE
|
||||
- `duration` (numeric): Length of the video in seconds
|
||||
- `duration_string` (string): Length of the video (HH:mm:ss)
|
||||
- `view_count` (numeric): How many users have watched the video on the platform
|
||||
- `concurrent_view_count` (numeric): How many users are currently watching the video on the platform.
|
||||
- `like_count` (numeric): Number of positive ratings of the video
|
||||
- `dislike_count` (numeric): Number of negative ratings of the video
|
||||
- `repost_count` (numeric): Number of reposts of the video
|
||||
@ -1286,7 +1365,7 @@ # OUTPUT TEMPLATE
|
||||
Available only when used in `--print`:
|
||||
|
||||
- `urls` (string): The URLs of all requested formats, one in each line
|
||||
- `filename` (string): Name of the video file. Note that the actual filename may be different due to post-processing. Use `--exec echo` to get the name after all postprocessing is complete
|
||||
- `filename` (string): Name of the video file. Note that the [actual filename may differ](#outtmpl-postprocess-note)
|
||||
- `formats_table` (table): The video format table as printed by `--list-formats`
|
||||
- `thumbnails_table` (table): The thumbnail format table as printed by `--list-thumbnails`
|
||||
- `subtitles_table` (table): The subtitle format table as printed by `--list-subs`
|
||||
@ -1297,14 +1376,15 @@ # OUTPUT TEMPLATE
|
||||
|
||||
- `start_time` (numeric): Start time of the chapter in seconds
|
||||
- `end_time` (numeric): End time of the chapter in seconds
|
||||
- `categories` (list): The SponsorBlock categories the chapter belongs to
|
||||
- `categories` (list): The [SponsorBlock categories](https://wiki.sponsor.ajay.app/w/Types#Category) the chapter belongs to
|
||||
- `category` (string): The smallest SponsorBlock category the chapter belongs to
|
||||
- `category_names` (list): Friendly names of the categories
|
||||
- `name` (string): Friendly name of the smallest category
|
||||
- `type` (string): The [SponsorBlock action type](https://wiki.sponsor.ajay.app/w/Types#Action_Type) of the chapter
|
||||
|
||||
Each aforementioned sequence when referenced in an output template will be replaced by the actual value corresponding to the sequence name. E.g. for `-o %(title)s-%(id)s.%(ext)s` and an mp4 video with title `yt-dlp test video` and id `BaW_jenozKc`, this will result in a `yt-dlp test video-BaW_jenozKc.mp4` file created in the current directory.
|
||||
|
||||
Note that some of the sequences are not guaranteed to be present since they depend on the metadata obtained by a particular extractor. Such sequences will be replaced with placeholder value provided with `--output-na-placeholder` (`NA` by default).
|
||||
**Note**: Some of the sequences are not guaranteed to be present since they depend on the metadata obtained by a particular extractor. Such sequences will be replaced with placeholder value provided with `--output-na-placeholder` (`NA` by default).
|
||||
|
||||
**Tip**: Look at the `-j` output to identify which fields are available for the particular URL
|
||||
|
||||
@ -1399,7 +1479,7 @@ # FORMAT SELECTION
|
||||
|
||||
You can select the n'th best format of a type by using `best<type>.<n>`. For example, `best.2` will select the 2nd best combined format. Similarly, `bv*.3` will select the 3rd best format that contains a video stream.
|
||||
|
||||
If you want to download multiple videos and they don't have the same formats available, you can specify the order of preference using slashes. Note that formats on the left hand side are preferred; e.g. `-f 22/17/18` will download format 22 if it's available, otherwise it will download format 17 if it's available, otherwise it will download format 18 if it's available, otherwise it will complain that no suitable formats are available for download.
|
||||
If you want to download multiple videos, and they don't have the same formats available, you can specify the order of preference using slashes. Note that formats on the left hand side are preferred; e.g. `-f 22/17/18` will download format 22 if it's available, otherwise it will download format 17 if it's available, otherwise it will download format 18 if it's available, otherwise it will complain that no suitable formats are available for download.
|
||||
|
||||
If you want to download several formats of the same video use a comma as a separator, e.g. `-f 22,17,18` will download all these three formats, of course if they are available. Or a more sophisticated example combined with the precedence feature: `-f 136/137/mp4/bestvideo,140/m4a/bestaudio`.
|
||||
|
||||
@ -1407,7 +1487,7 @@ # FORMAT SELECTION
|
||||
|
||||
**Deprecation warning**: Since the *below* described behavior is complex and counter-intuitive, this will be removed and multistreams will be enabled by default in the future. A new operator will be instead added to limit formats to single audio/video
|
||||
|
||||
Unless `--video-multistreams` is used, all formats with a video stream except the first one are ignored. Similarly, unless `--audio-multistreams` is used, all formats with an audio stream except the first one are ignored. E.g. `-f bestvideo+best+bestaudio --video-multistreams --audio-multistreams` will download and merge all 3 given formats. The resulting file will have 2 video streams and 2 audio streams. But `-f bestvideo+best+bestaudio --no-video-multistreams` will download and merge only `bestvideo` and `bestaudio`. `best` is ignored since another format containing a video stream (`bestvideo`) has already been selected. The order of the formats is therefore important. `-f best+bestaudio --no-audio-multistreams` will download and merge both formats while `-f bestaudio+best --no-audio-multistreams` will ignore `best` and download only `bestaudio`.
|
||||
Unless `--video-multistreams` is used, all formats with a video stream except the first one are ignored. Similarly, unless `--audio-multistreams` is used, all formats with an audio stream except the first one are ignored. E.g. `-f bestvideo+best+bestaudio --video-multistreams --audio-multistreams` will download and merge all 3 given formats. The resulting file will have 2 video streams and 2 audio streams. But `-f bestvideo+best+bestaudio --no-video-multistreams` will download and merge only `bestvideo` and `bestaudio`. `best` is ignored since another format containing a video stream (`bestvideo`) has already been selected. The order of the formats is therefore important. `-f best+bestaudio --no-audio-multistreams` will download only `best` while `-f bestaudio+best --no-audio-multistreams` will ignore `best` and download only `bestaudio`.
|
||||
|
||||
## Filtering Formats
|
||||
|
||||
@ -1419,6 +1499,7 @@ ## Filtering Formats
|
||||
- `filesize_approx`: An estimate for the number of bytes
|
||||
- `width`: Width of the video, if known
|
||||
- `height`: Height of the video, if known
|
||||
- `aspect_ratio`: Aspect ratio of the video, if known
|
||||
- `tbr`: Average bitrate of audio and video in KBit/s
|
||||
- `abr`: Average audio bitrate in KBit/s
|
||||
- `vbr`: Average video bitrate in KBit/s
|
||||
@ -1444,7 +1525,7 @@ ## Filtering Formats
|
||||
|
||||
Any string comparison may be prefixed with negation `!` in order to produce an opposite comparison, e.g. `!*=` (does not contain). The comparand of a string comparison needs to be quoted with either double or single quotes if it contains spaces or special characters other than `._-`.
|
||||
|
||||
Note that none of the aforementioned meta fields are guaranteed to be present since this solely depends on the metadata obtained by particular extractor, i.e. the metadata offered by the website. Any other field made available by the extractor can also be used for filtering.
|
||||
**Note**: None of the aforementioned meta fields are guaranteed to be present since this solely depends on the metadata obtained by particular extractor, i.e. the metadata offered by the website. Any other field made available by the extractor can also be used for filtering.
|
||||
|
||||
Formats for which the value is not known are excluded unless you put a question mark (`?`) after the operator. You can combine format filters, so `-f "[height<=?720][tbr>500]"` selects up to 720p videos (or videos where the height is not known) with a bitrate of at least 500 KBit/s. You can also use the filters with `all` to download all formats that satisfy the filter, e.g. `-f "all[vcodec=none]"` selects all audio-only formats.
|
||||
|
||||
@ -1456,17 +1537,17 @@ ## Sorting Formats
|
||||
|
||||
The available fields are:
|
||||
|
||||
- `hasvid`: Gives priority to formats that has a video stream
|
||||
- `hasaud`: Gives priority to formats that has a audio stream
|
||||
- `hasvid`: Gives priority to formats that have a video stream
|
||||
- `hasaud`: Gives priority to formats that have an audio stream
|
||||
- `ie_pref`: The format preference
|
||||
- `lang`: The language preference
|
||||
- `quality`: The quality of the format
|
||||
- `source`: The preference of the source
|
||||
- `proto`: Protocol used for download (`https`/`ftps` > `http`/`ftp` > `m3u8_native`/`m3u8` > `http_dash_segments`> `websocket_frag` > `mms`/`rtsp` > `f4f`/`f4m`)
|
||||
- `vcodec`: Video Codec (`av01` > `vp9.2` > `vp9` > `h265` > `h264` > `vp8` > `h263` > `theora` > other)
|
||||
- `acodec`: Audio Codec (`flac`/`alac` > `wav`/`aiff` > `opus` > `vorbis` > `aac` > `mp4a` > `mp3` > `eac3` > `ac3` > `dts` > other)
|
||||
- `acodec`: Audio Codec (`flac`/`alac` > `wav`/`aiff` > `opus` > `vorbis` > `aac` > `mp4a` > `mp3` > `ac4` > `eac3` > `ac3` > `dts` > other)
|
||||
- `codec`: Equivalent to `vcodec,acodec`
|
||||
- `vext`: Video Extension (`mp4` > `webm` > `flv` > other). If `--prefer-free-formats` is used, `webm` is preferred.
|
||||
- `vext`: Video Extension (`mp4` > `mov` > `webm` > `flv` > other). If `--prefer-free-formats` is used, `webm` is preferred.
|
||||
- `aext`: Audio Extension (`m4a` > `aac` > `mp3` > `ogg` > `opus` > `webm` > other). If `--prefer-free-formats` is used, the order changes to `ogg` > `opus` > `webm` > `mp3` > `m4a` > `aac`
|
||||
- `ext`: Equivalent to `vext,aext`
|
||||
- `filesize`: Exact filesize, if known in advance
|
||||
@ -1542,7 +1623,7 @@ # Download the best mp4 video available, or the best video if no mp4 available
|
||||
$ yt-dlp -f "bv*[ext=mp4]+ba[ext=m4a]/b[ext=mp4] / bv*+ba/b"
|
||||
|
||||
# Download the best video with the best extension
|
||||
# (For video, mp4 > webm > flv. For audio, m4a > aac > mp3 ...)
|
||||
# (For video, mp4 > mov > webm > flv. For audio, m4a > aac > mp3 ...)
|
||||
$ yt-dlp -S "ext"
|
||||
|
||||
|
||||
@ -1625,9 +1706,9 @@ # MODIFYING METADATA
|
||||
|
||||
`--replace-in-metadata FIELDS REGEX REPLACE` is used to replace text in any metadata field using [python regular expression](https://docs.python.org/3/library/re.html#regular-expression-syntax). [Backreferences](https://docs.python.org/3/library/re.html?highlight=backreferences#re.sub) can be used in the replace string for advanced use.
|
||||
|
||||
The general syntax of `--parse-metadata FROM:TO` is to give the name of a field or an [output template](#output-template) to extract data from, and the format to interpret it as, separated by a colon `:`. Either a [python regular expression](https://docs.python.org/3/library/re.html#regular-expression-syntax) with named capture groups or a similar syntax to the [output template](#output-template) (only `%(field)s` formatting is supported) can be used for `TO`. The option can be used multiple times to parse and modify various fields.
|
||||
The general syntax of `--parse-metadata FROM:TO` is to give the name of a field or an [output template](#output-template) to extract data from, and the format to interpret it as, separated by a colon `:`. Either a [python regular expression](https://docs.python.org/3/library/re.html#regular-expression-syntax) with named capture groups, a single field name, or a similar syntax to the [output template](#output-template) (only `%(field)s` formatting is supported) can be used for `TO`. The option can be used multiple times to parse and modify various fields.
|
||||
|
||||
Note that any field created by this can be used in the [output template](#output-template) and will also affect the media file's metadata added when using `--add-metadata`.
|
||||
Note that these options preserve their relative order, allowing replacements to be made in parsed fields and viceversa. Also, any field thus created can be used in the [output template](#output-template) and will also affect the media file's metadata added when using `--embed-metadata`.
|
||||
|
||||
This option also has a few special uses:
|
||||
|
||||
@ -1673,11 +1754,11 @@ # Set title as "Series name S01E05"
|
||||
$ yt-dlp --parse-metadata "%(series)s S%(season_number)02dE%(episode_number)02d:%(title)s"
|
||||
|
||||
# Prioritize uploader as the "artist" field in video metadata
|
||||
$ yt-dlp --parse-metadata "%(uploader|)s:%(meta_artist)s" --add-metadata
|
||||
$ yt-dlp --parse-metadata "%(uploader|)s:%(meta_artist)s" --embed-metadata
|
||||
|
||||
# Set "comment" field in video metadata using description instead of webpage_url,
|
||||
# handling multiple lines correctly
|
||||
$ yt-dlp --parse-metadata "description:(?s)(?P<meta_comment>.+)" --add-metadata
|
||||
$ yt-dlp --parse-metadata "description:(?s)(?P<meta_comment>.+)" --embed-metadata
|
||||
|
||||
# Do not set any "synopsis" in the video metadata
|
||||
$ yt-dlp --parse-metadata ":(?P<meta_synopsis>)"
|
||||
@ -1694,35 +1775,36 @@ # EXTRACTOR ARGUMENTS
|
||||
|
||||
Some extractors accept additional arguments which can be passed using `--extractor-args KEY:ARGS`. `ARGS` is a `;` (semicolon) separated string of `ARG=VAL1,VAL2`. E.g. `--extractor-args "youtube:player-client=android_embedded,web;include_live_dash" --extractor-args "funimation:version=uncut"`
|
||||
|
||||
Note: In CLI, `ARG` can use `-` instead of `_`; e.g. `youtube:player-client"` becomes `youtube:player_client"`
|
||||
|
||||
The following extractors use this feature:
|
||||
|
||||
#### youtube
|
||||
* `lang`: Prefer translated metadata (`title`, `description` etc) of this language code (case-sensitive). By default, the video primary language metadata is preferred, with a fallback to `en` translated. See [youtube.py](https://github.com/yt-dlp/yt-dlp/blob/c26f9b991a0681fd3ea548d535919cec1fbbd430/yt_dlp/extractor/youtube.py#L381-L390) for list of supported content language codes
|
||||
* `skip`: One or more of `hls`, `dash` or `translated_subs` to skip extraction of the m3u8 manifests, dash manifests and [auto-translated subtitles](https://github.com/yt-dlp/yt-dlp/issues/4090#issuecomment-1158102032) respectively
|
||||
* `player_client`: Clients to extract video data from. The main clients are `web`, `android` and `ios` with variants `_music`, `_embedded`, `_embedscreen`, `_creator` (e.g. `web_embedded`); and `mweb` and `tv_embedded` (agegate bypass) with no variants. By default, `android,web` is used, but `tv_embedded` and `creator` variants are added as required for age-gated videos. Similarly the music variants are added for `music.youtube.com` urls. You can use `all` to use all the clients, and `default` for the default clients.
|
||||
* `player_client`: Clients to extract video data from. The main clients are `web`, `android` and `ios` with variants `_music`, `_embedded`, `_embedscreen`, `_creator` (e.g. `web_embedded`); and `mweb` and `tv_embedded` (agegate bypass) with no variants. By default, `android,web` is used, but `tv_embedded` and `creator` variants are added as required for age-gated videos. Similarly, the music variants are added for `music.youtube.com` urls. You can use `all` to use all the clients, and `default` for the default clients.
|
||||
* `player_skip`: Skip some network requests that are generally needed for robust extraction. One or more of `configs` (skip client configs), `webpage` (skip initial webpage), `js` (skip js player). While these options can help reduce the number of requests needed or avoid some rate-limiting, they could cause some issues. See [#860](https://github.com/yt-dlp/yt-dlp/pull/860) for more details
|
||||
* `include_live_dash`: Include live dash formats even without `--live-from-start` (These formats don't download properly)
|
||||
* `comment_sort`: `top` or `new` (default) - choose comment sorting mode (on YouTube's side)
|
||||
* `max_comments`: Limit the amount of comments to gather. Comma-separated list of integers representing `max-comments,max-parents,max-replies,max-replies-per-thread`. Default is `all,all,all,all`
|
||||
* E.g. `all,all,1000,10` will get a maximum of 1000 replies total, with up to 10 replies per thread. `1000,all,100` will get a maximum of 1000 comments, with a maximum of 100 replies total
|
||||
* `include_incomplete_formats`: Extract formats that cannot be downloaded completely (live dash and post-live m3u8)
|
||||
* `innertube_host`: Innertube API host to use for all API requests; e.g. `studio.youtube.com`, `youtubei.googleapis.com`. Note that cookies exported from one subdomain will not work on others
|
||||
* `innertube_key`: Innertube API key to use for all API requests
|
||||
* `lang`: Language code to prefer translated metadata of this language (case-sensitive). By default, the video primary language metadata is preferred, with a fallback to `en` translated. See [youtube.py](https://github.com/yt-dlp/yt-dlp/blob/c26f9b991a0681fd3ea548d535919cec1fbbd430/yt_dlp/extractor/youtube.py#L381-L390) for list of supported content language codes
|
||||
|
||||
#### youtubetab (YouTube playlists, channels, feeds, etc.)
|
||||
* `skip`: One or more of `webpage` (skip initial webpage download), `authcheck` (allow the download of playlists requiring authentication when no initial webpage is downloaded. This may cause unwanted behavior, see [#1122](https://github.com/yt-dlp/yt-dlp/pull/1122) for more details)
|
||||
* `approximate_date`: Extract approximate `upload_date` in flat-playlist. This may cause date-based filters to be slightly off
|
||||
* `approximate_date`: Extract approximate `upload_date` and `timestamp` in flat-playlist. This may cause date-based filters to be slightly off
|
||||
|
||||
#### generic
|
||||
* `fragment_query`: Passthrough any query in mpd/m3u8 manifest URLs to their fragments. Does not apply to ffmpeg
|
||||
|
||||
#### funimation
|
||||
* `language`: Languages to extract, e.g. `funimation:language=english,japanese`
|
||||
* `language`: Audio languages to extract, e.g. `funimation:language=english,japanese`
|
||||
* `version`: The video version to extract - `uncut` or `simulcast`
|
||||
|
||||
#### crunchyroll
|
||||
* `language`: Languages to extract, e.g. `crunchyroll:language=jaJp`
|
||||
* `hardsub`: Which hard-sub versions to extract, e.g. `crunchyroll:hardsub=None,enUS`
|
||||
|
||||
#### crunchyrollbeta
|
||||
#### crunchyrollbeta (Crunchyroll)
|
||||
* `format`: Which stream type(s) to extract (default: `adaptive_hls`). Potentially useful values include `adaptive_hls`, `adaptive_dash`, `vo_adaptive_hls`, `vo_adaptive_dash`, `download_hls`, `download_dash`, `multitrack_adaptive_hls_v2`
|
||||
* `hardsub`: Preference order for which hardsub versions to extract (default: `None` = no hardsubs), e.g. `crunchyrollbeta:hardsub=en-US,None`
|
||||
* `hardsub`: Preference order for which hardsub versions to extract, or `all` (default: `None` = no hardsubs), e.g. `crunchyrollbeta:hardsub=en-US,None`
|
||||
|
||||
#### vikichannel
|
||||
* `video_types`: Types of videos to download - one or more of `episodes`, `movies`, `clips`, `trailers`
|
||||
@ -1742,33 +1824,88 @@ #### hotstar
|
||||
* `dr`: dynamic range to ignore - one or more of `sdr`, `hdr10`, `dv`
|
||||
|
||||
#### tiktok
|
||||
* `api_hostname`: Hostname to use for mobile API requests, e.g. `api-h2.tiktokv.com`
|
||||
* `app_version`: App version to call mobile APIs with - should be set along with `manifest_app_version`, e.g. `20.2.1`
|
||||
* `manifest_app_version`: Numeric app version to call mobile APIs with, e.g. `221`
|
||||
|
||||
#### rokfinchannel
|
||||
* `tab`: Which tab to download - one of `new`, `top`, `videos`, `podcasts`, `streams`, `stacks`
|
||||
|
||||
#### twitter
|
||||
* `force_graphql`: Force usage of the GraphQL API. By default it will only be used if login cookies are provided
|
||||
|
||||
NOTE: These options may be changed/removed in the future without concern for backward compatibility
|
||||
**Note**: These options may be changed/removed in the future without concern for backward compatibility
|
||||
|
||||
<!-- MANPAGE: MOVE "INSTALLATION" SECTION HERE -->
|
||||
|
||||
|
||||
# PLUGINS
|
||||
|
||||
Plugins are loaded from `<root-dir>/ytdlp_plugins/<type>/__init__.py`; where `<root-dir>` is the directory of the binary (`<root-dir>/yt-dlp`), or the root directory of the module if you are running directly from source-code (`<root dir>/yt_dlp/__main__.py`). Plugins are currently not supported for the `pip` version
|
||||
Note that **all** plugins are imported even if not invoked, and that **there are no checks** performed on plugin code. **Use plugins at your own risk and only if you trust the code!**
|
||||
|
||||
Plugins can be of `<type>`s `extractor` or `postprocessor`. Extractor plugins do not need to be enabled from the CLI and are automatically invoked when the input URL is suitable for it. Postprocessor plugins can be invoked using `--use-postprocessor NAME`.
|
||||
Plugins can be of `<type>`s `extractor` or `postprocessor`.
|
||||
- Extractor plugins do not need to be enabled from the CLI and are automatically invoked when the input URL is suitable for it.
|
||||
- Extractor plugins take priority over builtin extractors.
|
||||
- Postprocessor plugins can be invoked using `--use-postprocessor NAME`.
|
||||
|
||||
See [ytdlp_plugins](ytdlp_plugins) for example plugins.
|
||||
|
||||
Note that **all** plugins are imported even if not invoked, and that **there are no checks** performed on plugin code. Use plugins at your own risk and only if you trust the code
|
||||
Plugins are loaded from the namespace packages `yt_dlp_plugins.extractor` and `yt_dlp_plugins.postprocessor`.
|
||||
|
||||
If you are a plugin author, add [ytdlp-plugins](https://github.com/topics/ytdlp-plugins) as a topic to your repository for discoverability
|
||||
In other words, the file structure on the disk looks something like:
|
||||
|
||||
yt_dlp_plugins/
|
||||
extractor/
|
||||
myplugin.py
|
||||
postprocessor/
|
||||
myplugin.py
|
||||
|
||||
yt-dlp looks for these `yt_dlp_plugins` namespace folders in many locations (see below) and loads in plugins from **all** of them.
|
||||
|
||||
See the [wiki for some known plugins](https://github.com/yt-dlp/yt-dlp/wiki/Plugins)
|
||||
|
||||
## Installing Plugins
|
||||
|
||||
Plugins can be installed using various methods and locations.
|
||||
|
||||
1. **Configuration directories**:
|
||||
Plugin packages (containing a `yt_dlp_plugins` namespace folder) can be dropped into the following standard [configuration locations](#configuration):
|
||||
* **User Plugins**
|
||||
* `${XDG_CONFIG_HOME}/yt-dlp/plugins/<package name>/yt_dlp_plugins/` (recommended on Linux/macOS)
|
||||
* `${XDG_CONFIG_HOME}/yt-dlp-plugins/<package name>/yt_dlp_plugins/`
|
||||
* `${APPDATA}/yt-dlp/plugins/<package name>/yt_dlp_plugins/` (recommended on Windows)
|
||||
* `${APPDATA}/yt-dlp-plugins/<package name>/yt_dlp_plugins/`
|
||||
* `~/.yt-dlp/plugins/<package name>/yt_dlp_plugins/`
|
||||
* `~/yt-dlp-plugins/<package name>/yt_dlp_plugins/`
|
||||
* **System Plugins**
|
||||
* `/etc/yt-dlp/plugins/<package name>/yt_dlp_plugins/`
|
||||
* `/etc/yt-dlp-plugins/<package name>/yt_dlp_plugins/`
|
||||
2. **Executable location**: Plugin packages can similarly be installed in a `yt-dlp-plugins` directory under the executable location:
|
||||
* Binary: where `<root-dir>/yt-dlp.exe`, `<root-dir>/yt-dlp-plugins/<package name>/yt_dlp_plugins/`
|
||||
* Source: where `<root-dir>/yt_dlp/__main__.py`, `<root-dir>/yt-dlp-plugins/<package name>/yt_dlp_plugins/`
|
||||
|
||||
3. **pip and other locations in `PYTHONPATH`**
|
||||
* Plugin packages can be installed and managed using `pip`. See [yt-dlp-sample-plugins](https://github.com/yt-dlp/yt-dlp-sample-plugins) for an example.
|
||||
* Note: plugin files between plugin packages installed with pip must have unique filenames.
|
||||
* Any path in `PYTHONPATH` is searched in for the `yt_dlp_plugins` namespace folder.
|
||||
* Note: This does not apply for Pyinstaller/py2exe builds.
|
||||
|
||||
|
||||
`.zip`, `.egg` and `.whl` archives containing a `yt_dlp_plugins` namespace folder in their root are also supported as plugin packages.
|
||||
* e.g. `${XDG_CONFIG_HOME}/yt-dlp/plugins/mypluginpkg.zip` where `mypluginpkg.zip` contains `yt_dlp_plugins/<type>/myplugin.py`
|
||||
|
||||
Run yt-dlp with `--verbose` to check if the plugin has been loaded.
|
||||
|
||||
## Developing Plugins
|
||||
|
||||
See the [yt-dlp-sample-plugins](https://github.com/yt-dlp/yt-dlp-sample-plugins) repo for a template plugin package and the [Plugin Development](https://github.com/yt-dlp/yt-dlp/wiki/Plugin-Development) section of the wiki for a plugin development guide.
|
||||
|
||||
All public classes with a name ending in `IE`/`PP` are imported from each file for extractors and postprocessors repectively. This respects underscore prefix (e.g. `_MyBasePluginIE` is private) and `__all__`. Modules can similarly be excluded by prefixing the module name with an underscore (e.g. `_myplugin.py`).
|
||||
|
||||
To replace an existing extractor with a subclass of one, set the `plugin_name` class keyword argument (e.g. `class MyPluginIE(ABuiltInIE, plugin_name='myplugin')` will replace `ABuiltInIE` with `MyPluginIE`). Since the extractor replaces the parent, you should exclude the subclass extractor from being imported separately by making it private using one of the methods described above.
|
||||
|
||||
If you are a plugin author, add [yt-dlp-plugins](https://github.com/topics/yt-dlp-plugins) as a topic to your repository for discoverability.
|
||||
|
||||
See the [Developer Instructions](https://github.com/yt-dlp/yt-dlp/blob/master/CONTRIBUTING.md#developer-instructions) on how to write and test an extractor.
|
||||
|
||||
# EMBEDDING YT-DLP
|
||||
|
||||
@ -1786,7 +1923,7 @@ # EMBEDDING YT-DLP
|
||||
ydl.download(URLS)
|
||||
```
|
||||
|
||||
Most likely, you'll want to use various options. For a list of options available, have a look at [`yt_dlp/YoutubeDL.py`](yt_dlp/YoutubeDL.py#L180).
|
||||
Most likely, you'll want to use various options. For a list of options available, have a look at [`yt_dlp/YoutubeDL.py`](yt_dlp/YoutubeDL.py#L184).
|
||||
|
||||
**Tip**: If you are porting your code from youtube-dl to yt-dlp, one important point to look out for is that we do not guarantee the return value of `YoutubeDL.extract_info` to be json serializable, or even be a dictionary. It will be dictionary-like, but if you want to ensure it is a serializable dictionary, pass it through `YoutubeDL.sanitize_info` as shown in the [example below](#extracting-information)
|
||||
|
||||
@ -1996,6 +2133,7 @@ #### Redundant options
|
||||
--reject-title REGEX --match-filter "title !~= (?i)REGEX"
|
||||
--min-views COUNT --match-filter "view_count >=? COUNT"
|
||||
--max-views COUNT --match-filter "view_count <=? COUNT"
|
||||
--break-on-reject Use --break-match-filter
|
||||
--user-agent UA --add-header "User-Agent:UA"
|
||||
--referer URL --add-header "Referer:URL"
|
||||
--playlist-start NUMBER -I NUMBER:
|
||||
|
12
devscripts/changelog_override.json
Normal file
12
devscripts/changelog_override.json
Normal file
@ -0,0 +1,12 @@
|
||||
[
|
||||
{
|
||||
"action": "add",
|
||||
"when": "776d1c3f0c9b00399896dd2e40e78e9a43218109",
|
||||
"short": "[priority] **A new release type has been added!**\n * [`nightly`](https://github.com/yt-dlp/yt-dlp/releases/tag/nightly) builds will be made after each push, containing the latest fixes (but also possibly bugs).\n * When using `--update`/`-U`, a release binary will only update to its current channel (either `stable` or `nightly`).\n * The `--update-to` option has been added allowing the user more control over program upgrades (or downgrades).\n * `--update-to` can change the release channel (`stable`, `nightly`) and also upgrade or downgrade to specific tags.\n * **Usage**: `--update-to CHANNEL`, `--update-to TAG`, `--update-to CHANNEL@TAG`"
|
||||
},
|
||||
{
|
||||
"action": "add",
|
||||
"when": "776d1c3f0c9b00399896dd2e40e78e9a43218109",
|
||||
"short": "[priority] **YouTube throttling fixes!**"
|
||||
}
|
||||
]
|
96
devscripts/changelog_override.schema.json
Normal file
96
devscripts/changelog_override.schema.json
Normal file
@ -0,0 +1,96 @@
|
||||
{
|
||||
"$schema": "http://json-schema.org/draft/2020-12/schema",
|
||||
"type": "array",
|
||||
"uniqueItems": true,
|
||||
"items": {
|
||||
"type": "object",
|
||||
"oneOf": [
|
||||
{
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"action": {
|
||||
"enum": [
|
||||
"add"
|
||||
]
|
||||
},
|
||||
"when": {
|
||||
"type": "string",
|
||||
"pattern": "^([0-9a-f]{40}|\\d{4}\\.\\d{2}\\.\\d{2})$"
|
||||
},
|
||||
"hash": {
|
||||
"type": "string",
|
||||
"pattern": "^[0-9a-f]{40}$"
|
||||
},
|
||||
"short": {
|
||||
"type": "string"
|
||||
},
|
||||
"authors": {
|
||||
"type": "array",
|
||||
"items": {
|
||||
"type": "string"
|
||||
}
|
||||
}
|
||||
},
|
||||
"required": [
|
||||
"action",
|
||||
"short"
|
||||
]
|
||||
},
|
||||
{
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"action": {
|
||||
"enum": [
|
||||
"remove"
|
||||
]
|
||||
},
|
||||
"when": {
|
||||
"type": "string",
|
||||
"pattern": "^([0-9a-f]{40}|\\d{4}\\.\\d{2}\\.\\d{2})$"
|
||||
},
|
||||
"hash": {
|
||||
"type": "string",
|
||||
"pattern": "^[0-9a-f]{40}$"
|
||||
}
|
||||
},
|
||||
"required": [
|
||||
"action",
|
||||
"hash"
|
||||
]
|
||||
},
|
||||
{
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"action": {
|
||||
"enum": [
|
||||
"change"
|
||||
]
|
||||
},
|
||||
"when": {
|
||||
"type": "string",
|
||||
"pattern": "^([0-9a-f]{40}|\\d{4}\\.\\d{2}\\.\\d{2})$"
|
||||
},
|
||||
"hash": {
|
||||
"type": "string",
|
||||
"pattern": "^[0-9a-f]{40}$"
|
||||
},
|
||||
"short": {
|
||||
"type": "string"
|
||||
},
|
||||
"authors": {
|
||||
"type": "array",
|
||||
"items": {
|
||||
"type": "string"
|
||||
}
|
||||
}
|
||||
},
|
||||
"required": [
|
||||
"action",
|
||||
"hash",
|
||||
"short",
|
||||
"authors"
|
||||
]
|
||||
}
|
||||
]
|
||||
}
|
||||
}
|
@ -10,7 +10,7 @@
|
||||
)
|
||||
|
||||
# These bloat the lazy_extractors, so allow them to passthrough silently
|
||||
ALLOWED_CLASSMETHODS = {'get_testcases', 'extract_from_webpage'}
|
||||
ALLOWED_CLASSMETHODS = {'extract_from_webpage', 'get_testcases', 'get_webpage_testcases'}
|
||||
_WARNED = False
|
||||
|
||||
|
||||
|
470
devscripts/make_changelog.py
Normal file
470
devscripts/make_changelog.py
Normal file
@ -0,0 +1,470 @@
|
||||
from __future__ import annotations
|
||||
|
||||
# Allow direct execution
|
||||
import os
|
||||
import sys
|
||||
|
||||
sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
|
||||
|
||||
import enum
|
||||
import itertools
|
||||
import json
|
||||
import logging
|
||||
import re
|
||||
from collections import defaultdict
|
||||
from dataclasses import dataclass
|
||||
from functools import lru_cache
|
||||
from pathlib import Path
|
||||
|
||||
from devscripts.utils import read_file, run_process, write_file
|
||||
|
||||
BASE_URL = 'https://github.com'
|
||||
LOCATION_PATH = Path(__file__).parent
|
||||
HASH_LENGTH = 7
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
class CommitGroup(enum.Enum):
|
||||
UPSTREAM = None
|
||||
PRIORITY = 'Important'
|
||||
CORE = 'Core'
|
||||
EXTRACTOR = 'Extractor'
|
||||
DOWNLOADER = 'Downloader'
|
||||
POSTPROCESSOR = 'Postprocessor'
|
||||
MISC = 'Misc.'
|
||||
|
||||
@classmethod
|
||||
@lru_cache
|
||||
def commit_lookup(cls):
|
||||
return {
|
||||
name: group
|
||||
for group, names in {
|
||||
cls.PRIORITY: {''},
|
||||
cls.UPSTREAM: {'upstream'},
|
||||
cls.CORE: {
|
||||
'aes',
|
||||
'cache',
|
||||
'compat_utils',
|
||||
'compat',
|
||||
'cookies',
|
||||
'core',
|
||||
'dependencies',
|
||||
'jsinterp',
|
||||
'outtmpl',
|
||||
'plugins',
|
||||
'update',
|
||||
'utils',
|
||||
},
|
||||
cls.MISC: {
|
||||
'build',
|
||||
'cleanup',
|
||||
'devscripts',
|
||||
'docs',
|
||||
'misc',
|
||||
'test',
|
||||
},
|
||||
cls.EXTRACTOR: {'extractor', 'extractors'},
|
||||
cls.DOWNLOADER: {'downloader'},
|
||||
cls.POSTPROCESSOR: {'postprocessor'},
|
||||
}.items()
|
||||
for name in names
|
||||
}
|
||||
|
||||
@classmethod
|
||||
def get(cls, value):
|
||||
result = cls.commit_lookup().get(value)
|
||||
if result:
|
||||
logger.debug(f'Mapped {value!r} => {result.name}')
|
||||
return result
|
||||
|
||||
|
||||
@dataclass
|
||||
class Commit:
|
||||
hash: str | None
|
||||
short: str
|
||||
authors: list[str]
|
||||
|
||||
def __str__(self):
|
||||
result = f'{self.short!r}'
|
||||
|
||||
if self.hash:
|
||||
result += f' ({self.hash[:HASH_LENGTH]})'
|
||||
|
||||
if self.authors:
|
||||
authors = ', '.join(self.authors)
|
||||
result += f' by {authors}'
|
||||
|
||||
return result
|
||||
|
||||
|
||||
@dataclass
|
||||
class CommitInfo:
|
||||
details: str | None
|
||||
sub_details: tuple[str, ...]
|
||||
message: str
|
||||
issues: list[str]
|
||||
commit: Commit
|
||||
fixes: list[Commit]
|
||||
|
||||
def key(self):
|
||||
return ((self.details or '').lower(), self.sub_details, self.message)
|
||||
|
||||
|
||||
class Changelog:
|
||||
MISC_RE = re.compile(r'(?:^|\b)(?:lint(?:ing)?|misc|format(?:ting)?|fixes)(?:\b|$)', re.IGNORECASE)
|
||||
|
||||
def __init__(self, groups, repo):
|
||||
self._groups = groups
|
||||
self._repo = repo
|
||||
|
||||
def __str__(self):
|
||||
return '\n'.join(self._format_groups(self._groups)).replace('\t', ' ')
|
||||
|
||||
def _format_groups(self, groups):
|
||||
for item in CommitGroup:
|
||||
group = groups[item]
|
||||
if group:
|
||||
yield self.format_module(item.value, group)
|
||||
|
||||
def format_module(self, name, group):
|
||||
result = f'\n#### {name} changes\n' if name else '\n'
|
||||
return result + '\n'.join(self._format_group(group))
|
||||
|
||||
def _format_group(self, group):
|
||||
sorted_group = sorted(group, key=CommitInfo.key)
|
||||
detail_groups = itertools.groupby(sorted_group, lambda item: (item.details or '').lower())
|
||||
for _, items in detail_groups:
|
||||
items = list(items)
|
||||
details = items[0].details
|
||||
if not details:
|
||||
indent = ''
|
||||
else:
|
||||
yield f'- {details}'
|
||||
indent = '\t'
|
||||
|
||||
if details == 'cleanup':
|
||||
items, cleanup_misc_items = self._filter_cleanup_misc_items(items)
|
||||
|
||||
sub_detail_groups = itertools.groupby(items, lambda item: tuple(map(str.lower, item.sub_details)))
|
||||
for sub_details, entries in sub_detail_groups:
|
||||
if not sub_details:
|
||||
for entry in entries:
|
||||
yield f'{indent}- {self.format_single_change(entry)}'
|
||||
continue
|
||||
|
||||
entries = list(entries)
|
||||
prefix = f'{indent}- {", ".join(entries[0].sub_details)}'
|
||||
if len(entries) == 1:
|
||||
yield f'{prefix}: {self.format_single_change(entries[0])}'
|
||||
continue
|
||||
|
||||
yield prefix
|
||||
for entry in entries:
|
||||
yield f'{indent}\t- {self.format_single_change(entry)}'
|
||||
|
||||
if details == 'cleanup' and cleanup_misc_items:
|
||||
yield from self._format_cleanup_misc_sub_group(cleanup_misc_items)
|
||||
|
||||
def _filter_cleanup_misc_items(self, items):
|
||||
cleanup_misc_items = defaultdict(list)
|
||||
non_misc_items = []
|
||||
for item in items:
|
||||
if self.MISC_RE.search(item.message):
|
||||
cleanup_misc_items[tuple(item.commit.authors)].append(item)
|
||||
else:
|
||||
non_misc_items.append(item)
|
||||
|
||||
return non_misc_items, cleanup_misc_items
|
||||
|
||||
def _format_cleanup_misc_sub_group(self, group):
|
||||
prefix = '\t- Miscellaneous'
|
||||
if len(group) == 1:
|
||||
yield f'{prefix}: {next(self._format_cleanup_misc_items(group))}'
|
||||
return
|
||||
|
||||
yield prefix
|
||||
for message in self._format_cleanup_misc_items(group):
|
||||
yield f'\t\t- {message}'
|
||||
|
||||
def _format_cleanup_misc_items(self, group):
|
||||
for authors, infos in group.items():
|
||||
message = ', '.join(
|
||||
self._format_message_link(None, info.commit.hash)
|
||||
for info in sorted(infos, key=lambda item: item.commit.hash or ''))
|
||||
yield f'{message} by {self._format_authors(authors)}'
|
||||
|
||||
def format_single_change(self, info):
|
||||
message = self._format_message_link(info.message, info.commit.hash)
|
||||
if info.issues:
|
||||
message = f'{message} ({self._format_issues(info.issues)})'
|
||||
|
||||
if info.commit.authors:
|
||||
message = f'{message} by {self._format_authors(info.commit.authors)}'
|
||||
|
||||
if info.fixes:
|
||||
fix_message = ', '.join(f'{self._format_message_link(None, fix.hash)}' for fix in info.fixes)
|
||||
|
||||
authors = sorted({author for fix in info.fixes for author in fix.authors}, key=str.casefold)
|
||||
if authors != info.commit.authors:
|
||||
fix_message = f'{fix_message} by {self._format_authors(authors)}'
|
||||
|
||||
message = f'{message} (With fixes in {fix_message})'
|
||||
|
||||
return message
|
||||
|
||||
def _format_message_link(self, message, hash):
|
||||
assert message or hash, 'Improperly defined commit message or override'
|
||||
message = message if message else hash[:HASH_LENGTH]
|
||||
return f'[{message}]({self.repo_url}/commit/{hash})' if hash else message
|
||||
|
||||
def _format_issues(self, issues):
|
||||
return ', '.join(f'[#{issue}]({self.repo_url}/issues/{issue})' for issue in issues)
|
||||
|
||||
@staticmethod
|
||||
def _format_authors(authors):
|
||||
return ', '.join(f'[{author}]({BASE_URL}/{author})' for author in authors)
|
||||
|
||||
@property
|
||||
def repo_url(self):
|
||||
return f'{BASE_URL}/{self._repo}'
|
||||
|
||||
|
||||
class CommitRange:
|
||||
COMMAND = 'git'
|
||||
COMMIT_SEPARATOR = '-----'
|
||||
|
||||
AUTHOR_INDICATOR_RE = re.compile(r'Authored by:? ', re.IGNORECASE)
|
||||
MESSAGE_RE = re.compile(r'''
|
||||
(?:\[
|
||||
(?P<prefix>[^\]\/:,]+)
|
||||
(?:/(?P<details>[^\]:,]+))?
|
||||
(?:[:,](?P<sub_details>[^\]]+))?
|
||||
\]\ )?
|
||||
(?:(?P<sub_details_alt>`?[^:`]+`?): )?
|
||||
(?P<message>.+?)
|
||||
(?:\ \((?P<issues>\#\d+(?:,\ \#\d+)*)\))?
|
||||
''', re.VERBOSE | re.DOTALL)
|
||||
EXTRACTOR_INDICATOR_RE = re.compile(r'(?:Fix|Add)\s+Extractors?', re.IGNORECASE)
|
||||
FIXES_RE = re.compile(r'(?i:Fix(?:es)?(?:\s+bugs?)?(?:\s+in|\s+for)?|Revert)\s+([\da-f]{40})')
|
||||
UPSTREAM_MERGE_RE = re.compile(r'Update to ytdl-commit-([\da-f]+)')
|
||||
|
||||
def __init__(self, start, end, default_author=None):
|
||||
self._start, self._end = start, end
|
||||
self._commits, self._fixes = self._get_commits_and_fixes(default_author)
|
||||
self._commits_added = []
|
||||
|
||||
def __iter__(self):
|
||||
return iter(itertools.chain(self._commits.values(), self._commits_added))
|
||||
|
||||
def __len__(self):
|
||||
return len(self._commits) + len(self._commits_added)
|
||||
|
||||
def __contains__(self, commit):
|
||||
if isinstance(commit, Commit):
|
||||
if not commit.hash:
|
||||
return False
|
||||
commit = commit.hash
|
||||
|
||||
return commit in self._commits
|
||||
|
||||
def _get_commits_and_fixes(self, default_author):
|
||||
result = run_process(
|
||||
self.COMMAND, 'log', f'--format=%H%n%s%n%b%n{self.COMMIT_SEPARATOR}',
|
||||
f'{self._start}..{self._end}' if self._start else self._end).stdout
|
||||
|
||||
commits = {}
|
||||
fixes = defaultdict(list)
|
||||
lines = iter(result.splitlines(False))
|
||||
for i, commit_hash in enumerate(lines):
|
||||
short = next(lines)
|
||||
skip = short.startswith('Release ') or short == '[version] update'
|
||||
|
||||
authors = [default_author] if default_author else []
|
||||
for line in iter(lambda: next(lines), self.COMMIT_SEPARATOR):
|
||||
match = self.AUTHOR_INDICATOR_RE.match(line)
|
||||
if match:
|
||||
authors = sorted(map(str.strip, line[match.end():].split(',')), key=str.casefold)
|
||||
|
||||
commit = Commit(commit_hash, short, authors)
|
||||
if skip and (self._start or not i):
|
||||
logger.debug(f'Skipped commit: {commit}')
|
||||
continue
|
||||
elif skip:
|
||||
logger.debug(f'Reached Release commit, breaking: {commit}')
|
||||
break
|
||||
|
||||
fix_match = self.FIXES_RE.search(commit.short)
|
||||
if fix_match:
|
||||
commitish = fix_match.group(1)
|
||||
fixes[commitish].append(commit)
|
||||
|
||||
commits[commit.hash] = commit
|
||||
|
||||
for commitish, fix_commits in fixes.items():
|
||||
if commitish in commits:
|
||||
hashes = ', '.join(commit.hash[:HASH_LENGTH] for commit in fix_commits)
|
||||
logger.info(f'Found fix(es) for {commitish[:HASH_LENGTH]}: {hashes}')
|
||||
for fix_commit in fix_commits:
|
||||
del commits[fix_commit.hash]
|
||||
else:
|
||||
logger.debug(f'Commit with fixes not in changes: {commitish[:HASH_LENGTH]}')
|
||||
|
||||
return commits, fixes
|
||||
|
||||
def apply_overrides(self, overrides):
|
||||
for override in overrides:
|
||||
when = override.get('when')
|
||||
if when and when not in self and when != self._start:
|
||||
logger.debug(f'Ignored {when!r}, not in commits {self._start!r}')
|
||||
continue
|
||||
|
||||
override_hash = override.get('hash')
|
||||
if override['action'] == 'add':
|
||||
commit = Commit(override.get('hash'), override['short'], override.get('authors') or [])
|
||||
logger.info(f'ADD {commit}')
|
||||
self._commits_added.append(commit)
|
||||
|
||||
elif override['action'] == 'remove':
|
||||
if override_hash in self._commits:
|
||||
logger.info(f'REMOVE {self._commits[override_hash]}')
|
||||
del self._commits[override_hash]
|
||||
|
||||
elif override['action'] == 'change':
|
||||
if override_hash not in self._commits:
|
||||
continue
|
||||
commit = Commit(override_hash, override['short'], override['authors'])
|
||||
logger.info(f'CHANGE {self._commits[commit.hash]} -> {commit}')
|
||||
self._commits[commit.hash] = commit
|
||||
|
||||
self._commits = {key: value for key, value in reversed(self._commits.items())}
|
||||
|
||||
def groups(self):
|
||||
groups = defaultdict(list)
|
||||
for commit in self:
|
||||
upstream_re = self.UPSTREAM_MERGE_RE.match(commit.short)
|
||||
if upstream_re:
|
||||
commit.short = f'[upstream] Merge up to youtube-dl {upstream_re.group(1)}'
|
||||
|
||||
match = self.MESSAGE_RE.fullmatch(commit.short)
|
||||
if not match:
|
||||
logger.error(f'Error parsing short commit message: {commit.short!r}')
|
||||
continue
|
||||
|
||||
prefix, details, sub_details, sub_details_alt, message, issues = match.groups()
|
||||
group = None
|
||||
if prefix:
|
||||
if prefix == 'priority':
|
||||
prefix, _, details = (details or '').partition('/')
|
||||
logger.debug(f'Priority: {message!r}')
|
||||
group = CommitGroup.PRIORITY
|
||||
|
||||
if not details and prefix:
|
||||
if prefix not in ('core', 'downloader', 'extractor', 'misc', 'postprocessor', 'upstream'):
|
||||
logger.debug(f'Replaced details with {prefix!r}')
|
||||
details = prefix or None
|
||||
|
||||
if details == 'common':
|
||||
details = None
|
||||
|
||||
if details:
|
||||
details = details.strip()
|
||||
|
||||
else:
|
||||
group = CommitGroup.CORE
|
||||
|
||||
sub_details = f'{sub_details or ""},{sub_details_alt or ""}'.replace(':', ',')
|
||||
sub_details = tuple(filter(None, map(str.strip, sub_details.split(','))))
|
||||
|
||||
issues = [issue.strip()[1:] for issue in issues.split(',')] if issues else []
|
||||
|
||||
if not group:
|
||||
group = CommitGroup.get(prefix.lower())
|
||||
if not group:
|
||||
if self.EXTRACTOR_INDICATOR_RE.search(commit.short):
|
||||
group = CommitGroup.EXTRACTOR
|
||||
else:
|
||||
group = CommitGroup.POSTPROCESSOR
|
||||
logger.warning(f'Failed to map {commit.short!r}, selected {group.name}')
|
||||
|
||||
commit_info = CommitInfo(
|
||||
details, sub_details, message.strip(),
|
||||
issues, commit, self._fixes[commit.hash])
|
||||
logger.debug(f'Resolved {commit.short!r} to {commit_info!r}')
|
||||
groups[group].append(commit_info)
|
||||
|
||||
return groups
|
||||
|
||||
|
||||
def get_new_contributors(contributors_path, commits):
|
||||
contributors = set()
|
||||
if contributors_path.exists():
|
||||
for line in read_file(contributors_path).splitlines():
|
||||
author, _, _ = line.strip().partition(' (')
|
||||
authors = author.split('/')
|
||||
contributors.update(map(str.casefold, authors))
|
||||
|
||||
new_contributors = set()
|
||||
for commit in commits:
|
||||
for author in commit.authors:
|
||||
author_folded = author.casefold()
|
||||
if author_folded not in contributors:
|
||||
contributors.add(author_folded)
|
||||
new_contributors.add(author)
|
||||
|
||||
return sorted(new_contributors, key=str.casefold)
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
import argparse
|
||||
|
||||
parser = argparse.ArgumentParser(
|
||||
description='Create a changelog markdown from a git commit range')
|
||||
parser.add_argument(
|
||||
'commitish', default='HEAD', nargs='?',
|
||||
help='The commitish to create the range from (default: %(default)s)')
|
||||
parser.add_argument(
|
||||
'-v', '--verbosity', action='count', default=0,
|
||||
help='increase verbosity (can be used twice)')
|
||||
parser.add_argument(
|
||||
'-c', '--contributors', action='store_true',
|
||||
help='update CONTRIBUTORS file (default: %(default)s)')
|
||||
parser.add_argument(
|
||||
'--contributors-path', type=Path, default=LOCATION_PATH.parent / 'CONTRIBUTORS',
|
||||
help='path to the CONTRIBUTORS file')
|
||||
parser.add_argument(
|
||||
'--no-override', action='store_true',
|
||||
help='skip override json in commit generation (default: %(default)s)')
|
||||
parser.add_argument(
|
||||
'--override-path', type=Path, default=LOCATION_PATH / 'changelog_override.json',
|
||||
help='path to the changelog_override.json file')
|
||||
parser.add_argument(
|
||||
'--default-author', default='pukkandan',
|
||||
help='the author to use without a author indicator (default: %(default)s)')
|
||||
parser.add_argument(
|
||||
'--repo', default='yt-dlp/yt-dlp',
|
||||
help='the github repository to use for the operations (default: %(default)s)')
|
||||
args = parser.parse_args()
|
||||
|
||||
logging.basicConfig(
|
||||
datefmt='%Y-%m-%d %H-%M-%S', format='{asctime} | {levelname:<8} | {message}',
|
||||
level=logging.WARNING - 10 * args.verbosity, style='{', stream=sys.stderr)
|
||||
|
||||
commits = CommitRange(None, args.commitish, args.default_author)
|
||||
|
||||
if not args.no_override:
|
||||
if args.override_path.exists():
|
||||
overrides = json.loads(read_file(args.override_path))
|
||||
commits.apply_overrides(overrides)
|
||||
else:
|
||||
logger.warning(f'File {args.override_path.as_posix()} does not exist')
|
||||
|
||||
logger.info(f'Loaded {len(commits)} commits')
|
||||
|
||||
new_contributors = get_new_contributors(args.contributors_path, commits)
|
||||
if new_contributors:
|
||||
if args.contributors:
|
||||
write_file(args.contributors_path, '\n'.join(new_contributors) + '\n', mode='a')
|
||||
logger.info(f'New contributors: {", ".join(new_contributors)}')
|
||||
|
||||
print(Changelog(commits.groups(), args.repo))
|
@ -24,6 +24,8 @@
|
||||
options:
|
||||
- label: Run **your** yt-dlp command with **-vU** flag added (`yt-dlp -vU <your command line>`)
|
||||
required: true
|
||||
- label: "If using API, add `'verbose': True` to `YoutubeDL` params instead"
|
||||
required: false
|
||||
- label: Copy the WHOLE output (starting with `[debug] Command-line config`) and insert it below
|
||||
required: true
|
||||
- type: textarea
|
||||
@ -58,7 +60,7 @@
|
||||
label: DO NOT REMOVE OR SKIP THE ISSUE TEMPLATE
|
||||
description: Fill all fields even if you think it is irrelevant for the issue
|
||||
options:
|
||||
- label: I understand that I will be **blocked** if I remove or skip any mandatory\\* field
|
||||
- label: I understand that I will be **blocked** if I *intentionally* remove or skip any mandatory\\* field
|
||||
required: true
|
||||
'''.strip()
|
||||
|
||||
|
@ -2,6 +2,7 @@
|
||||
|
||||
# Allow direct execution
|
||||
import os
|
||||
import shutil
|
||||
import sys
|
||||
|
||||
sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
|
||||
@ -13,10 +14,17 @@
|
||||
|
||||
NO_ATTR = object()
|
||||
STATIC_CLASS_PROPERTIES = [
|
||||
'IE_NAME', 'IE_DESC', 'SEARCH_KEY', '_VALID_URL', '_WORKING', '_ENABLED', '_NETRC_MACHINE', 'age_limit'
|
||||
'IE_NAME', '_ENABLED', '_VALID_URL', # Used for URL matching
|
||||
'_WORKING', 'IE_DESC', '_NETRC_MACHINE', 'SEARCH_KEY', # Used for --extractor-descriptions
|
||||
'age_limit', # Used for --age-limit (evaluated)
|
||||
'_RETURN_TYPE', # Accessed in CLI only with instance (evaluated)
|
||||
]
|
||||
CLASS_METHODS = [
|
||||
'ie_key', 'working', 'description', 'suitable', '_match_valid_url', '_match_id', 'get_temp_id', 'is_suitable'
|
||||
'ie_key', 'suitable', '_match_valid_url', # Used for URL matching
|
||||
'working', 'get_temp_id', '_match_id', # Accessed just before instance creation
|
||||
'description', # Used for --extractor-descriptions
|
||||
'is_suitable', # Used for --age-limit
|
||||
'supports_login', 'is_single_video', # Accessed in CLI only with instance
|
||||
]
|
||||
IE_TEMPLATE = '''
|
||||
class {name}({bases}):
|
||||
@ -32,8 +40,12 @@ def main():
|
||||
|
||||
_ALL_CLASSES = get_all_ies() # Must be before import
|
||||
|
||||
import yt_dlp.plugins
|
||||
from yt_dlp.extractor.common import InfoExtractor, SearchInfoExtractor
|
||||
|
||||
# Filter out plugins
|
||||
_ALL_CLASSES = [cls for cls in _ALL_CLASSES if not cls.__module__.startswith(f'{yt_dlp.plugins.PACKAGE_NAME}.')]
|
||||
|
||||
DummyInfoExtractor = type('InfoExtractor', (InfoExtractor,), {'IE_NAME': NO_ATTR})
|
||||
module_src = '\n'.join((
|
||||
MODULE_TEMPLATE,
|
||||
@ -50,12 +62,13 @@ def get_all_ies():
|
||||
PLUGINS_DIRNAME = 'ytdlp_plugins'
|
||||
BLOCKED_DIRNAME = f'{PLUGINS_DIRNAME}_blocked'
|
||||
if os.path.exists(PLUGINS_DIRNAME):
|
||||
os.rename(PLUGINS_DIRNAME, BLOCKED_DIRNAME)
|
||||
# os.rename cannot be used, e.g. in Docker. See https://github.com/yt-dlp/yt-dlp/pull/4958
|
||||
shutil.move(PLUGINS_DIRNAME, BLOCKED_DIRNAME)
|
||||
try:
|
||||
from yt_dlp.extractor.extractors import _ALL_CLASSES
|
||||
finally:
|
||||
if os.path.exists(BLOCKED_DIRNAME):
|
||||
os.rename(BLOCKED_DIRNAME, PLUGINS_DIRNAME)
|
||||
shutil.move(BLOCKED_DIRNAME, PLUGINS_DIRNAME)
|
||||
return _ALL_CLASSES
|
||||
|
||||
|
||||
|
@ -45,33 +45,43 @@ def apply_patch(text, patch):
|
||||
delim = f'\n{" " * switch_col_width}'
|
||||
|
||||
PATCHES = (
|
||||
( # Standardize update message
|
||||
( # Standardize `--update` message
|
||||
r'(?m)^( -U, --update\s+).+(\n \s.+)*$',
|
||||
r'\1Update this program to the latest version',
|
||||
),
|
||||
( # Headings
|
||||
( # Headings
|
||||
r'(?m)^ (\w.+\n)( (?=\w))?',
|
||||
r'## \1'
|
||||
),
|
||||
( # Do not split URLs
|
||||
( # Fixup `--date` formatting
|
||||
rf'(?m)( --date DATE.+({delim}[^\[]+)*)\[.+({delim}.+)*$',
|
||||
(rf'\1[now|today|yesterday][-N[day|week|month|year]].{delim}'
|
||||
f'E.g. "--date today-2weeks" downloads only{delim}'
|
||||
'videos uploaded on the same day two weeks ago'),
|
||||
),
|
||||
( # Do not split URLs
|
||||
rf'({delim[:-1]})? (?P<label>\[\S+\] )?(?P<url>https?({delim})?:({delim})?/({delim})?/(({delim})?\S+)+)\s',
|
||||
lambda mobj: ''.join((delim, mobj.group('label') or '', re.sub(r'\s+', '', mobj.group('url')), '\n'))
|
||||
),
|
||||
( # Do not split "words"
|
||||
( # Do not split "words"
|
||||
rf'(?m)({delim}\S+)+$',
|
||||
lambda mobj: ''.join((delim, mobj.group(0).replace(delim, '')))
|
||||
),
|
||||
( # Allow overshooting last line
|
||||
( # Allow overshooting last line
|
||||
rf'(?m)^(?P<prev>.+)${delim}(?P<current>.+)$(?!{delim})',
|
||||
lambda mobj: (mobj.group().replace(delim, ' ')
|
||||
if len(mobj.group()) - len(delim) + 1 <= max_width + ALLOWED_OVERSHOOT
|
||||
else mobj.group())
|
||||
),
|
||||
( # Avoid newline when a space is available b/w switch and description
|
||||
( # Avoid newline when a space is available b/w switch and description
|
||||
DISABLE_PATCH, # This creates issues with prepare_manpage
|
||||
r'(?m)^(\s{4}-.{%d})(%s)' % (switch_col_width - 6, delim),
|
||||
r'\1 '
|
||||
),
|
||||
( # Replace brackets with a Markdown link
|
||||
r'SponsorBlock API \((http.+)\)',
|
||||
r'[SponsorBlock API](\1)'
|
||||
),
|
||||
)
|
||||
|
||||
readme = read_file(README_FILE)
|
||||
|
@ -7,16 +7,17 @@
|
||||
sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
|
||||
|
||||
|
||||
import argparse
|
||||
import contextlib
|
||||
import subprocess
|
||||
import sys
|
||||
from datetime import datetime
|
||||
|
||||
from devscripts.utils import read_version, write_file
|
||||
from devscripts.utils import read_version, run_process, write_file
|
||||
|
||||
|
||||
def get_new_version(revision):
|
||||
version = datetime.utcnow().strftime('%Y.%m.%d')
|
||||
def get_new_version(version, revision):
|
||||
if not version:
|
||||
version = datetime.utcnow().strftime('%Y.%m.%d')
|
||||
|
||||
if revision:
|
||||
assert revision.isdigit(), 'Revision must be a number'
|
||||
@ -30,25 +31,41 @@ def get_new_version(revision):
|
||||
|
||||
def get_git_head():
|
||||
with contextlib.suppress(Exception):
|
||||
sp = subprocess.Popen(['git', 'rev-parse', '--short', 'HEAD'], stdout=subprocess.PIPE)
|
||||
return sp.communicate()[0].decode().strip() or None
|
||||
return run_process('git', 'rev-parse', 'HEAD').stdout.strip()
|
||||
|
||||
|
||||
VERSION = get_new_version((sys.argv + [''])[1])
|
||||
GIT_HEAD = get_git_head()
|
||||
|
||||
VERSION_FILE = f'''\
|
||||
VERSION_TEMPLATE = '''\
|
||||
# Autogenerated by devscripts/update-version.py
|
||||
|
||||
__version__ = {VERSION!r}
|
||||
__version__ = {version!r}
|
||||
|
||||
RELEASE_GIT_HEAD = {GIT_HEAD!r}
|
||||
RELEASE_GIT_HEAD = {git_head!r}
|
||||
|
||||
VARIANT = None
|
||||
|
||||
UPDATE_HINT = None
|
||||
|
||||
CHANNEL = {channel!r}
|
||||
'''
|
||||
|
||||
write_file('yt_dlp/version.py', VERSION_FILE)
|
||||
print(f'::set-output name=ytdlp_version::{VERSION}')
|
||||
print(f'\nVersion = {VERSION}, Git HEAD = {GIT_HEAD}')
|
||||
if __name__ == '__main__':
|
||||
parser = argparse.ArgumentParser(description='Update the version.py file')
|
||||
parser.add_argument(
|
||||
'-c', '--channel', choices=['stable', 'nightly'], default='stable',
|
||||
help='Select update channel (default: %(default)s)')
|
||||
parser.add_argument(
|
||||
'-o', '--output', default='yt_dlp/version.py',
|
||||
help='The output file to write to (default: %(default)s)')
|
||||
parser.add_argument(
|
||||
'version', nargs='?', default=None,
|
||||
help='A version or revision to use instead of generating one')
|
||||
args = parser.parse_args()
|
||||
|
||||
git_head = get_git_head()
|
||||
version = (
|
||||
args.version if args.version and '.' in args.version
|
||||
else get_new_version(None, args.version))
|
||||
write_file(args.output, VERSION_TEMPLATE.format(
|
||||
version=version, git_head=git_head, channel=args.channel))
|
||||
|
||||
print(f'version={version} ({args.channel}), head={git_head}')
|
||||
|
@ -1,5 +1,6 @@
|
||||
import argparse
|
||||
import functools
|
||||
import subprocess
|
||||
|
||||
|
||||
def read_file(fname):
|
||||
@ -7,13 +8,13 @@ def read_file(fname):
|
||||
return f.read()
|
||||
|
||||
|
||||
def write_file(fname, content):
|
||||
with open(fname, 'w', encoding='utf-8') as f:
|
||||
def write_file(fname, content, mode='w'):
|
||||
with open(fname, mode, encoding='utf-8') as f:
|
||||
return f.write(content)
|
||||
|
||||
|
||||
# Get the version without importing the package
|
||||
def read_version(fname='yt_dlp/version.py'):
|
||||
"""Get the version without importing the package"""
|
||||
exec(compile(read_file(fname), fname, 'exec'))
|
||||
return locals()['__version__']
|
||||
|
||||
@ -33,3 +34,13 @@ def get_filename_args(has_infile=False, default_outfile=None):
|
||||
|
||||
def compose_functions(*functions):
|
||||
return lambda x: functools.reduce(lambda y, f: f(y), functions, x)
|
||||
|
||||
|
||||
def run_process(*args, **kwargs):
|
||||
kwargs.setdefault('text', True)
|
||||
kwargs.setdefault('check', True)
|
||||
kwargs.setdefault('capture_output', True)
|
||||
if kwargs['text']:
|
||||
kwargs.setdefault('encoding', 'utf-8')
|
||||
kwargs.setdefault('errors', 'replace')
|
||||
return subprocess.run(args, **kwargs)
|
||||
|
29
public.key
Normal file
29
public.key
Normal file
@ -0,0 +1,29 @@
|
||||
-----BEGIN PGP PUBLIC KEY BLOCK-----
|
||||
|
||||
mQINBGP78C4BEAD0rF9zjGPAt0thlt5C1ebzccAVX7Nb1v+eqQjk+WEZdTETVCg3
|
||||
WAM5ngArlHdm/fZqzUgO+pAYrB60GKeg7ffUDf+S0XFKEZdeRLYeAaqqKhSibVal
|
||||
DjvOBOztu3W607HLETQAqA7wTPuIt2WqmpL60NIcyr27LxqmgdN3mNvZ2iLO+bP0
|
||||
nKR/C+PgE9H4ytywDa12zMx6PmZCnVOOOu6XZEFmdUxxdQ9fFDqd9LcBKY2LDOcS
|
||||
Yo1saY0YWiZWHtzVoZu1kOzjnS5Fjq/yBHJLImDH7pNxHm7s/PnaurpmQFtDFruk
|
||||
t+2lhDnpKUmGr/I/3IHqH/X+9nPoS4uiqQ5HpblB8BK+4WfpaiEg75LnvuOPfZIP
|
||||
KYyXa/0A7QojMwgOrD88ozT+VCkKkkJ+ijXZ7gHNjmcBaUdKK7fDIEOYI63Lyc6Q
|
||||
WkGQTigFffSUXWHDCO9aXNhP3ejqFWgGMtCUsrbkcJkWuWY7q5ARy/05HbSM3K4D
|
||||
U9eqtnxmiV1WQ8nXuI9JgJQRvh5PTkny5LtxqzcmqvWO9TjHBbrs14BPEO9fcXxK
|
||||
L/CFBbzXDSvvAgArdqqlMoncQ/yicTlfL6qzJ8EKFiqW14QMTdAn6SuuZTodXCTi
|
||||
InwoT7WjjuFPKKdvfH1GP4bnqdzTnzLxCSDIEtfyfPsIX+9GI7Jkk/zZjQARAQAB
|
||||
tDdTaW1vbiBTYXdpY2tpICh5dC1kbHAgc2lnbmluZyBrZXkpIDxjb250YWN0QGdy
|
||||
dWI0ay54eXo+iQJOBBMBCgA4FiEErAy75oSNaoc0ZK9OV89lkztadYEFAmP78C4C
|
||||
GwMFCwkIBwIGFQoJCAsCBBYCAwECHgECF4AACgkQV89lkztadYEVqQ//cW7TxhXg
|
||||
7Xbh2EZQzXml0egn6j8QaV9KzGragMiShrlvTO2zXfLXqyizrFP4AspgjSn/4NrI
|
||||
8mluom+Yi+qr7DXT4BjQqIM9y3AjwZPdywe912Lxcw52NNoPZCm24I9T7ySc8lmR
|
||||
FQvZC0w4H/VTNj/2lgJ1dwMflpwvNRiWa5YzcFGlCUeDIPskLx9++AJE+xwU3LYm
|
||||
jQQsPBqpHHiTBEJzMLl+rfd9Fg4N+QNzpFkTDW3EPerLuvJniSBBwZthqxeAtw4M
|
||||
UiAXh6JvCc2hJkKCoygRfM281MeolvmsGNyQm+axlB0vyldiPP6BnaRgZlx+l6MU
|
||||
cPqgHblb7RW5j9lfr6OYL7SceBIHNv0CFrt1OnkGo/tVMwcs8LH3Ae4a7UJlIceL
|
||||
V54aRxSsZU7w4iX+PB79BWkEsQzwKrUuJVOeL4UDwWajp75OFaUqbS/slDDVXvK5
|
||||
OIeuth3mA/adjdvgjPxhRQjA3l69rRWIJDrqBSHldmRsnX6cvXTDy8wSXZgy51lP
|
||||
m4IVLHnCy9m4SaGGoAsfTZS0cC9FgjUIyTyrq9M67wOMpUxnuB0aRZgJE1DsI23E
|
||||
qdvcSNVlO+39xM/KPWUEh6b83wMn88QeW+DCVGWACQq5N3YdPnAJa50617fGbY6I
|
||||
gXIoRHXkDqe23PZ/jURYCv0sjVtjPoVC+bg=
|
||||
=bJkn
|
||||
-----END PGP PUBLIC KEY BLOCK-----
|
39
pyinst.py
39
pyinst.py
@ -12,9 +12,8 @@
|
||||
|
||||
from devscripts.utils import read_version
|
||||
|
||||
OS_NAME, MACHINE, ARCH = sys.platform, platform.machine(), platform.architecture()[0][:2]
|
||||
if MACHINE in ('x86_64', 'AMD64') or ('i' in MACHINE and '86' in MACHINE):
|
||||
# NB: Windows x86 has MACHINE = AMD64 irrespective of bitness
|
||||
OS_NAME, MACHINE, ARCH = sys.platform, platform.machine().lower(), platform.architecture()[0][:2]
|
||||
if MACHINE in ('x86', 'x86_64', 'amd64', 'i386', 'i686'):
|
||||
MACHINE = 'x86' if ARCH == '32' else ''
|
||||
|
||||
|
||||
@ -38,7 +37,7 @@ def main():
|
||||
'--icon=devscripts/logo.ico',
|
||||
'--upx-exclude=vcruntime140.dll',
|
||||
'--noconfirm',
|
||||
*dependency_options(),
|
||||
'--additional-hooks-dir=yt_dlp/__pyinstaller',
|
||||
*opts,
|
||||
'yt_dlp/__main__.py',
|
||||
]
|
||||
@ -63,7 +62,7 @@ def exe(onedir):
|
||||
name = '_'.join(filter(None, (
|
||||
'yt-dlp',
|
||||
{'win32': '', 'darwin': 'macos'}.get(OS_NAME, OS_NAME),
|
||||
MACHINE
|
||||
MACHINE,
|
||||
)))
|
||||
return name, ''.join(filter(None, (
|
||||
'dist/',
|
||||
@ -78,30 +77,6 @@ def version_to_list(version):
|
||||
return list(map(int, version_list)) + [0] * (4 - len(version_list))
|
||||
|
||||
|
||||
def dependency_options():
|
||||
# Due to the current implementation, these are auto-detected, but explicitly add them just in case
|
||||
dependencies = [pycryptodome_module(), 'mutagen', 'brotli', 'certifi', 'websockets']
|
||||
excluded_modules = ('youtube_dl', 'youtube_dlc', 'test', 'ytdlp_plugins', 'devscripts')
|
||||
|
||||
yield from (f'--hidden-import={module}' for module in dependencies)
|
||||
yield '--collect-submodules=websockets'
|
||||
yield from (f'--exclude-module={module}' for module in excluded_modules)
|
||||
|
||||
|
||||
def pycryptodome_module():
|
||||
try:
|
||||
import Cryptodome # noqa: F401
|
||||
except ImportError:
|
||||
try:
|
||||
import Crypto # noqa: F401
|
||||
print('WARNING: Using Crypto since Cryptodome is not available. '
|
||||
'Install with: pip install pycryptodomex', file=sys.stderr)
|
||||
return 'Crypto'
|
||||
except ImportError:
|
||||
pass
|
||||
return 'Cryptodome'
|
||||
|
||||
|
||||
def set_version_info(exe, version):
|
||||
if OS_NAME == 'win32':
|
||||
windows_set_version(exe, version)
|
||||
@ -110,7 +85,6 @@ def set_version_info(exe, version):
|
||||
def windows_set_version(exe, version):
|
||||
from PyInstaller.utils.win32.versioninfo import (
|
||||
FixedFileInfo,
|
||||
SetVersion,
|
||||
StringFileInfo,
|
||||
StringStruct,
|
||||
StringTable,
|
||||
@ -119,6 +93,11 @@ def windows_set_version(exe, version):
|
||||
VSVersionInfo,
|
||||
)
|
||||
|
||||
try:
|
||||
from PyInstaller.utils.win32.versioninfo import SetVersion
|
||||
except ImportError: # Pyinstaller >= 5.8
|
||||
from PyInstaller.utils.win32.versioninfo import write_version_info_to_executable as SetVersion
|
||||
|
||||
version_list = version_to_list(version)
|
||||
suffix = MACHINE and f'_{MACHINE}'
|
||||
SetVersion(exe, VSVersionInfo(
|
||||
|
5
pyproject.toml
Normal file
5
pyproject.toml
Normal file
@ -0,0 +1,5 @@
|
||||
[build-system]
|
||||
build-backend = 'setuptools.build_meta'
|
||||
# https://github.com/yt-dlp/yt-dlp/issues/5941
|
||||
# https://github.com/pypa/distutils/issues/17
|
||||
requires = ['setuptools > 50']
|
12
setup.cfg
12
setup.cfg
@ -10,6 +10,14 @@ per_file_ignores =
|
||||
devscripts/lazy_load_template.py: F401
|
||||
|
||||
|
||||
[autoflake]
|
||||
ignore-init-module-imports = true
|
||||
ignore-pass-after-docstring = true
|
||||
remove-all-unused-imports = true
|
||||
remove-duplicate-keys = true
|
||||
remove-unused-variables = true
|
||||
|
||||
|
||||
[tool:pytest]
|
||||
addopts = -ra -v --strict-markers
|
||||
markers =
|
||||
@ -18,12 +26,12 @@ markers =
|
||||
|
||||
[tox:tox]
|
||||
skipsdist = true
|
||||
envlist = py{36,37,38,39,310},pypy{36,37,38,39}
|
||||
envlist = py{36,37,38,39,310,311},pypy{36,37,38,39}
|
||||
skip_missing_interpreters = true
|
||||
|
||||
[testenv] # tox
|
||||
deps =
|
||||
pytest
|
||||
pytest
|
||||
commands = pytest {posargs:"-m not download"}
|
||||
passenv = HOME # For test_compat_expanduser
|
||||
setenv =
|
||||
|
139
setup.py
139
setup.py
@ -1,8 +1,12 @@
|
||||
#!/usr/bin/env python3
|
||||
|
||||
import os.path
|
||||
import subprocess
|
||||
# Allow execution from anywhere
|
||||
import os
|
||||
import sys
|
||||
|
||||
sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
|
||||
|
||||
import subprocess
|
||||
import warnings
|
||||
|
||||
try:
|
||||
@ -32,41 +36,38 @@ def packages():
|
||||
|
||||
return [
|
||||
'yt_dlp', 'yt_dlp.extractor', 'yt_dlp.downloader', 'yt_dlp.postprocessor', 'yt_dlp.compat',
|
||||
'yt_dlp.extractor.anvato_token_generator',
|
||||
]
|
||||
|
||||
|
||||
def py2exe_params():
|
||||
import py2exe # noqa: F401
|
||||
|
||||
warnings.warn(
|
||||
'py2exe builds do not support pycryptodomex and needs VC++14 to run. '
|
||||
'The recommended way is to use "pyinst.py" to build using pyinstaller')
|
||||
'It is recommended to run "pyinst.py" to build using pyinstaller instead')
|
||||
|
||||
return {
|
||||
'console': [{
|
||||
'script': './yt_dlp/__main__.py',
|
||||
'dest_base': 'yt-dlp',
|
||||
'icon_resources': [(1, 'devscripts/logo.ico')],
|
||||
}],
|
||||
'version_info': {
|
||||
'version': VERSION,
|
||||
'description': DESCRIPTION,
|
||||
'comments': LONG_DESCRIPTION.split('\n')[0],
|
||||
'product_name': 'yt-dlp',
|
||||
'product_version': VERSION,
|
||||
'icon_resources': [(1, 'devscripts/logo.ico')],
|
||||
}],
|
||||
'options': {
|
||||
'py2exe': {
|
||||
'bundle_files': 0,
|
||||
'compressed': 1,
|
||||
'optimize': 2,
|
||||
'dist_dir': './dist',
|
||||
'excludes': ['Crypto', 'Cryptodome'], # py2exe cannot import Crypto
|
||||
'dll_excludes': ['w9xpopen.exe', 'crypt32.dll'],
|
||||
# Modules that are only imported dynamically must be added here
|
||||
'includes': ['yt_dlp.compat._legacy'],
|
||||
}
|
||||
},
|
||||
'zipfile': None
|
||||
'options': {
|
||||
'bundle_files': 0,
|
||||
'compressed': 1,
|
||||
'optimize': 2,
|
||||
'dist_dir': './dist',
|
||||
'excludes': ['Crypto', 'Cryptodome'], # py2exe cannot import Crypto
|
||||
'dll_excludes': ['w9xpopen.exe', 'crypt32.dll'],
|
||||
# Modules that are only imported dynamically must be added here
|
||||
'includes': ['yt_dlp.compat._legacy'],
|
||||
},
|
||||
'zipfile': None,
|
||||
}
|
||||
|
||||
|
||||
@ -91,7 +92,10 @@ def build_params():
|
||||
params = {'data_files': data_files}
|
||||
|
||||
if setuptools_available:
|
||||
params['entry_points'] = {'console_scripts': ['yt-dlp = yt_dlp:main']}
|
||||
params['entry_points'] = {
|
||||
'console_scripts': ['yt-dlp = yt_dlp:main'],
|
||||
'pyinstaller40': ['hook-dirs = yt_dlp.__pyinstaller:get_hook_dirs'],
|
||||
}
|
||||
else:
|
||||
params['scripts'] = ['yt-dlp']
|
||||
return params
|
||||
@ -114,41 +118,58 @@ def run(self):
|
||||
subprocess.run([sys.executable, 'devscripts/make_lazy_extractors.py'])
|
||||
|
||||
|
||||
params = py2exe_params() if sys.argv[1:2] == ['py2exe'] else build_params()
|
||||
setup(
|
||||
name='yt-dlp',
|
||||
version=VERSION,
|
||||
maintainer='pukkandan',
|
||||
maintainer_email='pukkandan.ytdlp@gmail.com',
|
||||
description=DESCRIPTION,
|
||||
long_description=LONG_DESCRIPTION,
|
||||
long_description_content_type='text/markdown',
|
||||
url='https://github.com/yt-dlp/yt-dlp',
|
||||
packages=packages(),
|
||||
install_requires=REQUIREMENTS,
|
||||
python_requires='>=3.7',
|
||||
project_urls={
|
||||
'Documentation': 'https://github.com/yt-dlp/yt-dlp#readme',
|
||||
'Source': 'https://github.com/yt-dlp/yt-dlp',
|
||||
'Tracker': 'https://github.com/yt-dlp/yt-dlp/issues',
|
||||
'Funding': 'https://github.com/yt-dlp/yt-dlp/blob/master/Collaborators.md#collaborators',
|
||||
},
|
||||
classifiers=[
|
||||
'Topic :: Multimedia :: Video',
|
||||
'Development Status :: 5 - Production/Stable',
|
||||
'Environment :: Console',
|
||||
'Programming Language :: Python',
|
||||
'Programming Language :: Python :: 3.7',
|
||||
'Programming Language :: Python :: 3.8',
|
||||
'Programming Language :: Python :: 3.9',
|
||||
'Programming Language :: Python :: 3.10',
|
||||
'Programming Language :: Python :: 3.11',
|
||||
'Programming Language :: Python :: Implementation',
|
||||
'Programming Language :: Python :: Implementation :: CPython',
|
||||
'Programming Language :: Python :: Implementation :: PyPy',
|
||||
'License :: Public Domain',
|
||||
'Operating System :: OS Independent',
|
||||
],
|
||||
cmdclass={'build_lazy_extractors': build_lazy_extractors},
|
||||
**params
|
||||
)
|
||||
def main():
|
||||
if sys.argv[1:2] == ['py2exe']:
|
||||
params = py2exe_params()
|
||||
try:
|
||||
from py2exe import freeze
|
||||
except ImportError:
|
||||
import py2exe # noqa: F401
|
||||
warnings.warn('You are using an outdated version of py2exe. Support for this version will be removed in the future')
|
||||
params['console'][0].update(params.pop('version_info'))
|
||||
params['options'] = {'py2exe': params.pop('options')}
|
||||
else:
|
||||
return freeze(**params)
|
||||
else:
|
||||
params = build_params()
|
||||
|
||||
setup(
|
||||
name='yt-dlp',
|
||||
version=VERSION,
|
||||
maintainer='pukkandan',
|
||||
maintainer_email='pukkandan.ytdlp@gmail.com',
|
||||
description=DESCRIPTION,
|
||||
long_description=LONG_DESCRIPTION,
|
||||
long_description_content_type='text/markdown',
|
||||
url='https://github.com/yt-dlp/yt-dlp',
|
||||
packages=packages(),
|
||||
install_requires=REQUIREMENTS,
|
||||
python_requires='>=3.7',
|
||||
project_urls={
|
||||
'Documentation': 'https://github.com/yt-dlp/yt-dlp#readme',
|
||||
'Source': 'https://github.com/yt-dlp/yt-dlp',
|
||||
'Tracker': 'https://github.com/yt-dlp/yt-dlp/issues',
|
||||
'Funding': 'https://github.com/yt-dlp/yt-dlp/blob/master/Collaborators.md#collaborators',
|
||||
},
|
||||
classifiers=[
|
||||
'Topic :: Multimedia :: Video',
|
||||
'Development Status :: 5 - Production/Stable',
|
||||
'Environment :: Console',
|
||||
'Programming Language :: Python',
|
||||
'Programming Language :: Python :: 3.7',
|
||||
'Programming Language :: Python :: 3.8',
|
||||
'Programming Language :: Python :: 3.9',
|
||||
'Programming Language :: Python :: 3.10',
|
||||
'Programming Language :: Python :: 3.11',
|
||||
'Programming Language :: Python :: Implementation',
|
||||
'Programming Language :: Python :: Implementation :: CPython',
|
||||
'Programming Language :: Python :: Implementation :: PyPy',
|
||||
'License :: Public Domain',
|
||||
'Operating System :: OS Independent',
|
||||
],
|
||||
cmdclass={'build_lazy_extractors': build_lazy_extractors},
|
||||
**params
|
||||
)
|
||||
|
||||
|
||||
main()
|
||||
|
File diff suppressed because it is too large
Load Diff
@ -222,6 +222,10 @@ def sanitize(key, value):
|
||||
if test_info_dict.get('display_id') == test_info_dict.get('id'):
|
||||
test_info_dict.pop('display_id')
|
||||
|
||||
# Check url for flat entries
|
||||
if got_dict.get('_type', 'video') != 'video' and got_dict.get('url'):
|
||||
test_info_dict['url'] = got_dict['url']
|
||||
|
||||
return test_info_dict
|
||||
|
||||
|
||||
@ -235,8 +239,9 @@ def expect_info_dict(self, got_dict, expected_dict):
|
||||
for key in mandatory_fields:
|
||||
self.assertTrue(got_dict.get(key), 'Missing mandatory field %s' % key)
|
||||
# Check for mandatory fields that are automatically set by YoutubeDL
|
||||
for key in ['webpage_url', 'extractor', 'extractor_key']:
|
||||
self.assertTrue(got_dict.get(key), 'Missing field: %s' % key)
|
||||
if got_dict.get('_type', 'video') == 'video':
|
||||
for key in ['webpage_url', 'extractor', 'extractor_key']:
|
||||
self.assertTrue(got_dict.get(key), 'Missing field: %s' % key)
|
||||
|
||||
test_info_dict = sanitize_got_info_dict(got_dict)
|
||||
|
||||
@ -249,19 +254,16 @@ def _repr(v):
|
||||
return v.__name__
|
||||
else:
|
||||
return repr(v)
|
||||
info_dict_str = ''
|
||||
if len(missing_keys) != len(expected_dict):
|
||||
info_dict_str += ''.join(
|
||||
f' {_repr(k)}: {_repr(v)},\n'
|
||||
for k, v in test_info_dict.items() if k not in missing_keys)
|
||||
|
||||
if info_dict_str:
|
||||
info_dict_str += '\n'
|
||||
info_dict_str = ''.join(
|
||||
f' {_repr(k)}: {_repr(v)},\n'
|
||||
for k, v in test_info_dict.items() if k not in missing_keys)
|
||||
if info_dict_str:
|
||||
info_dict_str += '\n'
|
||||
info_dict_str += ''.join(
|
||||
f' {_repr(k)}: {_repr(test_info_dict[k])},\n'
|
||||
for k in missing_keys)
|
||||
write_string(
|
||||
'\n\'info_dict\': {\n' + info_dict_str + '},\n', out=sys.stderr)
|
||||
info_dict_str = '\n\'info_dict\': {\n' + info_dict_str + '},\n'
|
||||
write_string(info_dict_str.replace('\n', '\n '), out=sys.stderr)
|
||||
self.assertFalse(
|
||||
missing_keys,
|
||||
'Missing keys in test definition: %s' % (
|
||||
|
@ -44,5 +44,6 @@
|
||||
"writesubtitles": false,
|
||||
"allsubtitles": false,
|
||||
"listsubtitles": false,
|
||||
"fixup": "never"
|
||||
"fixup": "never",
|
||||
"allow_playlist_files": false
|
||||
}
|
||||
|
@ -41,7 +41,9 @@ def do_GET(self):
|
||||
|
||||
|
||||
class DummyIE(InfoExtractor):
|
||||
pass
|
||||
def _sort_formats(self, formats, field_preference=[]):
|
||||
self._downloader.sort_formats(
|
||||
{'formats': formats, '_format_sort_fields': field_preference})
|
||||
|
||||
|
||||
class TestInfoExtractor(unittest.TestCase):
|
||||
@ -67,6 +69,7 @@ def test_opengraph(self):
|
||||
<meta name="og:test1" content='foo > < bar'/>
|
||||
<meta name="og:test2" content="foo >//< bar"/>
|
||||
<meta property=og-test3 content='Ill-formatted opengraph'/>
|
||||
<meta property=og:test4 content=unquoted-value/>
|
||||
'''
|
||||
self.assertEqual(ie._og_search_title(html), 'Foo')
|
||||
self.assertEqual(ie._og_search_description(html), 'Some video\'s description ')
|
||||
@ -79,6 +82,7 @@ def test_opengraph(self):
|
||||
self.assertEqual(ie._og_search_property(('test0', 'test1'), html), 'foo > < bar')
|
||||
self.assertRaises(RegexNotFoundError, ie._og_search_property, 'test0', html, None, fatal=True)
|
||||
self.assertRaises(RegexNotFoundError, ie._og_search_property, ('test0', 'test00'), html, None, fatal=True)
|
||||
self.assertEqual(ie._og_search_property('test4', html), 'unquoted-value')
|
||||
|
||||
def test_html_search_meta(self):
|
||||
ie = self.ie
|
||||
@ -1567,6 +1571,292 @@ def test_parse_ism_formats(self):
|
||||
]
|
||||
},
|
||||
),
|
||||
(
|
||||
'ec-3_test',
|
||||
'https://smstr01.dmm.t-online.de/smooth24/smoothstream_m1/streaming/sony/9221438342941275747/636887760842957027/25_km_h-Trailer-9221571562372022953_deu_20_1300k_HD_H_264_ISMV.ism/Manifest',
|
||||
[{
|
||||
'format_id': 'audio_deu_1-224',
|
||||
'url': 'https://smstr01.dmm.t-online.de/smooth24/smoothstream_m1/streaming/sony/9221438342941275747/636887760842957027/25_km_h-Trailer-9221571562372022953_deu_20_1300k_HD_H_264_ISMV.ism/Manifest',
|
||||
'manifest_url': 'https://smstr01.dmm.t-online.de/smooth24/smoothstream_m1/streaming/sony/9221438342941275747/636887760842957027/25_km_h-Trailer-9221571562372022953_deu_20_1300k_HD_H_264_ISMV.ism/Manifest',
|
||||
'ext': 'isma',
|
||||
'tbr': 224,
|
||||
'asr': 48000,
|
||||
'vcodec': 'none',
|
||||
'acodec': 'EC-3',
|
||||
'protocol': 'ism',
|
||||
'_download_params':
|
||||
{
|
||||
'stream_type': 'audio',
|
||||
'duration': 370000000,
|
||||
'timescale': 10000000,
|
||||
'width': 0,
|
||||
'height': 0,
|
||||
'fourcc': 'EC-3',
|
||||
'language': 'deu',
|
||||
'codec_private_data': '00063F000000AF87FBA7022DFB42A4D405CD93843BDD0700200F00',
|
||||
'sampling_rate': 48000,
|
||||
'channels': 6,
|
||||
'bits_per_sample': 16,
|
||||
'nal_unit_length_field': 4
|
||||
},
|
||||
'audio_ext': 'isma',
|
||||
'video_ext': 'none',
|
||||
'abr': 224,
|
||||
}, {
|
||||
'format_id': 'audio_deu-127',
|
||||
'url': 'https://smstr01.dmm.t-online.de/smooth24/smoothstream_m1/streaming/sony/9221438342941275747/636887760842957027/25_km_h-Trailer-9221571562372022953_deu_20_1300k_HD_H_264_ISMV.ism/Manifest',
|
||||
'manifest_url': 'https://smstr01.dmm.t-online.de/smooth24/smoothstream_m1/streaming/sony/9221438342941275747/636887760842957027/25_km_h-Trailer-9221571562372022953_deu_20_1300k_HD_H_264_ISMV.ism/Manifest',
|
||||
'ext': 'isma',
|
||||
'tbr': 127,
|
||||
'asr': 48000,
|
||||
'vcodec': 'none',
|
||||
'acodec': 'AACL',
|
||||
'protocol': 'ism',
|
||||
'_download_params':
|
||||
{
|
||||
'stream_type': 'audio',
|
||||
'duration': 370000000,
|
||||
'timescale': 10000000,
|
||||
'width': 0,
|
||||
'height': 0,
|
||||
'fourcc': 'AACL',
|
||||
'language': 'deu',
|
||||
'codec_private_data': '1190',
|
||||
'sampling_rate': 48000,
|
||||
'channels': 2,
|
||||
'bits_per_sample': 16,
|
||||
'nal_unit_length_field': 4
|
||||
},
|
||||
'audio_ext': 'isma',
|
||||
'video_ext': 'none',
|
||||
'abr': 127,
|
||||
}, {
|
||||
'format_id': 'video_deu-23',
|
||||
'url': 'https://smstr01.dmm.t-online.de/smooth24/smoothstream_m1/streaming/sony/9221438342941275747/636887760842957027/25_km_h-Trailer-9221571562372022953_deu_20_1300k_HD_H_264_ISMV.ism/Manifest',
|
||||
'manifest_url': 'https://smstr01.dmm.t-online.de/smooth24/smoothstream_m1/streaming/sony/9221438342941275747/636887760842957027/25_km_h-Trailer-9221571562372022953_deu_20_1300k_HD_H_264_ISMV.ism/Manifest',
|
||||
'ext': 'ismv',
|
||||
'width': 384,
|
||||
'height': 216,
|
||||
'tbr': 23,
|
||||
'vcodec': 'AVC1',
|
||||
'acodec': 'none',
|
||||
'protocol': 'ism',
|
||||
'_download_params':
|
||||
{
|
||||
'stream_type': 'video',
|
||||
'duration': 370000000,
|
||||
'timescale': 10000000,
|
||||
'width': 384,
|
||||
'height': 216,
|
||||
'fourcc': 'AVC1',
|
||||
'language': 'deu',
|
||||
'codec_private_data': '000000016742C00CDB06077E5C05A808080A00000300020000030009C0C02EE0177CC6300F142AE00000000168CA8DC8',
|
||||
'channels': 2,
|
||||
'bits_per_sample': 16,
|
||||
'nal_unit_length_field': 4
|
||||
},
|
||||
'video_ext': 'ismv',
|
||||
'audio_ext': 'none',
|
||||
'vbr': 23,
|
||||
}, {
|
||||
'format_id': 'video_deu-403',
|
||||
'url': 'https://smstr01.dmm.t-online.de/smooth24/smoothstream_m1/streaming/sony/9221438342941275747/636887760842957027/25_km_h-Trailer-9221571562372022953_deu_20_1300k_HD_H_264_ISMV.ism/Manifest',
|
||||
'manifest_url': 'https://smstr01.dmm.t-online.de/smooth24/smoothstream_m1/streaming/sony/9221438342941275747/636887760842957027/25_km_h-Trailer-9221571562372022953_deu_20_1300k_HD_H_264_ISMV.ism/Manifest',
|
||||
'ext': 'ismv',
|
||||
'width': 400,
|
||||
'height': 224,
|
||||
'tbr': 403,
|
||||
'vcodec': 'AVC1',
|
||||
'acodec': 'none',
|
||||
'protocol': 'ism',
|
||||
'_download_params':
|
||||
{
|
||||
'stream_type': 'video',
|
||||
'duration': 370000000,
|
||||
'timescale': 10000000,
|
||||
'width': 400,
|
||||
'height': 224,
|
||||
'fourcc': 'AVC1',
|
||||
'language': 'deu',
|
||||
'codec_private_data': '00000001674D4014E98323B602D4040405000003000100000300320F1429380000000168EAECF2',
|
||||
'channels': 2,
|
||||
'bits_per_sample': 16,
|
||||
'nal_unit_length_field': 4
|
||||
},
|
||||
'video_ext': 'ismv',
|
||||
'audio_ext': 'none',
|
||||
'vbr': 403,
|
||||
}, {
|
||||
'format_id': 'video_deu-680',
|
||||
'url': 'https://smstr01.dmm.t-online.de/smooth24/smoothstream_m1/streaming/sony/9221438342941275747/636887760842957027/25_km_h-Trailer-9221571562372022953_deu_20_1300k_HD_H_264_ISMV.ism/Manifest',
|
||||
'manifest_url': 'https://smstr01.dmm.t-online.de/smooth24/smoothstream_m1/streaming/sony/9221438342941275747/636887760842957027/25_km_h-Trailer-9221571562372022953_deu_20_1300k_HD_H_264_ISMV.ism/Manifest',
|
||||
'ext': 'ismv',
|
||||
'width': 640,
|
||||
'height': 360,
|
||||
'tbr': 680,
|
||||
'vcodec': 'AVC1',
|
||||
'acodec': 'none',
|
||||
'protocol': 'ism',
|
||||
'_download_params':
|
||||
{
|
||||
'stream_type': 'video',
|
||||
'duration': 370000000,
|
||||
'timescale': 10000000,
|
||||
'width': 640,
|
||||
'height': 360,
|
||||
'fourcc': 'AVC1',
|
||||
'language': 'deu',
|
||||
'codec_private_data': '00000001674D401EE981405FF2E02D4040405000000300100000030320F162D3800000000168EAECF2',
|
||||
'channels': 2,
|
||||
'bits_per_sample': 16,
|
||||
'nal_unit_length_field': 4
|
||||
},
|
||||
'video_ext': 'ismv',
|
||||
'audio_ext': 'none',
|
||||
'vbr': 680,
|
||||
}, {
|
||||
'format_id': 'video_deu-1253',
|
||||
'url': 'https://smstr01.dmm.t-online.de/smooth24/smoothstream_m1/streaming/sony/9221438342941275747/636887760842957027/25_km_h-Trailer-9221571562372022953_deu_20_1300k_HD_H_264_ISMV.ism/Manifest',
|
||||
'manifest_url': 'https://smstr01.dmm.t-online.de/smooth24/smoothstream_m1/streaming/sony/9221438342941275747/636887760842957027/25_km_h-Trailer-9221571562372022953_deu_20_1300k_HD_H_264_ISMV.ism/Manifest',
|
||||
'ext': 'ismv',
|
||||
'width': 640,
|
||||
'height': 360,
|
||||
'tbr': 1253,
|
||||
'vcodec': 'AVC1',
|
||||
'acodec': 'none',
|
||||
'protocol': 'ism',
|
||||
'_download_params':
|
||||
{
|
||||
'stream_type': 'video',
|
||||
'duration': 370000000,
|
||||
'timescale': 10000000,
|
||||
'width': 640,
|
||||
'height': 360,
|
||||
'fourcc': 'AVC1',
|
||||
'language': 'deu',
|
||||
'codec_private_data': '00000001674D401EE981405FF2E02D4040405000000300100000030320F162D3800000000168EAECF2',
|
||||
'channels': 2,
|
||||
'bits_per_sample': 16,
|
||||
'nal_unit_length_field': 4
|
||||
},
|
||||
'video_ext': 'ismv',
|
||||
'audio_ext': 'none',
|
||||
'vbr': 1253,
|
||||
}, {
|
||||
'format_id': 'video_deu-2121',
|
||||
'url': 'https://smstr01.dmm.t-online.de/smooth24/smoothstream_m1/streaming/sony/9221438342941275747/636887760842957027/25_km_h-Trailer-9221571562372022953_deu_20_1300k_HD_H_264_ISMV.ism/Manifest',
|
||||
'manifest_url': 'https://smstr01.dmm.t-online.de/smooth24/smoothstream_m1/streaming/sony/9221438342941275747/636887760842957027/25_km_h-Trailer-9221571562372022953_deu_20_1300k_HD_H_264_ISMV.ism/Manifest',
|
||||
'ext': 'ismv',
|
||||
'width': 768,
|
||||
'height': 432,
|
||||
'tbr': 2121,
|
||||
'vcodec': 'AVC1',
|
||||
'acodec': 'none',
|
||||
'protocol': 'ism',
|
||||
'_download_params':
|
||||
{
|
||||
'stream_type': 'video',
|
||||
'duration': 370000000,
|
||||
'timescale': 10000000,
|
||||
'width': 768,
|
||||
'height': 432,
|
||||
'fourcc': 'AVC1',
|
||||
'language': 'deu',
|
||||
'codec_private_data': '00000001674D401EECA0601BD80B50101014000003000400000300C83C58B6580000000168E93B3C80',
|
||||
'channels': 2,
|
||||
'bits_per_sample': 16,
|
||||
'nal_unit_length_field': 4
|
||||
},
|
||||
'video_ext': 'ismv',
|
||||
'audio_ext': 'none',
|
||||
'vbr': 2121,
|
||||
}, {
|
||||
'format_id': 'video_deu-3275',
|
||||
'url': 'https://smstr01.dmm.t-online.de/smooth24/smoothstream_m1/streaming/sony/9221438342941275747/636887760842957027/25_km_h-Trailer-9221571562372022953_deu_20_1300k_HD_H_264_ISMV.ism/Manifest',
|
||||
'manifest_url': 'https://smstr01.dmm.t-online.de/smooth24/smoothstream_m1/streaming/sony/9221438342941275747/636887760842957027/25_km_h-Trailer-9221571562372022953_deu_20_1300k_HD_H_264_ISMV.ism/Manifest',
|
||||
'ext': 'ismv',
|
||||
'width': 1280,
|
||||
'height': 720,
|
||||
'tbr': 3275,
|
||||
'vcodec': 'AVC1',
|
||||
'acodec': 'none',
|
||||
'protocol': 'ism',
|
||||
'_download_params':
|
||||
{
|
||||
'stream_type': 'video',
|
||||
'duration': 370000000,
|
||||
'timescale': 10000000,
|
||||
'width': 1280,
|
||||
'height': 720,
|
||||
'fourcc': 'AVC1',
|
||||
'language': 'deu',
|
||||
'codec_private_data': '00000001674D4020ECA02802DD80B501010140000003004000000C83C60C65800000000168E93B3C80',
|
||||
'channels': 2,
|
||||
'bits_per_sample': 16,
|
||||
'nal_unit_length_field': 4
|
||||
},
|
||||
'video_ext': 'ismv',
|
||||
'audio_ext': 'none',
|
||||
'vbr': 3275,
|
||||
}, {
|
||||
'format_id': 'video_deu-5300',
|
||||
'url': 'https://smstr01.dmm.t-online.de/smooth24/smoothstream_m1/streaming/sony/9221438342941275747/636887760842957027/25_km_h-Trailer-9221571562372022953_deu_20_1300k_HD_H_264_ISMV.ism/Manifest',
|
||||
'manifest_url': 'https://smstr01.dmm.t-online.de/smooth24/smoothstream_m1/streaming/sony/9221438342941275747/636887760842957027/25_km_h-Trailer-9221571562372022953_deu_20_1300k_HD_H_264_ISMV.ism/Manifest',
|
||||
'ext': 'ismv',
|
||||
'width': 1920,
|
||||
'height': 1080,
|
||||
'tbr': 5300,
|
||||
'vcodec': 'AVC1',
|
||||
'acodec': 'none',
|
||||
'protocol': 'ism',
|
||||
'_download_params':
|
||||
{
|
||||
'stream_type': 'video',
|
||||
'duration': 370000000,
|
||||
'timescale': 10000000,
|
||||
'width': 1920,
|
||||
'height': 1080,
|
||||
'fourcc': 'AVC1',
|
||||
'language': 'deu',
|
||||
'codec_private_data': '00000001674D4028ECA03C0113F2E02D4040405000000300100000030320F18319600000000168E93B3C80',
|
||||
'channels': 2,
|
||||
'bits_per_sample': 16,
|
||||
'nal_unit_length_field': 4
|
||||
},
|
||||
'video_ext': 'ismv',
|
||||
'audio_ext': 'none',
|
||||
'vbr': 5300,
|
||||
}, {
|
||||
'format_id': 'video_deu-8079',
|
||||
'url': 'https://smstr01.dmm.t-online.de/smooth24/smoothstream_m1/streaming/sony/9221438342941275747/636887760842957027/25_km_h-Trailer-9221571562372022953_deu_20_1300k_HD_H_264_ISMV.ism/Manifest',
|
||||
'manifest_url': 'https://smstr01.dmm.t-online.de/smooth24/smoothstream_m1/streaming/sony/9221438342941275747/636887760842957027/25_km_h-Trailer-9221571562372022953_deu_20_1300k_HD_H_264_ISMV.ism/Manifest',
|
||||
'ext': 'ismv',
|
||||
'width': 1920,
|
||||
'height': 1080,
|
||||
'tbr': 8079,
|
||||
'vcodec': 'AVC1',
|
||||
'acodec': 'none',
|
||||
'protocol': 'ism',
|
||||
'_download_params':
|
||||
{
|
||||
'stream_type': 'video',
|
||||
'duration': 370000000,
|
||||
'timescale': 10000000,
|
||||
'width': 1920,
|
||||
'height': 1080,
|
||||
'fourcc': 'AVC1',
|
||||
'language': 'deu',
|
||||
'codec_private_data': '00000001674D4028ECA03C0113F2E02D4040405000000300100000030320F18319600000000168E93B3C80',
|
||||
'channels': 2,
|
||||
'bits_per_sample': 16,
|
||||
'nal_unit_length_field': 4
|
||||
},
|
||||
'video_ext': 'ismv',
|
||||
'audio_ext': 'none',
|
||||
'vbr': 8079,
|
||||
}],
|
||||
{},
|
||||
),
|
||||
]
|
||||
|
||||
for ism_file, ism_url, expected_formats, expected_subtitles in _TEST_CASES:
|
||||
|
@ -68,8 +68,7 @@ def test_prefer_free_formats(self):
|
||||
{'ext': 'mp4', 'height': 460, 'url': TEST_URL},
|
||||
]
|
||||
info_dict = _make_result(formats)
|
||||
yie = YoutubeIE(ydl)
|
||||
yie._sort_formats(info_dict['formats'])
|
||||
ydl.sort_formats(info_dict)
|
||||
ydl.process_ie_result(info_dict)
|
||||
downloaded = ydl.downloaded_info_dicts[0]
|
||||
self.assertEqual(downloaded['ext'], 'webm')
|
||||
@ -82,8 +81,7 @@ def test_prefer_free_formats(self):
|
||||
{'ext': 'mp4', 'height': 1080, 'url': TEST_URL},
|
||||
]
|
||||
info_dict['formats'] = formats
|
||||
yie = YoutubeIE(ydl)
|
||||
yie._sort_formats(info_dict['formats'])
|
||||
ydl.sort_formats(info_dict)
|
||||
ydl.process_ie_result(info_dict)
|
||||
downloaded = ydl.downloaded_info_dicts[0]
|
||||
self.assertEqual(downloaded['ext'], 'mp4')
|
||||
@ -97,8 +95,7 @@ def test_prefer_free_formats(self):
|
||||
{'ext': 'flv', 'height': 720, 'url': TEST_URL},
|
||||
]
|
||||
info_dict['formats'] = formats
|
||||
yie = YoutubeIE(ydl)
|
||||
yie._sort_formats(info_dict['formats'])
|
||||
ydl.sort_formats(info_dict)
|
||||
ydl.process_ie_result(info_dict)
|
||||
downloaded = ydl.downloaded_info_dicts[0]
|
||||
self.assertEqual(downloaded['ext'], 'mp4')
|
||||
@ -110,15 +107,14 @@ def test_prefer_free_formats(self):
|
||||
{'ext': 'webm', 'height': 720, 'url': TEST_URL},
|
||||
]
|
||||
info_dict['formats'] = formats
|
||||
yie = YoutubeIE(ydl)
|
||||
yie._sort_formats(info_dict['formats'])
|
||||
ydl.sort_formats(info_dict)
|
||||
ydl.process_ie_result(info_dict)
|
||||
downloaded = ydl.downloaded_info_dicts[0]
|
||||
self.assertEqual(downloaded['ext'], 'webm')
|
||||
|
||||
def test_format_selection(self):
|
||||
formats = [
|
||||
{'format_id': '35', 'ext': 'mp4', 'preference': 1, 'url': TEST_URL},
|
||||
{'format_id': '35', 'ext': 'mp4', 'preference': 0, 'url': TEST_URL},
|
||||
{'format_id': 'example-with-dashes', 'ext': 'webm', 'preference': 1, 'url': TEST_URL},
|
||||
{'format_id': '45', 'ext': 'webm', 'preference': 2, 'url': TEST_URL},
|
||||
{'format_id': '47', 'ext': 'webm', 'preference': 3, 'url': TEST_URL},
|
||||
@ -186,22 +182,19 @@ def test_format_selection_audio_exts(self):
|
||||
|
||||
info_dict = _make_result(formats)
|
||||
ydl = YDL({'format': 'best'})
|
||||
ie = YoutubeIE(ydl)
|
||||
ie._sort_formats(info_dict['formats'])
|
||||
ydl.sort_formats(info_dict)
|
||||
ydl.process_ie_result(copy.deepcopy(info_dict))
|
||||
downloaded = ydl.downloaded_info_dicts[0]
|
||||
self.assertEqual(downloaded['format_id'], 'aac-64')
|
||||
|
||||
ydl = YDL({'format': 'mp3'})
|
||||
ie = YoutubeIE(ydl)
|
||||
ie._sort_formats(info_dict['formats'])
|
||||
ydl.sort_formats(info_dict)
|
||||
ydl.process_ie_result(copy.deepcopy(info_dict))
|
||||
downloaded = ydl.downloaded_info_dicts[0]
|
||||
self.assertEqual(downloaded['format_id'], 'mp3-64')
|
||||
|
||||
ydl = YDL({'prefer_free_formats': True})
|
||||
ie = YoutubeIE(ydl)
|
||||
ie._sort_formats(info_dict['formats'])
|
||||
ydl.sort_formats(info_dict)
|
||||
ydl.process_ie_result(copy.deepcopy(info_dict))
|
||||
downloaded = ydl.downloaded_info_dicts[0]
|
||||
self.assertEqual(downloaded['format_id'], 'ogg-64')
|
||||
@ -346,8 +339,7 @@ def format_info(f_id):
|
||||
|
||||
info_dict = _make_result(list(formats_order), extractor='youtube')
|
||||
ydl = YDL({'format': 'bestvideo+bestaudio'})
|
||||
yie = YoutubeIE(ydl)
|
||||
yie._sort_formats(info_dict['formats'])
|
||||
ydl.sort_formats(info_dict)
|
||||
ydl.process_ie_result(info_dict)
|
||||
downloaded = ydl.downloaded_info_dicts[0]
|
||||
self.assertEqual(downloaded['format_id'], '248+172')
|
||||
@ -355,40 +347,35 @@ def format_info(f_id):
|
||||
|
||||
info_dict = _make_result(list(formats_order), extractor='youtube')
|
||||
ydl = YDL({'format': 'bestvideo[height>=999999]+bestaudio/best'})
|
||||
yie = YoutubeIE(ydl)
|
||||
yie._sort_formats(info_dict['formats'])
|
||||
ydl.sort_formats(info_dict)
|
||||
ydl.process_ie_result(info_dict)
|
||||
downloaded = ydl.downloaded_info_dicts[0]
|
||||
self.assertEqual(downloaded['format_id'], '38')
|
||||
|
||||
info_dict = _make_result(list(formats_order), extractor='youtube')
|
||||
ydl = YDL({'format': 'bestvideo/best,bestaudio'})
|
||||
yie = YoutubeIE(ydl)
|
||||
yie._sort_formats(info_dict['formats'])
|
||||
ydl.sort_formats(info_dict)
|
||||
ydl.process_ie_result(info_dict)
|
||||
downloaded_ids = [info['format_id'] for info in ydl.downloaded_info_dicts]
|
||||
self.assertEqual(downloaded_ids, ['137', '141'])
|
||||
|
||||
info_dict = _make_result(list(formats_order), extractor='youtube')
|
||||
ydl = YDL({'format': '(bestvideo[ext=mp4],bestvideo[ext=webm])+bestaudio'})
|
||||
yie = YoutubeIE(ydl)
|
||||
yie._sort_formats(info_dict['formats'])
|
||||
ydl.sort_formats(info_dict)
|
||||
ydl.process_ie_result(info_dict)
|
||||
downloaded_ids = [info['format_id'] for info in ydl.downloaded_info_dicts]
|
||||
self.assertEqual(downloaded_ids, ['137+141', '248+141'])
|
||||
|
||||
info_dict = _make_result(list(formats_order), extractor='youtube')
|
||||
ydl = YDL({'format': '(bestvideo[ext=mp4],bestvideo[ext=webm])[height<=720]+bestaudio'})
|
||||
yie = YoutubeIE(ydl)
|
||||
yie._sort_formats(info_dict['formats'])
|
||||
ydl.sort_formats(info_dict)
|
||||
ydl.process_ie_result(info_dict)
|
||||
downloaded_ids = [info['format_id'] for info in ydl.downloaded_info_dicts]
|
||||
self.assertEqual(downloaded_ids, ['136+141', '247+141'])
|
||||
|
||||
info_dict = _make_result(list(formats_order), extractor='youtube')
|
||||
ydl = YDL({'format': '(bestvideo[ext=none]/bestvideo[ext=webm])+bestaudio'})
|
||||
yie = YoutubeIE(ydl)
|
||||
yie._sort_formats(info_dict['formats'])
|
||||
ydl.sort_formats(info_dict)
|
||||
ydl.process_ie_result(info_dict)
|
||||
downloaded_ids = [info['format_id'] for info in ydl.downloaded_info_dicts]
|
||||
self.assertEqual(downloaded_ids, ['248+141'])
|
||||
@ -396,16 +383,14 @@ def format_info(f_id):
|
||||
for f1, f2 in zip(formats_order, formats_order[1:]):
|
||||
info_dict = _make_result([f1, f2], extractor='youtube')
|
||||
ydl = YDL({'format': 'best/bestvideo'})
|
||||
yie = YoutubeIE(ydl)
|
||||
yie._sort_formats(info_dict['formats'])
|
||||
ydl.sort_formats(info_dict)
|
||||
ydl.process_ie_result(info_dict)
|
||||
downloaded = ydl.downloaded_info_dicts[0]
|
||||
self.assertEqual(downloaded['format_id'], f1['format_id'])
|
||||
|
||||
info_dict = _make_result([f2, f1], extractor='youtube')
|
||||
ydl = YDL({'format': 'best/bestvideo'})
|
||||
yie = YoutubeIE(ydl)
|
||||
yie._sort_formats(info_dict['formats'])
|
||||
ydl.sort_formats(info_dict)
|
||||
ydl.process_ie_result(info_dict)
|
||||
downloaded = ydl.downloaded_info_dicts[0]
|
||||
self.assertEqual(downloaded['format_id'], f1['format_id'])
|
||||
@ -480,7 +465,7 @@ def test_format_filtering(self):
|
||||
for f in formats:
|
||||
f['url'] = 'http://_/'
|
||||
f['ext'] = 'unknown'
|
||||
info_dict = _make_result(formats)
|
||||
info_dict = _make_result(formats, _format_sort_fields=('id', ))
|
||||
|
||||
ydl = YDL({'format': 'best[filesize<3000]'})
|
||||
ydl.process_ie_result(info_dict)
|
||||
|
@ -11,7 +11,6 @@
|
||||
import base64
|
||||
|
||||
from yt_dlp.aes import (
|
||||
BLOCK_SIZE_BYTES,
|
||||
aes_cbc_decrypt,
|
||||
aes_cbc_decrypt_bytes,
|
||||
aes_cbc_encrypt,
|
||||
@ -27,7 +26,7 @@
|
||||
key_expansion,
|
||||
pad_block,
|
||||
)
|
||||
from yt_dlp.dependencies import Cryptodome_AES
|
||||
from yt_dlp.dependencies import Cryptodome
|
||||
from yt_dlp.utils import bytes_to_intlist, intlist_to_bytes
|
||||
|
||||
# the encrypted data can be generate with 'devscripts/generate_aes_testdata.py'
|
||||
@ -49,7 +48,7 @@ def test_cbc_decrypt(self):
|
||||
data = b'\x97\x92+\xe5\x0b\xc3\x18\x91ky9m&\xb3\xb5@\xe6\x27\xc2\x96.\xc8u\x88\xab9-[\x9e|\xf1\xcd'
|
||||
decrypted = intlist_to_bytes(aes_cbc_decrypt(bytes_to_intlist(data), self.key, self.iv))
|
||||
self.assertEqual(decrypted.rstrip(b'\x08'), self.secret_msg)
|
||||
if Cryptodome_AES:
|
||||
if Cryptodome.AES:
|
||||
decrypted = aes_cbc_decrypt_bytes(data, intlist_to_bytes(self.key), intlist_to_bytes(self.iv))
|
||||
self.assertEqual(decrypted.rstrip(b'\x08'), self.secret_msg)
|
||||
|
||||
@ -79,7 +78,7 @@ def test_gcm_decrypt(self):
|
||||
decrypted = intlist_to_bytes(aes_gcm_decrypt_and_verify(
|
||||
bytes_to_intlist(data), self.key, bytes_to_intlist(authentication_tag), self.iv[:12]))
|
||||
self.assertEqual(decrypted.rstrip(b'\x08'), self.secret_msg)
|
||||
if Cryptodome_AES:
|
||||
if Cryptodome.AES:
|
||||
decrypted = aes_gcm_decrypt_and_verify_bytes(
|
||||
data, intlist_to_bytes(self.key), authentication_tag, intlist_to_bytes(self.iv[:12]))
|
||||
self.assertEqual(decrypted.rstrip(b'\x08'), self.secret_msg)
|
||||
@ -103,8 +102,7 @@ def test_decrypt_text(self):
|
||||
|
||||
def test_ecb_encrypt(self):
|
||||
data = bytes_to_intlist(self.secret_msg)
|
||||
data += [0x08] * (BLOCK_SIZE_BYTES - len(data) % BLOCK_SIZE_BYTES)
|
||||
encrypted = intlist_to_bytes(aes_ecb_encrypt(data, self.key, self.iv))
|
||||
encrypted = intlist_to_bytes(aes_ecb_encrypt(data, self.key))
|
||||
self.assertEqual(
|
||||
encrypted,
|
||||
b'\xaa\x86]\x81\x97>\x02\x92\x9d\x1bR[[L/u\xd3&\xd1(h\xde{\x81\x94\xba\x02\xae\xbd\xa6\xd0:')
|
||||
|
@ -10,6 +10,7 @@
|
||||
|
||||
from test.helper import is_download_test, try_rm
|
||||
from yt_dlp import YoutubeDL
|
||||
from yt_dlp.utils import DownloadError
|
||||
|
||||
|
||||
def _download_restricted(url, filename, age):
|
||||
@ -25,10 +26,14 @@ def _download_restricted(url, filename, age):
|
||||
ydl.add_default_info_extractors()
|
||||
json_filename = os.path.splitext(filename)[0] + '.info.json'
|
||||
try_rm(json_filename)
|
||||
ydl.download([url])
|
||||
res = os.path.exists(json_filename)
|
||||
try_rm(json_filename)
|
||||
return res
|
||||
try:
|
||||
ydl.download([url])
|
||||
except DownloadError:
|
||||
pass
|
||||
else:
|
||||
return os.path.exists(json_filename)
|
||||
finally:
|
||||
try_rm(json_filename)
|
||||
|
||||
|
||||
@is_download_test
|
||||
@ -38,12 +43,12 @@ def _assert_restricted(self, url, filename, age, old_age=None):
|
||||
self.assertFalse(_download_restricted(url, filename, age))
|
||||
|
||||
def test_youtube(self):
|
||||
self._assert_restricted('07FYdnEawAQ', '07FYdnEawAQ.mp4', 10)
|
||||
self._assert_restricted('HtVdAasjOgU', 'HtVdAasjOgU.mp4', 10)
|
||||
|
||||
def test_youporn(self):
|
||||
self._assert_restricted(
|
||||
'http://www.youporn.com/watch/505835/sex-ed-is-it-safe-to-masturbate-daily/',
|
||||
'505835.mp4', 2, old_age=25)
|
||||
'https://www.youporn.com/watch/16715086/sex-ed-in-detention-18-asmr/',
|
||||
'16715086.mp4', 2, old_age=25)
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
|
@ -31,6 +31,9 @@ def test_compat_passthrough(self):
|
||||
# TODO: Test submodule
|
||||
# compat.asyncio.events # Must not raise error
|
||||
|
||||
with self.assertWarns(DeprecationWarning):
|
||||
compat.compat_pycrypto_AES # Must not raise error
|
||||
|
||||
def test_compat_expanduser(self):
|
||||
old_home = os.environ.get('HOME')
|
||||
test_str = R'C:\Documents and Settings\тест\Application Data'
|
||||
|
227
test/test_config.py
Normal file
227
test/test_config.py
Normal file
@ -0,0 +1,227 @@
|
||||
#!/usr/bin/env python3
|
||||
|
||||
# Allow direct execution
|
||||
import os
|
||||
import sys
|
||||
import unittest
|
||||
import unittest.mock
|
||||
|
||||
sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
|
||||
|
||||
import contextlib
|
||||
import itertools
|
||||
from pathlib import Path
|
||||
|
||||
from yt_dlp.compat import compat_expanduser
|
||||
from yt_dlp.options import create_parser, parseOpts
|
||||
from yt_dlp.utils import Config, get_executable_path
|
||||
|
||||
ENVIRON_DEFAULTS = {
|
||||
'HOME': None,
|
||||
'XDG_CONFIG_HOME': '/_xdg_config_home/',
|
||||
'USERPROFILE': 'C:/Users/testing/',
|
||||
'APPDATA': 'C:/Users/testing/AppData/Roaming/',
|
||||
'HOMEDRIVE': 'C:/',
|
||||
'HOMEPATH': 'Users/testing/',
|
||||
}
|
||||
|
||||
|
||||
@contextlib.contextmanager
|
||||
def set_environ(**kwargs):
|
||||
saved_environ = os.environ.copy()
|
||||
|
||||
for name, value in {**ENVIRON_DEFAULTS, **kwargs}.items():
|
||||
if value is None:
|
||||
os.environ.pop(name, None)
|
||||
else:
|
||||
os.environ[name] = value
|
||||
|
||||
yield
|
||||
|
||||
os.environ.clear()
|
||||
os.environ.update(saved_environ)
|
||||
|
||||
|
||||
def _generate_expected_groups():
|
||||
xdg_config_home = os.getenv('XDG_CONFIG_HOME') or compat_expanduser('~/.config')
|
||||
appdata_dir = os.getenv('appdata')
|
||||
home_dir = compat_expanduser('~')
|
||||
return {
|
||||
'Portable': [
|
||||
Path(get_executable_path(), 'yt-dlp.conf'),
|
||||
],
|
||||
'Home': [
|
||||
Path('yt-dlp.conf'),
|
||||
],
|
||||
'User': [
|
||||
Path(xdg_config_home, 'yt-dlp.conf'),
|
||||
Path(xdg_config_home, 'yt-dlp', 'config'),
|
||||
Path(xdg_config_home, 'yt-dlp', 'config.txt'),
|
||||
*((
|
||||
Path(appdata_dir, 'yt-dlp.conf'),
|
||||
Path(appdata_dir, 'yt-dlp', 'config'),
|
||||
Path(appdata_dir, 'yt-dlp', 'config.txt'),
|
||||
) if appdata_dir else ()),
|
||||
Path(home_dir, 'yt-dlp.conf'),
|
||||
Path(home_dir, 'yt-dlp.conf.txt'),
|
||||
Path(home_dir, '.yt-dlp', 'config'),
|
||||
Path(home_dir, '.yt-dlp', 'config.txt'),
|
||||
],
|
||||
'System': [
|
||||
Path('/etc/yt-dlp.conf'),
|
||||
Path('/etc/yt-dlp/config'),
|
||||
Path('/etc/yt-dlp/config.txt'),
|
||||
]
|
||||
}
|
||||
|
||||
|
||||
class TestConfig(unittest.TestCase):
|
||||
maxDiff = None
|
||||
|
||||
@set_environ()
|
||||
def test_config__ENVIRON_DEFAULTS_sanity(self):
|
||||
expected = make_expected()
|
||||
self.assertCountEqual(
|
||||
set(expected), expected,
|
||||
'ENVIRON_DEFAULTS produces non unique names')
|
||||
|
||||
def test_config_all_environ_values(self):
|
||||
for name, value in ENVIRON_DEFAULTS.items():
|
||||
for new_value in (None, '', '.', value or '/some/dir'):
|
||||
with set_environ(**{name: new_value}):
|
||||
self._simple_grouping_test()
|
||||
|
||||
def test_config_default_expected_locations(self):
|
||||
files, _ = self._simple_config_test()
|
||||
self.assertEqual(
|
||||
files, make_expected(),
|
||||
'Not all expected locations have been checked')
|
||||
|
||||
def test_config_default_grouping(self):
|
||||
self._simple_grouping_test()
|
||||
|
||||
def _simple_grouping_test(self):
|
||||
expected_groups = make_expected_groups()
|
||||
for name, group in expected_groups.items():
|
||||
for index, existing_path in enumerate(group):
|
||||
result, opts = self._simple_config_test(existing_path)
|
||||
expected = expected_from_expected_groups(expected_groups, existing_path)
|
||||
self.assertEqual(
|
||||
result, expected,
|
||||
f'The checked locations do not match the expected ({name}, {index})')
|
||||
self.assertEqual(
|
||||
opts.outtmpl['default'], '1',
|
||||
f'The used result value was incorrect ({name}, {index})')
|
||||
|
||||
def _simple_config_test(self, *stop_paths):
|
||||
encountered = 0
|
||||
paths = []
|
||||
|
||||
def read_file(filename, default=[]):
|
||||
nonlocal encountered
|
||||
path = Path(filename)
|
||||
paths.append(path)
|
||||
if path in stop_paths:
|
||||
encountered += 1
|
||||
return ['-o', f'{encountered}']
|
||||
|
||||
with ConfigMock(read_file):
|
||||
_, opts, _ = parseOpts([], False)
|
||||
|
||||
return paths, opts
|
||||
|
||||
@set_environ()
|
||||
def test_config_early_exit_commandline(self):
|
||||
self._early_exit_test(0, '--ignore-config')
|
||||
|
||||
@set_environ()
|
||||
def test_config_early_exit_files(self):
|
||||
for index, _ in enumerate(make_expected(), 1):
|
||||
self._early_exit_test(index)
|
||||
|
||||
def _early_exit_test(self, allowed_reads, *args):
|
||||
reads = 0
|
||||
|
||||
def read_file(filename, default=[]):
|
||||
nonlocal reads
|
||||
reads += 1
|
||||
|
||||
if reads > allowed_reads:
|
||||
self.fail('The remaining config was not ignored')
|
||||
elif reads == allowed_reads:
|
||||
return ['--ignore-config']
|
||||
|
||||
with ConfigMock(read_file):
|
||||
parseOpts(args, False)
|
||||
|
||||
@set_environ()
|
||||
def test_config_override_commandline(self):
|
||||
self._override_test(0, '-o', 'pass')
|
||||
|
||||
@set_environ()
|
||||
def test_config_override_files(self):
|
||||
for index, _ in enumerate(make_expected(), 1):
|
||||
self._override_test(index)
|
||||
|
||||
def _override_test(self, start_index, *args):
|
||||
index = 0
|
||||
|
||||
def read_file(filename, default=[]):
|
||||
nonlocal index
|
||||
index += 1
|
||||
|
||||
if index > start_index:
|
||||
return ['-o', 'fail']
|
||||
elif index == start_index:
|
||||
return ['-o', 'pass']
|
||||
|
||||
with ConfigMock(read_file):
|
||||
_, opts, _ = parseOpts(args, False)
|
||||
|
||||
self.assertEqual(
|
||||
opts.outtmpl['default'], 'pass',
|
||||
'The earlier group did not override the later ones')
|
||||
|
||||
|
||||
@contextlib.contextmanager
|
||||
def ConfigMock(read_file=None):
|
||||
with unittest.mock.patch('yt_dlp.options.Config') as mock:
|
||||
mock.return_value = Config(create_parser())
|
||||
if read_file is not None:
|
||||
mock.read_file = read_file
|
||||
|
||||
yield mock
|
||||
|
||||
|
||||
def make_expected(*filepaths):
|
||||
return expected_from_expected_groups(_generate_expected_groups(), *filepaths)
|
||||
|
||||
|
||||
def make_expected_groups(*filepaths):
|
||||
return _filter_expected_groups(_generate_expected_groups(), filepaths)
|
||||
|
||||
|
||||
def expected_from_expected_groups(expected_groups, *filepaths):
|
||||
return list(itertools.chain.from_iterable(
|
||||
_filter_expected_groups(expected_groups, filepaths).values()))
|
||||
|
||||
|
||||
def _filter_expected_groups(expected, filepaths):
|
||||
if not filepaths:
|
||||
return expected
|
||||
|
||||
result = {}
|
||||
for group, paths in expected.items():
|
||||
new_paths = []
|
||||
for path in paths:
|
||||
new_paths.append(path)
|
||||
if path in filepaths:
|
||||
break
|
||||
|
||||
result[group] = new_paths
|
||||
|
||||
return result
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
unittest.main()
|
@ -3,6 +3,7 @@
|
||||
|
||||
from yt_dlp import cookies
|
||||
from yt_dlp.cookies import (
|
||||
LenientSimpleCookie,
|
||||
LinuxChromeCookieDecryptor,
|
||||
MacChromeCookieDecryptor,
|
||||
WindowsChromeCookieDecryptor,
|
||||
@ -137,3 +138,163 @@ def test_safari_cookie_parsing(self):
|
||||
def test_pbkdf2_sha1(self):
|
||||
key = pbkdf2_sha1(b'peanuts', b' ' * 16, 1, 16)
|
||||
self.assertEqual(key, b'g\xe1\x8e\x0fQ\x1c\x9b\xf3\xc9`!\xaa\x90\xd9\xd34')
|
||||
|
||||
|
||||
class TestLenientSimpleCookie(unittest.TestCase):
|
||||
def _run_tests(self, *cases):
|
||||
for message, raw_cookie, expected in cases:
|
||||
cookie = LenientSimpleCookie(raw_cookie)
|
||||
|
||||
with self.subTest(message, expected=expected):
|
||||
self.assertEqual(cookie.keys(), expected.keys(), message)
|
||||
|
||||
for key, expected_value in expected.items():
|
||||
morsel = cookie[key]
|
||||
if isinstance(expected_value, tuple):
|
||||
expected_value, expected_attributes = expected_value
|
||||
else:
|
||||
expected_attributes = {}
|
||||
|
||||
attributes = {
|
||||
key: value
|
||||
for key, value in dict(morsel).items()
|
||||
if value != ""
|
||||
}
|
||||
self.assertEqual(attributes, expected_attributes, message)
|
||||
|
||||
self.assertEqual(morsel.value, expected_value, message)
|
||||
|
||||
def test_parsing(self):
|
||||
self._run_tests(
|
||||
# Copied from https://github.com/python/cpython/blob/v3.10.7/Lib/test/test_http_cookies.py
|
||||
(
|
||||
"Test basic cookie",
|
||||
"chips=ahoy; vienna=finger",
|
||||
{"chips": "ahoy", "vienna": "finger"},
|
||||
),
|
||||
(
|
||||
"Test quoted cookie",
|
||||
'keebler="E=mc2; L=\\"Loves\\"; fudge=\\012;"',
|
||||
{"keebler": 'E=mc2; L="Loves"; fudge=\012;'},
|
||||
),
|
||||
(
|
||||
"Allow '=' in an unquoted value",
|
||||
"keebler=E=mc2",
|
||||
{"keebler": "E=mc2"},
|
||||
),
|
||||
(
|
||||
"Allow cookies with ':' in their name",
|
||||
"key:term=value:term",
|
||||
{"key:term": "value:term"},
|
||||
),
|
||||
(
|
||||
"Allow '[' and ']' in cookie values",
|
||||
"a=b; c=[; d=r; f=h",
|
||||
{"a": "b", "c": "[", "d": "r", "f": "h"},
|
||||
),
|
||||
(
|
||||
"Test basic cookie attributes",
|
||||
'Customer="WILE_E_COYOTE"; Version=1; Path=/acme',
|
||||
{"Customer": ("WILE_E_COYOTE", {"version": "1", "path": "/acme"})},
|
||||
),
|
||||
(
|
||||
"Test flag only cookie attributes",
|
||||
'Customer="WILE_E_COYOTE"; HttpOnly; Secure',
|
||||
{"Customer": ("WILE_E_COYOTE", {"httponly": True, "secure": True})},
|
||||
),
|
||||
(
|
||||
"Test flag only attribute with values",
|
||||
"eggs=scrambled; httponly=foo; secure=bar; Path=/bacon",
|
||||
{"eggs": ("scrambled", {"httponly": "foo", "secure": "bar", "path": "/bacon"})},
|
||||
),
|
||||
(
|
||||
"Test special case for 'expires' attribute, 4 digit year",
|
||||
'Customer="W"; expires=Wed, 01 Jan 2010 00:00:00 GMT',
|
||||
{"Customer": ("W", {"expires": "Wed, 01 Jan 2010 00:00:00 GMT"})},
|
||||
),
|
||||
(
|
||||
"Test special case for 'expires' attribute, 2 digit year",
|
||||
'Customer="W"; expires=Wed, 01 Jan 98 00:00:00 GMT',
|
||||
{"Customer": ("W", {"expires": "Wed, 01 Jan 98 00:00:00 GMT"})},
|
||||
),
|
||||
(
|
||||
"Test extra spaces in keys and values",
|
||||
"eggs = scrambled ; secure ; path = bar ; foo=foo ",
|
||||
{"eggs": ("scrambled", {"secure": True, "path": "bar"}), "foo": "foo"},
|
||||
),
|
||||
(
|
||||
"Test quoted attributes",
|
||||
'Customer="WILE_E_COYOTE"; Version="1"; Path="/acme"',
|
||||
{"Customer": ("WILE_E_COYOTE", {"version": "1", "path": "/acme"})}
|
||||
),
|
||||
# Our own tests that CPython passes
|
||||
(
|
||||
"Allow ';' in quoted value",
|
||||
'chips="a;hoy"; vienna=finger',
|
||||
{"chips": "a;hoy", "vienna": "finger"},
|
||||
),
|
||||
(
|
||||
"Keep only the last set value",
|
||||
"a=c; a=b",
|
||||
{"a": "b"},
|
||||
),
|
||||
)
|
||||
|
||||
def test_lenient_parsing(self):
|
||||
self._run_tests(
|
||||
(
|
||||
"Ignore and try to skip invalid cookies",
|
||||
'chips={"ahoy;": 1}; vienna="finger;"',
|
||||
{"vienna": "finger;"},
|
||||
),
|
||||
(
|
||||
"Ignore cookies without a name",
|
||||
"a=b; unnamed; c=d",
|
||||
{"a": "b", "c": "d"},
|
||||
),
|
||||
(
|
||||
"Ignore '\"' cookie without name",
|
||||
'a=b; "; c=d',
|
||||
{"a": "b", "c": "d"},
|
||||
),
|
||||
(
|
||||
"Skip all space separated values",
|
||||
"x a=b c=d x; e=f",
|
||||
{"a": "b", "c": "d", "e": "f"},
|
||||
),
|
||||
(
|
||||
"Skip all space separated values",
|
||||
'x a=b; data={"complex": "json", "with": "key=value"}; x c=d x',
|
||||
{"a": "b", "c": "d"},
|
||||
),
|
||||
(
|
||||
"Expect quote mending",
|
||||
'a=b; invalid="; c=d',
|
||||
{"a": "b", "c": "d"},
|
||||
),
|
||||
(
|
||||
"Reset morsel after invalid to not capture attributes",
|
||||
"a=b; invalid; Version=1; c=d",
|
||||
{"a": "b", "c": "d"},
|
||||
),
|
||||
(
|
||||
"Reset morsel after invalid to not capture attributes",
|
||||
"a=b; $invalid; $Version=1; c=d",
|
||||
{"a": "b", "c": "d"},
|
||||
),
|
||||
(
|
||||
"Continue after non-flag attribute without value",
|
||||
"a=b; path; Version=1; c=d",
|
||||
{"a": "b", "c": "d"},
|
||||
),
|
||||
(
|
||||
"Allow cookie attributes with `$` prefix",
|
||||
'Customer="WILE_E_COYOTE"; $Version=1; $Secure; $Path=/acme',
|
||||
{"Customer": ("WILE_E_COYOTE", {"version": "1", "secure": True, "path": "/acme"})},
|
||||
),
|
||||
(
|
||||
"Invalid Morsel keys should not result in an error",
|
||||
"Key=Value; [Invalid]=Value; Another=Value",
|
||||
{"Key": "Value", "Another": "Value"},
|
||||
),
|
||||
)
|
||||
|
@ -106,7 +106,7 @@ def print_skipping(reason):
|
||||
params = tc.get('params', {})
|
||||
if not info_dict.get('id'):
|
||||
raise Exception(f'Test {tname} definition incorrect - "id" key is not present')
|
||||
elif not info_dict.get('ext'):
|
||||
elif not info_dict.get('ext') and info_dict.get('_type', 'video') == 'video':
|
||||
if params.get('skip_download') and params.get('ignore_no_formats_error'):
|
||||
continue
|
||||
raise Exception(f'Test {tname} definition incorrect - "ext" key must be present to define the output file')
|
||||
@ -122,7 +122,8 @@ def print_skipping(reason):
|
||||
params['outtmpl'] = tname + '_' + params['outtmpl']
|
||||
if is_playlist and 'playlist' not in test_case:
|
||||
params.setdefault('extract_flat', 'in_playlist')
|
||||
params.setdefault('playlistend', test_case.get('playlist_mincount'))
|
||||
params.setdefault('playlistend', test_case.get(
|
||||
'playlist_mincount', test_case.get('playlist_count', -2) + 1))
|
||||
params.setdefault('skip_download', True)
|
||||
|
||||
ydl = YoutubeDL(params, auto_init=False)
|
||||
@ -212,6 +213,8 @@ def try_rm_tcs_files(tcs=None):
|
||||
tc_res_dict = res_dict['entries'][tc_num]
|
||||
# First, check test cases' data against extracted data alone
|
||||
expect_info_dict(self, tc_res_dict, tc.get('info_dict', {}))
|
||||
if tc_res_dict.get('_type', 'video') != 'video':
|
||||
continue
|
||||
# Now, check downloaded file consistency
|
||||
tc_filename = get_tc_filename(tc)
|
||||
if not test_case.get('params', {}).get('skip_download', False):
|
||||
|
@ -155,6 +155,38 @@ def test_call(self):
|
||||
self.assertEqual(jsi.call_function('z'), 5)
|
||||
self.assertEqual(jsi.call_function('y'), 2)
|
||||
|
||||
def test_if(self):
|
||||
jsi = JSInterpreter('''
|
||||
function x() {
|
||||
let a = 9;
|
||||
if (0==0) {a++}
|
||||
return a
|
||||
}''')
|
||||
self.assertEqual(jsi.call_function('x'), 10)
|
||||
|
||||
jsi = JSInterpreter('''
|
||||
function x() {
|
||||
if (0==0) {return 10}
|
||||
}''')
|
||||
self.assertEqual(jsi.call_function('x'), 10)
|
||||
|
||||
jsi = JSInterpreter('''
|
||||
function x() {
|
||||
if (0!=0) {return 1}
|
||||
else {return 10}
|
||||
}''')
|
||||
self.assertEqual(jsi.call_function('x'), 10)
|
||||
|
||||
""" # Unsupported
|
||||
jsi = JSInterpreter('''
|
||||
function x() {
|
||||
if (0!=0) {return 1}
|
||||
else if (1==0) {return 2}
|
||||
else {return 10}
|
||||
}''')
|
||||
self.assertEqual(jsi.call_function('x'), 10)
|
||||
"""
|
||||
|
||||
def test_for_loop(self):
|
||||
jsi = JSInterpreter('''
|
||||
function x() { a=0; for (i=0; i-10; i++) {a++} return a }
|
||||
@ -392,6 +424,11 @@ def test_regex(self):
|
||||
''')
|
||||
self.assertEqual(jsi.call_function('x').pattern, r',][}",],()}(\[)')
|
||||
|
||||
jsi = JSInterpreter(R'''
|
||||
function x() { let a=[/[)\\]/]; return a[0]; }
|
||||
''')
|
||||
self.assertEqual(jsi.call_function('x').pattern, r'[)\\]')
|
||||
|
||||
def test_char_code_at(self):
|
||||
jsi = JSInterpreter('function x(i){return "test".charCodeAt(i)}')
|
||||
self.assertEqual(jsi.call_function('x', 0), 116)
|
||||
|
73
test/test_plugins.py
Normal file
73
test/test_plugins.py
Normal file
@ -0,0 +1,73 @@
|
||||
import importlib
|
||||
import os
|
||||
import shutil
|
||||
import sys
|
||||
import unittest
|
||||
from pathlib import Path
|
||||
|
||||
sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
|
||||
TEST_DATA_DIR = Path(os.path.dirname(os.path.abspath(__file__)), 'testdata')
|
||||
sys.path.append(str(TEST_DATA_DIR))
|
||||
importlib.invalidate_caches()
|
||||
|
||||
from yt_dlp.plugins import PACKAGE_NAME, directories, load_plugins
|
||||
|
||||
|
||||
class TestPlugins(unittest.TestCase):
|
||||
|
||||
TEST_PLUGIN_DIR = TEST_DATA_DIR / PACKAGE_NAME
|
||||
|
||||
def test_directories_containing_plugins(self):
|
||||
self.assertIn(self.TEST_PLUGIN_DIR, map(Path, directories()))
|
||||
|
||||
def test_extractor_classes(self):
|
||||
for module_name in tuple(sys.modules):
|
||||
if module_name.startswith(f'{PACKAGE_NAME}.extractor'):
|
||||
del sys.modules[module_name]
|
||||
plugins_ie = load_plugins('extractor', 'IE')
|
||||
|
||||
self.assertIn(f'{PACKAGE_NAME}.extractor.normal', sys.modules.keys())
|
||||
self.assertIn('NormalPluginIE', plugins_ie.keys())
|
||||
|
||||
# don't load modules with underscore prefix
|
||||
self.assertFalse(
|
||||
f'{PACKAGE_NAME}.extractor._ignore' in sys.modules.keys(),
|
||||
'loaded module beginning with underscore')
|
||||
self.assertNotIn('IgnorePluginIE', plugins_ie.keys())
|
||||
|
||||
# Don't load extractors with underscore prefix
|
||||
self.assertNotIn('_IgnoreUnderscorePluginIE', plugins_ie.keys())
|
||||
|
||||
# Don't load extractors not specified in __all__ (if supplied)
|
||||
self.assertNotIn('IgnoreNotInAllPluginIE', plugins_ie.keys())
|
||||
self.assertIn('InAllPluginIE', plugins_ie.keys())
|
||||
|
||||
def test_postprocessor_classes(self):
|
||||
plugins_pp = load_plugins('postprocessor', 'PP')
|
||||
self.assertIn('NormalPluginPP', plugins_pp.keys())
|
||||
|
||||
def test_importing_zipped_module(self):
|
||||
zip_path = TEST_DATA_DIR / 'zipped_plugins.zip'
|
||||
shutil.make_archive(str(zip_path)[:-4], 'zip', str(zip_path)[:-4])
|
||||
sys.path.append(str(zip_path)) # add zip to search paths
|
||||
importlib.invalidate_caches() # reset the import caches
|
||||
|
||||
try:
|
||||
for plugin_type in ('extractor', 'postprocessor'):
|
||||
package = importlib.import_module(f'{PACKAGE_NAME}.{plugin_type}')
|
||||
self.assertIn(zip_path / PACKAGE_NAME / plugin_type, map(Path, package.__path__))
|
||||
|
||||
plugins_ie = load_plugins('extractor', 'IE')
|
||||
self.assertIn('ZippedPluginIE', plugins_ie.keys())
|
||||
|
||||
plugins_pp = load_plugins('postprocessor', 'PP')
|
||||
self.assertIn('ZippedPluginPP', plugins_pp.keys())
|
||||
|
||||
finally:
|
||||
sys.path.remove(str(zip_path))
|
||||
os.remove(zip_path)
|
||||
importlib.invalidate_caches() # reset the import caches
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
unittest.main()
|
@ -16,6 +16,7 @@
|
||||
MetadataFromFieldPP,
|
||||
MetadataParserPP,
|
||||
ModifyChaptersPP,
|
||||
SponsorBlockPP,
|
||||
)
|
||||
|
||||
|
||||
@ -76,11 +77,15 @@ def setUp(self):
|
||||
self._pp = ModifyChaptersPP(YoutubeDL())
|
||||
|
||||
@staticmethod
|
||||
def _sponsor_chapter(start, end, cat, remove=False):
|
||||
c = {'start_time': start, 'end_time': end, '_categories': [(cat, start, end)]}
|
||||
if remove:
|
||||
c['remove'] = True
|
||||
return c
|
||||
def _sponsor_chapter(start, end, cat, remove=False, title=None):
|
||||
if title is None:
|
||||
title = SponsorBlockPP.CATEGORIES[cat]
|
||||
return {
|
||||
'start_time': start,
|
||||
'end_time': end,
|
||||
'_categories': [(cat, start, end, title)],
|
||||
**({'remove': True} if remove else {}),
|
||||
}
|
||||
|
||||
@staticmethod
|
||||
def _chapter(start, end, title=None, remove=False):
|
||||
@ -130,6 +135,19 @@ def test_remove_marked_arrange_sponsors_ChapterWithSponsors(self):
|
||||
'c', '[SponsorBlock]: Filler Tangent', 'c'])
|
||||
self._remove_marked_arrange_sponsors_test_impl(chapters, expected, [])
|
||||
|
||||
def test_remove_marked_arrange_sponsors_SponsorBlockChapters(self):
|
||||
chapters = self._chapters([70], ['c']) + [
|
||||
self._sponsor_chapter(10, 20, 'chapter', title='sb c1'),
|
||||
self._sponsor_chapter(15, 16, 'chapter', title='sb c2'),
|
||||
self._sponsor_chapter(30, 40, 'preview'),
|
||||
self._sponsor_chapter(50, 60, 'filler')]
|
||||
expected = self._chapters(
|
||||
[10, 15, 16, 20, 30, 40, 50, 60, 70],
|
||||
['c', '[SponsorBlock]: sb c1', '[SponsorBlock]: sb c1, sb c2', '[SponsorBlock]: sb c1',
|
||||
'c', '[SponsorBlock]: Preview/Recap',
|
||||
'c', '[SponsorBlock]: Filler Tangent', 'c'])
|
||||
self._remove_marked_arrange_sponsors_test_impl(chapters, expected, [])
|
||||
|
||||
def test_remove_marked_arrange_sponsors_UniqueNamesForOverlappingSponsors(self):
|
||||
chapters = self._chapters([120], ['c']) + [
|
||||
self._sponsor_chapter(10, 45, 'sponsor'), self._sponsor_chapter(20, 40, 'selfpromo'),
|
||||
@ -173,7 +191,7 @@ def test_remove_marked_arrange_sponsors_ChapterWithSponsorCutInTheMiddle(self):
|
||||
self._remove_marked_arrange_sponsors_test_impl(chapters, expected, cuts)
|
||||
|
||||
def test_remove_marked_arrange_sponsors_ChapterWithCutHidingSponsor(self):
|
||||
cuts = [self._sponsor_chapter(20, 50, 'selpromo', remove=True)]
|
||||
cuts = [self._sponsor_chapter(20, 50, 'selfpromo', remove=True)]
|
||||
chapters = self._chapters([60], ['c']) + [
|
||||
self._sponsor_chapter(10, 20, 'intro'),
|
||||
self._sponsor_chapter(30, 40, 'sponsor'),
|
||||
@ -199,7 +217,7 @@ def test_remove_marked_arrange_sponsors_ChapterWithAdjacentCuts(self):
|
||||
self._sponsor_chapter(10, 20, 'sponsor'),
|
||||
self._sponsor_chapter(20, 30, 'interaction', remove=True),
|
||||
self._chapter(30, 40, remove=True),
|
||||
self._sponsor_chapter(40, 50, 'selpromo', remove=True),
|
||||
self._sponsor_chapter(40, 50, 'selfpromo', remove=True),
|
||||
self._sponsor_chapter(50, 60, 'interaction')]
|
||||
expected = self._chapters([10, 20, 30, 40],
|
||||
['c', '[SponsorBlock]: Sponsor',
|
||||
@ -282,7 +300,7 @@ def test_remove_marked_arrange_sponsors_SponsorsNoLongerOverlapAfterCut(self):
|
||||
chapters = self._chapters([70], ['c']) + [
|
||||
self._sponsor_chapter(10, 30, 'sponsor'),
|
||||
self._sponsor_chapter(20, 50, 'interaction'),
|
||||
self._sponsor_chapter(30, 50, 'selpromo', remove=True),
|
||||
self._sponsor_chapter(30, 50, 'selfpromo', remove=True),
|
||||
self._sponsor_chapter(40, 60, 'sponsor'),
|
||||
self._sponsor_chapter(50, 60, 'interaction')]
|
||||
expected = self._chapters(
|
||||
|
@ -2,6 +2,7 @@
|
||||
|
||||
# Allow direct execution
|
||||
import os
|
||||
import re
|
||||
import sys
|
||||
import unittest
|
||||
|
||||
@ -104,11 +105,13 @@
|
||||
sanitized_Request,
|
||||
shell_quote,
|
||||
smuggle_url,
|
||||
str_or_none,
|
||||
str_to_int,
|
||||
strip_jsonp,
|
||||
strip_or_none,
|
||||
subtitles_filename,
|
||||
timeconvert,
|
||||
traverse_obj,
|
||||
unescapeHTML,
|
||||
unified_strdate,
|
||||
unified_timestamp,
|
||||
@ -952,6 +955,85 @@ def test_escape_url(self):
|
||||
)
|
||||
self.assertEqual(escape_url('http://vimeo.com/56015672#at=0'), 'http://vimeo.com/56015672#at=0')
|
||||
|
||||
def test_js_to_json_vars_strings(self):
|
||||
self.assertDictEqual(
|
||||
json.loads(js_to_json(
|
||||
'''{
|
||||
'null': a,
|
||||
'nullStr': b,
|
||||
'true': c,
|
||||
'trueStr': d,
|
||||
'false': e,
|
||||
'falseStr': f,
|
||||
'unresolvedVar': g,
|
||||
}''',
|
||||
{
|
||||
'a': 'null',
|
||||
'b': '"null"',
|
||||
'c': 'true',
|
||||
'd': '"true"',
|
||||
'e': 'false',
|
||||
'f': '"false"',
|
||||
'g': 'var',
|
||||
}
|
||||
)),
|
||||
{
|
||||
'null': None,
|
||||
'nullStr': 'null',
|
||||
'true': True,
|
||||
'trueStr': 'true',
|
||||
'false': False,
|
||||
'falseStr': 'false',
|
||||
'unresolvedVar': 'var'
|
||||
}
|
||||
)
|
||||
|
||||
self.assertDictEqual(
|
||||
json.loads(js_to_json(
|
||||
'''{
|
||||
'int': a,
|
||||
'intStr': b,
|
||||
'float': c,
|
||||
'floatStr': d,
|
||||
}''',
|
||||
{
|
||||
'a': '123',
|
||||
'b': '"123"',
|
||||
'c': '1.23',
|
||||
'd': '"1.23"',
|
||||
}
|
||||
)),
|
||||
{
|
||||
'int': 123,
|
||||
'intStr': '123',
|
||||
'float': 1.23,
|
||||
'floatStr': '1.23',
|
||||
}
|
||||
)
|
||||
|
||||
self.assertDictEqual(
|
||||
json.loads(js_to_json(
|
||||
'''{
|
||||
'object': a,
|
||||
'objectStr': b,
|
||||
'array': c,
|
||||
'arrayStr': d,
|
||||
}''',
|
||||
{
|
||||
'a': '{}',
|
||||
'b': '"{}"',
|
||||
'c': '[]',
|
||||
'd': '"[]"',
|
||||
}
|
||||
)),
|
||||
{
|
||||
'object': {},
|
||||
'objectStr': '{}',
|
||||
'array': [],
|
||||
'arrayStr': '[]',
|
||||
}
|
||||
)
|
||||
|
||||
def test_js_to_json_realworld(self):
|
||||
inp = '''{
|
||||
'clip':{'provider':'pseudo'}
|
||||
@ -1098,6 +1180,12 @@ def test_js_to_json_edgecases(self):
|
||||
on = js_to_json('[1,//{},\n2]')
|
||||
self.assertEqual(json.loads(on), [1, 2])
|
||||
|
||||
on = js_to_json(R'"\^\$\#"')
|
||||
self.assertEqual(json.loads(on), R'^$#', msg='Unnecessary escapes should be stripped')
|
||||
|
||||
on = js_to_json('\'"\\""\'')
|
||||
self.assertEqual(json.loads(on), '"""', msg='Unnecessary quote escape should be escaped')
|
||||
|
||||
def test_js_to_json_malformed(self):
|
||||
self.assertEqual(js_to_json('42a1'), '42"a1"')
|
||||
self.assertEqual(js_to_json('42a-1'), '42"a"-1')
|
||||
@ -1677,6 +1765,9 @@ def test_get_elements_text_and_html_by_attribute(self):
|
||||
self.assertEqual(list(get_elements_text_and_html_by_attribute('class', 'foo', html)), [])
|
||||
self.assertEqual(list(get_elements_text_and_html_by_attribute('class', 'no-such-foo', html)), [])
|
||||
|
||||
self.assertEqual(list(get_elements_text_and_html_by_attribute(
|
||||
'class', 'foo', '<a class="foo">nice</a><span class="foo">nice</span>', tag='a')), [('nice', '<a class="foo">nice</a>')])
|
||||
|
||||
GET_ELEMENT_BY_TAG_TEST_STRING = '''
|
||||
random text lorem ipsum</p>
|
||||
<div>
|
||||
@ -1863,6 +1954,8 @@ def test_get_compatible_ext(self):
|
||||
vcodecs=[None], acodecs=[None], vexts=['webm'], aexts=['m4a']), 'mkv')
|
||||
self.assertEqual(get_compatible_ext(
|
||||
vcodecs=[None], acodecs=[None], vexts=['webm'], aexts=['webm']), 'webm')
|
||||
self.assertEqual(get_compatible_ext(
|
||||
vcodecs=[None], acodecs=[None], vexts=['webm'], aexts=['weba']), 'webm')
|
||||
|
||||
self.assertEqual(get_compatible_ext(
|
||||
vcodecs=['h264'], acodecs=['mp4a'], vexts=['mov'], aexts=['m4a']), 'mp4')
|
||||
@ -1874,6 +1967,301 @@ def test_get_compatible_ext(self):
|
||||
self.assertEqual(get_compatible_ext(
|
||||
vcodecs=['av1'], acodecs=['mp4a'], vexts=['webm'], aexts=['m4a'], preferences=('webm', 'mkv')), 'mkv')
|
||||
|
||||
def test_traverse_obj(self):
|
||||
_TEST_DATA = {
|
||||
100: 100,
|
||||
1.2: 1.2,
|
||||
'str': 'str',
|
||||
'None': None,
|
||||
'...': ...,
|
||||
'urls': [
|
||||
{'index': 0, 'url': 'https://www.example.com/0'},
|
||||
{'index': 1, 'url': 'https://www.example.com/1'},
|
||||
],
|
||||
'data': (
|
||||
{'index': 2},
|
||||
{'index': 3},
|
||||
),
|
||||
'dict': {},
|
||||
}
|
||||
|
||||
# Test base functionality
|
||||
self.assertEqual(traverse_obj(_TEST_DATA, ('str',)), 'str',
|
||||
msg='allow tuple path')
|
||||
self.assertEqual(traverse_obj(_TEST_DATA, ['str']), 'str',
|
||||
msg='allow list path')
|
||||
self.assertEqual(traverse_obj(_TEST_DATA, (value for value in ("str",))), 'str',
|
||||
msg='allow iterable path')
|
||||
self.assertEqual(traverse_obj(_TEST_DATA, 'str'), 'str',
|
||||
msg='single items should be treated as a path')
|
||||
self.assertEqual(traverse_obj(_TEST_DATA, None), _TEST_DATA)
|
||||
self.assertEqual(traverse_obj(_TEST_DATA, 100), 100)
|
||||
self.assertEqual(traverse_obj(_TEST_DATA, 1.2), 1.2)
|
||||
|
||||
# Test Ellipsis behavior
|
||||
self.assertCountEqual(traverse_obj(_TEST_DATA, ...),
|
||||
(item for item in _TEST_DATA.values() if item not in (None, {})),
|
||||
msg='`...` should give all non discarded values')
|
||||
self.assertCountEqual(traverse_obj(_TEST_DATA, ('urls', 0, ...)), _TEST_DATA['urls'][0].values(),
|
||||
msg='`...` selection for dicts should select all values')
|
||||
self.assertEqual(traverse_obj(_TEST_DATA, (..., ..., 'url')),
|
||||
['https://www.example.com/0', 'https://www.example.com/1'],
|
||||
msg='nested `...` queries should work')
|
||||
self.assertCountEqual(traverse_obj(_TEST_DATA, (..., ..., 'index')), range(4),
|
||||
msg='`...` query result should be flattened')
|
||||
|
||||
# Test function as key
|
||||
self.assertEqual(traverse_obj(_TEST_DATA, lambda x, y: x == 'urls' and isinstance(y, list)),
|
||||
[_TEST_DATA['urls']],
|
||||
msg='function as query key should perform a filter based on (key, value)')
|
||||
self.assertCountEqual(traverse_obj(_TEST_DATA, lambda _, x: isinstance(x[0], str)), {'str'},
|
||||
msg='exceptions in the query function should be catched')
|
||||
if __debug__:
|
||||
with self.assertRaises(Exception, msg='Wrong function signature should raise in debug'):
|
||||
traverse_obj(_TEST_DATA, lambda a: ...)
|
||||
with self.assertRaises(Exception, msg='Wrong function signature should raise in debug'):
|
||||
traverse_obj(_TEST_DATA, lambda a, b, c: ...)
|
||||
|
||||
# Test set as key (transformation/type, like `expected_type`)
|
||||
self.assertEqual(traverse_obj(_TEST_DATA, (..., {str.upper}, )), ['STR'],
|
||||
msg='Function in set should be a transformation')
|
||||
self.assertEqual(traverse_obj(_TEST_DATA, (..., {str})), ['str'],
|
||||
msg='Type in set should be a type filter')
|
||||
self.assertEqual(traverse_obj(_TEST_DATA, {dict}), _TEST_DATA,
|
||||
msg='A single set should be wrapped into a path')
|
||||
self.assertEqual(traverse_obj(_TEST_DATA, (..., {str.upper})), ['STR'],
|
||||
msg='Transformation function should not raise')
|
||||
self.assertEqual(traverse_obj(_TEST_DATA, (..., {str_or_none})),
|
||||
[item for item in map(str_or_none, _TEST_DATA.values()) if item is not None],
|
||||
msg='Function in set should be a transformation')
|
||||
if __debug__:
|
||||
with self.assertRaises(Exception, msg='Sets with length != 1 should raise in debug'):
|
||||
traverse_obj(_TEST_DATA, set())
|
||||
with self.assertRaises(Exception, msg='Sets with length != 1 should raise in debug'):
|
||||
traverse_obj(_TEST_DATA, {str.upper, str})
|
||||
|
||||
# Test alternative paths
|
||||
self.assertEqual(traverse_obj(_TEST_DATA, 'fail', 'str'), 'str',
|
||||
msg='multiple `paths` should be treated as alternative paths')
|
||||
self.assertEqual(traverse_obj(_TEST_DATA, 'str', 100), 'str',
|
||||
msg='alternatives should exit early')
|
||||
self.assertEqual(traverse_obj(_TEST_DATA, 'fail', 'fail'), None,
|
||||
msg='alternatives should return `default` if exhausted')
|
||||
self.assertEqual(traverse_obj(_TEST_DATA, (..., 'fail'), 100), 100,
|
||||
msg='alternatives should track their own branching return')
|
||||
self.assertEqual(traverse_obj(_TEST_DATA, ('dict', ...), ('data', ...)), list(_TEST_DATA['data']),
|
||||
msg='alternatives on empty objects should search further')
|
||||
|
||||
# Test branch and path nesting
|
||||
self.assertEqual(traverse_obj(_TEST_DATA, ('urls', (3, 0), 'url')), ['https://www.example.com/0'],
|
||||
msg='tuple as key should be treated as branches')
|
||||
self.assertEqual(traverse_obj(_TEST_DATA, ('urls', [3, 0], 'url')), ['https://www.example.com/0'],
|
||||
msg='list as key should be treated as branches')
|
||||
self.assertEqual(traverse_obj(_TEST_DATA, ('urls', ((1, 'fail'), (0, 'url')))), ['https://www.example.com/0'],
|
||||
msg='double nesting in path should be treated as paths')
|
||||
self.assertEqual(traverse_obj(['0', [1, 2]], [(0, 1), 0]), [1],
|
||||
msg='do not fail early on branching')
|
||||
self.assertCountEqual(traverse_obj(_TEST_DATA, ('urls', ((1, ('fail', 'url')), (0, 'url')))),
|
||||
['https://www.example.com/0', 'https://www.example.com/1'],
|
||||
msg='tripple nesting in path should be treated as branches')
|
||||
self.assertEqual(traverse_obj(_TEST_DATA, ('urls', ('fail', (..., 'url')))),
|
||||
['https://www.example.com/0', 'https://www.example.com/1'],
|
||||
msg='ellipsis as branch path start gets flattened')
|
||||
|
||||
# Test dictionary as key
|
||||
self.assertEqual(traverse_obj(_TEST_DATA, {0: 100, 1: 1.2}), {0: 100, 1: 1.2},
|
||||
msg='dict key should result in a dict with the same keys')
|
||||
self.assertEqual(traverse_obj(_TEST_DATA, {0: ('urls', 0, 'url')}),
|
||||
{0: 'https://www.example.com/0'},
|
||||
msg='dict key should allow paths')
|
||||
self.assertEqual(traverse_obj(_TEST_DATA, {0: ('urls', (3, 0), 'url')}),
|
||||
{0: ['https://www.example.com/0']},
|
||||
msg='tuple in dict path should be treated as branches')
|
||||
self.assertEqual(traverse_obj(_TEST_DATA, {0: ('urls', ((1, 'fail'), (0, 'url')))}),
|
||||
{0: ['https://www.example.com/0']},
|
||||
msg='double nesting in dict path should be treated as paths')
|
||||
self.assertEqual(traverse_obj(_TEST_DATA, {0: ('urls', ((1, ('fail', 'url')), (0, 'url')))}),
|
||||
{0: ['https://www.example.com/1', 'https://www.example.com/0']},
|
||||
msg='tripple nesting in dict path should be treated as branches')
|
||||
self.assertEqual(traverse_obj(_TEST_DATA, {0: 'fail'}), {},
|
||||
msg='remove `None` values when top level dict key fails')
|
||||
self.assertEqual(traverse_obj(_TEST_DATA, {0: 'fail'}, default=...), {0: ...},
|
||||
msg='use `default` if key fails and `default`')
|
||||
self.assertEqual(traverse_obj(_TEST_DATA, {0: 'dict'}), {},
|
||||
msg='remove empty values when dict key')
|
||||
self.assertEqual(traverse_obj(_TEST_DATA, {0: 'dict'}, default=...), {0: ...},
|
||||
msg='use `default` when dict key and `default`')
|
||||
self.assertEqual(traverse_obj(_TEST_DATA, {0: {0: 'fail'}}), {},
|
||||
msg='remove empty values when nested dict key fails')
|
||||
self.assertEqual(traverse_obj(None, {0: 'fail'}), {},
|
||||
msg='default to dict if pruned')
|
||||
self.assertEqual(traverse_obj(None, {0: 'fail'}, default=...), {0: ...},
|
||||
msg='default to dict if pruned and default is given')
|
||||
self.assertEqual(traverse_obj(_TEST_DATA, {0: {0: 'fail'}}, default=...), {0: {0: ...}},
|
||||
msg='use nested `default` when nested dict key fails and `default`')
|
||||
self.assertEqual(traverse_obj(_TEST_DATA, {0: ('dict', ...)}), {},
|
||||
msg='remove key if branch in dict key not successful')
|
||||
|
||||
# Testing default parameter behavior
|
||||
_DEFAULT_DATA = {'None': None, 'int': 0, 'list': []}
|
||||
self.assertEqual(traverse_obj(_DEFAULT_DATA, 'fail'), None,
|
||||
msg='default value should be `None`')
|
||||
self.assertEqual(traverse_obj(_DEFAULT_DATA, 'fail', 'fail', default=...), ...,
|
||||
msg='chained fails should result in default')
|
||||
self.assertEqual(traverse_obj(_DEFAULT_DATA, 'None', 'int'), 0,
|
||||
msg='should not short cirquit on `None`')
|
||||
self.assertEqual(traverse_obj(_DEFAULT_DATA, 'fail', default=1), 1,
|
||||
msg='invalid dict key should result in `default`')
|
||||
self.assertEqual(traverse_obj(_DEFAULT_DATA, 'None', default=1), 1,
|
||||
msg='`None` is a deliberate sentinel and should become `default`')
|
||||
self.assertEqual(traverse_obj(_DEFAULT_DATA, ('list', 10)), None,
|
||||
msg='`IndexError` should result in `default`')
|
||||
self.assertEqual(traverse_obj(_DEFAULT_DATA, (..., 'fail'), default=1), 1,
|
||||
msg='if branched but not successful return `default` if defined, not `[]`')
|
||||
self.assertEqual(traverse_obj(_DEFAULT_DATA, (..., 'fail'), default=None), None,
|
||||
msg='if branched but not successful return `default` even if `default` is `None`')
|
||||
self.assertEqual(traverse_obj(_DEFAULT_DATA, (..., 'fail')), [],
|
||||
msg='if branched but not successful return `[]`, not `default`')
|
||||
self.assertEqual(traverse_obj(_DEFAULT_DATA, ('list', ...)), [],
|
||||
msg='if branched but object is empty return `[]`, not `default`')
|
||||
self.assertEqual(traverse_obj(None, ...), [],
|
||||
msg='if branched but object is `None` return `[]`, not `default`')
|
||||
self.assertEqual(traverse_obj({0: None}, (0, ...)), [],
|
||||
msg='if branched but state is `None` return `[]`, not `default`')
|
||||
|
||||
branching_paths = [
|
||||
('fail', ...),
|
||||
(..., 'fail'),
|
||||
100 * ('fail',) + (...,),
|
||||
(...,) + 100 * ('fail',),
|
||||
]
|
||||
for branching_path in branching_paths:
|
||||
self.assertEqual(traverse_obj({}, branching_path), [],
|
||||
msg='if branched but state is `None`, return `[]` (not `default`)')
|
||||
self.assertEqual(traverse_obj({}, 'fail', branching_path), [],
|
||||
msg='if branching in last alternative and previous did not match, return `[]` (not `default`)')
|
||||
self.assertEqual(traverse_obj({0: 'x'}, 0, branching_path), 'x',
|
||||
msg='if branching in last alternative and previous did match, return single value')
|
||||
self.assertEqual(traverse_obj({0: 'x'}, branching_path, 0), 'x',
|
||||
msg='if branching in first alternative and non-branching path does match, return single value')
|
||||
self.assertEqual(traverse_obj({}, branching_path, 'fail'), None,
|
||||
msg='if branching in first alternative and non-branching path does not match, return `default`')
|
||||
|
||||
# Testing expected_type behavior
|
||||
_EXPECTED_TYPE_DATA = {'str': 'str', 'int': 0}
|
||||
self.assertEqual(traverse_obj(_EXPECTED_TYPE_DATA, 'str', expected_type=str),
|
||||
'str', msg='accept matching `expected_type` type')
|
||||
self.assertEqual(traverse_obj(_EXPECTED_TYPE_DATA, 'str', expected_type=int),
|
||||
None, msg='reject non matching `expected_type` type')
|
||||
self.assertEqual(traverse_obj(_EXPECTED_TYPE_DATA, 'int', expected_type=lambda x: str(x)),
|
||||
'0', msg='transform type using type function')
|
||||
self.assertEqual(traverse_obj(_EXPECTED_TYPE_DATA, 'str', expected_type=lambda _: 1 / 0),
|
||||
None, msg='wrap expected_type fuction in try_call')
|
||||
self.assertEqual(traverse_obj(_EXPECTED_TYPE_DATA, ..., expected_type=str),
|
||||
['str'], msg='eliminate items that expected_type fails on')
|
||||
self.assertEqual(traverse_obj(_TEST_DATA, {0: 100, 1: 1.2}, expected_type=int),
|
||||
{0: 100}, msg='type as expected_type should filter dict values')
|
||||
self.assertEqual(traverse_obj(_TEST_DATA, {0: 100, 1: 1.2, 2: 'None'}, expected_type=str_or_none),
|
||||
{0: '100', 1: '1.2'}, msg='function as expected_type should transform dict values')
|
||||
self.assertEqual(traverse_obj(_TEST_DATA, ({0: 1.2}, 0, {int_or_none}), expected_type=int),
|
||||
1, msg='expected_type should not filter non final dict values')
|
||||
self.assertEqual(traverse_obj(_TEST_DATA, {0: {0: 100, 1: 'str'}}, expected_type=int),
|
||||
{0: {0: 100}}, msg='expected_type should transform deep dict values')
|
||||
self.assertEqual(traverse_obj(_TEST_DATA, [({0: '...'}, {0: '...'})], expected_type=type(...)),
|
||||
[{0: ...}, {0: ...}], msg='expected_type should transform branched dict values')
|
||||
self.assertEqual(traverse_obj({1: {3: 4}}, [(1, 2), 3], expected_type=int),
|
||||
[4], msg='expected_type regression for type matching in tuple branching')
|
||||
self.assertEqual(traverse_obj(_TEST_DATA, ['data', ...], expected_type=int),
|
||||
[], msg='expected_type regression for type matching in dict result')
|
||||
|
||||
# Test get_all behavior
|
||||
_GET_ALL_DATA = {'key': [0, 1, 2]}
|
||||
self.assertEqual(traverse_obj(_GET_ALL_DATA, ('key', ...), get_all=False), 0,
|
||||
msg='if not `get_all`, return only first matching value')
|
||||
self.assertEqual(traverse_obj(_GET_ALL_DATA, ..., get_all=False), [0, 1, 2],
|
||||
msg='do not overflatten if not `get_all`')
|
||||
|
||||
# Test casesense behavior
|
||||
_CASESENSE_DATA = {
|
||||
'KeY': 'value0',
|
||||
0: {
|
||||
'KeY': 'value1',
|
||||
0: {'KeY': 'value2'},
|
||||
},
|
||||
}
|
||||
self.assertEqual(traverse_obj(_CASESENSE_DATA, 'key'), None,
|
||||
msg='dict keys should be case sensitive unless `casesense`')
|
||||
self.assertEqual(traverse_obj(_CASESENSE_DATA, 'keY',
|
||||
casesense=False), 'value0',
|
||||
msg='allow non matching key case if `casesense`')
|
||||
self.assertEqual(traverse_obj(_CASESENSE_DATA, (0, ('keY',)),
|
||||
casesense=False), ['value1'],
|
||||
msg='allow non matching key case in branch if `casesense`')
|
||||
self.assertEqual(traverse_obj(_CASESENSE_DATA, (0, ((0, 'keY'),)),
|
||||
casesense=False), ['value2'],
|
||||
msg='allow non matching key case in branch path if `casesense`')
|
||||
|
||||
# Test traverse_string behavior
|
||||
_TRAVERSE_STRING_DATA = {'str': 'str', 1.2: 1.2}
|
||||
self.assertEqual(traverse_obj(_TRAVERSE_STRING_DATA, ('str', 0)), None,
|
||||
msg='do not traverse into string if not `traverse_string`')
|
||||
self.assertEqual(traverse_obj(_TRAVERSE_STRING_DATA, ('str', 0),
|
||||
traverse_string=True), 's',
|
||||
msg='traverse into string if `traverse_string`')
|
||||
self.assertEqual(traverse_obj(_TRAVERSE_STRING_DATA, (1.2, 1),
|
||||
traverse_string=True), '.',
|
||||
msg='traverse into converted data if `traverse_string`')
|
||||
self.assertEqual(traverse_obj(_TRAVERSE_STRING_DATA, ('str', ...),
|
||||
traverse_string=True), 'str',
|
||||
msg='`...` should result in string (same value) if `traverse_string`')
|
||||
self.assertEqual(traverse_obj(_TRAVERSE_STRING_DATA, ('str', slice(0, None, 2)),
|
||||
traverse_string=True), 'sr',
|
||||
msg='`slice` should result in string if `traverse_string`')
|
||||
self.assertEqual(traverse_obj(_TRAVERSE_STRING_DATA, ('str', lambda i, v: i or v == "s"),
|
||||
traverse_string=True), 'str',
|
||||
msg='function should result in string if `traverse_string`')
|
||||
self.assertEqual(traverse_obj(_TRAVERSE_STRING_DATA, ('str', (0, 2)),
|
||||
traverse_string=True), ['s', 'r'],
|
||||
msg='branching should result in list if `traverse_string`')
|
||||
|
||||
# Test is_user_input behavior
|
||||
_IS_USER_INPUT_DATA = {'range8': list(range(8))}
|
||||
self.assertEqual(traverse_obj(_IS_USER_INPUT_DATA, ('range8', '3'),
|
||||
is_user_input=True), 3,
|
||||
msg='allow for string indexing if `is_user_input`')
|
||||
self.assertCountEqual(traverse_obj(_IS_USER_INPUT_DATA, ('range8', '3:'),
|
||||
is_user_input=True), tuple(range(8))[3:],
|
||||
msg='allow for string slice if `is_user_input`')
|
||||
self.assertCountEqual(traverse_obj(_IS_USER_INPUT_DATA, ('range8', ':4:2'),
|
||||
is_user_input=True), tuple(range(8))[:4:2],
|
||||
msg='allow step in string slice if `is_user_input`')
|
||||
self.assertCountEqual(traverse_obj(_IS_USER_INPUT_DATA, ('range8', ':'),
|
||||
is_user_input=True), range(8),
|
||||
msg='`:` should be treated as `...` if `is_user_input`')
|
||||
with self.assertRaises(TypeError, msg='too many params should result in error'):
|
||||
traverse_obj(_IS_USER_INPUT_DATA, ('range8', ':::'), is_user_input=True)
|
||||
|
||||
# Test re.Match as input obj
|
||||
mobj = re.fullmatch(r'0(12)(?P<group>3)(4)?', '0123')
|
||||
self.assertEqual(traverse_obj(mobj, ...), [x for x in mobj.groups() if x is not None],
|
||||
msg='`...` on a `re.Match` should give its `groups()`')
|
||||
self.assertEqual(traverse_obj(mobj, lambda k, _: k in (0, 2)), ['0123', '3'],
|
||||
msg='function on a `re.Match` should give groupno, value starting at 0')
|
||||
self.assertEqual(traverse_obj(mobj, 'group'), '3',
|
||||
msg='str key on a `re.Match` should give group with that name')
|
||||
self.assertEqual(traverse_obj(mobj, 2), '3',
|
||||
msg='int key on a `re.Match` should give group with that name')
|
||||
self.assertEqual(traverse_obj(mobj, 'gRoUp', casesense=False), '3',
|
||||
msg='str key on a `re.Match` should respect casesense')
|
||||
self.assertEqual(traverse_obj(mobj, 'fail'), None,
|
||||
msg='failing str key on a `re.Match` should return `default`')
|
||||
self.assertEqual(traverse_obj(mobj, 'gRoUpS', casesense=False), None,
|
||||
msg='failing str key on a `re.Match` should return `default`')
|
||||
self.assertEqual(traverse_obj(mobj, 8), None,
|
||||
msg='failing int key on a `re.Match` should return `default`')
|
||||
self.assertEqual(traverse_obj(mobj, lambda k, _: k in (0, 'group')), ['0123', '3'],
|
||||
msg='function on a `re.Match` should give group name as well')
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
unittest.main()
|
||||
|
@ -10,6 +10,7 @@
|
||||
|
||||
from test.helper import FakeYDL, is_download_test
|
||||
from yt_dlp.extractor import YoutubeIE, YoutubeTabIE
|
||||
from yt_dlp.utils import ExtractorError
|
||||
|
||||
|
||||
@is_download_test
|
||||
@ -53,6 +54,18 @@ def test_youtube_flat_playlist_extraction(self):
|
||||
self.assertEqual(video['duration'], 10)
|
||||
self.assertEqual(video['uploader'], 'Philipp Hagemeister')
|
||||
|
||||
def test_youtube_channel_no_uploads(self):
|
||||
dl = FakeYDL()
|
||||
dl.params['extract_flat'] = True
|
||||
ie = YoutubeTabIE(dl)
|
||||
# no uploads
|
||||
with self.assertRaisesRegex(ExtractorError, r'no uploads'):
|
||||
ie.extract('https://www.youtube.com/channel/UC2yXPzFejc422buOIzn_0CA')
|
||||
|
||||
# no uploads and no UCID given
|
||||
with self.assertRaisesRegex(ExtractorError, r'no uploads'):
|
||||
ie.extract('https://www.youtube.com/news')
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
unittest.main()
|
||||
|
@ -66,6 +66,10 @@
|
||||
]
|
||||
|
||||
_NSIG_TESTS = [
|
||||
(
|
||||
'https://www.youtube.com/s/player/7862ca1f/player_ias.vflset/en_US/base.js',
|
||||
'X_LCxVDjAavgE5t', 'yxJ1dM6iz5ogUg',
|
||||
),
|
||||
(
|
||||
'https://www.youtube.com/s/player/9216d1f7/player_ias.vflset/en_US/base.js',
|
||||
'SLp9F5bwjAdhE9F-', 'gWnb9IK2DJ8Q1w',
|
||||
@ -130,6 +134,14 @@
|
||||
'https://www.youtube.com/s/player/5a3b6271/player_ias.vflset/en_US/base.js',
|
||||
'B2j7f_UPT4rfje85Lu_e', 'm5DmNymaGQ5RdQ',
|
||||
),
|
||||
(
|
||||
'https://www.youtube.com/s/player/7a062b77/player_ias.vflset/en_US/base.js',
|
||||
'NRcE3y3mVtm_cV-W', 'VbsCYUATvqlt5w',
|
||||
),
|
||||
(
|
||||
'https://www.youtube.com/s/player/dac945fd/player_ias.vflset/en_US/base.js',
|
||||
'o8BkRxXhuYsBCWi6RplPdP', '3Lx32v_hmzTm6A',
|
||||
),
|
||||
]
|
||||
|
||||
|
||||
|
1
test/testdata/ism/ec-3_test.Manifest
vendored
Normal file
1
test/testdata/ism/ec-3_test.Manifest
vendored
Normal file
@ -0,0 +1 @@
|
||||
<?xml version="1.0" encoding="utf-8"?><!--Transformed by VSMT using XSL stylesheet for rule Identity--><!-- Created with Unified Streaming Platform (version=1.10.12-18737) --><SmoothStreamingMedia MajorVersion="2" MinorVersion="0" TimeScale="10000000" Duration="370000000"><StreamIndex Type="audio" QualityLevels="1" TimeScale="10000000" Language="deu" Name="audio_deu" Chunks="19" Url="QualityLevels({bitrate})/Fragments(audio_deu={start time})?noStreamProfile=1"><QualityLevel Index="0" Bitrate="127802" CodecPrivateData="1190" SamplingRate="48000" Channels="2" BitsPerSample="16" PacketSize="4" AudioTag="255" FourCC="AACL" /><c t="0" d="20053333" /><c d="20053334" /><c d="20053333" /><c d="19840000" /><c d="20053333" /><c d="20053334" /><c d="20053333" /><c d="19840000" /><c d="20053333" /><c d="20053334" /><c d="20053333" /><c d="19840000" /><c d="20053333" /><c d="20053334" /><c d="20053333" /><c d="19840000" /><c d="20053333" /><c d="20053334" /><c d="7253333" /></StreamIndex><StreamIndex Type="audio" QualityLevels="1" TimeScale="10000000" Language="deu" Name="audio_deu_1" Chunks="19" Url="QualityLevels({bitrate})/Fragments(audio_deu_1={start time})?noStreamProfile=1"><QualityLevel Index="0" Bitrate="224000" CodecPrivateData="00063F000000AF87FBA7022DFB42A4D405CD93843BDD0700200F00" FourCCData="0700200F00" SamplingRate="48000" Channels="6" BitsPerSample="16" PacketSize="896" AudioTag="65534" FourCC="EC-3" /><c t="0" d="20160000" /><c d="19840000" /><c d="20160000" /><c d="19840000" /><c d="20160000" /><c d="19840000" /><c d="20160000" /><c d="19840000" /><c d="20160000" /><c d="19840000" /><c d="20160000" /><c d="19840000" /><c d="20160000" /><c d="19840000" /><c d="20160000" /><c d="19840000" /><c d="20160000" /><c d="19840000" /><c d="8320000" /></StreamIndex><StreamIndex Type="video" QualityLevels="8" TimeScale="10000000" Language="deu" Name="video_deu" Chunks="19" Url="QualityLevels({bitrate})/Fragments(video_deu={start time})?noStreamProfile=1" MaxWidth="1920" MaxHeight="1080" DisplayWidth="1920" DisplayHeight="1080"><QualityLevel Index="0" Bitrate="23909" CodecPrivateData="000000016742C00CDB06077E5C05A808080A00000300020000030009C0C02EE0177CC6300F142AE00000000168CA8DC8" MaxWidth="384" MaxHeight="216" FourCC="AVC1" /><QualityLevel Index="1" Bitrate="403188" CodecPrivateData="00000001674D4014E98323B602D4040405000003000100000300320F1429380000000168EAECF2" MaxWidth="400" MaxHeight="224" FourCC="AVC1" /><QualityLevel Index="2" Bitrate="680365" CodecPrivateData="00000001674D401EE981405FF2E02D4040405000000300100000030320F162D3800000000168EAECF2" MaxWidth="640" MaxHeight="360" FourCC="AVC1" /><QualityLevel Index="3" Bitrate="1253465" CodecPrivateData="00000001674D401EE981405FF2E02D4040405000000300100000030320F162D3800000000168EAECF2" MaxWidth="640" MaxHeight="360" FourCC="AVC1" /><QualityLevel Index="4" Bitrate="2121558" CodecPrivateData="00000001674D401EECA0601BD80B50101014000003000400000300C83C58B6580000000168E93B3C80" MaxWidth="768" MaxHeight="432" FourCC="AVC1" /><QualityLevel Index="5" Bitrate="3275545" CodecPrivateData="00000001674D4020ECA02802DD80B501010140000003004000000C83C60C65800000000168E93B3C80" MaxWidth="1280" MaxHeight="720" FourCC="AVC1" /><QualityLevel Index="6" Bitrate="5300196" CodecPrivateData="00000001674D4028ECA03C0113F2E02D4040405000000300100000030320F18319600000000168E93B3C80" MaxWidth="1920" MaxHeight="1080" FourCC="AVC1" /><QualityLevel Index="7" Bitrate="8079312" CodecPrivateData="00000001674D4028ECA03C0113F2E02D4040405000000300100000030320F18319600000000168E93B3C80" MaxWidth="1920" MaxHeight="1080" FourCC="AVC1" /><c t="0" d="20000000" /><c d="20000000" /><c d="20000000" /><c d="20000000" /><c d="20000000" /><c d="20000000" /><c d="20000000" /><c d="20000000" /><c d="20000000" /><c d="20000000" /><c d="20000000" /><c d="20000000" /><c d="20000000" /><c d="20000000" /><c d="20000000" /><c d="20000000" /><c d="20000000" /><c d="20000000" /><c d="10000000" /></StreamIndex></SmoothStreamingMedia>
|
5
test/testdata/yt_dlp_plugins/extractor/_ignore.py
vendored
Normal file
5
test/testdata/yt_dlp_plugins/extractor/_ignore.py
vendored
Normal file
@ -0,0 +1,5 @@
|
||||
from yt_dlp.extractor.common import InfoExtractor
|
||||
|
||||
|
||||
class IgnorePluginIE(InfoExtractor):
|
||||
pass
|
12
test/testdata/yt_dlp_plugins/extractor/ignore.py
vendored
Normal file
12
test/testdata/yt_dlp_plugins/extractor/ignore.py
vendored
Normal file
@ -0,0 +1,12 @@
|
||||
from yt_dlp.extractor.common import InfoExtractor
|
||||
|
||||
|
||||
class IgnoreNotInAllPluginIE(InfoExtractor):
|
||||
pass
|
||||
|
||||
|
||||
class InAllPluginIE(InfoExtractor):
|
||||
pass
|
||||
|
||||
|
||||
__all__ = ['InAllPluginIE']
|
9
test/testdata/yt_dlp_plugins/extractor/normal.py
vendored
Normal file
9
test/testdata/yt_dlp_plugins/extractor/normal.py
vendored
Normal file
@ -0,0 +1,9 @@
|
||||
from yt_dlp.extractor.common import InfoExtractor
|
||||
|
||||
|
||||
class NormalPluginIE(InfoExtractor):
|
||||
pass
|
||||
|
||||
|
||||
class _IgnoreUnderscorePluginIE(InfoExtractor):
|
||||
pass
|
5
test/testdata/yt_dlp_plugins/postprocessor/normal.py
vendored
Normal file
5
test/testdata/yt_dlp_plugins/postprocessor/normal.py
vendored
Normal file
@ -0,0 +1,5 @@
|
||||
from yt_dlp.postprocessor.common import PostProcessor
|
||||
|
||||
|
||||
class NormalPluginPP(PostProcessor):
|
||||
pass
|
5
test/testdata/zipped_plugins/yt_dlp_plugins/extractor/zipped.py
vendored
Normal file
5
test/testdata/zipped_plugins/yt_dlp_plugins/extractor/zipped.py
vendored
Normal file
@ -0,0 +1,5 @@
|
||||
from yt_dlp.extractor.common import InfoExtractor
|
||||
|
||||
|
||||
class ZippedPluginIE(InfoExtractor):
|
||||
pass
|
5
test/testdata/zipped_plugins/yt_dlp_plugins/postprocessor/zipped.py
vendored
Normal file
5
test/testdata/zipped_plugins/yt_dlp_plugins/postprocessor/zipped.py
vendored
Normal file
@ -0,0 +1,5 @@
|
||||
from yt_dlp.postprocessor.common import PostProcessor
|
||||
|
||||
|
||||
class ZippedPluginPP(PostProcessor):
|
||||
pass
|
@ -32,7 +32,8 @@
|
||||
from .extractor.common import UnsupportedURLIE
|
||||
from .extractor.openload import PhantomJSwrapper
|
||||
from .minicurses import format_text
|
||||
from .postprocessor import _PLUGIN_CLASSES as plugin_postprocessors
|
||||
from .plugins import directories as plugin_directories
|
||||
from .postprocessor import _PLUGIN_CLASSES as plugin_pps
|
||||
from .postprocessor import (
|
||||
EmbedThumbnailPP,
|
||||
FFmpegFixupDuplicateMoovPP,
|
||||
@ -67,6 +68,7 @@
|
||||
EntryNotInPlaylist,
|
||||
ExistingVideoReached,
|
||||
ExtractorError,
|
||||
FormatSorter,
|
||||
GeoRestrictedError,
|
||||
HEADRequest,
|
||||
ISO3166Utils,
|
||||
@ -148,7 +150,7 @@
|
||||
write_json_file,
|
||||
write_string,
|
||||
)
|
||||
from .version import RELEASE_GIT_HEAD, VARIANT, __version__
|
||||
from .version import CHANNEL, RELEASE_GIT_HEAD, VARIANT, __version__
|
||||
|
||||
if compat_os_name == 'nt':
|
||||
import ctypes
|
||||
@ -298,8 +300,6 @@ class YoutubeDL:
|
||||
Videos already present in the file are not downloaded again.
|
||||
break_on_existing: Stop the download process after attempting to download a
|
||||
file that is in the archive.
|
||||
break_on_reject: Stop the download process when encountering a video that
|
||||
has been filtered out.
|
||||
break_per_url: Whether break_on_reject and break_on_existing
|
||||
should act on each input URL as opposed to for the entire queue
|
||||
cookiefile: File name or text stream from where cookies should be read and dumped to
|
||||
@ -316,6 +316,7 @@ class YoutubeDL:
|
||||
If not provided and the key is encrypted, yt-dlp will ask interactively
|
||||
prefer_insecure: Use HTTP instead of HTTPS to retrieve information.
|
||||
(Only supported by some extractors)
|
||||
enable_file_urls: Enable file:// URLs. This is disabled by default for security reasons.
|
||||
http_headers: A dictionary of custom headers to be used for all requests
|
||||
proxy: URL of the proxy server to use
|
||||
geo_verification_proxy: URL of the proxy to use for IP address verification
|
||||
@ -411,6 +412,8 @@ class YoutubeDL:
|
||||
- If it returns None, the video is downloaded.
|
||||
- If it returns utils.NO_DEFAULT, the user is interactively
|
||||
asked whether to download the video.
|
||||
- Raise utils.DownloadCancelled(msg) to abort remaining
|
||||
downloads when a video is rejected.
|
||||
match_filter_func in utils.py is one example for this.
|
||||
no_color: Do not emit color codes in output.
|
||||
geo_bypass: Bypass geographic restriction via faking X-Forwarded-For
|
||||
@ -480,6 +483,9 @@ class YoutubeDL:
|
||||
|
||||
The following options are deprecated and may be removed in the future:
|
||||
|
||||
break_on_reject: Stop the download process when encountering a video that
|
||||
has been filtered out.
|
||||
- `raise DownloadCancelled(msg)` in match_filter instead
|
||||
force_generic_extractor: Force downloader to use the generic extractor
|
||||
- Use allowed_extractors = ['generic', 'default']
|
||||
playliststart: - Use playlist_items
|
||||
@ -547,11 +553,11 @@ class YoutubeDL:
|
||||
_format_fields = {
|
||||
# NB: Keep in sync with the docstring of extractor/common.py
|
||||
'url', 'manifest_url', 'manifest_stream_number', 'ext', 'format', 'format_id', 'format_note',
|
||||
'width', 'height', 'resolution', 'dynamic_range', 'tbr', 'abr', 'acodec', 'asr', 'audio_channels',
|
||||
'vbr', 'fps', 'vcodec', 'container', 'filesize', 'filesize_approx',
|
||||
'width', 'height', 'aspect_ratio', 'resolution', 'dynamic_range', 'tbr', 'abr', 'acodec', 'asr', 'audio_channels',
|
||||
'vbr', 'fps', 'vcodec', 'container', 'filesize', 'filesize_approx', 'rows', 'columns',
|
||||
'player_url', 'protocol', 'fragment_base_url', 'fragments', 'is_from_start',
|
||||
'preference', 'language', 'language_preference', 'quality', 'source_preference',
|
||||
'http_headers', 'stretched_ratio', 'no_resume', 'has_drm', 'downloader_options',
|
||||
'http_headers', 'stretched_ratio', 'no_resume', 'has_drm', 'extra_param_to_segment_url', 'hls_aes', 'downloader_options',
|
||||
'page_url', 'app', 'play_path', 'tc_url', 'flash_version', 'rtmp_live', 'rtmp_conn', 'rtmp_protocol', 'rtmp_real_time'
|
||||
}
|
||||
_format_selection_exts = {
|
||||
@ -583,7 +589,6 @@ def __init__(self, params=None, auto_init=True):
|
||||
self._playlist_urls = set()
|
||||
self.cache = Cache(self)
|
||||
|
||||
windows_enable_vt_mode()
|
||||
stdout = sys.stderr if self.params.get('logtostderr') else sys.stdout
|
||||
self._out_files = Namespace(
|
||||
out=stdout,
|
||||
@ -592,6 +597,12 @@ def __init__(self, params=None, auto_init=True):
|
||||
console=None if compat_os_name == 'nt' else next(
|
||||
filter(supports_terminal_sequences, (sys.stderr, sys.stdout)), None)
|
||||
)
|
||||
|
||||
try:
|
||||
windows_enable_vt_mode()
|
||||
except Exception as e:
|
||||
self.write_debug(f'Failed to enable VT mode: {e}')
|
||||
|
||||
self._allow_colors = Namespace(**{
|
||||
type_: not self.params.get('no_color') and supports_terminal_sequences(stream)
|
||||
for type_, stream in self._out_files.items_ if type_ != 'console'
|
||||
@ -606,7 +617,7 @@ def __init__(self, params=None, auto_init=True):
|
||||
'\n You will no longer receive updates on this version')
|
||||
if current_version < MIN_SUPPORTED:
|
||||
msg = 'Python version %d.%d is no longer supported'
|
||||
self.deprecation_warning(
|
||||
self.deprecated_feature(
|
||||
f'{msg}! Please update to Python %d.%d or above' % (*current_version, *MIN_RECOMMENDED))
|
||||
|
||||
if self.params.get('allow_unplayable_formats'):
|
||||
@ -616,46 +627,6 @@ def __init__(self, params=None, auto_init=True):
|
||||
' If you experience any issues while using this option, '
|
||||
f'{self._format_err("DO NOT", self.Styles.ERROR)} open a bug report')
|
||||
|
||||
def check_deprecated(param, option, suggestion):
|
||||
if self.params.get(param) is not None:
|
||||
self.report_warning(f'{option} is deprecated. Use {suggestion} instead')
|
||||
return True
|
||||
return False
|
||||
|
||||
if check_deprecated('cn_verification_proxy', '--cn-verification-proxy', '--geo-verification-proxy'):
|
||||
if self.params.get('geo_verification_proxy') is None:
|
||||
self.params['geo_verification_proxy'] = self.params['cn_verification_proxy']
|
||||
|
||||
check_deprecated('autonumber', '--auto-number', '-o "%(autonumber)s-%(title)s.%(ext)s"')
|
||||
check_deprecated('usetitle', '--title', '-o "%(title)s-%(id)s.%(ext)s"')
|
||||
check_deprecated('useid', '--id', '-o "%(id)s.%(ext)s"')
|
||||
|
||||
for msg in self.params.get('_warnings', []):
|
||||
self.report_warning(msg)
|
||||
for msg in self.params.get('_deprecation_warnings', []):
|
||||
self.deprecated_feature(msg)
|
||||
|
||||
self.params['compat_opts'] = set(self.params.get('compat_opts', ()))
|
||||
if 'list-formats' in self.params['compat_opts']:
|
||||
self.params['listformats_table'] = False
|
||||
|
||||
if 'overwrites' not in self.params and self.params.get('nooverwrites') is not None:
|
||||
# nooverwrites was unnecessarily changed to overwrites
|
||||
# in 0c3d0f51778b153f65c21906031c2e091fcfb641
|
||||
# This ensures compatibility with both keys
|
||||
self.params['overwrites'] = not self.params['nooverwrites']
|
||||
elif self.params.get('overwrites') is None:
|
||||
self.params.pop('overwrites', None)
|
||||
else:
|
||||
self.params['nooverwrites'] = not self.params['overwrites']
|
||||
|
||||
self.params.setdefault('forceprint', {})
|
||||
self.params.setdefault('print_to_file', {})
|
||||
|
||||
# Compatibility with older syntax
|
||||
if not isinstance(params['forceprint'], dict):
|
||||
self.params['forceprint'] = {'video': params['forceprint']}
|
||||
|
||||
if self.params.get('bidi_workaround', False):
|
||||
try:
|
||||
import pty
|
||||
@ -676,9 +647,57 @@ def check_deprecated(param, option, suggestion):
|
||||
else:
|
||||
raise
|
||||
|
||||
self.params['compat_opts'] = set(self.params.get('compat_opts', ()))
|
||||
if auto_init and auto_init != 'no_verbose_header':
|
||||
self.print_debug_header()
|
||||
|
||||
def check_deprecated(param, option, suggestion):
|
||||
if self.params.get(param) is not None:
|
||||
self.report_warning(f'{option} is deprecated. Use {suggestion} instead')
|
||||
return True
|
||||
return False
|
||||
|
||||
if check_deprecated('cn_verification_proxy', '--cn-verification-proxy', '--geo-verification-proxy'):
|
||||
if self.params.get('geo_verification_proxy') is None:
|
||||
self.params['geo_verification_proxy'] = self.params['cn_verification_proxy']
|
||||
|
||||
check_deprecated('autonumber', '--auto-number', '-o "%(autonumber)s-%(title)s.%(ext)s"')
|
||||
check_deprecated('usetitle', '--title', '-o "%(title)s-%(id)s.%(ext)s"')
|
||||
check_deprecated('useid', '--id', '-o "%(id)s.%(ext)s"')
|
||||
|
||||
for msg in self.params.get('_warnings', []):
|
||||
self.report_warning(msg)
|
||||
for msg in self.params.get('_deprecation_warnings', []):
|
||||
self.deprecated_feature(msg)
|
||||
|
||||
if 'list-formats' in self.params['compat_opts']:
|
||||
self.params['listformats_table'] = False
|
||||
|
||||
if 'overwrites' not in self.params and self.params.get('nooverwrites') is not None:
|
||||
# nooverwrites was unnecessarily changed to overwrites
|
||||
# in 0c3d0f51778b153f65c21906031c2e091fcfb641
|
||||
# This ensures compatibility with both keys
|
||||
self.params['overwrites'] = not self.params['nooverwrites']
|
||||
elif self.params.get('overwrites') is None:
|
||||
self.params.pop('overwrites', None)
|
||||
else:
|
||||
self.params['nooverwrites'] = not self.params['overwrites']
|
||||
|
||||
if self.params.get('simulate') is None and any((
|
||||
self.params.get('list_thumbnails'),
|
||||
self.params.get('listformats'),
|
||||
self.params.get('listsubtitles'),
|
||||
)):
|
||||
self.params['simulate'] = 'list_only'
|
||||
|
||||
self.params.setdefault('forceprint', {})
|
||||
self.params.setdefault('print_to_file', {})
|
||||
|
||||
# Compatibility with older syntax
|
||||
if not isinstance(params['forceprint'], dict):
|
||||
self.params['forceprint'] = {'video': params['forceprint']}
|
||||
|
||||
if auto_init:
|
||||
if auto_init != 'no_verbose_header':
|
||||
self.print_debug_header()
|
||||
self.add_default_info_extractors()
|
||||
|
||||
if (sys.platform != 'win32'
|
||||
@ -846,7 +865,7 @@ def to_stdout(self, message, skip_eol=False, quiet=None):
|
||||
'Use "YoutubeDL.to_screen" instead')
|
||||
self._write_string(f'{self._bidi_workaround(message)}\n', self._out_files.out)
|
||||
|
||||
def to_screen(self, message, skip_eol=False, quiet=None):
|
||||
def to_screen(self, message, skip_eol=False, quiet=None, only_once=False):
|
||||
"""Print message to screen if not in quiet mode"""
|
||||
if self.params.get('logger'):
|
||||
self.params['logger'].debug(message)
|
||||
@ -855,7 +874,7 @@ def to_screen(self, message, skip_eol=False, quiet=None):
|
||||
return
|
||||
self._write_string(
|
||||
'%s%s' % (self._bidi_workaround(message), ('' if skip_eol else '\n')),
|
||||
self._out_files.screen)
|
||||
self._out_files.screen, only_once=only_once)
|
||||
|
||||
def to_stderr(self, message, only_once=False):
|
||||
"""Print message to stderr"""
|
||||
@ -1059,7 +1078,7 @@ def _outtmpl_expandpath(outtmpl):
|
||||
# correspondingly that is not what we want since we need to keep
|
||||
# '%%' intact for template dict substitution step. Working around
|
||||
# with boundary-alike separator hack.
|
||||
sep = ''.join([random.choice(ascii_letters) for _ in range(32)])
|
||||
sep = ''.join(random.choices(ascii_letters, k=32))
|
||||
outtmpl = outtmpl.replace('%%', f'%{sep}%').replace('$$', f'${sep}$')
|
||||
|
||||
# outtmpl should be expand_path'ed before template dict substitution
|
||||
@ -1249,7 +1268,7 @@ def create_key(outer_mobj):
|
||||
elif fmt[-1] == 'j': # json
|
||||
value, fmt = json.dumps(
|
||||
value, default=_dumpjson_default,
|
||||
indent=4 if '#' in flags else None, ensure_ascii=False), str_fmt
|
||||
indent=4 if '#' in flags else None, ensure_ascii='+' not in flags), str_fmt
|
||||
elif fmt[-1] == 'h': # html
|
||||
value, fmt = escapeHTML(str(value)), str_fmt
|
||||
elif fmt[-1] == 'q': # quoted
|
||||
@ -1349,11 +1368,19 @@ def prepare_filename(self, info_dict, dir_type='', *, outtmpl=None, warn=False):
|
||||
return self.get_output_path(dir_type, filename)
|
||||
|
||||
def _match_entry(self, info_dict, incomplete=False, silent=False):
|
||||
""" Returns None if the file should be downloaded """
|
||||
"""Returns None if the file should be downloaded"""
|
||||
_type = info_dict.get('_type', 'video')
|
||||
assert incomplete or _type == 'video', 'Only video result can be considered complete'
|
||||
|
||||
video_title = info_dict.get('title', info_dict.get('id', 'entry'))
|
||||
|
||||
def check_filter():
|
||||
if _type in ('playlist', 'multi_video'):
|
||||
return
|
||||
elif _type in ('url', 'url_transparent') and not try_call(
|
||||
lambda: self.get_info_extractor(info_dict['ie_key']).is_single_video(info_dict['url'])):
|
||||
return
|
||||
|
||||
if 'title' in info_dict:
|
||||
# This can happen when we're just evaluating the playlist
|
||||
title = info_dict['title']
|
||||
@ -1365,6 +1392,7 @@ def check_filter():
|
||||
if rejecttitle:
|
||||
if re.search(rejecttitle, title, re.IGNORECASE):
|
||||
return '"' + title + '" title matched reject pattern "' + rejecttitle + '"'
|
||||
|
||||
date = info_dict.get('upload_date')
|
||||
if date is not None:
|
||||
dateRange = self.params.get('daterange', DateRange())
|
||||
@ -1382,31 +1410,44 @@ def check_filter():
|
||||
return 'Skipping "%s" because it is age restricted' % video_title
|
||||
|
||||
match_filter = self.params.get('match_filter')
|
||||
if match_filter is not None:
|
||||
if match_filter is None:
|
||||
return None
|
||||
|
||||
cancelled = None
|
||||
try:
|
||||
try:
|
||||
ret = match_filter(info_dict, incomplete=incomplete)
|
||||
except TypeError:
|
||||
# For backward compatibility
|
||||
ret = None if incomplete else match_filter(info_dict)
|
||||
if ret is NO_DEFAULT:
|
||||
while True:
|
||||
filename = self._format_screen(self.prepare_filename(info_dict), self.Styles.FILENAME)
|
||||
reply = input(self._format_screen(
|
||||
f'Download "{filename}"? (Y/n): ', self.Styles.EMPHASIS)).lower().strip()
|
||||
if reply in {'y', ''}:
|
||||
return None
|
||||
elif reply == 'n':
|
||||
return f'Skipping {video_title}'
|
||||
elif ret is not None:
|
||||
return ret
|
||||
return None
|
||||
except DownloadCancelled as err:
|
||||
if err.msg is not NO_DEFAULT:
|
||||
raise
|
||||
ret, cancelled = err.msg, err
|
||||
|
||||
if ret is NO_DEFAULT:
|
||||
while True:
|
||||
filename = self._format_screen(self.prepare_filename(info_dict), self.Styles.FILENAME)
|
||||
reply = input(self._format_screen(
|
||||
f'Download "{filename}"? (Y/n): ', self.Styles.EMPHASIS)).lower().strip()
|
||||
if reply in {'y', ''}:
|
||||
return None
|
||||
elif reply == 'n':
|
||||
if cancelled:
|
||||
raise type(cancelled)(f'Skipping {video_title}')
|
||||
return f'Skipping {video_title}'
|
||||
return ret
|
||||
|
||||
if self.in_download_archive(info_dict):
|
||||
reason = '%s has already been recorded in the archive' % video_title
|
||||
break_opt, break_err = 'break_on_existing', ExistingVideoReached
|
||||
else:
|
||||
reason = check_filter()
|
||||
break_opt, break_err = 'break_on_reject', RejectedVideoReached
|
||||
try:
|
||||
reason = check_filter()
|
||||
except DownloadCancelled as e:
|
||||
reason, break_opt, break_err = e.msg, 'match_filter', type(e)
|
||||
else:
|
||||
break_opt, break_err = 'break_on_reject', RejectedVideoReached
|
||||
if reason is not None:
|
||||
if not silent:
|
||||
self.to_screen('[download] ' + reason)
|
||||
@ -1608,8 +1649,8 @@ def process_ie_result(self, ie_result, download=True, extra_info=None):
|
||||
if result_type in ('url', 'url_transparent'):
|
||||
ie_result['url'] = sanitize_url(
|
||||
ie_result['url'], scheme='http' if self.params.get('prefer_insecure') else 'https')
|
||||
if ie_result.get('original_url'):
|
||||
extra_info.setdefault('original_url', ie_result['original_url'])
|
||||
if ie_result.get('original_url') and not extra_info.get('original_url'):
|
||||
extra_info = {'original_url': ie_result['original_url'], **extra_info}
|
||||
|
||||
extract_flat = self.params.get('extract_flat', False)
|
||||
if ((extract_flat == 'in_playlist' and 'playlist' in extra_info)
|
||||
@ -1621,6 +1662,7 @@ def process_ie_result(self, ie_result, download=True, extra_info=None):
|
||||
self.add_default_extra_info(info_copy, ie, ie_result['url'])
|
||||
self.add_extra_info(info_copy, extra_info)
|
||||
info_copy, _ = self.pre_process(info_copy)
|
||||
self._fill_common_fields(info_copy, False)
|
||||
self.__forced_printings(info_copy, self.prepare_filename(info_copy), incomplete=True)
|
||||
self._raise_pending_errors(info_copy)
|
||||
if self.params.get('force_write_download_archive', False):
|
||||
@ -1687,8 +1729,8 @@ def process_ie_result(self, ie_result, download=True, extra_info=None):
|
||||
elif result_type in ('playlist', 'multi_video'):
|
||||
# Protect from infinite recursion due to recursively nested playlists
|
||||
# (see https://github.com/ytdl-org/youtube-dl/issues/27833)
|
||||
webpage_url = ie_result['webpage_url']
|
||||
if webpage_url in self._playlist_urls:
|
||||
webpage_url = ie_result.get('webpage_url') # Playlists maynot have webpage_url
|
||||
if webpage_url and webpage_url in self._playlist_urls:
|
||||
self.to_screen(
|
||||
'[download] Skipping already downloaded playlist: %s'
|
||||
% ie_result.get('title') or ie_result.get('id'))
|
||||
@ -1742,14 +1784,17 @@ def _playlist_infodict(ie_result, strict=False, **kwargs):
|
||||
}
|
||||
if strict:
|
||||
return info
|
||||
if ie_result.get('webpage_url'):
|
||||
info.update({
|
||||
'webpage_url': ie_result['webpage_url'],
|
||||
'webpage_url_basename': url_basename(ie_result['webpage_url']),
|
||||
'webpage_url_domain': get_domain(ie_result['webpage_url']),
|
||||
})
|
||||
return {
|
||||
**info,
|
||||
'playlist_index': 0,
|
||||
'__last_playlist_index': max(ie_result['requested_entries'] or (0, 0)),
|
||||
'__last_playlist_index': max(ie_result.get('requested_entries') or (0, 0)),
|
||||
'extractor': ie_result['extractor'],
|
||||
'webpage_url': ie_result['webpage_url'],
|
||||
'webpage_url_basename': url_basename(ie_result['webpage_url']),
|
||||
'webpage_url_domain': get_domain(ie_result['webpage_url']),
|
||||
'extractor_key': ie_result['extractor_key'],
|
||||
}
|
||||
|
||||
@ -1804,7 +1849,7 @@ def __process_playlist(self, ie_result, download):
|
||||
elif self.params.get('playlistrandom'):
|
||||
random.shuffle(entries)
|
||||
|
||||
self.to_screen(f'[{ie_result["extractor"]}] Playlist {title}: Downloading {n_entries} videos'
|
||||
self.to_screen(f'[{ie_result["extractor"]}] Playlist {title}: Downloading {n_entries} items'
|
||||
f'{format_field(ie_result, "playlist_count", " of %s")}')
|
||||
|
||||
keep_resolved_entries = self.params.get('extract_flat') != 'discard'
|
||||
@ -1837,14 +1882,13 @@ def __process_playlist(self, ie_result, download):
|
||||
resolved_entries[i] = (playlist_index, NO_DEFAULT)
|
||||
continue
|
||||
|
||||
self.to_screen('[download] Downloading video %s of %s' % (
|
||||
self.to_screen('[download] Downloading item %s of %s' % (
|
||||
self._format_screen(i + 1, self.Styles.ID), self._format_screen(n_entries, self.Styles.EMPHASIS)))
|
||||
|
||||
extra.update({
|
||||
entry_result = self.__process_iterable_entry(entry, download, collections.ChainMap({
|
||||
'playlist_index': playlist_index,
|
||||
'playlist_autonumber': i + 1,
|
||||
})
|
||||
entry_result = self.__process_iterable_entry(entry, download, extra)
|
||||
}, extra))
|
||||
if not entry_result:
|
||||
failures += 1
|
||||
if failures >= max_failures:
|
||||
@ -1855,8 +1899,11 @@ def __process_playlist(self, ie_result, download):
|
||||
resolved_entries[i] = (playlist_index, entry_result)
|
||||
|
||||
# Update with processed data
|
||||
ie_result['requested_entries'] = [i for i, e in resolved_entries if e is not NO_DEFAULT]
|
||||
ie_result['entries'] = [e for _, e in resolved_entries if e is not NO_DEFAULT]
|
||||
ie_result['requested_entries'] = [i for i, e in resolved_entries if e is not NO_DEFAULT]
|
||||
if ie_result['requested_entries'] == try_call(lambda: list(range(1, ie_result['playlist_count'] + 1))):
|
||||
# Do not set for full playlist
|
||||
ie_result.pop('requested_entries')
|
||||
|
||||
# Write the updated info to json
|
||||
if _infojson_written is True and self._write_info_json(
|
||||
@ -2162,6 +2209,7 @@ def _merge(formats_pair):
|
||||
'vcodec': the_only_video.get('vcodec'),
|
||||
'vbr': the_only_video.get('vbr'),
|
||||
'stretched_ratio': the_only_video.get('stretched_ratio'),
|
||||
'aspect_ratio': the_only_video.get('aspect_ratio'),
|
||||
})
|
||||
|
||||
if the_only_audio:
|
||||
@ -2376,15 +2424,10 @@ def check_thumbnails(thumbnails):
|
||||
else:
|
||||
info_dict['thumbnails'] = thumbnails
|
||||
|
||||
def _fill_common_fields(self, info_dict, is_video=True):
|
||||
def _fill_common_fields(self, info_dict, final=True):
|
||||
# TODO: move sanitization here
|
||||
if is_video:
|
||||
# playlists are allowed to lack "title"
|
||||
title = info_dict.get('title', NO_DEFAULT)
|
||||
if title is NO_DEFAULT:
|
||||
raise ExtractorError('Missing "title" field in extractor result',
|
||||
video_id=info_dict['id'], ie=info_dict['extractor'])
|
||||
info_dict['fulltitle'] = title
|
||||
if final:
|
||||
title = info_dict['fulltitle'] = info_dict.get('title')
|
||||
if not title:
|
||||
if title == '':
|
||||
self.write_debug('Extractor gave empty title. Creating a generic title')
|
||||
@ -2423,11 +2466,13 @@ def _fill_common_fields(self, info_dict, is_video=True):
|
||||
for key in live_keys:
|
||||
if info_dict.get(key) is None:
|
||||
info_dict[key] = (live_status == key)
|
||||
if live_status == 'post_live':
|
||||
info_dict['was_live'] = True
|
||||
|
||||
# Auto generate title fields corresponding to the *_number fields when missing
|
||||
# in order to always have clean titles. This is very common for TV series.
|
||||
for field in ('chapter', 'season', 'episode'):
|
||||
if info_dict.get('%s_number' % field) is not None and not info_dict.get(field):
|
||||
if final and info_dict.get('%s_number' % field) is not None and not info_dict.get(field):
|
||||
info_dict[field] = '%s %d' % (field.capitalize(), info_dict['%s_number' % field])
|
||||
|
||||
def _raise_pending_errors(self, info):
|
||||
@ -2435,6 +2480,11 @@ def _raise_pending_errors(self, info):
|
||||
if err:
|
||||
self.report_error(err, tb=False)
|
||||
|
||||
def sort_formats(self, info_dict):
|
||||
formats = self._get_formats(info_dict)
|
||||
formats.sort(key=FormatSorter(
|
||||
self, info_dict.get('_format_sort_fields') or []).calculate_preference)
|
||||
|
||||
def process_video_result(self, info_dict, download=True):
|
||||
assert info_dict.get('_type', 'video') == 'video'
|
||||
self._num_videos += 1
|
||||
@ -2520,11 +2570,12 @@ def sanitize_numeric_fields(info):
|
||||
info_dict['requested_subtitles'] = self.process_subtitles(
|
||||
info_dict['id'], subtitles, automatic_captions)
|
||||
|
||||
if info_dict.get('formats') is None:
|
||||
# There's only one format available
|
||||
formats = [info_dict]
|
||||
else:
|
||||
formats = info_dict['formats']
|
||||
formats = self._get_formats(info_dict)
|
||||
|
||||
# Backward compatibility with InfoExtractor._sort_formats
|
||||
field_preference = (formats or [{}])[0].pop('__sort_fields', None)
|
||||
if field_preference:
|
||||
info_dict['_format_sort_fields'] = field_preference
|
||||
|
||||
# or None ensures --clean-infojson removes it
|
||||
info_dict['_has_drm'] = any(f.get('has_drm') for f in formats) or None
|
||||
@ -2563,22 +2614,43 @@ def is_wellformed(f):
|
||||
if not formats:
|
||||
self.raise_no_formats(info_dict)
|
||||
|
||||
formats_dict = {}
|
||||
|
||||
# We check that all the formats have the format and format_id fields
|
||||
for i, format in enumerate(formats):
|
||||
for format in formats:
|
||||
sanitize_string_field(format, 'format_id')
|
||||
sanitize_numeric_fields(format)
|
||||
format['url'] = sanitize_url(format['url'])
|
||||
if format.get('ext') is None:
|
||||
format['ext'] = determine_ext(format['url']).lower()
|
||||
if format.get('protocol') is None:
|
||||
format['protocol'] = determine_protocol(format)
|
||||
if format.get('resolution') is None:
|
||||
format['resolution'] = self.format_resolution(format, default=None)
|
||||
if format.get('dynamic_range') is None and format.get('vcodec') != 'none':
|
||||
format['dynamic_range'] = 'SDR'
|
||||
if format.get('aspect_ratio') is None:
|
||||
format['aspect_ratio'] = try_call(lambda: round(format['width'] / format['height'], 2))
|
||||
if (info_dict.get('duration') and format.get('tbr')
|
||||
and not format.get('filesize') and not format.get('filesize_approx')):
|
||||
format['filesize_approx'] = int(info_dict['duration'] * format['tbr'] * (1024 / 8))
|
||||
format['http_headers'] = self._calc_headers(collections.ChainMap(format, info_dict))
|
||||
|
||||
# This is copied to http_headers by the above _calc_headers and can now be removed
|
||||
if '__x_forwarded_for_ip' in info_dict:
|
||||
del info_dict['__x_forwarded_for_ip']
|
||||
|
||||
self.sort_formats({
|
||||
'formats': formats,
|
||||
'_format_sort_fields': info_dict.get('_format_sort_fields')
|
||||
})
|
||||
|
||||
# Sanitize and group by format_id
|
||||
formats_dict = {}
|
||||
for i, format in enumerate(formats):
|
||||
if not format.get('format_id'):
|
||||
format['format_id'] = str(i)
|
||||
else:
|
||||
# Sanitize format_id from characters used in format selector expression
|
||||
format['format_id'] = re.sub(r'[\s,/+\[\]()]', '_', format['format_id'])
|
||||
format_id = format['format_id']
|
||||
if format_id not in formats_dict:
|
||||
formats_dict[format_id] = []
|
||||
formats_dict[format_id].append(format)
|
||||
formats_dict.setdefault(format['format_id'], []).append(format)
|
||||
|
||||
# Make sure all formats have unique format_id
|
||||
common_exts = set(itertools.chain(*self._format_selection_exts.values()))
|
||||
@ -2587,38 +2659,17 @@ def is_wellformed(f):
|
||||
for i, format in enumerate(ambiguous_formats):
|
||||
if ambigious_id:
|
||||
format['format_id'] = '%s-%d' % (format_id, i)
|
||||
if format.get('ext') is None:
|
||||
format['ext'] = determine_ext(format['url']).lower()
|
||||
# Ensure there is no conflict between id and ext in format selection
|
||||
# See https://github.com/yt-dlp/yt-dlp/issues/1282
|
||||
if format['format_id'] != format['ext'] and format['format_id'] in common_exts:
|
||||
format['format_id'] = 'f%s' % format['format_id']
|
||||
|
||||
for i, format in enumerate(formats):
|
||||
if format.get('format') is None:
|
||||
format['format'] = '{id} - {res}{note}'.format(
|
||||
id=format['format_id'],
|
||||
res=self.format_resolution(format),
|
||||
note=format_field(format, 'format_note', ' (%s)'),
|
||||
)
|
||||
if format.get('protocol') is None:
|
||||
format['protocol'] = determine_protocol(format)
|
||||
if format.get('resolution') is None:
|
||||
format['resolution'] = self.format_resolution(format, default=None)
|
||||
if format.get('dynamic_range') is None and format.get('vcodec') != 'none':
|
||||
format['dynamic_range'] = 'SDR'
|
||||
if (info_dict.get('duration') and format.get('tbr')
|
||||
and not format.get('filesize') and not format.get('filesize_approx')):
|
||||
format['filesize_approx'] = int(info_dict['duration'] * format['tbr'] * (1024 / 8))
|
||||
|
||||
# Add HTTP headers, so that external programs can use them from the
|
||||
# json output
|
||||
full_format_info = info_dict.copy()
|
||||
full_format_info.update(format)
|
||||
format['http_headers'] = self._calc_headers(full_format_info)
|
||||
# Remove private housekeeping stuff
|
||||
if '__x_forwarded_for_ip' in info_dict:
|
||||
del info_dict['__x_forwarded_for_ip']
|
||||
if format.get('format') is None:
|
||||
format['format'] = '{id} - {res}{note}'.format(
|
||||
id=format['format_id'],
|
||||
res=self.format_resolution(format),
|
||||
note=format_field(format, 'format_note', ' (%s)'),
|
||||
)
|
||||
|
||||
if self.params.get('check_formats') is True:
|
||||
formats = LazyList(self._check_formats(formats[::-1]), reverse=True)
|
||||
@ -2639,10 +2690,9 @@ def is_wellformed(f):
|
||||
info_dict, _ = self.pre_process(info_dict, 'after_filter')
|
||||
|
||||
# The pre-processors may have modified the formats
|
||||
formats = info_dict.get('formats', [info_dict])
|
||||
formats = self._get_formats(info_dict)
|
||||
|
||||
list_only = self.params.get('simulate') is None and (
|
||||
self.params.get('list_thumbnails') or self.params.get('listformats') or self.params.get('listsubtitles'))
|
||||
list_only = self.params.get('simulate') == 'list_only'
|
||||
interactive_format_selection = not list_only and self.format_selector == '-'
|
||||
if self.params.get('list_thumbnails'):
|
||||
self.list_thumbnails(info_dict)
|
||||
@ -2697,31 +2747,30 @@ def is_wellformed(f):
|
||||
# Process what we can, even without any available formats.
|
||||
formats_to_download = [{}]
|
||||
|
||||
requested_ranges = self.params.get('download_ranges')
|
||||
if requested_ranges:
|
||||
requested_ranges = tuple(requested_ranges(info_dict, self))
|
||||
|
||||
requested_ranges = tuple(self.params.get('download_ranges', lambda *_: [{}])(info_dict, self))
|
||||
best_format, downloaded_formats = formats_to_download[-1], []
|
||||
if download:
|
||||
if best_format:
|
||||
if best_format and requested_ranges:
|
||||
def to_screen(*msg):
|
||||
self.to_screen(f'[info] {info_dict["id"]}: {" ".join(", ".join(variadic(m)) for m in msg)}')
|
||||
|
||||
to_screen(f'Downloading {len(formats_to_download)} format(s):',
|
||||
(f['format_id'] for f in formats_to_download))
|
||||
if requested_ranges:
|
||||
if requested_ranges != ({}, ):
|
||||
to_screen(f'Downloading {len(requested_ranges)} time ranges:',
|
||||
(f'{int(c["start_time"])}-{int(c["end_time"])}' for c in requested_ranges))
|
||||
(f'{c["start_time"]:.1f}-{c["end_time"]:.1f}' for c in requested_ranges))
|
||||
max_downloads_reached = False
|
||||
|
||||
for fmt, chapter in itertools.product(formats_to_download, requested_ranges or [{}]):
|
||||
for fmt, chapter in itertools.product(formats_to_download, requested_ranges):
|
||||
new_info = self._copy_infodict(info_dict)
|
||||
new_info.update(fmt)
|
||||
offset, duration = info_dict.get('section_start') or 0, info_dict.get('duration') or float('inf')
|
||||
end_time = offset + min(chapter.get('end_time', duration), duration)
|
||||
if chapter or offset:
|
||||
new_info.update({
|
||||
'section_start': offset + chapter.get('start_time', 0),
|
||||
'section_end': offset + min(chapter.get('end_time', duration), duration),
|
||||
# duration may not be accurate. So allow deviations <1sec
|
||||
'section_end': end_time if end_time <= offset + duration + 1 else None,
|
||||
'section_title': chapter.get('title'),
|
||||
'section_number': chapter.get('index'),
|
||||
})
|
||||
@ -2777,10 +2826,14 @@ def process_subtitles(self, video_id, normal_subtitles, automatic_captions):
|
||||
self.params.get('subtitleslangs'), {'all': all_sub_langs}, use_regex=True)
|
||||
except re.error as e:
|
||||
raise ValueError(f'Wrong regex for subtitlelangs: {e.pattern}')
|
||||
elif normal_sub_langs:
|
||||
requested_langs = ['en'] if 'en' in normal_sub_langs else normal_sub_langs[:1]
|
||||
else:
|
||||
requested_langs = ['en'] if 'en' in all_sub_langs else all_sub_langs[:1]
|
||||
requested_langs = LazyList(itertools.chain(
|
||||
['en'] if 'en' in normal_sub_langs else [],
|
||||
filter(lambda f: f.startswith('en'), normal_sub_langs),
|
||||
['en'] if 'en' in all_sub_langs else [],
|
||||
filter(lambda f: f.startswith('en'), all_sub_langs),
|
||||
normal_sub_langs, all_sub_langs,
|
||||
))[:1]
|
||||
if requested_langs:
|
||||
self.to_screen(f'[info] {video_id}: Downloading subtitles: {", ".join(requested_langs)}')
|
||||
|
||||
@ -2935,14 +2988,22 @@ def process_info(self, info_dict):
|
||||
if 'format' not in info_dict and 'ext' in info_dict:
|
||||
info_dict['format'] = info_dict['ext']
|
||||
|
||||
# This is mostly just for backward compatibility of process_info
|
||||
# As a side-effect, this allows for format-specific filters
|
||||
if self._match_entry(info_dict) is not None:
|
||||
info_dict['__write_download_archive'] = 'ignore'
|
||||
return
|
||||
|
||||
# Does nothing under normal operation - for backward compatibility of process_info
|
||||
self.post_extract(info_dict)
|
||||
|
||||
def replace_info_dict(new_info):
|
||||
nonlocal info_dict
|
||||
if new_info == info_dict:
|
||||
return
|
||||
info_dict.clear()
|
||||
info_dict.update(new_info)
|
||||
|
||||
new_info, _ = self.pre_process(info_dict, 'video')
|
||||
replace_info_dict(new_info)
|
||||
self._num_downloads += 1
|
||||
|
||||
# info_dict['_filename'] needs to be set for backward compatibility
|
||||
@ -3056,13 +3117,6 @@ def _write_link_file(link_type):
|
||||
for link_type, should_write in write_links.items()):
|
||||
return
|
||||
|
||||
def replace_info_dict(new_info):
|
||||
nonlocal info_dict
|
||||
if new_info == info_dict:
|
||||
return
|
||||
info_dict.clear()
|
||||
info_dict.update(new_info)
|
||||
|
||||
new_info, files_to_move = self.pre_process(info_dict, 'before_dl', files_to_move)
|
||||
replace_info_dict(new_info)
|
||||
|
||||
@ -3089,7 +3143,7 @@ def existing_video_file(*filepaths):
|
||||
fd, success = None, True
|
||||
if info_dict.get('protocol') or info_dict.get('url'):
|
||||
fd = get_suitable_downloader(info_dict, self.params, to_stdout=temp_filename == '-')
|
||||
if fd is not FFmpegFD and (
|
||||
if fd is not FFmpegFD and 'no-direct-merge' not in self.params['compat_opts'] and (
|
||||
info_dict.get('section_start') or info_dict.get('section_end')):
|
||||
msg = ('This format cannot be partially downloaded' if FFmpegFD.available()
|
||||
else 'You have requested downloading the video partially, but ffmpeg is not installed')
|
||||
@ -3354,6 +3408,7 @@ def sanitize_info(info_dict, remove_private_keys=False):
|
||||
reject = lambda k, v: v is None or k.startswith('__') or k in {
|
||||
'requested_downloads', 'requested_formats', 'requested_subtitles', 'requested_entries',
|
||||
'entries', 'filepath', '_filename', 'infojson_filename', 'original_url', 'playlist_autonumber',
|
||||
'_format_sort_fields',
|
||||
}
|
||||
else:
|
||||
reject = lambda k, v: False
|
||||
@ -3423,7 +3478,8 @@ def run_pp(self, pp, infodict):
|
||||
return infodict
|
||||
|
||||
def run_all_pps(self, key, info, *, additional_pps=None):
|
||||
self._forceprint(key, info)
|
||||
if key != 'video':
|
||||
self._forceprint(key, info)
|
||||
for pp in (additional_pps or []) + self._pps[key]:
|
||||
info = self.run_pp(pp, info)
|
||||
return info
|
||||
@ -3568,11 +3624,17 @@ def _format_note(self, fdict):
|
||||
res += '~' + format_bytes(fdict['filesize_approx'])
|
||||
return res
|
||||
|
||||
def render_formats_table(self, info_dict):
|
||||
if not info_dict.get('formats') and not info_dict.get('url'):
|
||||
return None
|
||||
def _get_formats(self, info_dict):
|
||||
if info_dict.get('formats') is None:
|
||||
if info_dict.get('url') and info_dict.get('_type', 'video') == 'video':
|
||||
return [info_dict]
|
||||
return []
|
||||
return info_dict['formats']
|
||||
|
||||
formats = info_dict.get('formats', [info_dict])
|
||||
def render_formats_table(self, info_dict):
|
||||
formats = self._get_formats(info_dict)
|
||||
if not formats:
|
||||
return
|
||||
if not self.params.get('listformats_table', True) is not False:
|
||||
table = [
|
||||
[
|
||||
@ -3580,7 +3642,7 @@ def render_formats_table(self, info_dict):
|
||||
format_field(f, 'ext'),
|
||||
self.format_resolution(f),
|
||||
self._format_note(f)
|
||||
] for f in formats if f.get('preference') is None or f['preference'] >= -1000]
|
||||
] for f in formats if (f.get('preference') or 0) >= -1000]
|
||||
return render_table(['format code', 'extension', 'resolution', 'note'], table, extra_gap=1)
|
||||
|
||||
def simplified_codec(f, field):
|
||||
@ -3619,6 +3681,7 @@ def simplified_codec(f, field):
|
||||
format_field(f, 'asr', '\t%s', func=format_decimal_suffix),
|
||||
join_nonempty(
|
||||
self._format_out('UNSUPPORTED', 'light red') if f.get('ext') in ('f4f', 'f4m') else None,
|
||||
self._format_out('DRM', 'light red') if f.get('has_drm') else None,
|
||||
format_field(f, 'language', '[%s]'),
|
||||
join_nonempty(format_field(f, 'format_note'),
|
||||
format_field(f, 'container', ignore=(None, f.get('ext'))),
|
||||
@ -3639,7 +3702,7 @@ def render_thumbnails_table(self, info_dict):
|
||||
return None
|
||||
return render_table(
|
||||
self._list_format_headers('ID', 'Width', 'Height', 'URL'),
|
||||
[[t.get('id'), t.get('width', 'unknown'), t.get('height', 'unknown'), t['url']] for t in thumbnails])
|
||||
[[t.get('id'), t.get('width') or 'unknown', t.get('height') or 'unknown', t['url']] for t in thumbnails])
|
||||
|
||||
def render_subtitles_table(self, video_id, subtitles):
|
||||
def _row(lang, formats):
|
||||
@ -3682,9 +3745,14 @@ def print_debug_header(self):
|
||||
if not self.params.get('verbose'):
|
||||
return
|
||||
|
||||
from . import _IN_CLI # Must be delayed import
|
||||
|
||||
# These imports can be slow. So import them only as needed
|
||||
from .extractor.extractors import _LAZY_LOADER
|
||||
from .extractor.extractors import _PLUGIN_CLASSES as plugin_extractors
|
||||
from .extractor.extractors import (
|
||||
_PLUGIN_CLASSES as plugin_ies,
|
||||
_PLUGIN_OVERRIDES as plugin_ie_overrides
|
||||
)
|
||||
|
||||
def get_encoding(stream):
|
||||
ret = str(getattr(stream, 'encoding', 'missing (%s)' % type(stream).__name__))
|
||||
@ -3713,21 +3781,23 @@ def get_encoding(stream):
|
||||
source = detect_variant()
|
||||
if VARIANT not in (None, 'pip'):
|
||||
source += '*'
|
||||
klass = type(self)
|
||||
write_debug(join_nonempty(
|
||||
f'{"yt-dlp" if REPOSITORY == "yt-dlp/yt-dlp" else REPOSITORY} version',
|
||||
__version__,
|
||||
f'[{RELEASE_GIT_HEAD}]' if RELEASE_GIT_HEAD else '',
|
||||
f'{CHANNEL}@{__version__}',
|
||||
f'[{RELEASE_GIT_HEAD[:9]}]' if RELEASE_GIT_HEAD else '',
|
||||
'' if source == 'unknown' else f'({source})',
|
||||
'' if _IN_CLI else 'API' if klass == YoutubeDL else f'API:{self.__module__}.{klass.__qualname__}',
|
||||
delim=' '))
|
||||
|
||||
if not _IN_CLI:
|
||||
write_debug(f'params: {self.params}')
|
||||
|
||||
if not _LAZY_LOADER:
|
||||
if os.environ.get('YTDLP_NO_LAZY_EXTRACTORS'):
|
||||
write_debug('Lazy loading extractors is forcibly disabled')
|
||||
else:
|
||||
write_debug('Lazy loading extractors is disabled')
|
||||
if plugin_extractors or plugin_postprocessors:
|
||||
write_debug('Plugins: %s' % [
|
||||
'%s%s' % (klass.__name__, '' if klass.__name__ == name else f' as {name}')
|
||||
for name, klass in itertools.chain(plugin_extractors.items(), plugin_postprocessors.items())])
|
||||
if self.params['compat_opts']:
|
||||
write_debug('Compatibility options: %s' % ', '.join(self.params['compat_opts']))
|
||||
|
||||
@ -3761,6 +3831,21 @@ def get_encoding(stream):
|
||||
proxy_map.update(handler.proxies)
|
||||
write_debug(f'Proxy map: {proxy_map}')
|
||||
|
||||
for plugin_type, plugins in {'Extractor': plugin_ies, 'Post-Processor': plugin_pps}.items():
|
||||
display_list = ['%s%s' % (
|
||||
klass.__name__, '' if klass.__name__ == name else f' as {name}')
|
||||
for name, klass in plugins.items()]
|
||||
if plugin_type == 'Extractor':
|
||||
display_list.extend(f'{plugins[-1].IE_NAME.partition("+")[2]} ({parent.__name__})'
|
||||
for parent, plugins in plugin_ie_overrides.items())
|
||||
if not display_list:
|
||||
continue
|
||||
write_debug(f'{plugin_type} Plugins: {", ".join(sorted(display_list))}')
|
||||
|
||||
plugin_dirs = plugin_directories()
|
||||
if plugin_dirs:
|
||||
write_debug(f'Plugin directories: {plugin_dirs}')
|
||||
|
||||
# Not implemented
|
||||
if False and self.params.get('call_home'):
|
||||
ipaddr = self.urlopen('https://yt-dl.org/ip').read().decode()
|
||||
@ -3810,9 +3895,12 @@ def _setup_opener(self):
|
||||
# https://github.com/ytdl-org/youtube-dl/issues/8227)
|
||||
file_handler = urllib.request.FileHandler()
|
||||
|
||||
def file_open(*args, **kwargs):
|
||||
raise urllib.error.URLError('file:// scheme is explicitly disabled in yt-dlp for security reasons')
|
||||
file_handler.file_open = file_open
|
||||
if not self.params.get('enable_file_urls'):
|
||||
def file_open(*args, **kwargs):
|
||||
raise urllib.error.URLError(
|
||||
'file:// URLs are explicitly disabled in yt-dlp for security reasons. '
|
||||
'Use --enable-file-urls to enable at your own risk.')
|
||||
file_handler.file_open = file_open
|
||||
|
||||
opener = urllib.request.build_opener(
|
||||
proxy_handler, https_handler, cookie_processor, ydlh, redirect_handler, data_handler, file_handler)
|
||||
@ -3874,7 +3962,7 @@ def _write_description(self, label, ie_result, descfn):
|
||||
elif not self.params.get('overwrites', True) and os.path.exists(descfn):
|
||||
self.to_screen(f'[info] {label.title()} description is already present')
|
||||
elif ie_result.get('description') is None:
|
||||
self.report_warning(f'There\'s no {label} description to write')
|
||||
self.to_screen(f'[info] There\'s no {label} description to write')
|
||||
return False
|
||||
else:
|
||||
try:
|
||||
@ -3890,15 +3978,18 @@ def _write_subtitles(self, info_dict, filename):
|
||||
''' Write subtitles to file and return list of (sub_filename, final_sub_filename); or None if error'''
|
||||
ret = []
|
||||
subtitles = info_dict.get('requested_subtitles')
|
||||
if not subtitles or not (self.params.get('writesubtitles') or self.params.get('writeautomaticsub')):
|
||||
if not (self.params.get('writesubtitles') or self.params.get('writeautomaticsub')):
|
||||
# subtitles download errors are already managed as troubles in relevant IE
|
||||
# that way it will silently go on when used with unsupporting IE
|
||||
return ret
|
||||
|
||||
elif not subtitles:
|
||||
self.to_screen('[info] There\'s no subtitles for the requested languages')
|
||||
return ret
|
||||
sub_filename_base = self.prepare_filename(info_dict, 'subtitle')
|
||||
if not sub_filename_base:
|
||||
self.to_screen('[info] Skipping writing video subtitles')
|
||||
return ret
|
||||
|
||||
for sub_lang, sub_info in subtitles.items():
|
||||
sub_format = sub_info['ext']
|
||||
sub_filename = subtitles_filename(filename, sub_lang, sub_format, info_dict.get('ext'))
|
||||
@ -3945,6 +4036,9 @@ def _write_thumbnails(self, label, info_dict, filename, thumb_filename_base=None
|
||||
thumbnails, ret = [], []
|
||||
if write_all or self.params.get('writethumbnail', False):
|
||||
thumbnails = info_dict.get('thumbnails') or []
|
||||
if not thumbnails:
|
||||
self.to_screen(f'[info] There\'s no {label} thumbnails to download')
|
||||
return ret
|
||||
multiple = write_all and len(thumbnails) > 1
|
||||
|
||||
if thumb_filename_base is None:
|
||||
|
@ -16,11 +16,9 @@
|
||||
|
||||
from .compat import compat_shlex_quote
|
||||
from .cookies import SUPPORTED_BROWSERS, SUPPORTED_KEYRINGS
|
||||
from .downloader import FileDownloader
|
||||
from .downloader.external import get_external_downloader
|
||||
from .extractor import list_extractor_classes
|
||||
from .extractor.adobepass import MSO_INFO
|
||||
from .extractor.common import InfoExtractor
|
||||
from .options import parseOpts
|
||||
from .postprocessor import (
|
||||
FFmpegExtractAudioPP,
|
||||
@ -40,6 +38,7 @@
|
||||
DateRange,
|
||||
DownloadCancelled,
|
||||
DownloadError,
|
||||
FormatSorter,
|
||||
GeoUtils,
|
||||
PlaylistEntries,
|
||||
SameFileError,
|
||||
@ -50,6 +49,7 @@
|
||||
format_field,
|
||||
int_or_none,
|
||||
match_filter_func,
|
||||
parse_bytes,
|
||||
parse_duration,
|
||||
preferredencoding,
|
||||
read_batch_urls,
|
||||
@ -91,12 +91,11 @@ def get_urls(urls, batchfile, verbose):
|
||||
|
||||
|
||||
def print_extractor_information(opts, urls):
|
||||
# Importing GenericIE is currently slow since it imports other extractors
|
||||
# TODO: Move this back to module level after generalization of embed detection
|
||||
from .extractor.generic import GenericIE
|
||||
|
||||
out = ''
|
||||
if opts.list_extractors:
|
||||
# Importing GenericIE is currently slow since it imports YoutubeIE
|
||||
from .extractor.generic import GenericIE
|
||||
|
||||
urls = dict.fromkeys(urls, False)
|
||||
for ie in list_extractor_classes(opts.age_limit):
|
||||
out += ie.IE_NAME + (' (CURRENTLY BROKEN)' if not ie.working() else '') + '\n'
|
||||
@ -152,7 +151,7 @@ def set_default_compat(compat_name, opt_name, default=True, remove_compat=True):
|
||||
else:
|
||||
opts.embed_infojson = False
|
||||
if 'format-sort' in opts.compat_opts:
|
||||
opts.format_sort.extend(InfoExtractor.FormatSort.ytdl_default)
|
||||
opts.format_sort.extend(FormatSorter.ytdl_default)
|
||||
_video_multistreams_set = set_default_compat('multistreams', 'allow_multiple_video_streams', False, remove_compat=False)
|
||||
_audio_multistreams_set = set_default_compat('multistreams', 'allow_multiple_audio_streams', False, remove_compat=False)
|
||||
if _video_multistreams_set is False and _audio_multistreams_set is False:
|
||||
@ -227,7 +226,7 @@ def validate_minmax(min_val, max_val, min_name, max_name=None):
|
||||
|
||||
# Format sort
|
||||
for f in opts.format_sort:
|
||||
validate_regex('format sorting', f, InfoExtractor.FormatSort.regex)
|
||||
validate_regex('format sorting', f, FormatSorter.regex)
|
||||
|
||||
# Postprocessor formats
|
||||
validate_regex('merge output format', opts.merge_output_format,
|
||||
@ -281,19 +280,19 @@ def parse_sleep_func(expr):
|
||||
raise ValueError(f'invalid {key} retry sleep expression {expr!r}')
|
||||
|
||||
# Bytes
|
||||
def parse_bytes(name, value):
|
||||
def validate_bytes(name, value):
|
||||
if value is None:
|
||||
return None
|
||||
numeric_limit = FileDownloader.parse_bytes(value)
|
||||
numeric_limit = parse_bytes(value)
|
||||
validate(numeric_limit is not None, 'rate limit', value)
|
||||
return numeric_limit
|
||||
|
||||
opts.ratelimit = parse_bytes('rate limit', opts.ratelimit)
|
||||
opts.throttledratelimit = parse_bytes('throttled rate limit', opts.throttledratelimit)
|
||||
opts.min_filesize = parse_bytes('min filesize', opts.min_filesize)
|
||||
opts.max_filesize = parse_bytes('max filesize', opts.max_filesize)
|
||||
opts.buffersize = parse_bytes('buffer size', opts.buffersize)
|
||||
opts.http_chunk_size = parse_bytes('http chunk size', opts.http_chunk_size)
|
||||
opts.ratelimit = validate_bytes('rate limit', opts.ratelimit)
|
||||
opts.throttledratelimit = validate_bytes('throttled rate limit', opts.throttledratelimit)
|
||||
opts.min_filesize = validate_bytes('min filesize', opts.min_filesize)
|
||||
opts.max_filesize = validate_bytes('max filesize', opts.max_filesize)
|
||||
opts.buffersize = validate_bytes('buffer size', opts.buffersize)
|
||||
opts.http_chunk_size = validate_bytes('http chunk size', opts.http_chunk_size)
|
||||
|
||||
# Output templates
|
||||
def validate_outtmpl(tmpl, msg):
|
||||
@ -319,21 +318,18 @@ def validate_outtmpl(tmpl, msg):
|
||||
if outtmpl_default == '':
|
||||
opts.skip_download = None
|
||||
del opts.outtmpl['default']
|
||||
if outtmpl_default and not os.path.splitext(outtmpl_default)[1] and opts.extractaudio:
|
||||
raise ValueError(
|
||||
'Cannot download a video and extract audio into the same file! '
|
||||
f'Use "{outtmpl_default}.%(ext)s" instead of "{outtmpl_default}" as the output template')
|
||||
|
||||
def parse_chapters(name, value):
|
||||
chapters, ranges = [], []
|
||||
parse_timestamp = lambda x: float('inf') if x in ('inf', 'infinite') else parse_duration(x)
|
||||
for regex in value or []:
|
||||
if regex.startswith('*'):
|
||||
for range in regex[1:].split(','):
|
||||
dur = tuple(map(parse_duration, range.strip().split('-')))
|
||||
if len(dur) == 2 and all(t is not None for t in dur):
|
||||
ranges.append(dur)
|
||||
else:
|
||||
raise ValueError(f'invalid {name} time range "{regex}". Must be of the form *start-end')
|
||||
for range_ in map(str.strip, regex[1:].split(',')):
|
||||
mobj = range_ != '-' and re.fullmatch(r'([^-]+)?\s*-\s*([^-]+)?', range_)
|
||||
dur = mobj and (parse_timestamp(mobj.group(1) or '0'), parse_timestamp(mobj.group(2) or 'inf'))
|
||||
if None in (dur or [None]):
|
||||
raise ValueError(f'invalid {name} time range "{regex}". Must be of the form "*start-end"')
|
||||
ranges.append(dur)
|
||||
continue
|
||||
try:
|
||||
chapters.append(re.compile(regex))
|
||||
@ -350,7 +346,7 @@ def parse_chapters(name, value):
|
||||
mobj = re.fullmatch(r'''(?x)
|
||||
(?P<name>[^+:]+)
|
||||
(?:\s*\+\s*(?P<keyring>[^:]+))?
|
||||
(?:\s*:\s*(?P<profile>.+?))?
|
||||
(?:\s*:\s*(?!:)(?P<profile>.+?))?
|
||||
(?:\s*::\s*(?P<container>.+))?
|
||||
''', opts.cookiesfrombrowser)
|
||||
if mobj is None:
|
||||
@ -386,10 +382,12 @@ def metadataparser_actions(f):
|
||||
raise ValueError(f'{cmd} is invalid; {err}')
|
||||
yield action
|
||||
|
||||
parse_metadata = opts.parse_metadata or []
|
||||
if opts.metafromtitle is not None:
|
||||
parse_metadata.append('title:%s' % opts.metafromtitle)
|
||||
opts.parse_metadata = list(itertools.chain(*map(metadataparser_actions, parse_metadata)))
|
||||
opts.parse_metadata.setdefault('pre_process', []).append('title:%s' % opts.metafromtitle)
|
||||
opts.parse_metadata = {
|
||||
k: list(itertools.chain(*map(metadataparser_actions, v)))
|
||||
for k, v in opts.parse_metadata.items()
|
||||
}
|
||||
|
||||
# Other options
|
||||
if opts.playlist_items is not None:
|
||||
@ -405,11 +403,14 @@ def metadataparser_actions(f):
|
||||
except Exception:
|
||||
raise ValueError('unsupported geo-bypass country or ip-block')
|
||||
|
||||
opts.match_filter = match_filter_func(opts.match_filter)
|
||||
opts.match_filter = match_filter_func(opts.match_filter, opts.breaking_match_filter)
|
||||
|
||||
if opts.download_archive is not None:
|
||||
opts.download_archive = expand_path(opts.download_archive)
|
||||
|
||||
if opts.ffmpeg_location is not None:
|
||||
opts.ffmpeg_location = expand_path(opts.ffmpeg_location)
|
||||
|
||||
if opts.user_agent is not None:
|
||||
opts.headers.setdefault('User-Agent', opts.user_agent)
|
||||
if opts.referer is not None:
|
||||
@ -485,7 +486,7 @@ def report_conflict(arg1, opt1, arg2='--allow-unplayable-formats', opt2='allow_u
|
||||
val1=opts.sponskrub and opts.sponskrub_cut)
|
||||
|
||||
# Conflicts with --allow-unplayable-formats
|
||||
report_conflict('--add-metadata', 'addmetadata')
|
||||
report_conflict('--embed-metadata', 'addmetadata')
|
||||
report_conflict('--embed-chapters', 'addchapters')
|
||||
report_conflict('--embed-info-json', 'embed_infojson')
|
||||
report_conflict('--embed-subs', 'embedsubtitles')
|
||||
@ -558,11 +559,11 @@ def report_deprecation(val, old, new=None):
|
||||
def get_postprocessors(opts):
|
||||
yield from opts.add_postprocessors
|
||||
|
||||
if opts.parse_metadata:
|
||||
for when, actions in opts.parse_metadata.items():
|
||||
yield {
|
||||
'key': 'MetadataParser',
|
||||
'actions': opts.parse_metadata,
|
||||
'when': 'pre_process'
|
||||
'actions': actions,
|
||||
'when': when
|
||||
}
|
||||
sponsorblock_query = opts.sponsorblock_mark | opts.sponsorblock_remove
|
||||
if sponsorblock_query:
|
||||
@ -698,11 +699,12 @@ def parse_options(argv=None):
|
||||
|
||||
postprocessors = list(get_postprocessors(opts))
|
||||
|
||||
print_only = bool(opts.forceprint) and all(k not in opts.forceprint for k in POSTPROCESS_WHEN[2:])
|
||||
print_only = bool(opts.forceprint) and all(k not in opts.forceprint for k in POSTPROCESS_WHEN[3:])
|
||||
any_getting = any(getattr(opts, k) for k in (
|
||||
'dumpjson', 'dump_single_json', 'getdescription', 'getduration', 'getfilename',
|
||||
'getformat', 'getid', 'getthumbnail', 'gettitle', 'geturl'
|
||||
))
|
||||
opts.quiet = opts.quiet or any_getting or opts.print_json or bool(opts.forceprint)
|
||||
|
||||
playlist_pps = [pp for pp in postprocessors if pp.get('when') == 'playlist']
|
||||
write_playlist_infojson = (opts.writeinfojson and not opts.clean_infojson
|
||||
@ -738,7 +740,7 @@ def parse_options(argv=None):
|
||||
'client_certificate': opts.client_certificate,
|
||||
'client_certificate_key': opts.client_certificate_key,
|
||||
'client_certificate_password': opts.client_certificate_password,
|
||||
'quiet': opts.quiet or any_getting or opts.print_json or bool(opts.forceprint),
|
||||
'quiet': opts.quiet,
|
||||
'no_warnings': opts.no_warnings,
|
||||
'forceurl': opts.geturl,
|
||||
'forcetitle': opts.gettitle,
|
||||
@ -850,6 +852,7 @@ def parse_options(argv=None):
|
||||
'legacyserverconnect': opts.legacy_server_connect,
|
||||
'nocheckcertificate': opts.no_check_certificate,
|
||||
'prefer_insecure': opts.prefer_insecure,
|
||||
'enable_file_urls': opts.enable_file_urls,
|
||||
'http_headers': opts.headers,
|
||||
'proxy': opts.proxy,
|
||||
'socket_timeout': opts.socket_timeout,
|
||||
@ -928,7 +931,7 @@ def _real_main(argv=None):
|
||||
if opts.rm_cachedir:
|
||||
ydl.cache.remove()
|
||||
|
||||
updater = Updater(ydl)
|
||||
updater = Updater(ydl, opts.update_self if isinstance(opts.update_self, str) else None)
|
||||
if opts.update_self and updater.update() and actual_use:
|
||||
if updater.cmd:
|
||||
return updater.restart()
|
||||
@ -958,6 +961,8 @@ def _real_main(argv=None):
|
||||
|
||||
|
||||
def main(argv=None):
|
||||
global _IN_CLI
|
||||
_IN_CLI = True
|
||||
try:
|
||||
_exit(*variadic(_real_main(argv)))
|
||||
except DownloadError:
|
||||
|
@ -5,7 +5,7 @@
|
||||
|
||||
import sys
|
||||
|
||||
if __package__ is None and not hasattr(sys, 'frozen'):
|
||||
if __package__ is None and not getattr(sys, 'frozen', False):
|
||||
# direct call of __main__.py
|
||||
import os.path
|
||||
path = os.path.realpath(os.path.abspath(__file__))
|
||||
@ -14,5 +14,4 @@
|
||||
import yt_dlp
|
||||
|
||||
if __name__ == '__main__':
|
||||
yt_dlp._IN_CLI = True
|
||||
yt_dlp.main()
|
||||
|
5
yt_dlp/__pyinstaller/__init__.py
Normal file
5
yt_dlp/__pyinstaller/__init__.py
Normal file
@ -0,0 +1,5 @@
|
||||
import os
|
||||
|
||||
|
||||
def get_hook_dirs():
|
||||
return [os.path.dirname(__file__)]
|
31
yt_dlp/__pyinstaller/hook-yt_dlp.py
Normal file
31
yt_dlp/__pyinstaller/hook-yt_dlp.py
Normal file
@ -0,0 +1,31 @@
|
||||
import sys
|
||||
|
||||
from PyInstaller.utils.hooks import collect_submodules
|
||||
|
||||
|
||||
def pycryptodome_module():
|
||||
try:
|
||||
import Cryptodome # noqa: F401
|
||||
except ImportError:
|
||||
try:
|
||||
import Crypto # noqa: F401
|
||||
print('WARNING: Using Crypto since Cryptodome is not available. '
|
||||
'Install with: pip install pycryptodomex', file=sys.stderr)
|
||||
return 'Crypto'
|
||||
except ImportError:
|
||||
pass
|
||||
return 'Cryptodome'
|
||||
|
||||
|
||||
def get_hidden_imports():
|
||||
yield 'yt_dlp.compat._legacy'
|
||||
yield pycryptodome_module()
|
||||
yield from collect_submodules('websockets')
|
||||
# These are auto-detected, but explicitly add them just in case
|
||||
yield from ('mutagen', 'brotli', 'certifi')
|
||||
|
||||
|
||||
hiddenimports = list(get_hidden_imports())
|
||||
print(f'Adding imports: {hiddenimports}')
|
||||
|
||||
excludedimports = ['youtube_dl', 'youtube_dlc', 'test', 'ytdlp_plugins', 'devscripts']
|
@ -2,17 +2,17 @@
|
||||
from math import ceil
|
||||
|
||||
from .compat import compat_ord
|
||||
from .dependencies import Cryptodome_AES
|
||||
from .dependencies import Cryptodome
|
||||
from .utils import bytes_to_intlist, intlist_to_bytes
|
||||
|
||||
if Cryptodome_AES:
|
||||
if Cryptodome.AES:
|
||||
def aes_cbc_decrypt_bytes(data, key, iv):
|
||||
""" Decrypt bytes with AES-CBC using pycryptodome """
|
||||
return Cryptodome_AES.new(key, Cryptodome_AES.MODE_CBC, iv).decrypt(data)
|
||||
return Cryptodome.AES.new(key, Cryptodome.AES.MODE_CBC, iv).decrypt(data)
|
||||
|
||||
def aes_gcm_decrypt_and_verify_bytes(data, key, tag, nonce):
|
||||
""" Decrypt bytes with AES-GCM using pycryptodome """
|
||||
return Cryptodome_AES.new(key, Cryptodome_AES.MODE_GCM, nonce).decrypt_and_verify(data, tag)
|
||||
return Cryptodome.AES.new(key, Cryptodome.AES.MODE_GCM, nonce).decrypt_and_verify(data, tag)
|
||||
|
||||
else:
|
||||
def aes_cbc_decrypt_bytes(data, key, iv):
|
||||
@ -28,11 +28,23 @@ def aes_cbc_encrypt_bytes(data, key, iv, **kwargs):
|
||||
return intlist_to_bytes(aes_cbc_encrypt(*map(bytes_to_intlist, (data, key, iv)), **kwargs))
|
||||
|
||||
|
||||
BLOCK_SIZE_BYTES = 16
|
||||
|
||||
|
||||
def unpad_pkcs7(data):
|
||||
return data[:-compat_ord(data[-1])]
|
||||
|
||||
|
||||
BLOCK_SIZE_BYTES = 16
|
||||
def pkcs7_padding(data):
|
||||
"""
|
||||
PKCS#7 padding
|
||||
|
||||
@param {int[]} data cleartext
|
||||
@returns {int[]} padding data
|
||||
"""
|
||||
|
||||
remaining_length = BLOCK_SIZE_BYTES - len(data) % BLOCK_SIZE_BYTES
|
||||
return data + [remaining_length] * remaining_length
|
||||
|
||||
|
||||
def pad_block(block, padding_mode):
|
||||
@ -64,7 +76,7 @@ def pad_block(block, padding_mode):
|
||||
|
||||
def aes_ecb_encrypt(data, key, iv=None):
|
||||
"""
|
||||
Encrypt with aes in ECB mode
|
||||
Encrypt with aes in ECB mode. Using PKCS#7 padding
|
||||
|
||||
@param {int[]} data cleartext
|
||||
@param {int[]} key 16/24/32-Byte cipher key
|
||||
@ -77,8 +89,7 @@ def aes_ecb_encrypt(data, key, iv=None):
|
||||
encrypted_data = []
|
||||
for i in range(block_count):
|
||||
block = data[i * BLOCK_SIZE_BYTES: (i + 1) * BLOCK_SIZE_BYTES]
|
||||
encrypted_data += aes_encrypt(block, expanded_key)
|
||||
encrypted_data = encrypted_data[:len(data)]
|
||||
encrypted_data += aes_encrypt(pkcs7_padding(block), expanded_key)
|
||||
|
||||
return encrypted_data
|
||||
|
||||
@ -551,5 +562,6 @@ def ghash(subkey, data):
|
||||
|
||||
'key_expansion',
|
||||
'pad_block',
|
||||
'pkcs7_padding',
|
||||
'unpad_pkcs7',
|
||||
]
|
||||
|
@ -1,10 +1,10 @@
|
||||
import contextlib
|
||||
import errno
|
||||
import json
|
||||
import os
|
||||
import re
|
||||
import shutil
|
||||
import traceback
|
||||
import urllib.parse
|
||||
|
||||
from .utils import expand_path, traverse_obj, version_tuple, write_json_file
|
||||
from .version import __version__
|
||||
@ -22,11 +22,9 @@ def _get_root_dir(self):
|
||||
return expand_path(res)
|
||||
|
||||
def _get_cache_fn(self, section, key, dtype):
|
||||
assert re.match(r'^[a-zA-Z0-9_.-]+$', section), \
|
||||
'invalid section %r' % section
|
||||
assert re.match(r'^[a-zA-Z0-9_.-]+$', key), 'invalid key %r' % key
|
||||
return os.path.join(
|
||||
self._get_root_dir(), section, f'{key}.{dtype}')
|
||||
assert re.match(r'^[\w.-]+$', section), f'invalid section {section!r}'
|
||||
key = urllib.parse.quote(key, safe='').replace('%', ',') # encode non-ascii characters
|
||||
return os.path.join(self._get_root_dir(), section, f'{key}.{dtype}')
|
||||
|
||||
@property
|
||||
def enabled(self):
|
||||
@ -40,11 +38,7 @@ def store(self, section, key, data, dtype='json'):
|
||||
|
||||
fn = self._get_cache_fn(section, key, dtype)
|
||||
try:
|
||||
try:
|
||||
os.makedirs(os.path.dirname(fn))
|
||||
except OSError as ose:
|
||||
if ose.errno != errno.EEXIST:
|
||||
raise
|
||||
os.makedirs(os.path.dirname(fn), exist_ok=True)
|
||||
self._ydl.write_debug(f'Saving {section}.{key} to cache')
|
||||
write_json_file({'yt-dlp_version': __version__, 'data': data}, fn)
|
||||
except Exception:
|
||||
|
@ -8,13 +8,13 @@
|
||||
|
||||
# XXX: Implement this the same way as other DeprecationWarnings without circular import
|
||||
passthrough_module(__name__, '._legacy', callback=lambda attr: warnings.warn(
|
||||
DeprecationWarning(f'{__name__}.{attr} is deprecated'), stacklevel=3))
|
||||
DeprecationWarning(f'{__name__}.{attr} is deprecated'), stacklevel=5))
|
||||
|
||||
|
||||
# HTMLParseError has been deprecated in Python 3.3 and removed in
|
||||
# Python 3.5. Introducing dummy exception for Python >3.5 for compatible
|
||||
# and uniform cross-version exception handling
|
||||
class compat_HTMLParseError(Exception):
|
||||
class compat_HTMLParseError(ValueError):
|
||||
pass
|
||||
|
||||
|
||||
@ -70,9 +70,3 @@ def compat_expanduser(path):
|
||||
return userhome + path[i:]
|
||||
else:
|
||||
compat_expanduser = os.path.expanduser
|
||||
|
||||
|
||||
# NB: Add modules that are imported dynamically here so that PyInstaller can find them
|
||||
# See https://github.com/pyinstaller/pyinstaller-hooks-contrib/issues/438
|
||||
if False:
|
||||
from . import _legacy # noqa: F401
|
||||
|
@ -1,5 +1,6 @@
|
||||
""" Do not use! """
|
||||
|
||||
import base64
|
||||
import collections
|
||||
import ctypes
|
||||
import getpass
|
||||
@ -29,10 +30,11 @@
|
||||
from re import Pattern as compat_Pattern # noqa: F401
|
||||
from re import match as compat_Match # noqa: F401
|
||||
|
||||
from . import compat_expanduser, compat_HTMLParseError, compat_realpath
|
||||
from .compat_utils import passthrough_module
|
||||
from ..dependencies import Cryptodome_AES as compat_pycrypto_AES # noqa: F401
|
||||
from ..dependencies import brotli as compat_brotli # noqa: F401
|
||||
from ..dependencies import websockets as compat_websockets # noqa: F401
|
||||
from ..dependencies.Cryptodome import AES as compat_pycrypto_AES # noqa: F401
|
||||
|
||||
passthrough_module(__name__, '...utils', ('WINDOWS_VT_MODE', 'windows_enable_vt_mode'))
|
||||
|
||||
@ -47,22 +49,25 @@ def compat_setenv(key, value, env=os.environ):
|
||||
env[key] = value
|
||||
|
||||
|
||||
compat_base64_b64decode = base64.b64decode
|
||||
compat_basestring = str
|
||||
compat_casefold = str.casefold
|
||||
compat_chr = chr
|
||||
compat_collections_abc = collections.abc
|
||||
compat_cookiejar = http.cookiejar
|
||||
compat_cookiejar_Cookie = http.cookiejar.Cookie
|
||||
compat_cookies = http.cookies
|
||||
compat_cookies_SimpleCookie = http.cookies.SimpleCookie
|
||||
compat_etree_Element = etree.Element
|
||||
compat_etree_register_namespace = etree.register_namespace
|
||||
compat_cookiejar = compat_http_cookiejar = http.cookiejar
|
||||
compat_cookiejar_Cookie = compat_http_cookiejar_Cookie = http.cookiejar.Cookie
|
||||
compat_cookies = compat_http_cookies = http.cookies
|
||||
compat_cookies_SimpleCookie = compat_http_cookies_SimpleCookie = http.cookies.SimpleCookie
|
||||
compat_etree_Element = compat_xml_etree_ElementTree_Element = etree.Element
|
||||
compat_etree_register_namespace = compat_xml_etree_register_namespace = etree.register_namespace
|
||||
compat_filter = filter
|
||||
compat_get_terminal_size = shutil.get_terminal_size
|
||||
compat_getenv = os.getenv
|
||||
compat_getpass = getpass.getpass
|
||||
compat_getpass = compat_getpass_getpass = getpass.getpass
|
||||
compat_html_entities = html.entities
|
||||
compat_html_entities_html5 = html.entities.html5
|
||||
compat_HTMLParser = html.parser.HTMLParser
|
||||
compat_html_parser_HTMLParseError = compat_HTMLParseError
|
||||
compat_HTMLParser = compat_html_parser_HTMLParser = html.parser.HTMLParser
|
||||
compat_http_client = http.client
|
||||
compat_http_server = http.server
|
||||
compat_input = input
|
||||
@ -71,6 +76,8 @@ def compat_setenv(key, value, env=os.environ):
|
||||
compat_kwargs = lambda kwargs: kwargs
|
||||
compat_map = map
|
||||
compat_numeric_types = (int, float, complex)
|
||||
compat_os_path_expanduser = compat_expanduser
|
||||
compat_os_path_realpath = compat_realpath
|
||||
compat_print = print
|
||||
compat_shlex_split = shlex.split
|
||||
compat_socket_create_connection = socket.create_connection
|
||||
@ -80,7 +87,9 @@ def compat_setenv(key, value, env=os.environ):
|
||||
compat_subprocess_get_DEVNULL = lambda: DEVNULL
|
||||
compat_tokenize_tokenize = tokenize.tokenize
|
||||
compat_urllib_error = urllib.error
|
||||
compat_urllib_HTTPError = urllib.error.HTTPError
|
||||
compat_urllib_parse = urllib.parse
|
||||
compat_urllib_parse_parse_qs = urllib.parse.parse_qs
|
||||
compat_urllib_parse_quote = urllib.parse.quote
|
||||
compat_urllib_parse_quote_plus = urllib.parse.quote_plus
|
||||
compat_urllib_parse_unquote_plus = urllib.parse.unquote_plus
|
||||
@ -89,8 +98,10 @@ def compat_setenv(key, value, env=os.environ):
|
||||
compat_urllib_request = urllib.request
|
||||
compat_urllib_request_DataHandler = urllib.request.DataHandler
|
||||
compat_urllib_response = urllib.response
|
||||
compat_urlretrieve = urllib.request.urlretrieve
|
||||
compat_xml_parse_error = etree.ParseError
|
||||
compat_urlretrieve = compat_urllib_request_urlretrieve = urllib.request.urlretrieve
|
||||
compat_xml_parse_error = compat_xml_etree_ElementTree_ParseError = etree.ParseError
|
||||
compat_xpath = lambda xpath: xpath
|
||||
compat_zip = zip
|
||||
workaround_optparse_bug9161 = lambda: None
|
||||
|
||||
legacy = []
|
||||
|
@ -1,5 +1,6 @@
|
||||
import collections
|
||||
import contextlib
|
||||
import functools
|
||||
import importlib
|
||||
import sys
|
||||
import types
|
||||
@ -10,61 +11,73 @@
|
||||
|
||||
|
||||
def get_package_info(module):
|
||||
parent = module.__name__.split('.')[0]
|
||||
parent_module = None
|
||||
with contextlib.suppress(ImportError):
|
||||
parent_module = importlib.import_module(parent)
|
||||
|
||||
for attr in ('__version__', 'version_string', 'version'):
|
||||
version = getattr(parent_module, attr, None)
|
||||
if version is not None:
|
||||
break
|
||||
return _Package(getattr(module, '_yt_dlp__identifier', parent), str(version))
|
||||
return _Package(
|
||||
name=getattr(module, '_yt_dlp__identifier', module.__name__),
|
||||
version=str(next(filter(None, (
|
||||
getattr(module, attr, None)
|
||||
for attr in ('__version__', 'version_string', 'version')
|
||||
)), None)))
|
||||
|
||||
|
||||
def _is_package(module):
|
||||
try:
|
||||
module.__getattribute__('__path__')
|
||||
except AttributeError:
|
||||
return False
|
||||
return True
|
||||
return '__path__' in vars(module)
|
||||
|
||||
|
||||
def passthrough_module(parent, child, allowed_attributes=None, *, callback=lambda _: None):
|
||||
parent_module = importlib.import_module(parent)
|
||||
child_module = None # Import child module only as needed
|
||||
def _is_dunder(name):
|
||||
return name.startswith('__') and name.endswith('__')
|
||||
|
||||
class PassthroughModule(types.ModuleType):
|
||||
def __getattr__(self, attr):
|
||||
if _is_package(parent_module):
|
||||
with contextlib.suppress(ImportError):
|
||||
return importlib.import_module(f'.{attr}', parent)
|
||||
|
||||
ret = self.__from_child(attr)
|
||||
if ret is _NO_ATTRIBUTE:
|
||||
raise AttributeError(f'module {parent} has no attribute {attr}')
|
||||
callback(attr)
|
||||
return ret
|
||||
class EnhancedModule(types.ModuleType):
|
||||
def __bool__(self):
|
||||
return vars(self).get('__bool__', lambda: True)()
|
||||
|
||||
def __from_child(self, attr):
|
||||
if allowed_attributes is None:
|
||||
if attr.startswith('__') and attr.endswith('__'):
|
||||
return _NO_ATTRIBUTE
|
||||
elif attr not in allowed_attributes:
|
||||
def __getattribute__(self, attr):
|
||||
try:
|
||||
ret = super().__getattribute__(attr)
|
||||
except AttributeError:
|
||||
if _is_dunder(attr):
|
||||
raise
|
||||
getter = getattr(self, '__getattr__', None)
|
||||
if not getter:
|
||||
raise
|
||||
ret = getter(attr)
|
||||
return ret.fget() if isinstance(ret, property) else ret
|
||||
|
||||
|
||||
def passthrough_module(parent, child, allowed_attributes=(..., ), *, callback=lambda _: None):
|
||||
"""Passthrough parent module into a child module, creating the parent if necessary"""
|
||||
def __getattr__(attr):
|
||||
if _is_package(parent):
|
||||
with contextlib.suppress(ModuleNotFoundError):
|
||||
return importlib.import_module(f'.{attr}', parent.__name__)
|
||||
|
||||
ret = from_child(attr)
|
||||
if ret is _NO_ATTRIBUTE:
|
||||
raise AttributeError(f'module {parent.__name__} has no attribute {attr}')
|
||||
callback(attr)
|
||||
return ret
|
||||
|
||||
@functools.lru_cache(maxsize=None)
|
||||
def from_child(attr):
|
||||
nonlocal child
|
||||
if attr not in allowed_attributes:
|
||||
if ... not in allowed_attributes or _is_dunder(attr):
|
||||
return _NO_ATTRIBUTE
|
||||
|
||||
nonlocal child_module
|
||||
child_module = child_module or importlib.import_module(child, parent)
|
||||
if isinstance(child, str):
|
||||
child = importlib.import_module(child, parent.__name__)
|
||||
|
||||
with contextlib.suppress(AttributeError):
|
||||
return getattr(child_module, attr)
|
||||
if _is_package(child):
|
||||
with contextlib.suppress(ImportError):
|
||||
return passthrough_module(f'{parent.__name__}.{attr}',
|
||||
importlib.import_module(f'.{attr}', child.__name__))
|
||||
|
||||
if _is_package(child_module):
|
||||
with contextlib.suppress(ImportError):
|
||||
return importlib.import_module(f'.{attr}', child)
|
||||
with contextlib.suppress(AttributeError):
|
||||
return getattr(child, attr)
|
||||
|
||||
return _NO_ATTRIBUTE
|
||||
return _NO_ATTRIBUTE
|
||||
|
||||
# Python 3.6 does not have module level __getattr__
|
||||
# https://peps.python.org/pep-0562/
|
||||
sys.modules[parent].__class__ = PassthroughModule
|
||||
parent = sys.modules.get(parent, types.ModuleType(parent))
|
||||
parent.__class__ = EnhancedModule
|
||||
parent.__getattr__ = __getattr__
|
||||
return parent
|
||||
|
30
yt_dlp/compat/shutil.py
Normal file
30
yt_dlp/compat/shutil.py
Normal file
@ -0,0 +1,30 @@
|
||||
# flake8: noqa: F405
|
||||
from shutil import * # noqa: F403
|
||||
|
||||
from .compat_utils import passthrough_module
|
||||
|
||||
passthrough_module(__name__, 'shutil')
|
||||
del passthrough_module
|
||||
|
||||
|
||||
import sys
|
||||
|
||||
if sys.platform.startswith('freebsd'):
|
||||
import errno
|
||||
import os
|
||||
import shutil
|
||||
|
||||
# Workaround for PermissionError when using restricted ACL mode on FreeBSD
|
||||
def copy2(src, dst, *args, **kwargs):
|
||||
if os.path.isdir(dst):
|
||||
dst = os.path.join(dst, os.path.basename(src))
|
||||
shutil.copyfile(src, dst, *args, **kwargs)
|
||||
try:
|
||||
shutil.copystat(src, dst, *args, **kwargs)
|
||||
except PermissionError as e:
|
||||
if e.errno != getattr(errno, 'EPERM', None):
|
||||
raise
|
||||
return dst
|
||||
|
||||
def move(*args, copy_function=copy2, **kwargs):
|
||||
return shutil.move(*args, copy_function=copy_function, **kwargs)
|
@ -1,6 +1,7 @@
|
||||
import base64
|
||||
import contextlib
|
||||
import http.cookiejar
|
||||
import http.cookies
|
||||
import json
|
||||
import os
|
||||
import re
|
||||
@ -844,12 +845,15 @@ def _get_linux_keyring_password(browser_keyring_name, keyring, logger):
|
||||
def _get_mac_keyring_password(browser_keyring_name, logger):
|
||||
logger.debug('using find-generic-password to obtain password from OSX keychain')
|
||||
try:
|
||||
stdout, _, _ = Popen.run(
|
||||
stdout, _, returncode = Popen.run(
|
||||
['security', 'find-generic-password',
|
||||
'-w', # write password to stdout
|
||||
'-a', browser_keyring_name, # match 'account'
|
||||
'-s', f'{browser_keyring_name} Safe Storage'], # match 'service'
|
||||
stdout=subprocess.PIPE, stderr=subprocess.DEVNULL)
|
||||
if returncode:
|
||||
logger.warning('find-generic-password failed')
|
||||
return None
|
||||
return stdout.rstrip(b'\n')
|
||||
except Exception as e:
|
||||
logger.warning(f'exception running find-generic-password: {error_to_str(e)}')
|
||||
@ -987,6 +991,97 @@ def _parse_browser_specification(browser_name, profile=None, keyring=None, conta
|
||||
raise ValueError(f'unsupported browser: "{browser_name}"')
|
||||
if keyring not in (None, *SUPPORTED_KEYRINGS):
|
||||
raise ValueError(f'unsupported keyring: "{keyring}"')
|
||||
if profile is not None and _is_path(profile):
|
||||
profile = os.path.expanduser(profile)
|
||||
if profile is not None and _is_path(expand_path(profile)):
|
||||
profile = expand_path(profile)
|
||||
return browser_name, profile, keyring, container
|
||||
|
||||
|
||||
class LenientSimpleCookie(http.cookies.SimpleCookie):
|
||||
"""More lenient version of http.cookies.SimpleCookie"""
|
||||
# From https://github.com/python/cpython/blob/v3.10.7/Lib/http/cookies.py
|
||||
# We use Morsel's legal key chars to avoid errors on setting values
|
||||
_LEGAL_KEY_CHARS = r'\w\d' + re.escape('!#$%&\'*+-.:^_`|~')
|
||||
_LEGAL_VALUE_CHARS = _LEGAL_KEY_CHARS + re.escape('(),/<=>?@[]{}')
|
||||
|
||||
_RESERVED = {
|
||||
"expires",
|
||||
"path",
|
||||
"comment",
|
||||
"domain",
|
||||
"max-age",
|
||||
"secure",
|
||||
"httponly",
|
||||
"version",
|
||||
"samesite",
|
||||
}
|
||||
|
||||
_FLAGS = {"secure", "httponly"}
|
||||
|
||||
# Added 'bad' group to catch the remaining value
|
||||
_COOKIE_PATTERN = re.compile(r"""
|
||||
\s* # Optional whitespace at start of cookie
|
||||
(?P<key> # Start of group 'key'
|
||||
[""" + _LEGAL_KEY_CHARS + r"""]+?# Any word of at least one letter
|
||||
) # End of group 'key'
|
||||
( # Optional group: there may not be a value.
|
||||
\s*=\s* # Equal Sign
|
||||
( # Start of potential value
|
||||
(?P<val> # Start of group 'val'
|
||||
"(?:[^\\"]|\\.)*" # Any doublequoted string
|
||||
| # or
|
||||
\w{3},\s[\w\d\s-]{9,11}\s[\d:]{8}\sGMT # Special case for "expires" attr
|
||||
| # or
|
||||
[""" + _LEGAL_VALUE_CHARS + r"""]* # Any word or empty string
|
||||
) # End of group 'val'
|
||||
| # or
|
||||
(?P<bad>(?:\\;|[^;])*?) # 'bad' group fallback for invalid values
|
||||
) # End of potential value
|
||||
)? # End of optional value group
|
||||
\s* # Any number of spaces.
|
||||
(\s+|;|$) # Ending either at space, semicolon, or EOS.
|
||||
""", re.ASCII | re.VERBOSE)
|
||||
|
||||
def load(self, data):
|
||||
# Workaround for https://github.com/yt-dlp/yt-dlp/issues/4776
|
||||
if not isinstance(data, str):
|
||||
return super().load(data)
|
||||
|
||||
morsel = None
|
||||
for match in self._COOKIE_PATTERN.finditer(data):
|
||||
if match.group('bad'):
|
||||
morsel = None
|
||||
continue
|
||||
|
||||
key, value = match.group('key', 'val')
|
||||
|
||||
is_attribute = False
|
||||
if key.startswith('$'):
|
||||
key = key[1:]
|
||||
is_attribute = True
|
||||
|
||||
lower_key = key.lower()
|
||||
if lower_key in self._RESERVED:
|
||||
if morsel is None:
|
||||
continue
|
||||
|
||||
if value is None:
|
||||
if lower_key not in self._FLAGS:
|
||||
morsel = None
|
||||
continue
|
||||
value = True
|
||||
else:
|
||||
value, _ = self.value_decode(value)
|
||||
|
||||
morsel[key] = value
|
||||
|
||||
elif is_attribute:
|
||||
morsel = None
|
||||
|
||||
elif value is not None:
|
||||
morsel = self.get(key, http.cookies.Morsel())
|
||||
real_value, coded_value = self.value_decode(value)
|
||||
morsel.set(key, real_value, coded_value)
|
||||
self[key] = morsel
|
||||
|
||||
else:
|
||||
morsel = None
|
||||
|
36
yt_dlp/dependencies/Cryptodome.py
Normal file
36
yt_dlp/dependencies/Cryptodome.py
Normal file
@ -0,0 +1,36 @@
|
||||
import types
|
||||
|
||||
try:
|
||||
import Cryptodome as _parent
|
||||
except ImportError:
|
||||
try:
|
||||
import Crypto as _parent
|
||||
except (ImportError, SyntaxError): # Old Crypto gives SyntaxError in newer Python
|
||||
_parent = types.ModuleType('no_Cryptodome')
|
||||
__bool__ = lambda: False
|
||||
|
||||
__version__ = ''
|
||||
AES = PKCS1_v1_5 = Blowfish = PKCS1_OAEP = SHA1 = CMAC = RSA = None
|
||||
try:
|
||||
if _parent.__name__ == 'Cryptodome':
|
||||
from Cryptodome import __version__
|
||||
from Cryptodome.Cipher import AES, PKCS1_OAEP, Blowfish, PKCS1_v1_5
|
||||
from Cryptodome.Hash import CMAC, SHA1
|
||||
from Cryptodome.PublicKey import RSA
|
||||
elif _parent.__name__ == 'Crypto':
|
||||
from Crypto import __version__
|
||||
from Crypto.Cipher import AES, PKCS1_OAEP, Blowfish, PKCS1_v1_5 # noqa: F401
|
||||
from Crypto.Hash import CMAC, SHA1 # noqa: F401
|
||||
from Crypto.PublicKey import RSA # noqa: F401
|
||||
except ImportError:
|
||||
__version__ = f'broken {__version__}'.strip()
|
||||
|
||||
|
||||
_yt_dlp__identifier = _parent.__name__
|
||||
if AES and _yt_dlp__identifier == 'Crypto':
|
||||
try:
|
||||
# In pycrypto, mode defaults to ECB. See:
|
||||
# https://www.pycryptodome.org/en/latest/src/vs_pycrypto.html#:~:text=not%20have%20ECB%20as%20default%20mode
|
||||
AES.new(b'abcdefghijklmnop')
|
||||
except TypeError:
|
||||
_yt_dlp__identifier = 'pycrypto'
|
@ -23,24 +23,6 @@
|
||||
certifi = None
|
||||
|
||||
|
||||
try:
|
||||
from Cryptodome.Cipher import AES as Cryptodome_AES
|
||||
except ImportError:
|
||||
try:
|
||||
from Crypto.Cipher import AES as Cryptodome_AES
|
||||
except (ImportError, SyntaxError): # Old Crypto gives SyntaxError in newer Python
|
||||
Cryptodome_AES = None
|
||||
else:
|
||||
try:
|
||||
# In pycrypto, mode defaults to ECB. See:
|
||||
# https://www.pycryptodome.org/en/latest/src/vs_pycrypto.html#:~:text=not%20have%20ECB%20as%20default%20mode
|
||||
Cryptodome_AES.new(b'abcdefghijklmnop')
|
||||
except TypeError:
|
||||
pass
|
||||
else:
|
||||
Cryptodome_AES._yt_dlp__identifier = 'pycrypto'
|
||||
|
||||
|
||||
try:
|
||||
import mutagen
|
||||
except ImportError:
|
||||
@ -84,12 +66,16 @@
|
||||
xattr._yt_dlp__identifier = 'pyxattr'
|
||||
|
||||
|
||||
from . import Cryptodome
|
||||
|
||||
all_dependencies = {k: v for k, v in globals().items() if not k.startswith('_')}
|
||||
|
||||
|
||||
available_dependencies = {k: v for k, v in all_dependencies.items() if v}
|
||||
|
||||
|
||||
# Deprecated
|
||||
Cryptodome_AES = Cryptodome.AES
|
||||
|
||||
|
||||
__all__ = [
|
||||
'all_dependencies',
|
||||
'available_dependencies',
|
@ -15,15 +15,17 @@
|
||||
from ..utils import (
|
||||
IDENTITY,
|
||||
NO_DEFAULT,
|
||||
NUMBER_RE,
|
||||
LockingUnsupportedError,
|
||||
Namespace,
|
||||
RetryManager,
|
||||
classproperty,
|
||||
decodeArgument,
|
||||
deprecation_warning,
|
||||
encodeFilename,
|
||||
format_bytes,
|
||||
join_nonempty,
|
||||
parse_bytes,
|
||||
remove_start,
|
||||
sanitize_open,
|
||||
shell_quote,
|
||||
timeconvert,
|
||||
@ -120,11 +122,11 @@ def format_seconds(seconds):
|
||||
time = timetuple_from_msec(seconds * 1000)
|
||||
if time.hours > 99:
|
||||
return '--:--:--'
|
||||
if not time.hours:
|
||||
return ' %02d:%02d' % time[1:-1]
|
||||
return '%02d:%02d:%02d' % time[:-1]
|
||||
|
||||
format_eta = format_seconds
|
||||
@classmethod
|
||||
def format_eta(cls, seconds):
|
||||
return f'{remove_start(cls.format_seconds(seconds), "00:"):>8s}'
|
||||
|
||||
@staticmethod
|
||||
def calc_percent(byte_counter, data_len):
|
||||
@ -179,12 +181,9 @@ def best_block_size(elapsed_time, bytes):
|
||||
@staticmethod
|
||||
def parse_bytes(bytestr):
|
||||
"""Parse a string indicating a byte quantity into an integer."""
|
||||
matchobj = re.match(rf'(?i)^({NUMBER_RE})([kMGTPEZY]?)$', bytestr)
|
||||
if matchobj is None:
|
||||
return None
|
||||
number = float(matchobj.group(1))
|
||||
multiplier = 1024.0 ** 'bkmgtpezy'.index(matchobj.group(2).lower())
|
||||
return int(round(number * multiplier))
|
||||
deprecation_warning('yt_dlp.FileDownloader.parse_bytes is deprecated and '
|
||||
'may be removed in the future. Use yt_dlp.utils.parse_bytes instead')
|
||||
return parse_bytes(bytestr)
|
||||
|
||||
def slow_down(self, start_time, now, byte_counter):
|
||||
"""Sleep if the download speed is over the rate limit."""
|
||||
@ -332,6 +331,8 @@ def with_fields(*tups, default=''):
|
||||
return tmpl
|
||||
return default
|
||||
|
||||
_format_bytes = lambda k: f'{format_bytes(s.get(k)):>10s}'
|
||||
|
||||
if s['status'] == 'finished':
|
||||
if self.params.get('noprogress'):
|
||||
self.to_screen('[download] Download completed')
|
||||
@ -339,7 +340,7 @@ def with_fields(*tups, default=''):
|
||||
s.update({
|
||||
'speed': speed,
|
||||
'_speed_str': self.format_speed(speed).strip(),
|
||||
'_total_bytes_str': format_bytes(s.get('total_bytes')),
|
||||
'_total_bytes_str': _format_bytes('total_bytes'),
|
||||
'_elapsed_str': self.format_seconds(s.get('elapsed')),
|
||||
'_percent_str': self.format_percent(100),
|
||||
})
|
||||
@ -354,15 +355,15 @@ def with_fields(*tups, default=''):
|
||||
return
|
||||
|
||||
s.update({
|
||||
'_eta_str': self.format_eta(s.get('eta')),
|
||||
'_eta_str': self.format_eta(s.get('eta')).strip(),
|
||||
'_speed_str': self.format_speed(s.get('speed')),
|
||||
'_percent_str': self.format_percent(try_call(
|
||||
lambda: 100 * s['downloaded_bytes'] / s['total_bytes'],
|
||||
lambda: 100 * s['downloaded_bytes'] / s['total_bytes_estimate'],
|
||||
lambda: s['downloaded_bytes'] == 0 and 0)),
|
||||
'_total_bytes_str': format_bytes(s.get('total_bytes')),
|
||||
'_total_bytes_estimate_str': format_bytes(s.get('total_bytes_estimate')),
|
||||
'_downloaded_bytes_str': format_bytes(s.get('downloaded_bytes')),
|
||||
'_total_bytes_str': _format_bytes('total_bytes'),
|
||||
'_total_bytes_estimate_str': _format_bytes('total_bytes_estimate'),
|
||||
'_downloaded_bytes_str': _format_bytes('downloaded_bytes'),
|
||||
'_elapsed_str': self.format_seconds(s.get('elapsed')),
|
||||
})
|
||||
|
||||
|
@ -1,8 +1,9 @@
|
||||
import time
|
||||
import urllib.parse
|
||||
|
||||
from . import get_suitable_downloader
|
||||
from .fragment import FragmentFD
|
||||
from ..utils import urljoin
|
||||
from ..utils import update_url_query, urljoin
|
||||
|
||||
|
||||
class DashSegmentsFD(FragmentFD):
|
||||
@ -40,7 +41,12 @@ def real_download(self, filename, info_dict):
|
||||
self._prepare_and_start_frag_download(ctx, fmt)
|
||||
ctx['start'] = real_start
|
||||
|
||||
fragments_to_download = self._get_fragments(fmt, ctx)
|
||||
extra_query = None
|
||||
extra_param_to_segment_url = info_dict.get('extra_param_to_segment_url')
|
||||
if extra_param_to_segment_url:
|
||||
extra_query = urllib.parse.parse_qs(extra_param_to_segment_url)
|
||||
|
||||
fragments_to_download = self._get_fragments(fmt, ctx, extra_query)
|
||||
|
||||
if real_downloader:
|
||||
self.to_screen(
|
||||
@ -51,13 +57,13 @@ def real_download(self, filename, info_dict):
|
||||
|
||||
args.append([ctx, fragments_to_download, fmt])
|
||||
|
||||
return self.download_and_append_fragments_multiple(*args)
|
||||
return self.download_and_append_fragments_multiple(*args, is_fatal=lambda idx: idx == 0)
|
||||
|
||||
def _resolve_fragments(self, fragments, ctx):
|
||||
fragments = fragments(ctx) if callable(fragments) else fragments
|
||||
return [next(iter(fragments))] if self.params.get('test') else fragments
|
||||
|
||||
def _get_fragments(self, fmt, ctx):
|
||||
def _get_fragments(self, fmt, ctx, extra_query):
|
||||
fragment_base_url = fmt.get('fragment_base_url')
|
||||
fragments = self._resolve_fragments(fmt['fragments'], ctx)
|
||||
|
||||
@ -70,6 +76,8 @@ def _get_fragments(self, fmt, ctx):
|
||||
if not fragment_url:
|
||||
assert fragment_base_url
|
||||
fragment_url = urljoin(fragment_base_url, fragment['path'])
|
||||
if extra_query:
|
||||
fragment_url = update_url_query(fragment_url, extra_query)
|
||||
|
||||
yield {
|
||||
'frag_index': frag_index,
|
||||
|
@ -1,9 +1,11 @@
|
||||
import enum
|
||||
import json
|
||||
import os.path
|
||||
import re
|
||||
import subprocess
|
||||
import sys
|
||||
import time
|
||||
import uuid
|
||||
|
||||
from .fragment import FragmentFD
|
||||
from ..compat import functools
|
||||
@ -20,8 +22,10 @@
|
||||
determine_ext,
|
||||
encodeArgument,
|
||||
encodeFilename,
|
||||
find_available_port,
|
||||
handle_youtubedl_headers,
|
||||
remove_end,
|
||||
sanitized_Request,
|
||||
traverse_obj,
|
||||
)
|
||||
|
||||
@ -60,7 +64,6 @@ def real_download(self, filename, info_dict):
|
||||
}
|
||||
if filename != '-':
|
||||
fsize = os.path.getsize(encodeFilename(tmpfilename))
|
||||
self.to_screen(f'\r[{self.get_basename()}] Downloaded {fsize} bytes')
|
||||
self.try_rename(tmpfilename, filename)
|
||||
status.update({
|
||||
'downloaded_bytes': fsize,
|
||||
@ -101,6 +104,7 @@ def supports(cls, info_dict):
|
||||
return all((
|
||||
not info_dict.get('to_stdout') or Features.TO_STDOUT in cls.SUPPORTED_FEATURES,
|
||||
'+' not in info_dict['protocol'] or Features.MULTIPLE_FORMATS in cls.SUPPORTED_FEATURES,
|
||||
not traverse_obj(info_dict, ('hls_aes', ...), 'extra_param_to_segment_url'),
|
||||
all(proto in cls.SUPPORTED_PROTOCOLS for proto in info_dict['protocol'].split('+')),
|
||||
))
|
||||
|
||||
@ -129,8 +133,7 @@ def _call_downloader(self, tmpfilename, info_dict):
|
||||
self._debug_cmd(cmd)
|
||||
|
||||
if 'fragments' not in info_dict:
|
||||
_, stderr, returncode = Popen.run(
|
||||
cmd, text=True, stderr=subprocess.PIPE if self._CAPTURE_STDERR else None)
|
||||
_, stderr, returncode = self._call_process(cmd, info_dict)
|
||||
if returncode and stderr:
|
||||
self.to_stderr(stderr)
|
||||
return returncode
|
||||
@ -140,7 +143,7 @@ def _call_downloader(self, tmpfilename, info_dict):
|
||||
retry_manager = RetryManager(self.params.get('fragment_retries'), self.report_retry,
|
||||
frag_index=None, fatal=not skip_unavailable_fragments)
|
||||
for retry in retry_manager:
|
||||
_, stderr, returncode = Popen.run(cmd, text=True, stderr=subprocess.PIPE)
|
||||
_, stderr, returncode = self._call_process(cmd, info_dict)
|
||||
if not returncode:
|
||||
break
|
||||
# TODO: Decide whether to retry based on error code
|
||||
@ -172,6 +175,9 @@ def _call_downloader(self, tmpfilename, info_dict):
|
||||
self.try_remove(encodeFilename('%s.frag.urls' % tmpfilename))
|
||||
return 0
|
||||
|
||||
def _call_process(self, cmd, info_dict):
|
||||
return Popen.run(cmd, text=True, stderr=subprocess.PIPE)
|
||||
|
||||
|
||||
class CurlFD(ExternalFD):
|
||||
AVAILABLE_OPT = '-V'
|
||||
@ -252,6 +258,19 @@ def supports_manifest(manifest):
|
||||
check_results = (not re.search(feature, manifest) for feature in UNSUPPORTED_FEATURES)
|
||||
return all(check_results)
|
||||
|
||||
@staticmethod
|
||||
def _aria2c_filename(fn):
|
||||
return fn if os.path.isabs(fn) else f'.{os.path.sep}{fn}'
|
||||
|
||||
def _call_downloader(self, tmpfilename, info_dict):
|
||||
# FIXME: Disabled due to https://github.com/yt-dlp/yt-dlp/issues/5931
|
||||
if False and 'no-external-downloader-progress' not in self.params.get('compat_opts', []):
|
||||
info_dict['__rpc'] = {
|
||||
'port': find_available_port() or 19190,
|
||||
'secret': str(uuid.uuid4()),
|
||||
}
|
||||
return super()._call_downloader(tmpfilename, info_dict)
|
||||
|
||||
def _make_cmd(self, tmpfilename, info_dict):
|
||||
cmd = [self.exe, '-c',
|
||||
'--console-log-level=warn', '--summary-interval=0', '--download-result=hide',
|
||||
@ -272,6 +291,12 @@ def _make_cmd(self, tmpfilename, info_dict):
|
||||
cmd += self._bool_option('--show-console-readout', 'noprogress', 'false', 'true', '=')
|
||||
cmd += self._configuration_args()
|
||||
|
||||
if '__rpc' in info_dict:
|
||||
cmd += [
|
||||
'--enable-rpc',
|
||||
f'--rpc-listen-port={info_dict["__rpc"]["port"]}',
|
||||
f'--rpc-secret={info_dict["__rpc"]["secret"]}']
|
||||
|
||||
# aria2c strips out spaces from the beginning/end of filenames and paths.
|
||||
# We work around this issue by adding a "./" to the beginning of the
|
||||
# filename and relative path, and adding a "/" at the end of the path.
|
||||
@ -280,11 +305,9 @@ def _make_cmd(self, tmpfilename, info_dict):
|
||||
# https://github.com/aria2/aria2/issues/1373
|
||||
dn = os.path.dirname(tmpfilename)
|
||||
if dn:
|
||||
if not os.path.isabs(dn):
|
||||
dn = f'.{os.path.sep}{dn}'
|
||||
cmd += ['--dir', dn + os.path.sep]
|
||||
cmd += ['--dir', self._aria2c_filename(dn) + os.path.sep]
|
||||
if 'fragments' not in info_dict:
|
||||
cmd += ['--out', f'.{os.path.sep}{os.path.basename(tmpfilename)}']
|
||||
cmd += ['--out', self._aria2c_filename(os.path.basename(tmpfilename))]
|
||||
cmd += ['--auto-file-renaming=false']
|
||||
|
||||
if 'fragments' in info_dict:
|
||||
@ -293,15 +316,97 @@ def _make_cmd(self, tmpfilename, info_dict):
|
||||
url_list = []
|
||||
for frag_index, fragment in enumerate(info_dict['fragments']):
|
||||
fragment_filename = '%s-Frag%d' % (os.path.basename(tmpfilename), frag_index)
|
||||
url_list.append('%s\n\tout=%s' % (fragment['url'], fragment_filename))
|
||||
url_list.append('%s\n\tout=%s' % (fragment['url'], self._aria2c_filename(fragment_filename)))
|
||||
stream, _ = self.sanitize_open(url_list_file, 'wb')
|
||||
stream.write('\n'.join(url_list).encode())
|
||||
stream.close()
|
||||
cmd += ['-i', url_list_file]
|
||||
cmd += ['-i', self._aria2c_filename(url_list_file)]
|
||||
else:
|
||||
cmd += ['--', info_dict['url']]
|
||||
return cmd
|
||||
|
||||
def aria2c_rpc(self, rpc_port, rpc_secret, method, params=()):
|
||||
# Does not actually need to be UUID, just unique
|
||||
sanitycheck = str(uuid.uuid4())
|
||||
d = json.dumps({
|
||||
'jsonrpc': '2.0',
|
||||
'id': sanitycheck,
|
||||
'method': method,
|
||||
'params': [f'token:{rpc_secret}', *params],
|
||||
}).encode('utf-8')
|
||||
request = sanitized_Request(
|
||||
f'http://localhost:{rpc_port}/jsonrpc',
|
||||
data=d, headers={
|
||||
'Content-Type': 'application/json',
|
||||
'Content-Length': f'{len(d)}',
|
||||
'Ytdl-request-proxy': '__noproxy__',
|
||||
})
|
||||
with self.ydl.urlopen(request) as r:
|
||||
resp = json.load(r)
|
||||
assert resp.get('id') == sanitycheck, 'Something went wrong with RPC server'
|
||||
return resp['result']
|
||||
|
||||
def _call_process(self, cmd, info_dict):
|
||||
if '__rpc' not in info_dict:
|
||||
return super()._call_process(cmd, info_dict)
|
||||
|
||||
send_rpc = functools.partial(self.aria2c_rpc, info_dict['__rpc']['port'], info_dict['__rpc']['secret'])
|
||||
started = time.time()
|
||||
|
||||
fragmented = 'fragments' in info_dict
|
||||
frag_count = len(info_dict['fragments']) if fragmented else 1
|
||||
status = {
|
||||
'filename': info_dict.get('_filename'),
|
||||
'status': 'downloading',
|
||||
'elapsed': 0,
|
||||
'downloaded_bytes': 0,
|
||||
'fragment_count': frag_count if fragmented else None,
|
||||
'fragment_index': 0 if fragmented else None,
|
||||
}
|
||||
self._hook_progress(status, info_dict)
|
||||
|
||||
def get_stat(key, *obj, average=False):
|
||||
val = tuple(filter(None, map(float, traverse_obj(obj, (..., ..., key))))) or [0]
|
||||
return sum(val) / (len(val) if average else 1)
|
||||
|
||||
with Popen(cmd, text=True, stdout=subprocess.DEVNULL, stderr=subprocess.PIPE) as p:
|
||||
# Add a small sleep so that RPC client can receive response,
|
||||
# or the connection stalls infinitely
|
||||
time.sleep(0.2)
|
||||
retval = p.poll()
|
||||
while retval is None:
|
||||
# We don't use tellStatus as we won't know the GID without reading stdout
|
||||
# Ref: https://aria2.github.io/manual/en/html/aria2c.html#aria2.tellActive
|
||||
active = send_rpc('aria2.tellActive')
|
||||
completed = send_rpc('aria2.tellStopped', [0, frag_count])
|
||||
|
||||
downloaded = get_stat('totalLength', completed) + get_stat('completedLength', active)
|
||||
speed = get_stat('downloadSpeed', active)
|
||||
total = frag_count * get_stat('totalLength', active, completed, average=True)
|
||||
if total < downloaded:
|
||||
total = None
|
||||
|
||||
status.update({
|
||||
'downloaded_bytes': int(downloaded),
|
||||
'speed': speed,
|
||||
'total_bytes': None if fragmented else total,
|
||||
'total_bytes_estimate': total,
|
||||
'eta': (total - downloaded) / (speed or 1),
|
||||
'fragment_index': min(frag_count, len(completed) + 1) if fragmented else None,
|
||||
'elapsed': time.time() - started
|
||||
})
|
||||
self._hook_progress(status, info_dict)
|
||||
|
||||
if not active and len(completed) >= frag_count:
|
||||
send_rpc('aria2.shutdown')
|
||||
retval = p.wait()
|
||||
break
|
||||
|
||||
time.sleep(0.1)
|
||||
retval = p.poll()
|
||||
|
||||
return '', p.stderr.read(), retval
|
||||
|
||||
|
||||
class HttpieFD(ExternalFD):
|
||||
AVAILABLE_OPT = '--version'
|
||||
@ -340,7 +445,6 @@ def can_merge_formats(cls, info_dict, params):
|
||||
and cls.can_download(info_dict))
|
||||
|
||||
def _call_downloader(self, tmpfilename, info_dict):
|
||||
urls = [f['url'] for f in info_dict.get('requested_formats', [])] or [info_dict['url']]
|
||||
ffpp = FFmpegPostProcessor(downloader=self)
|
||||
if not ffpp.available:
|
||||
self.report_error('m3u8 download detected but ffmpeg could not be found. Please install')
|
||||
@ -370,16 +474,6 @@ def _call_downloader(self, tmpfilename, info_dict):
|
||||
# http://trac.ffmpeg.org/ticket/6125#comment:10
|
||||
args += ['-seekable', '1' if seekable else '0']
|
||||
|
||||
http_headers = None
|
||||
if info_dict.get('http_headers'):
|
||||
youtubedl_headers = handle_youtubedl_headers(info_dict['http_headers'])
|
||||
http_headers = [
|
||||
# Trailing \r\n after each HTTP header is important to prevent warning from ffmpeg/avconv:
|
||||
# [http @ 00000000003d2fa0] No trailing CRLF found in HTTP header.
|
||||
'-headers',
|
||||
''.join(f'{key}: {val}\r\n' for key, val in youtubedl_headers.items())
|
||||
]
|
||||
|
||||
env = None
|
||||
proxy = self.params.get('proxy')
|
||||
if proxy:
|
||||
@ -432,21 +526,26 @@ def _call_downloader(self, tmpfilename, info_dict):
|
||||
|
||||
start_time, end_time = info_dict.get('section_start') or 0, info_dict.get('section_end')
|
||||
|
||||
for i, url in enumerate(urls):
|
||||
if http_headers is not None and re.match(r'^https?://', url):
|
||||
args += http_headers
|
||||
selected_formats = info_dict.get('requested_formats') or [info_dict]
|
||||
for i, fmt in enumerate(selected_formats):
|
||||
if fmt.get('http_headers') and re.match(r'^https?://', fmt['url']):
|
||||
headers_dict = handle_youtubedl_headers(fmt['http_headers'])
|
||||
# Trailing \r\n after each HTTP header is important to prevent warning from ffmpeg/avconv:
|
||||
# [http @ 00000000003d2fa0] No trailing CRLF found in HTTP header.
|
||||
args.extend(['-headers', ''.join(f'{key}: {val}\r\n' for key, val in headers_dict.items())])
|
||||
|
||||
if start_time:
|
||||
args += ['-ss', str(start_time)]
|
||||
if end_time:
|
||||
args += ['-t', str(end_time - start_time)]
|
||||
|
||||
args += self._configuration_args((f'_i{i + 1}', '_i')) + ['-i', url]
|
||||
args += self._configuration_args((f'_i{i + 1}', '_i')) + ['-i', fmt['url']]
|
||||
|
||||
if not (start_time or end_time) or not self.params.get('force_keyframes_at_cuts'):
|
||||
args += ['-c', 'copy']
|
||||
|
||||
if info_dict.get('requested_formats') or protocol == 'http_dash_segments':
|
||||
for (i, fmt) in enumerate(info_dict.get('requested_formats') or [info_dict]):
|
||||
for i, fmt in enumerate(selected_formats):
|
||||
stream_number = fmt.get('manifest_stream_number', 0)
|
||||
args.extend(['-map', f'{i}:{stream_number}'])
|
||||
|
||||
@ -486,8 +585,9 @@ def _call_downloader(self, tmpfilename, info_dict):
|
||||
args.append(encodeFilename(ffpp._ffmpeg_filename_argument(tmpfilename), True))
|
||||
self._debug_cmd(args)
|
||||
|
||||
piped = any(fmt['url'] in ('-', 'pipe:') for fmt in selected_formats)
|
||||
with Popen(args, stdin=subprocess.PIPE, env=env) as proc:
|
||||
if url in ('-', 'pipe:'):
|
||||
if piped:
|
||||
self.on_process_started(proc, proc.stdin)
|
||||
try:
|
||||
retval = proc.wait()
|
||||
@ -497,7 +597,7 @@ def _call_downloader(self, tmpfilename, info_dict):
|
||||
# produces a file that is playable (this is mostly useful for live
|
||||
# streams). Note that Windows is not affected and produces playable
|
||||
# files (see https://github.com/ytdl-org/youtube-dl/issues/8300).
|
||||
if isinstance(e, KeyboardInterrupt) and sys.platform != 'win32' and url not in ('-', 'pipe:'):
|
||||
if isinstance(e, KeyboardInterrupt) and sys.platform != 'win32' and not piped:
|
||||
proc.communicate_or_kill(b'q')
|
||||
else:
|
||||
proc.kill(timeout=None)
|
||||
|
@ -424,6 +424,4 @@ def real_download(self, filename, info_dict):
|
||||
msg = 'Missed %d fragments' % (fragments_list[0][1] - (frag_i + 1))
|
||||
self.report_warning(msg)
|
||||
|
||||
self._finish_frag_download(ctx, info_dict)
|
||||
|
||||
return True
|
||||
return self._finish_frag_download(ctx, info_dict)
|
||||
|
@ -295,16 +295,23 @@ def _finish_frag_download(self, ctx, info_dict):
|
||||
self.try_remove(ytdl_filename)
|
||||
elapsed = time.time() - ctx['started']
|
||||
|
||||
if ctx['tmpfilename'] == '-':
|
||||
downloaded_bytes = ctx['complete_frags_downloaded_bytes']
|
||||
to_file = ctx['tmpfilename'] != '-'
|
||||
if to_file:
|
||||
downloaded_bytes = os.path.getsize(encodeFilename(ctx['tmpfilename']))
|
||||
else:
|
||||
downloaded_bytes = ctx['complete_frags_downloaded_bytes']
|
||||
|
||||
if not downloaded_bytes:
|
||||
if to_file:
|
||||
self.try_remove(ctx['tmpfilename'])
|
||||
self.report_error('The downloaded file is empty')
|
||||
return False
|
||||
elif to_file:
|
||||
self.try_rename(ctx['tmpfilename'], ctx['filename'])
|
||||
if self.params.get('updatetime', True):
|
||||
filetime = ctx.get('fragment_filetime')
|
||||
if filetime:
|
||||
with contextlib.suppress(Exception):
|
||||
os.utime(ctx['filename'], (time.time(), filetime))
|
||||
downloaded_bytes = os.path.getsize(encodeFilename(ctx['filename']))
|
||||
filetime = ctx.get('fragment_filetime')
|
||||
if self.params.get('updatetime', True) and filetime:
|
||||
with contextlib.suppress(Exception):
|
||||
os.utime(ctx['filename'], (time.time(), filetime))
|
||||
|
||||
self._hook_progress({
|
||||
'downloaded_bytes': downloaded_bytes,
|
||||
@ -316,6 +323,7 @@ def _finish_frag_download(self, ctx, info_dict):
|
||||
'max_progress': ctx.get('max_progress'),
|
||||
'progress_idx': ctx.get('progress_idx'),
|
||||
}, info_dict)
|
||||
return True
|
||||
|
||||
def _prepare_external_frag_download(self, ctx):
|
||||
if 'live' not in ctx:
|
||||
@ -352,7 +360,8 @@ def decrypt_fragment(fragment, frag_content):
|
||||
if not decrypt_info or decrypt_info['METHOD'] != 'AES-128':
|
||||
return frag_content
|
||||
iv = decrypt_info.get('IV') or struct.pack('>8xq', fragment['media_sequence'])
|
||||
decrypt_info['KEY'] = decrypt_info.get('KEY') or _get_key(info_dict.get('_decryption_key_url') or decrypt_info['URI'])
|
||||
decrypt_info['KEY'] = (decrypt_info.get('KEY')
|
||||
or _get_key(traverse_obj(info_dict, ('hls_aes', 'uri')) or decrypt_info['URI']))
|
||||
# Don't decrypt the content in tests since the data is explicitly truncated and it's not to a valid block
|
||||
# size (see https://github.com/ytdl-org/youtube-dl/pull/27660). Tests only care that the correct data downloaded,
|
||||
# not what it decrypts to.
|
||||
@ -362,7 +371,7 @@ def decrypt_fragment(fragment, frag_content):
|
||||
|
||||
return decrypt_fragment
|
||||
|
||||
def download_and_append_fragments_multiple(self, *args, pack_func=None, finish_func=None):
|
||||
def download_and_append_fragments_multiple(self, *args, **kwargs):
|
||||
'''
|
||||
@params (ctx1, fragments1, info_dict1), (ctx2, fragments2, info_dict2), ...
|
||||
all args must be either tuple or list
|
||||
@ -370,18 +379,17 @@ def download_and_append_fragments_multiple(self, *args, pack_func=None, finish_f
|
||||
interrupt_trigger = [True]
|
||||
max_progress = len(args)
|
||||
if max_progress == 1:
|
||||
return self.download_and_append_fragments(*args[0], pack_func=pack_func, finish_func=finish_func)
|
||||
return self.download_and_append_fragments(*args[0], **kwargs)
|
||||
max_workers = self.params.get('concurrent_fragment_downloads', 1)
|
||||
if max_progress > 1:
|
||||
self._prepare_multiline_status(max_progress)
|
||||
is_live = any(traverse_obj(args, (..., 2, 'is_live'), default=[]))
|
||||
is_live = any(traverse_obj(args, (..., 2, 'is_live')))
|
||||
|
||||
def thread_func(idx, ctx, fragments, info_dict, tpe):
|
||||
ctx['max_progress'] = max_progress
|
||||
ctx['progress_idx'] = idx
|
||||
return self.download_and_append_fragments(
|
||||
ctx, fragments, info_dict, pack_func=pack_func, finish_func=finish_func,
|
||||
tpe=tpe, interrupt_trigger=interrupt_trigger)
|
||||
ctx, fragments, info_dict, **kwargs, tpe=tpe, interrupt_trigger=interrupt_trigger)
|
||||
|
||||
class FTPE(concurrent.futures.ThreadPoolExecutor):
|
||||
# has to stop this or it's going to wait on the worker thread itself
|
||||
@ -428,17 +436,12 @@ def interrupt_trigger_iter(fg):
|
||||
return result
|
||||
|
||||
def download_and_append_fragments(
|
||||
self, ctx, fragments, info_dict, *, pack_func=None, finish_func=None,
|
||||
tpe=None, interrupt_trigger=None):
|
||||
if not interrupt_trigger:
|
||||
interrupt_trigger = (True, )
|
||||
self, ctx, fragments, info_dict, *, is_fatal=(lambda idx: False),
|
||||
pack_func=(lambda content, idx: content), finish_func=None,
|
||||
tpe=None, interrupt_trigger=(True, )):
|
||||
|
||||
is_fatal = (
|
||||
((lambda _: False) if info_dict.get('is_live') else (lambda idx: idx == 0))
|
||||
if self.params.get('skip_unavailable_fragments', True) else (lambda _: True))
|
||||
|
||||
if not pack_func:
|
||||
pack_func = lambda frag_content, _: frag_content
|
||||
if not self.params.get('skip_unavailable_fragments', True):
|
||||
is_fatal = lambda _: True
|
||||
|
||||
def download_fragment(fragment, ctx):
|
||||
if not interrupt_trigger[0]:
|
||||
@ -463,7 +466,8 @@ def error_callback(err, count, retries):
|
||||
for retry in RetryManager(self.params.get('fragment_retries'), error_callback):
|
||||
try:
|
||||
ctx['fragment_count'] = fragment.get('fragment_count')
|
||||
if not self._download_fragment(ctx, fragment['url'], info_dict, headers):
|
||||
if not self._download_fragment(
|
||||
ctx, fragment['url'], info_dict, headers, info_dict.get('request_data')):
|
||||
return
|
||||
except (urllib.error.HTTPError, http.client.IncompleteRead) as err:
|
||||
retry.error = err
|
||||
@ -493,7 +497,7 @@ def _download_fragment(fragment):
|
||||
download_fragment(fragment, ctx_copy)
|
||||
return fragment, fragment['frag_index'], ctx_copy.get('fragment_filename_sanitized')
|
||||
|
||||
self.report_warning('The download speed shown is only of one thread. This is a known issue and patches are welcome')
|
||||
self.report_warning('The download speed shown is only of one thread. This is a known issue')
|
||||
with tpe or concurrent.futures.ThreadPoolExecutor(max_workers) as pool:
|
||||
try:
|
||||
for fragment, frag_index, frag_filename in pool.map(_download_fragment, fragments):
|
||||
@ -527,5 +531,4 @@ def _download_fragment(fragment):
|
||||
if finish_func is not None:
|
||||
ctx['dest_stream'].write(finish_func())
|
||||
ctx['dest_stream'].flush()
|
||||
self._finish_frag_download(ctx, info_dict)
|
||||
return True
|
||||
return self._finish_frag_download(ctx, info_dict)
|
||||
|
@ -7,8 +7,15 @@
|
||||
from .external import FFmpegFD
|
||||
from .fragment import FragmentFD
|
||||
from .. import webvtt
|
||||
from ..dependencies import Cryptodome_AES
|
||||
from ..utils import bug_reports_message, parse_m3u8_attributes, update_url_query
|
||||
from ..dependencies import Cryptodome
|
||||
from ..utils import (
|
||||
bug_reports_message,
|
||||
parse_m3u8_attributes,
|
||||
remove_start,
|
||||
traverse_obj,
|
||||
update_url_query,
|
||||
urljoin,
|
||||
)
|
||||
|
||||
|
||||
class HlsFD(FragmentFD):
|
||||
@ -63,7 +70,7 @@ def real_download(self, filename, info_dict):
|
||||
can_download, message = self.can_download(s, info_dict, self.params.get('allow_unplayable_formats')), None
|
||||
if can_download:
|
||||
has_ffmpeg = FFmpegFD.available()
|
||||
no_crypto = not Cryptodome_AES and '#EXT-X-KEY:METHOD=AES-128' in s
|
||||
no_crypto = not Cryptodome.AES and '#EXT-X-KEY:METHOD=AES-128' in s
|
||||
if no_crypto and has_ffmpeg:
|
||||
can_download, message = False, 'The stream has AES-128 encryption and pycryptodomex is not available'
|
||||
elif no_crypto:
|
||||
@ -150,6 +157,13 @@ def is_ad_fragment_end(s):
|
||||
i = 0
|
||||
media_sequence = 0
|
||||
decrypt_info = {'METHOD': 'NONE'}
|
||||
external_aes_key = traverse_obj(info_dict, ('hls_aes', 'key'))
|
||||
if external_aes_key:
|
||||
external_aes_key = binascii.unhexlify(remove_start(external_aes_key, '0x'))
|
||||
assert len(external_aes_key) in (16, 24, 32), 'Invalid length for HLS AES-128 key'
|
||||
external_aes_iv = traverse_obj(info_dict, ('hls_aes', 'iv'))
|
||||
if external_aes_iv:
|
||||
external_aes_iv = binascii.unhexlify(remove_start(external_aes_iv, '0x').zfill(32))
|
||||
byte_range = {}
|
||||
discontinuity_count = 0
|
||||
frag_index = 0
|
||||
@ -165,10 +179,7 @@ def is_ad_fragment_end(s):
|
||||
frag_index += 1
|
||||
if frag_index <= ctx['fragment_index']:
|
||||
continue
|
||||
frag_url = (
|
||||
line
|
||||
if re.match(r'^https?://', line)
|
||||
else urllib.parse.urljoin(man_url, line))
|
||||
frag_url = urljoin(man_url, line)
|
||||
if extra_query:
|
||||
frag_url = update_url_query(frag_url, extra_query)
|
||||
|
||||
@ -190,10 +201,7 @@ def is_ad_fragment_end(s):
|
||||
return False
|
||||
frag_index += 1
|
||||
map_info = parse_m3u8_attributes(line[11:])
|
||||
frag_url = (
|
||||
map_info.get('URI')
|
||||
if re.match(r'^https?://', map_info.get('URI'))
|
||||
else urllib.parse.urljoin(man_url, map_info.get('URI')))
|
||||
frag_url = urljoin(man_url, map_info.get('URI'))
|
||||
if extra_query:
|
||||
frag_url = update_url_query(frag_url, extra_query)
|
||||
|
||||
@ -218,15 +226,18 @@ def is_ad_fragment_end(s):
|
||||
decrypt_url = decrypt_info.get('URI')
|
||||
decrypt_info = parse_m3u8_attributes(line[11:])
|
||||
if decrypt_info['METHOD'] == 'AES-128':
|
||||
if 'IV' in decrypt_info:
|
||||
if external_aes_iv:
|
||||
decrypt_info['IV'] = external_aes_iv
|
||||
elif 'IV' in decrypt_info:
|
||||
decrypt_info['IV'] = binascii.unhexlify(decrypt_info['IV'][2:].zfill(32))
|
||||
if not re.match(r'^https?://', decrypt_info['URI']):
|
||||
decrypt_info['URI'] = urllib.parse.urljoin(
|
||||
man_url, decrypt_info['URI'])
|
||||
if extra_query:
|
||||
decrypt_info['URI'] = update_url_query(decrypt_info['URI'], extra_query)
|
||||
if decrypt_url != decrypt_info['URI']:
|
||||
decrypt_info['KEY'] = None
|
||||
if external_aes_key:
|
||||
decrypt_info['KEY'] = external_aes_key
|
||||
else:
|
||||
decrypt_info['URI'] = urljoin(man_url, decrypt_info['URI'])
|
||||
if extra_query:
|
||||
decrypt_info['URI'] = update_url_query(decrypt_info['URI'], extra_query)
|
||||
if decrypt_url != decrypt_info['URI']:
|
||||
decrypt_info['KEY'] = None
|
||||
|
||||
elif line.startswith('#EXT-X-MEDIA-SEQUENCE'):
|
||||
media_sequence = int(line[22:])
|
||||
|
@ -211,7 +211,12 @@ def close_stream():
|
||||
ctx.stream = None
|
||||
|
||||
def download():
|
||||
data_len = ctx.data.info().get('Content-length', None)
|
||||
data_len = ctx.data.info().get('Content-length')
|
||||
|
||||
if ctx.data.info().get('Content-encoding'):
|
||||
# Content-encoding is present, Content-length is not reliable anymore as we are
|
||||
# doing auto decompression. (See: https://github.com/yt-dlp/yt-dlp/pull/6176)
|
||||
data_len = None
|
||||
|
||||
# Range HTTP header may be ignored/unsupported by a webserver
|
||||
# (e.g. extractor/scivee.py, extractor/bambuser.py).
|
||||
|
@ -138,6 +138,8 @@ def write_piff_header(stream, params):
|
||||
|
||||
if fourcc == 'AACL':
|
||||
sample_entry_box = box(b'mp4a', sample_entry_payload)
|
||||
if fourcc == 'EC-3':
|
||||
sample_entry_box = box(b'ec-3', sample_entry_payload)
|
||||
elif stream_type == 'video':
|
||||
sample_entry_payload += u16.pack(0) # pre defined
|
||||
sample_entry_payload += u16.pack(0) # reserved
|
||||
@ -278,5 +280,4 @@ def real_download(self, filename, info_dict):
|
||||
return False
|
||||
self.report_skip_fragment(frag_index)
|
||||
|
||||
self._finish_frag_download(ctx, info_dict)
|
||||
return True
|
||||
return self._finish_frag_download(ctx, info_dict)
|
||||
|
@ -186,5 +186,4 @@ def real_download(self, filename, info_dict):
|
||||
|
||||
ctx['dest_stream'].write(
|
||||
b'--%b--\r\n\r\n' % frag_boundary.encode('us-ascii'))
|
||||
self._finish_frag_download(ctx, info_dict)
|
||||
return True
|
||||
return self._finish_frag_download(ctx, info_dict)
|
||||
|
@ -191,8 +191,7 @@ def download_and_parse_fragment(url, frag_index, request_data=None, headers=None
|
||||
if test:
|
||||
break
|
||||
|
||||
self._finish_frag_download(ctx, info_dict)
|
||||
return True
|
||||
return self._finish_frag_download(ctx, info_dict)
|
||||
|
||||
@staticmethod
|
||||
def parse_live_timestamp(action):
|
||||
|
@ -21,6 +21,8 @@
|
||||
YoutubeYtBeIE,
|
||||
YoutubeYtUserIE,
|
||||
YoutubeWatchLaterIE,
|
||||
YoutubeShortsAudioPivotIE,
|
||||
YoutubeConsentRedirectIE,
|
||||
)
|
||||
|
||||
from .abc import (
|
||||
@ -64,12 +66,21 @@
|
||||
HistoryPlayerIE,
|
||||
BiographyIE,
|
||||
)
|
||||
from .aeonco import AeonCoIE
|
||||
from .afreecatv import (
|
||||
AfreecaTVIE,
|
||||
AfreecaTVLiveIE,
|
||||
AfreecaTVUserIE,
|
||||
)
|
||||
from .agora import (
|
||||
TokFMAuditionIE,
|
||||
TokFMPodcastIE,
|
||||
WyborczaPodcastIE,
|
||||
WyborczaVideoIE,
|
||||
)
|
||||
from .airmozilla import AirMozillaIE
|
||||
from .airtv import AirTVIE
|
||||
from .aitube import AitubeKZVideoIE
|
||||
from .aljazeera import AlJazeeraIE
|
||||
from .alphaporno import AlphaPornoIE
|
||||
from .amara import AmaraIE
|
||||
@ -78,11 +89,20 @@
|
||||
AluraCourseIE
|
||||
)
|
||||
from .amcnetworks import AMCNetworksIE
|
||||
from .amazon import AmazonStoreIE
|
||||
from .amazon import (
|
||||
AmazonStoreIE,
|
||||
AmazonReviewsIE,
|
||||
)
|
||||
from .amazonminitv import (
|
||||
AmazonMiniTVIE,
|
||||
AmazonMiniTVSeasonIE,
|
||||
AmazonMiniTVSeriesIE,
|
||||
)
|
||||
from .americastestkitchen import (
|
||||
AmericasTestKitchenIE,
|
||||
AmericasTestKitchenSeasonIE,
|
||||
)
|
||||
from .anchorfm import AnchorFMEpisodeIE
|
||||
from .angel import AngelIE
|
||||
from .anvato import AnvatoIE
|
||||
from .aol import AolIE
|
||||
@ -103,6 +123,7 @@
|
||||
from .archiveorg import (
|
||||
ArchiveOrgIE,
|
||||
YoutubeWebArchiveIE,
|
||||
VLiveWebArchiveIE,
|
||||
)
|
||||
from .arcpublishing import ArcPublishingIE
|
||||
from .arkena import ArkenaIE
|
||||
@ -170,7 +191,12 @@
|
||||
from .beeg import BeegIE
|
||||
from .behindkink import BehindKinkIE
|
||||
from .bellmedia import BellMediaIE
|
||||
from .beatbump import (
|
||||
BeatBumpVideoIE,
|
||||
BeatBumpPlaylistIE,
|
||||
)
|
||||
from .beatport import BeatportIE
|
||||
from .berufetv import BerufeTVIE
|
||||
from .bet import BetIE
|
||||
from .bfi import BFIPlayerIE
|
||||
from .bfmtv import (
|
||||
@ -184,13 +210,16 @@
|
||||
from .bild import BildIE
|
||||
from .bilibili import (
|
||||
BiliBiliIE,
|
||||
BiliBiliBangumiIE,
|
||||
BiliBiliBangumiMediaIE,
|
||||
BiliBiliSearchIE,
|
||||
BilibiliCategoryIE,
|
||||
BiliBiliBangumiIE,
|
||||
BilibiliAudioIE,
|
||||
BilibiliAudioAlbumIE,
|
||||
BiliBiliPlayerIE,
|
||||
BilibiliChannelIE,
|
||||
BilibiliSpaceVideoIE,
|
||||
BilibiliSpaceAudioIE,
|
||||
BilibiliSpacePlaylistIE,
|
||||
BiliIntlIE,
|
||||
BiliIntlSeriesIE,
|
||||
BiliLiveIE,
|
||||
@ -210,12 +239,15 @@
|
||||
BleacherReportIE,
|
||||
BleacherReportCMSIE,
|
||||
)
|
||||
from .blerp import BlerpIE
|
||||
from .blogger import BloggerIE
|
||||
from .bloomberg import BloombergIE
|
||||
from .bokecc import BokeCCIE
|
||||
from .bongacams import BongaCamsIE
|
||||
from .bostonglobe import BostonGlobeIE
|
||||
from .box import BoxIE
|
||||
from .boxcast import BoxCastVideoIE
|
||||
from .booyah import BooyahClipsIE
|
||||
from .bpb import BpbIE
|
||||
from .br import (
|
||||
BRIE,
|
||||
@ -229,6 +261,7 @@
|
||||
BrightcoveNewIE,
|
||||
)
|
||||
from .businessinsider import BusinessInsiderIE
|
||||
from .bundesliga import BundesligaIE
|
||||
from .buzzfeed import BuzzFeedIE
|
||||
from .byutv import BYUtvIE
|
||||
from .c56 import C56IE
|
||||
@ -241,6 +274,7 @@
|
||||
CamdemyFolderIE
|
||||
)
|
||||
from .cammodels import CamModelsIE
|
||||
from .camsoda import CamsodaIE
|
||||
from .camtasia import CamtasiaEmbedIE
|
||||
from .camwithher import CamWithHerIE
|
||||
from .canalalpha import CanalAlphaIE
|
||||
@ -304,6 +338,7 @@
|
||||
)
|
||||
from .cinchcast import CinchcastIE
|
||||
from .cinemax import CinemaxIE
|
||||
from .cinetecamilano import CinetecaMilanoIE
|
||||
from .ciscolive import (
|
||||
CiscoLiveSessionIE,
|
||||
CiscoLiveSearchIE,
|
||||
@ -328,6 +363,7 @@
|
||||
CNNIE,
|
||||
CNNBlogsIE,
|
||||
CNNArticleIE,
|
||||
CNNIndonesiaIE,
|
||||
)
|
||||
from .coub import CoubIE
|
||||
from .comedycentral import (
|
||||
@ -357,8 +393,6 @@
|
||||
CrowdBunkerChannelIE,
|
||||
)
|
||||
from .crunchyroll import (
|
||||
CrunchyrollIE,
|
||||
CrunchyrollShowPlaylistIE,
|
||||
CrunchyrollBetaIE,
|
||||
CrunchyrollBetaShowIE,
|
||||
)
|
||||
@ -410,7 +444,7 @@
|
||||
DeezerShowIE,
|
||||
)
|
||||
from .democracynow import DemocracynowIE
|
||||
from .detik import Detik20IE
|
||||
from .detik import DetikEmbedIE
|
||||
from .dfb import DFBIE
|
||||
from .dhm import DHMIE
|
||||
from .digg import DiggIE
|
||||
@ -437,6 +471,7 @@
|
||||
AnimalPlanetIE,
|
||||
TLCIE,
|
||||
MotorTrendIE,
|
||||
MotorTrendOnDemandIE,
|
||||
DiscoveryPlusIndiaIE,
|
||||
DiscoveryNetworksDeIE,
|
||||
DiscoveryPlusItalyIE,
|
||||
@ -449,6 +484,8 @@
|
||||
from .drtv import (
|
||||
DRTVIE,
|
||||
DRTVLiveIE,
|
||||
DRTVSeasonIE,
|
||||
DRTVSeriesIE,
|
||||
)
|
||||
from .dtube import DTubeIE
|
||||
from .dvtv import DVTVIE
|
||||
@ -458,11 +495,14 @@
|
||||
)
|
||||
from .dumpert import DumpertIE
|
||||
from .defense import DefenseGouvFrIE
|
||||
from .deuxm import (
|
||||
DeuxMIE,
|
||||
DeuxMNewsIE
|
||||
)
|
||||
from .digitalconcerthall import DigitalConcertHallIE
|
||||
from .discovery import DiscoveryIE
|
||||
from .disney import DisneyIE
|
||||
from .dispeak import DigitallySpeakingIE
|
||||
from .doodstream import DoodStreamIE
|
||||
from .dropbox import DropboxIE
|
||||
from .dropout import (
|
||||
DropoutSeasonIE,
|
||||
@ -474,6 +514,7 @@
|
||||
)
|
||||
from .eagleplatform import EaglePlatformIE, ClipYouEmbedIE
|
||||
from .ebaumsworld import EbaumsWorldIE
|
||||
from .ebay import EbayIE
|
||||
from .echomsk import EchoMskIE
|
||||
from .egghead import (
|
||||
EggheadCourseIE,
|
||||
@ -516,7 +557,7 @@
|
||||
ESPNCricInfoIE,
|
||||
)
|
||||
from .esri import EsriVideoIE
|
||||
from .europa import EuropaIE
|
||||
from .europa import EuropaIE, EuroParlWebstreamIE
|
||||
from .europeantour import EuropeanTourIE
|
||||
from .eurosport import EurosportIE
|
||||
from .euscreen import EUScreenIE
|
||||
@ -574,6 +615,7 @@
|
||||
from .foxnews import (
|
||||
FoxNewsIE,
|
||||
FoxNewsArticleIE,
|
||||
FoxNewsVideoIE,
|
||||
)
|
||||
from .foxsports import FoxSportsIE
|
||||
from .fptplay import FptplayIE
|
||||
@ -624,6 +666,10 @@
|
||||
from .gdcvault import GDCVaultIE
|
||||
from .gedidigital import GediDigitalIE
|
||||
from .generic import GenericIE
|
||||
from .genius import (
|
||||
GeniusIE,
|
||||
GeniusLyricsIE,
|
||||
)
|
||||
from .gettr import (
|
||||
GettrIE,
|
||||
GettrStreamingIE,
|
||||
@ -651,6 +697,7 @@
|
||||
)
|
||||
from .googlesearch import GoogleSearchIE
|
||||
from .gopro import GoProIE
|
||||
from .goplay import GoPlayIE
|
||||
from .goshgay import GoshgayIE
|
||||
from .gotostage import GoToStageIE
|
||||
from .gputechconf import GPUTechConfIE
|
||||
@ -679,6 +726,7 @@
|
||||
HotStarIE,
|
||||
HotStarPrefixIE,
|
||||
HotStarPlaylistIE,
|
||||
HotStarSeasonIE,
|
||||
HotStarSeriesIE,
|
||||
)
|
||||
from .howcast import HowcastIE
|
||||
@ -692,7 +740,10 @@
|
||||
HSEShowIE,
|
||||
HSEProductIE,
|
||||
)
|
||||
from .genericembeds import HTML5MediaEmbedIE
|
||||
from .genericembeds import (
|
||||
HTML5MediaEmbedIE,
|
||||
QuotedHTMLIE,
|
||||
)
|
||||
from .huajiao import HuajiaoIE
|
||||
from .huya import HuyaLiveIE
|
||||
from .huffpost import HuffPostIE
|
||||
@ -702,6 +753,7 @@
|
||||
HungamaAlbumPlaylistIE,
|
||||
)
|
||||
from .hypem import HypemIE
|
||||
from .hypergryph import MonsterSirenHypergryphMusicIE
|
||||
from .hytale import HytaleIE
|
||||
from .icareus import IcareusIE
|
||||
from .ichinanalive import (
|
||||
@ -717,6 +769,7 @@
|
||||
IHeartRadioIE,
|
||||
IHeartRadioPodcastIE,
|
||||
)
|
||||
from .iltalehti import IltalehtiIE
|
||||
from .imdb import (
|
||||
ImdbIE,
|
||||
ImdbListIE
|
||||
@ -752,6 +805,7 @@
|
||||
IslamChannelIE,
|
||||
IslamChannelSeriesIE,
|
||||
)
|
||||
from .israelnationalnews import IsraelNationalNewsIE
|
||||
from .itprotv import (
|
||||
ITProTVIE,
|
||||
ITProTVCourseIE
|
||||
@ -780,12 +834,21 @@
|
||||
JamendoIE,
|
||||
JamendoAlbumIE,
|
||||
)
|
||||
from .japandiet import (
|
||||
ShugiinItvLiveIE,
|
||||
ShugiinItvLiveRoomIE,
|
||||
ShugiinItvVodIE,
|
||||
SangiinInstructionIE,
|
||||
SangiinIE,
|
||||
)
|
||||
from .jeuxvideo import JeuxVideoIE
|
||||
from .jove import JoveIE
|
||||
from .joj import JojIE
|
||||
from .jwplatform import JWPlatformIE
|
||||
from .kakao import KakaoIE
|
||||
from .kaltura import KalturaIE
|
||||
from .kanal2 import Kanal2IE
|
||||
from .kankanews import KankaNewsIE
|
||||
from .karaoketv import KaraoketvIE
|
||||
from .karrierevideos import KarriereVideosIE
|
||||
from .keezmovies import KeezMoviesIE
|
||||
@ -795,10 +858,15 @@
|
||||
KhanAcademyIE,
|
||||
KhanAcademyUnitIE,
|
||||
)
|
||||
from .kick import (
|
||||
KickIE,
|
||||
KickVODIE,
|
||||
)
|
||||
from .kicker import KickerIE
|
||||
from .kickstarter import KickStarterIE
|
||||
from .kinja import KinjaEmbedIE
|
||||
from .kinopoisk import KinoPoiskIE
|
||||
from .kommunetv import KommunetvIE
|
||||
from .kompas import KompasVideoIE
|
||||
from .konserthusetplay import KonserthusetPlayIE
|
||||
from .koo import KooIE
|
||||
@ -850,6 +918,10 @@
|
||||
LePlaylistIE,
|
||||
LetvCloudIE,
|
||||
)
|
||||
from .lefigaro import (
|
||||
LeFigaroVideoEmbedIE,
|
||||
LeFigaroVideoSectionIE,
|
||||
)
|
||||
from .lego import LEGOIE
|
||||
from .lemonde import LemondeIE
|
||||
from .lenta import LentaIE
|
||||
@ -879,6 +951,7 @@
|
||||
)
|
||||
from .linuxacademy import LinuxAcademyIE
|
||||
from .liputan6 import Liputan6IE
|
||||
from .listennotes import ListenNotesIE
|
||||
from .litv import LiTVIE
|
||||
from .livejournal import LiveJournalIE
|
||||
from .livestream import (
|
||||
@ -897,6 +970,9 @@
|
||||
LRTVODIE,
|
||||
LRTStreamIE
|
||||
)
|
||||
from .lumni import (
|
||||
LumniIE
|
||||
)
|
||||
from .lynda import (
|
||||
LyndaIE,
|
||||
LyndaCourseIE
|
||||
@ -941,6 +1017,10 @@
|
||||
MediasiteCatalogIE,
|
||||
MediasiteNamedCatalogIE,
|
||||
)
|
||||
from .mediastream import (
|
||||
MediaStreamIE,
|
||||
WinSportsVideoIE,
|
||||
)
|
||||
from .mediaworksnz import MediaWorksNZVODIE
|
||||
from .medici import MediciIE
|
||||
from .megaphone import MegaphoneIE
|
||||
@ -957,6 +1037,7 @@
|
||||
MicrosoftVirtualAcademyIE,
|
||||
MicrosoftVirtualAcademyCourseIE,
|
||||
)
|
||||
from .microsoftembed import MicrosoftEmbedIE
|
||||
from .mildom import (
|
||||
MildomIE,
|
||||
MildomVodIE,
|
||||
@ -991,6 +1072,7 @@
|
||||
MLBIE,
|
||||
MLBVideoIE,
|
||||
MLBTVIE,
|
||||
MLBArticleIE,
|
||||
)
|
||||
from .mlssoccer import MLSSoccerIE
|
||||
from .mnet import MnetIE
|
||||
@ -1078,6 +1160,7 @@
|
||||
NBCSportsIE,
|
||||
NBCSportsStreamIE,
|
||||
NBCSportsVPlayerIE,
|
||||
NBCStationsIE,
|
||||
)
|
||||
from .ndr import (
|
||||
NDRIE,
|
||||
@ -1106,6 +1189,7 @@
|
||||
from .netverse import (
|
||||
NetverseIE,
|
||||
NetversePlaylistIE,
|
||||
NetverseSearchIE,
|
||||
)
|
||||
from .newgrounds import (
|
||||
NewgroundsIE,
|
||||
@ -1130,6 +1214,8 @@
|
||||
from .nfl import (
|
||||
NFLIE,
|
||||
NFLArticleIE,
|
||||
NFLPlusEpisodeIE,
|
||||
NFLPlusReplayIE,
|
||||
)
|
||||
from .nhk import (
|
||||
NhkVodIE,
|
||||
@ -1167,11 +1253,13 @@
|
||||
from .nitter import NitterIE
|
||||
from .njpwworld import NJPWWorldIE
|
||||
from .nobelprize import NobelPrizeIE
|
||||
from .noice import NoicePodcastIE
|
||||
from .nonktube import NonkTubeIE
|
||||
from .noodlemagazine import NoodleMagazineIE
|
||||
from .noovo import NoovoIE
|
||||
from .normalboots import NormalbootsIE
|
||||
from .nosvideo import NosVideoIE
|
||||
from .nosnl import NOSNLArticleIE
|
||||
from .nova import (
|
||||
NovaEmbedIE,
|
||||
NovaIE,
|
||||
@ -1211,6 +1299,7 @@
|
||||
from .ntvcojp import NTVCoJpCUIE
|
||||
from .ntvde import NTVDeIE
|
||||
from .ntvru import NTVRuIE
|
||||
from .nubilesporn import NubilesPornIE
|
||||
from .nytimes import (
|
||||
NYTimesIE,
|
||||
NYTimesArticleIE,
|
||||
@ -1218,14 +1307,22 @@
|
||||
)
|
||||
from .nuvid import NuvidIE
|
||||
from .nzherald import NZHeraldIE
|
||||
from .nzonscreen import NZOnScreenIE
|
||||
from .nzz import NZZIE
|
||||
from .odatv import OdaTVIE
|
||||
from .odkmedia import OnDemandChinaEpisodeIE
|
||||
from .odnoklassniki import OdnoklassnikiIE
|
||||
from .oftv import (
|
||||
OfTVIE,
|
||||
OfTVPlaylistIE
|
||||
)
|
||||
from .oktoberfesttv import OktoberfestTVIE
|
||||
from .olympics import OlympicsReplayIE
|
||||
from .on24 import On24IE
|
||||
from .ondemandkorea import OnDemandKoreaIE
|
||||
from .onefootball import OneFootballIE
|
||||
from .onenewsnz import OneNewsNZIE
|
||||
from .oneplace import OnePlacePodcastIE
|
||||
from .onet import (
|
||||
OnetIE,
|
||||
OnetChannelIE,
|
||||
@ -1334,6 +1431,7 @@
|
||||
PluralsightIE,
|
||||
PluralsightCourseIE,
|
||||
)
|
||||
from .podbayfm import PodbayFMIE, PodbayFMChannelIE
|
||||
from .podchaser import PodchaserIE
|
||||
from .podomatic import PodomaticIE
|
||||
from .pokemon import (
|
||||
@ -1347,6 +1445,8 @@
|
||||
from .polsatgo import PolsatGoIE
|
||||
from .polskieradio import (
|
||||
PolskieRadioIE,
|
||||
PolskieRadioLegacyIE,
|
||||
PolskieRadioAuditionIE,
|
||||
PolskieRadioCategoryIE,
|
||||
PolskieRadioPlayerIE,
|
||||
PolskieRadioPodcastIE,
|
||||
@ -1374,6 +1474,8 @@
|
||||
PuhuTVIE,
|
||||
PuhuTVSerieIE,
|
||||
)
|
||||
from .pr0gramm import Pr0grammStaticIE, Pr0grammIE
|
||||
from .prankcast import PrankCastIE
|
||||
from .premiershiprugby import PremiershipRugbyIE
|
||||
from .presstv import PressTVIE
|
||||
from .projectveritas import ProjectVeritasIE
|
||||
@ -1387,6 +1489,7 @@
|
||||
)
|
||||
from .puls4 import Puls4IE
|
||||
from .pyvideo import PyvideoIE
|
||||
from .qingting import QingTingIE
|
||||
from .qqmusic import (
|
||||
QQMusicIE,
|
||||
QQMusicSingerIE,
|
||||
@ -1433,6 +1536,10 @@
|
||||
RayWenderlichCourseIE,
|
||||
)
|
||||
from .rbmaradio import RBMARadioIE
|
||||
from .rbgtum import (
|
||||
RbgTumIE,
|
||||
RbgTumCourseIE,
|
||||
)
|
||||
from .rcs import (
|
||||
RCSIE,
|
||||
RCSEmbedsIE,
|
||||
@ -1477,7 +1584,10 @@
|
||||
)
|
||||
from .roosterteeth import RoosterTeethIE, RoosterTeethSeriesIE
|
||||
from .rottentomatoes import RottenTomatoesIE
|
||||
from .rozhlas import RozhlasIE
|
||||
from .rozhlas import (
|
||||
RozhlasIE,
|
||||
RozhlasVltavaIE,
|
||||
)
|
||||
from .rte import RteIE, RteRadioIE
|
||||
from .rtlnl import (
|
||||
RtlNlIE,
|
||||
@ -1514,6 +1624,7 @@
|
||||
from .rule34video import Rule34VideoIE
|
||||
from .rumble import (
|
||||
RumbleEmbedIE,
|
||||
RumbleIE,
|
||||
RumbleChannelIE,
|
||||
)
|
||||
from .rutube import (
|
||||
@ -1554,7 +1665,9 @@
|
||||
from .sapo import SapoIE
|
||||
from .savefrom import SaveFromIE
|
||||
from .sbs import SBSIE
|
||||
from .screen9 import Screen9IE
|
||||
from .screencast import ScreencastIE
|
||||
from .screencastify import ScreencastifyIE
|
||||
from .screencastomatic import ScreencastOMaticIE
|
||||
from .scrippsnetworks import (
|
||||
ScrippsNetworksWatchIE,
|
||||
@ -1584,6 +1697,7 @@
|
||||
VivoIE,
|
||||
)
|
||||
from .sharevideos import ShareVideosEmbedIE
|
||||
from .sibnet import SibnetEmbedIE
|
||||
from .shemaroome import ShemarooMeIE
|
||||
from .showroomlive import ShowRoomLiveIE
|
||||
from .simplecast import (
|
||||
@ -1599,7 +1713,6 @@
|
||||
SkyItVideoIE,
|
||||
SkyItVideoLiveIE,
|
||||
SkyItIE,
|
||||
SkyItAcademyIE,
|
||||
SkyItArteIE,
|
||||
CieloTVItIE,
|
||||
TV8ItIE,
|
||||
@ -1619,6 +1732,7 @@
|
||||
from .slideshare import SlideshareIE
|
||||
from .slideslive import SlidesLiveIE
|
||||
from .slutload import SlutloadIE
|
||||
from .smotrim import SmotrimIE
|
||||
from .snotr import SnotrIE
|
||||
from .sohu import SohuIE
|
||||
from .sonyliv import (
|
||||
@ -1631,6 +1745,7 @@
|
||||
SoundcloudSetIE,
|
||||
SoundcloudRelatedIE,
|
||||
SoundcloudUserIE,
|
||||
SoundcloudUserPermalinkIE,
|
||||
SoundcloudTrackStationIE,
|
||||
SoundcloudPlaylistIE,
|
||||
SoundcloudSearchIE,
|
||||
@ -1718,6 +1833,7 @@
|
||||
SVTPlayIE,
|
||||
SVTSeriesIE,
|
||||
)
|
||||
from .swearnet import SwearnetEpisodeIE
|
||||
from .swrmediathek import SWRMediathekIE
|
||||
from .syvdk import SYVDKIE
|
||||
from .syfy import SyfyIE
|
||||
@ -1735,7 +1851,10 @@
|
||||
TeacherTubeUserIE,
|
||||
)
|
||||
from .teachingchannel import TeachingChannelIE
|
||||
from .teamcoco import TeamcocoIE
|
||||
from .teamcoco import (
|
||||
TeamcocoIE,
|
||||
ConanClassicIE,
|
||||
)
|
||||
from .teamtreehouse import TeamTreeHouseIE
|
||||
from .techtalks import TechTalksIE
|
||||
from .ted import (
|
||||
@ -1747,6 +1866,7 @@
|
||||
from .tele5 import Tele5IE
|
||||
from .tele13 import Tele13IE
|
||||
from .telebruxelles import TeleBruxellesIE
|
||||
from .telecaribe import TelecaribePlayIE
|
||||
from .telecinco import TelecincoIE
|
||||
from .telegraaf import TelegraafIE
|
||||
from .telegram import TelegramEmbedIE
|
||||
@ -1761,8 +1881,10 @@
|
||||
)
|
||||
from .teletask import TeleTaskIE
|
||||
from .telewebion import TelewebionIE
|
||||
from .tempo import TempoIE
|
||||
from .tempo import TempoIE, IVXPlayerIE
|
||||
from .tencent import (
|
||||
IflixEpisodeIE,
|
||||
IflixSeriesIE,
|
||||
VQQSeriesIE,
|
||||
VQQVideoIE,
|
||||
WeTvEpisodeIE,
|
||||
@ -1789,6 +1911,11 @@
|
||||
from .thisamericanlife import ThisAmericanLifeIE
|
||||
from .thisav import ThisAVIE
|
||||
from .thisoldhouse import ThisOldHouseIE
|
||||
from .thisvid import (
|
||||
ThisVidIE,
|
||||
ThisVidMemberIE,
|
||||
ThisVidPlaylistIE,
|
||||
)
|
||||
from .threespeak import (
|
||||
ThreeSpeakIE,
|
||||
ThreeSpeakUserIE,
|
||||
@ -1801,6 +1928,7 @@
|
||||
TikTokEffectIE,
|
||||
TikTokTagIE,
|
||||
TikTokVMIE,
|
||||
TikTokLiveIE,
|
||||
DouyinIE,
|
||||
)
|
||||
from .tinypic import TinyPicIE
|
||||
@ -1838,6 +1966,7 @@
|
||||
TrovoChannelVodIE,
|
||||
TrovoChannelClipIE,
|
||||
)
|
||||
from .trtcocuk import TrtCocukVideoIE
|
||||
from .trueid import TrueIDIE
|
||||
from .trunews import TruNewsIE
|
||||
from .truth import TruthIE
|
||||
@ -1850,10 +1979,9 @@
|
||||
)
|
||||
from .tumblr import TumblrIE
|
||||
from .tunein import (
|
||||
TuneInClipIE,
|
||||
TuneInStationIE,
|
||||
TuneInProgramIE,
|
||||
TuneInTopicIE,
|
||||
TuneInPodcastIE,
|
||||
TuneInPodcastEpisodeIE,
|
||||
TuneInShortenerIE,
|
||||
)
|
||||
from .tunepk import TunePkIE
|
||||
@ -1864,6 +1992,9 @@
|
||||
KatsomoIE,
|
||||
MTVUutisetArticleIE,
|
||||
)
|
||||
from .tv24ua import (
|
||||
TV24UAVideoIE,
|
||||
)
|
||||
from .tv2dk import (
|
||||
TV2DKIE,
|
||||
TV2DKBornholmPlayIE,
|
||||
@ -1913,7 +2044,8 @@
|
||||
TVPEmbedIE,
|
||||
TVPIE,
|
||||
TVPStreamIE,
|
||||
TVPWebsiteIE,
|
||||
TVPVODSeriesIE,
|
||||
TVPVODVideoIE,
|
||||
)
|
||||
from .tvplay import (
|
||||
TVPlayIE,
|
||||
@ -1944,8 +2076,13 @@
|
||||
TwitterIE,
|
||||
TwitterAmplifyIE,
|
||||
TwitterBroadcastIE,
|
||||
TwitterSpacesIE,
|
||||
TwitterShortenerIE,
|
||||
)
|
||||
from .txxx import (
|
||||
TxxxIE,
|
||||
PornTopIE,
|
||||
)
|
||||
from .udemy import (
|
||||
UdemyIE,
|
||||
UdemyCourseIE
|
||||
@ -1966,6 +2103,8 @@
|
||||
from .umg import UMGDeIE
|
||||
from .unistra import UnistraIE
|
||||
from .unity import UnityIE
|
||||
from .unscripted import UnscriptedNewsVideoIE
|
||||
from .unsupported import KnownDRMIE, KnownPiracyIE
|
||||
from .uol import UOLIE
|
||||
from .uplynk import (
|
||||
UplynkIE,
|
||||
@ -1985,7 +2124,10 @@
|
||||
from .vbox7 import Vbox7IE
|
||||
from .veehd import VeeHDIE
|
||||
from .veo import VeoIE
|
||||
from .veoh import VeohIE
|
||||
from .veoh import (
|
||||
VeohIE,
|
||||
VeohUserIE
|
||||
)
|
||||
from .vesti import VestiIE
|
||||
from .vevo import (
|
||||
VevoIE,
|
||||
@ -2011,6 +2153,13 @@
|
||||
)
|
||||
from .videodetective import VideoDetectiveIE
|
||||
from .videofyme import VideofyMeIE
|
||||
from .videoken import (
|
||||
VideoKenIE,
|
||||
VideoKenPlayerIE,
|
||||
VideoKenPlaylistIE,
|
||||
VideoKenCategoryIE,
|
||||
VideoKenTopicIE,
|
||||
)
|
||||
from .videomore import (
|
||||
VideomoreIE,
|
||||
VideomoreVideoIE,
|
||||
@ -2023,7 +2172,6 @@
|
||||
VidioLiveIE
|
||||
)
|
||||
from .vidlii import VidLiiIE
|
||||
from .vier import VierIE, VierVideosIE
|
||||
from .viewlift import (
|
||||
ViewLiftIE,
|
||||
ViewLiftEmbedIE,
|
||||
@ -2036,6 +2184,7 @@
|
||||
VimeoGroupsIE,
|
||||
VimeoLikesIE,
|
||||
VimeoOndemandIE,
|
||||
VimeoProIE,
|
||||
VimeoReviewIE,
|
||||
VimeoUserIE,
|
||||
VimeoWatchLaterIE,
|
||||
@ -2059,17 +2208,14 @@
|
||||
ViuIE,
|
||||
ViuPlaylistIE,
|
||||
ViuOTTIE,
|
||||
ViuOTTIndonesiaIE,
|
||||
)
|
||||
from .vk import (
|
||||
VKIE,
|
||||
VKUserVideosIE,
|
||||
VKWallPostIE,
|
||||
)
|
||||
from .vlive import (
|
||||
VLiveIE,
|
||||
VLivePostIE,
|
||||
VLiveChannelIE,
|
||||
)
|
||||
from .vocaroo import VocarooIE
|
||||
from .vodlocker import VodlockerIE
|
||||
from .vodpl import VODPlIE
|
||||
from .vodplatform import VODPlatformIE
|
||||
@ -2078,6 +2224,7 @@
|
||||
VoicyIE,
|
||||
VoicyChannelIE,
|
||||
)
|
||||
from .volejtv import VolejTVIE
|
||||
from .voot import (
|
||||
VootIE,
|
||||
VootSeriesIE,
|
||||
@ -2123,6 +2270,7 @@
|
||||
WDRElefantIE,
|
||||
WDRMobileIE,
|
||||
)
|
||||
from .webcamerapl import WebcameraplIE
|
||||
from .webcaster import (
|
||||
WebcasterIE,
|
||||
WebcasterFeedIE,
|
||||
@ -2143,17 +2291,27 @@
|
||||
from .wistia import (
|
||||
WistiaIE,
|
||||
WistiaPlaylistIE,
|
||||
WistiaChannelIE,
|
||||
)
|
||||
from .wordpress import (
|
||||
WordpressPlaylistEmbedIE,
|
||||
WordpressMiniAudioPlayerEmbedIE,
|
||||
)
|
||||
from .worldstarhiphop import WorldStarHipHopIE
|
||||
from .wppilot import (
|
||||
WPPilotIE,
|
||||
WPPilotChannelsIE,
|
||||
)
|
||||
from .wrestleuniverse import (
|
||||
WrestleUniverseVODIE,
|
||||
WrestleUniversePPVIE,
|
||||
)
|
||||
from .wsj import (
|
||||
WSJIE,
|
||||
WSJArticleIE,
|
||||
)
|
||||
from .wwe import WWEIE
|
||||
from .xanimu import XanimuIE
|
||||
from .xbef import XBefIE
|
||||
from .xboxclips import XboxClipsIE
|
||||
from .xfileshare import XFileShareIE
|
||||
@ -2162,12 +2320,6 @@
|
||||
XHamsterEmbedIE,
|
||||
XHamsterUserIE,
|
||||
)
|
||||
from .xiami import (
|
||||
XiamiSongIE,
|
||||
XiamiAlbumIE,
|
||||
XiamiArtistIE,
|
||||
XiamiCollectionIE
|
||||
)
|
||||
from .ximalaya import (
|
||||
XimalayaIE,
|
||||
XimalayaAlbumIE
|
||||
@ -2178,7 +2330,10 @@
|
||||
from .xstream import XstreamIE
|
||||
from .xtube import XTubeUserIE, XTubeIE
|
||||
from .xuite import XuiteIE
|
||||
from .xvideos import XVideosIE
|
||||
from .xvideos import (
|
||||
XVideosIE,
|
||||
XVideosQuickiesIE
|
||||
)
|
||||
from .xxxymovies import XXXYMoviesIE
|
||||
from .yahoo import (
|
||||
YahooIE,
|
||||
@ -2202,8 +2357,10 @@
|
||||
ZenYandexChannelIE,
|
||||
)
|
||||
from .yapfiles import YapFilesIE
|
||||
from .yappy import YappyIE
|
||||
from .yesjapan import YesJapanIE
|
||||
from .yinyuetai import YinYueTaiIE
|
||||
from .yle_areena import YleAreenaIE
|
||||
from .ynet import YnetIE
|
||||
from .youjizz import YouJizzIE
|
||||
from .youku import (
|
||||
@ -2266,6 +2423,7 @@
|
||||
Zee5IE,
|
||||
Zee5SeriesIE,
|
||||
)
|
||||
from .zeenews import ZeeNewsIE
|
||||
from .zhihu import ZhihuIE
|
||||
from .zingmp3 import (
|
||||
ZingMp3IE,
|
||||
|
@ -155,8 +155,6 @@ def _real_extract(self, url):
|
||||
'format_id': format_id
|
||||
})
|
||||
|
||||
self._sort_formats(formats)
|
||||
|
||||
return {
|
||||
'id': video_id,
|
||||
'title': self._og_search_title(webpage),
|
||||
@ -221,7 +219,6 @@ def tokenize_url(url, token):
|
||||
entry_protocol='m3u8_native', m3u8_id='hls', fatal=False)
|
||||
if formats:
|
||||
break
|
||||
self._sort_formats(formats)
|
||||
|
||||
subtitles = {}
|
||||
src_vtt = stream.get('captions', {}).get('src-vtt')
|
||||
|
@ -78,7 +78,6 @@ def _real_extract(self, url):
|
||||
'url': mp4_url,
|
||||
'width': 640,
|
||||
})
|
||||
self._sort_formats(formats)
|
||||
|
||||
image = video.get('image') or {}
|
||||
|
||||
@ -119,7 +118,6 @@ def _real_extract(self, url):
|
||||
title = video_data['title']
|
||||
formats = self._extract_m3u8_formats(
|
||||
video_data['videoURL'].split('?')[0], video_id, 'mp4')
|
||||
self._sort_formats(formats)
|
||||
|
||||
return {
|
||||
'id': video_id,
|
||||
|
@ -156,7 +156,7 @@ class AbemaTVBaseIE(InfoExtractor):
|
||||
def _generate_aks(cls, deviceid):
|
||||
deviceid = deviceid.encode('utf-8')
|
||||
# add 1 hour and then drop minute and secs
|
||||
ts_1hour = int((time_seconds(hours=9) // 3600 + 1) * 3600)
|
||||
ts_1hour = int((time_seconds() // 3600 + 1) * 3600)
|
||||
time_struct = time.gmtime(ts_1hour)
|
||||
ts_1hour_str = str(ts_1hour).encode('utf-8')
|
||||
|
||||
@ -190,6 +190,16 @@ def _get_device_token(self):
|
||||
if self._USERTOKEN:
|
||||
return self._USERTOKEN
|
||||
|
||||
username, _ = self._get_login_info()
|
||||
AbemaTVBaseIE._USERTOKEN = username and self.cache.load(self._NETRC_MACHINE, username)
|
||||
if AbemaTVBaseIE._USERTOKEN:
|
||||
# try authentication with locally stored token
|
||||
try:
|
||||
self._get_media_token(True)
|
||||
return
|
||||
except ExtractorError as e:
|
||||
self.report_warning(f'Failed to login with cached user token; obtaining a fresh one ({e})')
|
||||
|
||||
AbemaTVBaseIE._DEVICE_ID = str(uuid.uuid4())
|
||||
aks = self._generate_aks(self._DEVICE_ID)
|
||||
user_data = self._download_json(
|
||||
@ -300,6 +310,11 @@ class AbemaTVIE(AbemaTVBaseIE):
|
||||
_TIMETABLE = None
|
||||
|
||||
def _perform_login(self, username, password):
|
||||
self._get_device_token()
|
||||
if self.cache.load(self._NETRC_MACHINE, username) and self._get_media_token():
|
||||
self.write_debug('Skipping logging in')
|
||||
return
|
||||
|
||||
if '@' in username: # don't strictly check if it's email address or not
|
||||
ep, method = 'user/email', 'email'
|
||||
else:
|
||||
@ -319,6 +334,7 @@ def _perform_login(self, username, password):
|
||||
|
||||
AbemaTVBaseIE._USERTOKEN = login_response['token']
|
||||
self._get_media_token(True)
|
||||
self.cache.store(self._NETRC_MACHINE, username, AbemaTVBaseIE._USERTOKEN)
|
||||
|
||||
def _real_extract(self, url):
|
||||
# starting download using infojson from this extractor is undefined behavior,
|
||||
@ -416,7 +432,7 @@ def _real_extract(self, url):
|
||||
f'https://api.abema.io/v1/video/programs/{video_id}', video_id,
|
||||
note='Checking playability',
|
||||
headers=headers)
|
||||
ondemand_types = traverse_obj(api_response, ('terms', ..., 'onDemandType'), default=[])
|
||||
ondemand_types = traverse_obj(api_response, ('terms', ..., 'onDemandType'))
|
||||
if 3 not in ondemand_types:
|
||||
# cannot acquire decryption key for these streams
|
||||
self.report_warning('This is a premium-only stream')
|
||||
@ -489,7 +505,7 @@ def _fetch_page(self, playlist_id, series_version, page):
|
||||
})
|
||||
yield from (
|
||||
self.url_result(f'https://abema.tv/video/episode/{x}')
|
||||
for x in traverse_obj(programs, ('programs', ..., 'id'), default=[]))
|
||||
for x in traverse_obj(programs, ('programs', ..., 'id')))
|
||||
|
||||
def _entries(self, playlist_id, series_version):
|
||||
return OnDemandPagedList(
|
||||
|
@ -27,7 +27,6 @@ def _extract_metadata(self, video_id, video_info):
|
||||
**parse_codecs(video.get('codecs', ''))
|
||||
})
|
||||
|
||||
self._sort_formats(formats)
|
||||
return {
|
||||
'id': video_id,
|
||||
'formats': formats,
|
||||
@ -84,7 +83,7 @@ def _real_extract(self, url):
|
||||
video_id = self._match_id(url)
|
||||
|
||||
webpage = self._download_webpage(url, video_id)
|
||||
json_all = self._search_json(r'window.videoInfo\s*=\s*', webpage, 'videoInfo', video_id)
|
||||
json_all = self._search_json(r'window.videoInfo\s*=', webpage, 'videoInfo', video_id)
|
||||
|
||||
title = json_all.get('title')
|
||||
video_list = json_all.get('videoList') or []
|
||||
@ -161,10 +160,10 @@ class AcFunBangumiIE(AcFunVideoBaseIE):
|
||||
def _real_extract(self, url):
|
||||
video_id = self._match_id(url)
|
||||
ac_idx = parse_qs(url).get('ac', [None])[-1]
|
||||
video_id = f'{video_id}{format_field(ac_idx, template="__%s")}'
|
||||
video_id = f'{video_id}{format_field(ac_idx, None, "__%s")}'
|
||||
|
||||
webpage = self._download_webpage(url, video_id)
|
||||
json_bangumi_data = self._search_json(r'window.bangumiData\s*=\s*', webpage, 'bangumiData', video_id)
|
||||
json_bangumi_data = self._search_json(r'window.bangumiData\s*=', webpage, 'bangumiData', video_id)
|
||||
|
||||
if ac_idx:
|
||||
video_info = json_bangumi_data['hlVideoInfo']
|
||||
@ -181,7 +180,7 @@ def _real_extract(self, url):
|
||||
if v.get('id') == season_id), 1)
|
||||
|
||||
json_bangumi_list = self._search_json(
|
||||
r'window\.bangumiList\s*=\s*', webpage, 'bangumiList', video_id, fatal=False)
|
||||
r'window\.bangumiList\s*=', webpage, 'bangumiList', video_id, fatal=False)
|
||||
video_internal_id = int_or_none(traverse_obj(json_bangumi_data, ('currentVideoInfo', 'id')))
|
||||
episode_number = video_internal_id and next((
|
||||
idx for idx, v in enumerate(json_bangumi_list.get('items') or [], 1)
|
||||
|
@ -28,30 +28,34 @@
|
||||
|
||||
|
||||
class ADNIE(InfoExtractor):
|
||||
IE_DESC = 'Anime Digital Network'
|
||||
_VALID_URL = r'https?://(?:www\.)?animedigitalnetwork\.fr/video/[^/]+/(?P<id>\d+)'
|
||||
_TEST = {
|
||||
'url': 'http://animedigitalnetwork.fr/video/blue-exorcist-kyoto-saga/7778-episode-1-debut-des-hostilites',
|
||||
'md5': '0319c99885ff5547565cacb4f3f9348d',
|
||||
IE_DESC = 'Animation Digital Network'
|
||||
_VALID_URL = r'https?://(?:www\.)?(?:animation|anime)digitalnetwork\.fr/video/[^/]+/(?P<id>\d+)'
|
||||
_TESTS = [{
|
||||
'url': 'https://animationdigitalnetwork.fr/video/fruits-basket/9841-episode-1-a-ce-soir',
|
||||
'md5': '1c9ef066ceb302c86f80c2b371615261',
|
||||
'info_dict': {
|
||||
'id': '7778',
|
||||
'id': '9841',
|
||||
'ext': 'mp4',
|
||||
'title': 'Blue Exorcist - Kyôto Saga - Episode 1',
|
||||
'description': 'md5:2f7b5aa76edbc1a7a92cedcda8a528d5',
|
||||
'series': 'Blue Exorcist - Kyôto Saga',
|
||||
'duration': 1467,
|
||||
'release_date': '20170106',
|
||||
'title': 'Fruits Basket - Episode 1',
|
||||
'description': 'md5:14be2f72c3c96809b0ca424b0097d336',
|
||||
'series': 'Fruits Basket',
|
||||
'duration': 1437,
|
||||
'release_date': '20190405',
|
||||
'comment_count': int,
|
||||
'average_rating': float,
|
||||
'season_number': 2,
|
||||
'episode': 'Début des hostilités',
|
||||
'season_number': 1,
|
||||
'episode': 'À ce soir !',
|
||||
'episode_number': 1,
|
||||
}
|
||||
}
|
||||
},
|
||||
'skip': 'Only available in region (FR, ...)',
|
||||
}, {
|
||||
'url': 'http://animedigitalnetwork.fr/video/blue-exorcist-kyoto-saga/7778-episode-1-debut-des-hostilites',
|
||||
'only_matching': True,
|
||||
}]
|
||||
|
||||
_NETRC_MACHINE = 'animedigitalnetwork'
|
||||
_BASE_URL = 'http://animedigitalnetwork.fr'
|
||||
_API_BASE_URL = 'https://gw.api.animedigitalnetwork.fr/'
|
||||
_NETRC_MACHINE = 'animationdigitalnetwork'
|
||||
_BASE = 'animationdigitalnetwork.fr'
|
||||
_API_BASE_URL = 'https://gw.api.' + _BASE + '/'
|
||||
_PLAYER_BASE_URL = _API_BASE_URL + 'player/'
|
||||
_HEADERS = {}
|
||||
_LOGIN_ERR_MESSAGE = 'Unable to log in'
|
||||
@ -75,11 +79,11 @@ def _get_subtitles(self, sub_url, video_id):
|
||||
if subtitle_location:
|
||||
enc_subtitles = self._download_webpage(
|
||||
subtitle_location, video_id, 'Downloading subtitles data',
|
||||
fatal=False, headers={'Origin': 'https://animedigitalnetwork.fr'})
|
||||
fatal=False, headers={'Origin': 'https://' + self._BASE})
|
||||
if not enc_subtitles:
|
||||
return None
|
||||
|
||||
# http://animedigitalnetwork.fr/components/com_vodvideo/videojs/adn-vjs.min.js
|
||||
# http://animationdigitalnetwork.fr/components/com_vodvideo/videojs/adn-vjs.min.js
|
||||
dec_subtitles = unpad_pkcs7(aes_cbc_decrypt_bytes(
|
||||
compat_b64decode(enc_subtitles[24:]),
|
||||
binascii.unhexlify(self._K + '7fac1178830cfe0c'),
|
||||
@ -164,7 +168,7 @@ def _real_extract(self, url):
|
||||
}, data=b'')['token']
|
||||
|
||||
links_url = try_get(options, lambda x: x['video']['url']) or (video_base_url + 'link')
|
||||
self._K = ''.join([random.choice('0123456789abcdef') for _ in range(16)])
|
||||
self._K = ''.join(random.choices('0123456789abcdef', k=16))
|
||||
message = bytes_to_intlist(json.dumps({
|
||||
'k': self._K,
|
||||
't': token,
|
||||
@ -231,7 +235,6 @@ def _real_extract(self, url):
|
||||
for f in m3u8_formats:
|
||||
f['language'] = 'fr'
|
||||
formats.extend(m3u8_formats)
|
||||
self._sort_formats(formats)
|
||||
|
||||
video = (self._download_json(
|
||||
self._API_BASE_URL + 'video/%s' % video_id, video_id,
|
||||
|
@ -1352,7 +1352,7 @@
|
||||
}
|
||||
|
||||
|
||||
class AdobePassIE(InfoExtractor):
|
||||
class AdobePassIE(InfoExtractor): # XXX: Conventionally, base classes should end with BaseIE/InfoExtractor
|
||||
_SERVICE_PROVIDER_TEMPLATE = 'https://sp.auth.adobe.com/adobe-services/%s'
|
||||
_USER_AGENT = 'Mozilla/5.0 (X11; Linux i686; rv:47.0) Gecko/20100101 Firefox/47.0'
|
||||
_MVPD_CACHE = 'ap-mvpd'
|
||||
|
@ -70,7 +70,6 @@ def _parse_video_data(self, video_data):
|
||||
})
|
||||
s3_extracted = True
|
||||
formats.append(f)
|
||||
self._sort_formats(formats)
|
||||
|
||||
return {
|
||||
'id': video_id,
|
||||
@ -269,7 +268,6 @@ def _real_extract(self, url):
|
||||
'width': int_or_none(source.get('width') or None),
|
||||
'url': source_src,
|
||||
})
|
||||
self._sort_formats(formats)
|
||||
|
||||
# For both metadata and downloaded files the duration varies among
|
||||
# formats. I just pick the max one
|
||||
|
@ -180,7 +180,6 @@ def _real_extract(self, url):
|
||||
info['subtitles'].setdefault('en', []).append({
|
||||
'url': asset_url,
|
||||
})
|
||||
self._sort_formats(info['formats'])
|
||||
|
||||
return info
|
||||
else:
|
||||
|
@ -8,7 +8,7 @@
|
||||
)
|
||||
|
||||
|
||||
class AENetworksBaseIE(ThePlatformIE):
|
||||
class AENetworksBaseIE(ThePlatformIE): # XXX: Do not subclass from concrete IE
|
||||
_BASE_URL_REGEX = r'''(?x)https?://
|
||||
(?:(?:www|play|watch)\.)?
|
||||
(?P<domain>
|
||||
@ -62,7 +62,6 @@ def _extract_aen_smil(self, smil_url, video_id, auth=None):
|
||||
subtitles = self._merge_subtitles(subtitles, tp_subtitles)
|
||||
if last_e and not formats:
|
||||
raise last_e
|
||||
self._sort_formats(formats)
|
||||
return {
|
||||
'id': video_id,
|
||||
'formats': formats,
|
||||
@ -304,7 +303,6 @@ def _real_extract(self, url):
|
||||
class HistoryPlayerIE(AENetworksBaseIE):
|
||||
IE_NAME = 'history:player'
|
||||
_VALID_URL = r'https?://(?:www\.)?(?P<domain>(?:history|biography)\.com)/player/(?P<id>\d+)'
|
||||
_TESTS = []
|
||||
|
||||
def _real_extract(self, url):
|
||||
domain, video_id = self._match_valid_url(url).groups()
|
||||
|
40
yt_dlp/extractor/aeonco.py
Normal file
40
yt_dlp/extractor/aeonco.py
Normal file
@ -0,0 +1,40 @@
|
||||
from .common import InfoExtractor
|
||||
from .vimeo import VimeoIE
|
||||
|
||||
|
||||
class AeonCoIE(InfoExtractor):
|
||||
_VALID_URL = r'https?://(?:www\.)?aeon\.co/videos/(?P<id>[^/?]+)'
|
||||
_TESTS = [{
|
||||
'url': 'https://aeon.co/videos/raw-solar-storm-footage-is-the-punk-rock-antidote-to-sleek-james-webb-imagery',
|
||||
'md5': 'e5884d80552c9b6ea8d268a258753362',
|
||||
'info_dict': {
|
||||
'id': '1284717',
|
||||
'ext': 'mp4',
|
||||
'title': 'Brilliant Noise',
|
||||
'thumbnail': 'https://i.vimeocdn.com/video/21006315-1a1e49da8b07fd908384a982b4ba9ff0268c509a474576ebdf7b1392f4acae3b-d_960',
|
||||
'uploader': 'Semiconductor',
|
||||
'uploader_id': 'semiconductor',
|
||||
'uploader_url': 'https://vimeo.com/semiconductor',
|
||||
'duration': 348
|
||||
}
|
||||
}, {
|
||||
'url': 'https://aeon.co/videos/dazzling-timelapse-shows-how-microbes-spoil-our-food-and-sometimes-enrich-it',
|
||||
'md5': '4e5f3dad9dbda0dbfa2da41a851e631e',
|
||||
'info_dict': {
|
||||
'id': '728595228',
|
||||
'ext': 'mp4',
|
||||
'title': 'Wrought',
|
||||
'thumbnail': 'https://i.vimeocdn.com/video/1484618528-c91452611f9a4e4497735a533da60d45b2fe472deb0c880f0afaab0cd2efb22a-d_1280',
|
||||
'uploader': 'Biofilm Productions',
|
||||
'uploader_id': 'user140352216',
|
||||
'uploader_url': 'https://vimeo.com/user140352216',
|
||||
'duration': 1344
|
||||
}
|
||||
}]
|
||||
|
||||
def _real_extract(self, url):
|
||||
video_id = self._match_id(url)
|
||||
webpage = self._download_webpage(url, video_id)
|
||||
vimeo_id = self._search_regex(r'hosterId":\s*"(?P<id>[0-9]+)', webpage, 'vimeo id')
|
||||
vimeo_url = VimeoIE._smuggle_referrer(f'https://player.vimeo.com/video/{vimeo_id}', 'https://aeon.co')
|
||||
return self.url_result(vimeo_url, VimeoIE)
|
Some files were not shown because too many files have changed in this diff Show More
Loading…
Reference in New Issue
Block a user