198 Commits

Author	SHA1	Message	Date
pukkandan	981052c9c6	Some minor fixes and refactoring (see desc) ... * [utils] Fix issues with reversal * check_formats should catch `DownloadError`, not `ExtractorError` * Simplify format selectors with `LazyList` and `yield from`	2021-07-02 08:17:37 +05:30
pukkandan	49c258e18d	[youtube] Fix subtitle names for age-gated videos ... Related: https://github.com/iv-org/invidious/pull/2205#issuecomment-868680486	2021-06-25 23:10:31 +05:30
pukkandan	d3f62c1967	Fix --throttled-rate when using --load-info-json	2021-06-25 22:57:17 +05:30
pukkandan	5d3a0e794b	Add --extractor-args to pass extractor-specific arguments	2021-06-25 20:10:28 +05:30
Adrik	4513a41a72	Process videos when using --ignore-no-formats-error (#441) ... Authored by: krichbanana	2021-06-24 22:23:34 +05:30
pukkandan	6033d9808d	Fix --flat-playlist when entry has no ie_key	2021-06-24 22:23:34 +05:30
pukkandan	51d9739f80	Add option --throttled-rate below which video data is re-extracted ... Currently only for HTTP downloads Closes #430, workaround for https://github.com/ytdl-org/youtube-dl/issues/29326	2021-06-23 05:29:58 +05:30
pukkandan	e36d50c5dd	[websockets] Add WebSocketFragmentFD (#399) ... Necessary for #392 Co-authored by: nao20010128nao, pukkandan	2021-06-21 22:56:36 +05:30
pukkandan	f89b3e2d7a	Skip fixup of existing files and add --fixup force to force it	2021-06-20 04:59:34 +05:30
pukkandan	fd7cfb6444	[cleanup] Refactor fixup	2021-06-20 04:26:11 +05:30
pukkandan	9fea350f0d	Fix id sanitization in filenames ... Closes #415	2021-06-17 02:32:24 +05:30
pukkandan	e858a9d6d3	[EmbedThumbnail] Add compat-option embed-thumbnail-atomicparsley ... to force use of atomicparsley for embedding thumbnails in mp4 Related: #411	2021-06-16 22:33:32 +05:30
pukkandan	551f93885e	Ignore images formats from merge	2021-06-13 04:16:42 +05:30
pukkandan	8326b00aab	Allow images formats ... Necessary for #343. * They are identified by `vcodec=acodec='none'` * These formats show as the worst in `-F` * Any postprocessor that expects audio/video will be skipped * `b*` and all related selectors will skip such formats * This commit also does not add any selector for downloading such formats. They have to be explicitly requested by the `format_id`. Implementation of a selector is left for when #389 is resolved	2021-06-13 03:45:53 +05:30
pukkandan	b0249bcaf0	Expand --check-formats to thumbnails ... Closes #402	2021-06-13 03:45:53 +05:30
pukkandan	21cd8fae49	Use NamedTemporaryFile for --check-formats	2021-06-13 03:45:53 +05:30
pukkandan	8ba8714880	[EmbedThumbnail] Fix for already downloaded thumbnail	2021-06-11 19:13:24 +05:30
pukkandan	187986a857	Better error handling of syntax errors in -f	2021-06-11 19:13:22 +05:30
pukkandan	0181adefc6	[build] Build Windows x86 version with py3.7 ... and remove redundant tests Closes #390 :ci skip Co-authored by: pukkandan, shirt-dev	2021-06-10 01:41:04 +05:30
pukkandan	385a27fad1	Improve offset parsing in outtmpl	2021-06-09 20:01:57 +05:30
pukkandan	639f1cea92	Fix %d and empty default in outtmpl ... Closes #388	2021-06-09 15:37:15 +05:30
pukkandan	b5c5d84f60	Revert "[build] Build Windows x86 version with py3.8" ... This reverts commit aa75e51f99. See #390 This is being reverted instead of modified due to #388	2021-06-09 15:37:15 +05:30
pukkandan	aa75e51f99	[build] Build Windows x86 version with py3.8 ... and remove redundant tests :ci skip Ao-authored by: pukkandan, shirt-dev	2021-06-09 02:18:55 +05:30
pukkandan	76a264ac9e	Make outtmpl more robust and catch errors early	2021-06-08 20:11:00 +05:30
pukkandan	324ad82006	[utils] Generalize traverse_dict to traverse_obj	2021-06-08 19:26:44 +05:30
pukkandan	bd99f6e648	Add field original_url with the user-inputted URL ... So that they can be processed by `--parse-metadata` for example `webpage_url` is the same, but may be modified by the extractor	2021-06-08 00:20:06 +05:30
pukkandan	ecb5419149	Make more fields available for --print when used with --flat-playlist	2021-06-08 00:17:53 +05:30
pukkandan	87ea7dfc04	Fix filename sanitization ... Bug from 752cda3880	2021-06-06 19:36:28 +05:30
pukkandan	752cda3880	Fix and refactor prepare_outtmpl ... The following tests would have failed previously: %(id)d %(id)r %(ext)s-%(ext|def)d %(width|)d %(id)r %(height)r %(formats.0)r %s	2021-06-06 00:59:04 +05:30
felix	cc52de4356	[cleanup] Point all shebang to python3 (#372) ... Authored by: fstirlitz	2021-06-06 00:59:04 +05:30
pukkandan	885cc0b75c	[embedthumbnail] Embed if any thumbnail was downloaded, not just the best	2021-05-29 02:31:14 +05:30
pukkandan	ae8f99e648	Remove None values from info.json	2021-05-29 02:31:13 +05:30
pukkandan	835a1478b4	Write messages to stderr when both quiet and verbose	2021-05-29 02:31:10 +05:30
pukkandan	120fe5134a	Pre-check archive and filters during playlist extraction ... This makes `--break-on-existing` much faster. It also helps `--break-on-reject` if the playlist extractor can extract the relevant fields	2021-05-29 02:12:09 +05:30
pukkandan	56a8fb4f77	Refactor __process_playlist using LazyList	2021-05-29 02:12:09 +05:30
pukkandan	c77495e3a4	[cleanup] _match_entry	2021-05-29 02:12:07 +05:30
pukkandan	c19bc311cb	[cleanup] Refactor updater ... The updater now uses `.update.run_update` and not `.update.update_self`. Although I don't expect anyone to be using the updater via API, a wrapper `update_self` is provided for compatibility just in case	2021-05-26 01:13:08 +05:30
pukkandan	bc516a3f3c	Sanitize and sort playlist thumbnails ... Closes #341	2021-05-23 17:28:15 +05:30
pukkandan	18e674b4f6	[ffmpeg] Download and merge in a single step if possible	2021-05-23 03:53:18 +05:30
pukkandan	28b0eb0f65	[cleanup] See desc ... * Remove struct from `embedthumbnail` * Use bullet lists in readme where numbered list don't make sense * Fix error introduced in 9c2b75b561 when `ie_result` is `None`	2021-05-20 18:02:58 +05:30
pukkandan	9c2b75b561	Field additional_urls to download additional videos from metadata	2021-05-19 18:11:15 +05:30
pukkandan	4ec82a72bb	Ensure post_extract and pre_process only run once ... Previously, they ran once for each format requested	2021-05-19 16:48:22 +05:30
pukkandan	07cce701de	[cleanup] linter, code formatting and readme	2021-05-19 16:48:20 +05:30
pukkandan	ff2751ac9c	[youtube] Always extract maxresdefault thumbnail ... Fixes: https://github.com/ytdl-org/youtube-dl/issues/29049	2021-05-18 19:31:17 +05:30
pukkandan	a06916d98e	[extractor] Add write_debug and get_param	2021-05-17 18:59:51 +05:30
pukkandan	681de68e9d	Write thumbnail of playlist ... Related: https://github.com/ytdl-org/youtube-dl/pull/28872, https://github.com/ytdl-org/youtube-dl/pull/28860 This is slightly different from the above PRs in that this downloads the playlist's thumbnail instead of the uploader's profile picture. But for youtube channel URLs these are the same	2021-05-17 18:24:17 +05:30
pukkandan	7aee40c13c	Fix bug in listing subtitles ... Bug introduced by: 2412044c90	2021-05-17 18:24:16 +05:30
pukkandan	2412044c90	Add field name for subtitles ... Co-authored by: pukkandan, tpikonen Based on: #310, https://github.com/ytdl-org/youtube-dl/pull/26112	2021-05-15 19:27:52 +05:30
pukkandan	fe346461ff	Fix --check-formats when there is network error	2021-05-15 19:26:01 +05:30
pukkandan	53c18592d3	Add option --print ... Deprecates: `--get-description`, `--get-duration`, `--get-filename`, `--get-format`, `--get-id`, `--get-thumbnail`, `--get-title`, `--get-url` Closes #295	2021-05-14 13:15:47 +05:30
pukkandan	e632bce2e4	[options] Refactor callbacks	2021-05-14 13:15:47 +05:30
pukkandan	0760b0a7e2	Standardize write_debug	2021-05-14 13:15:29 +05:30
pukkandan	d908aa636a	[cleanup] Fix typos	2021-05-11 23:34:40 +05:30
pukkandan	4171221823	Add compat-option no-attach-infojson	2021-05-11 14:25:31 +05:30
pukkandan	53ed7066ab	Option --compat-options to revert some of yt-dlp's changes ... * Deprecates `--list-formats-as-table`, `--list-formats-old`	2021-05-11 13:30:48 +05:30
pukkandan	a61f4b287b	Deprecate support for python versions < 3.6 ... Closes #267	2021-05-09 04:32:23 +05:30
pukkandan	41d1cca328	Update to ytdl-commit-a726009 ... [blinkx] Remove extractor a726009987	2021-05-06 21:31:20 +05:30
pukkandan	717297545b	Fix playlist_index and add playlist_autonumber (#302) ... Now `playlist_index` is always the position of the video in the actual playlist and `playlist_autonumber` is the position of the item in the playlist queue	2021-05-06 20:56:19 +05:30
pukkandan	e8e738406a	Add experimental option --check-formats to test the URLs before format selection	2021-05-06 20:50:44 +05:30
pukkandan	e625be0d10	Improve output template internal formatting ... * Allow slicing lists/strings using `field.start🔚step` * A field can also be used as offset like `field1+num+field2` * A default value can be given using `field|default` * Capture all format strings and set it to `None` if invalid. This prevents invalid fields from causing errors	2021-05-06 20:28:58 +05:30
pukkandan	848887eb7a	[downloader] Fix quiet and to_stderr	2021-05-04 22:38:10 +05:30
pukkandan	3158150cb7	[utils] Add network_exceptions	2021-05-04 22:36:18 +05:30
pukkandan	f59ae58163	Fix number of digits in %(playlist_index)s ... When used with `--playlist-(items|start|end)`, the number of digits should depend on the last index in the playlist, not number of items	2021-05-03 22:49:05 +05:30
pukkandan	0d1bb027aa	Move option warnings to YoutubeDL ... Previously, these warnings did not obey `--no-warnings` and did not output colors	2021-05-03 22:49:04 +05:30
pukkandan	4cd0a709aa	Fix preload_download_archive writing verbose message to stdout ... * And move it after all deprecated warnings	2021-05-03 22:49:03 +05:30
pukkandan	58f197b76c	Revert "[core] be able to hand over id and title using url_result" ... This reverts commit 0704d2224b. This is a commit from `youtube-dlc`. It is not clear what the original purpose of this was. It seems to be a way for extractors to pass `title` and `id` through when the entry is processed by another extractor * But `title` can already be passed through using `url_transparent` * `id` is never supposed to be passed through since it could cause issues with archiving	2021-04-28 19:18:06 +05:30
pukkandan	598d185db1	Fix case sensitivity of format selector ... Bug introduced in f8d4ad9ab0	2021-04-26 10:56:56 +05:30
pukkandan	a439a3a45c	Improve output template (see desc) ... * Objects can be traversed like `%(field.key1.key2)s` * A number can be added to the field as `%(field+n)s` * Deprecates `--autonumber-start`	2021-04-22 04:19:33 +05:30
pukkandan	26e2805c3f	Add option --skip-playlist-after-errors ... Allows to skip the rest of a playlist after a given number of errors are encountered	2021-04-22 02:16:31 +05:30
pukkandan	c32b0aab8a	Improve --sub-langs (see desc) ... * Treat `--sub-langs` entries as regex * `all` can be used to refer to all the subtitles * the language code can be prefixed with `-` to exclude it * Deprecates `--all-subs` Closes #253	2021-04-20 02:58:03 +05:30
pukkandan	b7da73eb19	Add option --ignore-no-formats-error ... * Ignores the "no video format" and similar errors * Experimental - Some extractors may still throw these errors	2021-04-17 08:40:30 +05:30
pukkandan	921b76cab8	Ensure mergeall selects best format when multistreams are disabled	2021-04-13 10:53:25 +05:30
pukkandan	56d868dbb7	Allow running some postprocessors before actual download	2021-04-12 02:12:46 +05:30
pukkandan	e01d6aa435	Fix mergeall when requested formats are unavailable	2021-04-10 22:29:58 +05:30
pukkandan	52a8a1e1b9	Option to choose different downloader for different protocols ... * Renamed `--external-downloader-args` to `--downloader-args` * Added `native` as an option for the downloader * Use similar syntax to `--downloader-args` etc. Eg: `--downloader dash:native --downloader aria2c` * Deprecated `--hls-prefer-native` and `--hls-prefer-ffmpeg` since the same can now be done with `--downloader "m3u8:native"` and `m3u8:ffmpeg` respectively * Split `frag_urls` protocol into `m3u8_frag_urls` and `dash_frag_urls` * Standardize shortening of protocol names with `downloader.shorten_protocol_name`	2021-04-10 20:57:52 +05:30
pukkandan	f8d4ad9ab0	Format selector mergeall to download and merge all formats	2021-04-10 20:57:27 +05:30
pukkandan	eff635394a	Ability to select nth best format ... Eg: -f `bv*.2` * Also cleaned up the related format selection code	2021-04-02 22:16:52 +05:30
pukkandan	b050d210df	Fix some typos and linter	2021-04-01 14:16:10 +05:30
pukkandan	143db31d48	Parse metadata from multiple fields ... Closes #196	2021-03-25 03:32:15 +05:30
pukkandan	498f560638	Ability to load playlist infojson ... * If `--no-clean-infojson` is given, the video ids are saved/loaded from in the infojson along with their playlist index * If a video entry that was not saved is requested, we fallback to using `webpage_url` to re-extract the entries Related: https://github.com/yt-dlp/yt-dlp/issues/190#issuecomment-804921024	2021-03-24 01:57:50 +05:30
pukkandan	394dcd4486	Write current epoch to infojson when using --no-clean-infojson ... Closes #191	2021-03-24 00:34:33 +05:30
pukkandan	4d971a16b8	Merge webm formats into mkv if thumbnails are to be embedded (#173) ... Co-authored by: Damiano Amatruda <damiano.amatruda@outlook.com>	2021-03-20 09:34:52 +05:30
Damiano Amatruda	3561530776	Parse resolution in info dictionary (#173)	2021-03-20 09:34:52 +05:30
Damiano Amatruda	4690688658	Use headers and cookies when downloading subtitles (#173)	2021-03-20 09:34:52 +05:30
pukkandan	23c1a66730	Make sure post_hook gets the final filename	2021-03-20 09:31:53 +05:30
pukkandan	a515a78dd3	fix some typos and linter	2021-03-18 22:19:31 +05:30
pukkandan	75d43ca080	Option to keep private keys in the infojson ... Options: --clean-infojson, --no-clean-infojson Related: https://github.com/yt-dlp/yt-dlp/issues/42#issuecomment-800778391	2021-03-18 21:00:45 +05:30
pukkandan	5226731e2d	Recursively remove private keys from infojson ... Related: https://github.com/yt-dlp/yt-dlp/issues/42	2021-03-18 21:00:45 +05:30
pukkandan	dcf64d43e0	[movefiles] Fix bugs and make more robust	2021-03-18 20:54:53 +05:30
pukkandan	10db0d2f57	Update to ytdl-commit-3be0980 ... 3be098010f	2021-03-15 04:52:06 +05:30
pukkandan	e92caff5d5	Refactor (See desc) ... * Create `FFmpegPostProcessor.real_run_ffmpeg` that can accept multiple input/output files along with switches for each * Rewrite `cli_configuration_args` and related functions * Create `YoutubeDL._ensure_dir_exists` - this was previously defined in multiple places	2021-03-14 20:02:55 +05:30
pukkandan	0a473f2f0f	More improvements to HLS/DASH external downloader code ... * Fix error when there is no `protocol` in `info_dict` * Move HLS byte range detection to `Aria2cFD` so that the download will fall back to the native downloader instead of ffmpeg * Fix bug with getting no fragments in DASH * Convert `check_results` in `can_download` to a generator	2021-03-11 22:07:42 +05:30
shirt-dev	d7009caa03	Improve HLS/DASH external downloader code (#162) ... Authored by: shirt	2021-03-10 20:09:40 +05:30
pukkandan	5ef7d9bdd8	Release 2021.03.01	2021-03-01 05:39:50 +05:30
pukkandan	62bff2c170	Add option --extractor-retries to retry on known extractor errors ... * Currently only used by youtube Fixes https://github.com/ytdl-org/youtube-dl/issues/28194 Possibly also fixes: https://github.com/ytdl-org/youtube-dl/issues/28289 (can not confirm since the issue isn't reliably reproducible)	2021-03-01 05:18:37 +05:30
pukkandan	277d6ff5f2	Extract comments only when needed #95 (Closes #94)	2021-02-28 20:26:08 +05:30
pukkandan	1cf376f55a	Add option --sleep-requests to sleep b/w requests (Closes #106) ... * Also fix documentation of `sleep_interval_subtitles` Related issues: https://github.com/blackjack4494/yt-dlc/issues/158 https://github.com/blackjack4494/youtube-dlc/issues/195 https://github.com/ytdl-org/youtube-dl/pull/28270 https://github.com/ytdl-org/youtube-dl/pull/28144 https://github.com/ytdl-org/youtube-dl/issues/27767 https://github.com/ytdl-org/youtube-dl/issues/23638 https://github.com/ytdl-org/youtube-dl/issues/26287 https://github.com/ytdl-org/youtube-dl/issues/26319	2021-02-27 18:14:42 +05:30
Pccode66	7a5c1cfe93	Completely change project name to yt-dlp (#85) ... * All modules and binary names are changed * All documentation references changed * yt-dlp no longer loads youtube-dlc config files * All URLs changed to point to organization account Co-authored-by: Pccode66 Co-authored-by: pukkandan	2021-02-25 00:15:56 +05:30