diff --git a/CHANGELOG.md b/CHANGELOG.md index 8ef05628..ea8078fe 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -21,7 +21,7 @@ That's it! **No need** to set up thread pools (that don't even work in PHP), use synchronization primitives, and so on... Just `yield $MadelineProto->messages->sendMessage` instead of `$MadelineProto->messages->sendMessage`. -~~~~ +*** Naturally, async is not the only feature present in MadelineProto 4.0: to implement async in MadelineProto, I rewrote the **entire codebase two times** over the course of the last six months, as shown in the diff: ``` @@ -40,7 +40,9 @@ The simplified async buffered implementation of AMPHP sockets I created allowed Obfuscation is now handled via a separate setting, and can be enabled for **all** transports, not just `tcp_abridged` (it can be used to prevent ISP blocks in combination with the padded protocol, for example). I also added support for different **MTProto transports**, including the brand new **websocket transport** and **secure websocket transport (HTTPS)**, until now only implemented in tdlib! -If you want all communcations with telegram to be **double** or **triple**-encrypted using HTTPS+MTProto+obfuscation, you can enable the `wss` transport. +If you want all communcations with telegram to be **triple**-encrypted using HTTPS+MTProto+obfuscation, you can enable the `wss` transport (the third layer of encryption w/ obfuscation will be enabled automatically). +Note: the **websocket HTTPS MTProto transport** is different from the **HTTPS MTProto protocol** (both are supported by MadelineProto). +The **websocket HTTPS MTProto transport** is more reliable and faster than the **HTTPS MTProto protocol**, since MadelineProto does not have to handle HTTP long polls. (the websocket transport may or may not also be used in future to implement MadelineProto in the browser with webassembly ;) @@ -48,37 +50,70 @@ The new [proxy API](https://docs.madelineproto.xyz/docs/PROXY.html) is also base You can still use the socks5 and HTTP transports if you want. [Writing your custom proxies](https://docs.madelineproto.xyz/docs/PROXY.html#build-your-proxy) now is now a LOT easier, thanks to the neat structure of the stream APIs and the abundant PHPDOC comments. -~~~ +*** -Streams and proxies aren't the only things that have changed in this release: -I have written a **unique** socket message handler API based on [MadelineProto's loop API](https://docs.madelineproto.xyz/docs/ASYNC.html#async-loop-apis): and greatly simplifies +Streams and proxies aren't the only things that have been completely rewritten in this release: +I have written a **unique** socket message handler API based on [MadelineProto's loop API](https://docs.madelineproto.xyz/docs/ASYNC.html#async-loop-apis), and guarantees **maximum reliability** while communicating with telegram. +By the way, the new async background loop API can also be used by **you**! It is perfect for repeating tasks in a cron-like manner, running multiple resumable event loops and much more! More on that later. + +The new **message handler loops** run in the background, and guarantee that every single method call you make gets delivered to telegram. +**Two signal loops** running in two separate green threads take care of writing and reading data from the socket asynchronously. +A **third signal loop** uses state request messages to make sure that the method calls were received by telegram, and queries replies/resend method calls if something's wrong. +A **fourth signal loop** takes care of HTTP long polling. + +This guarantees maximum stability even if telegram's having server issues. +The write loop also greatly reduces overhead, increasing performances by automatically wrapping in containers multiple method calls: this is especially useful when making multiple method calls simultaneously with async (more on that later). + +*** + +Possibly the most __exciting__ thing to work on in this version of MadelineProto was the new **update management system**: I whipped it up in merely two days a few weeks ago, and it has **absolutely improved** the overall reliability of MadelineProto. +Huge thanks to Aliaksei Levin, the developer of tdlib, for explaining to me how exactly does the MTProto update API work: he saved me a lot of time, and was really nice <3<3<3. + +While thinking of an easy way I could implement the new update system, I had an inspiration: + +I created a **unique**, **reliable**, **fast** and **extremely simple** update handling system based on [MadelineProto's loop API](https://docs.madelineproto.xyz/docs/ASYNC.html#async-loop-apis), **not present in any** other MTProto client, **not even tdlib**. + +Instead of messing around with various synchronization locks, checks and cluttered update receivers, **I implemented three simple feed loops**. +Updates are fed to the first update loop, where some simple duplicate/hole checks are done. +Updates are then fed to a second update loop, where secondary duplicate/hole checks are done. +A third loop type takes care of eventual holes and periodically fetches updates for all supergroups, channels. + +All of this is implemented without **any** kind of additional synchronization or locking due to the nature of the **MadelineProto loop APIs**, with **1%** of the complexity of the official clients (which means less bugs and more pony time for me). -Let's elaborate on this: when I say **thread**, I actually mean **green thread**, often called **strand**. -**Strands** are behave exactly like normal **threads**, except that they're implemented in user-space, and they're much **faster**, **more reliable**, and **do not suffer** from synchronization issues present with normal threads. +*** + +Even if most of MadelineProto's logic is now concentrated in the new **loop** and **stream** APIs, some parts like the TL parser are still there, albeit with many changes. +For example: now the TL parser is fully asynchronous (that may sound a bit weird to some of you, but for dynamic TL parsers, this greatly increases performances and allows parallelism). + +The TL deserializer now uses yet another well-structured API called the **TLCallback API** to automatically populate internal databases directly on deserialization: again, this paradigm greatly **reduces complexity**, **increases performance** and is **absolutely unique** to MadelineProto; no other MTProto client has it, not even official clients. + +One of the **completely new** modules that I created for MadelineProto async is the **file reference database**: a **very complex** module, required to handle files in the newer versions of the Telegram API. +It makes use of the **TLCallback** API to map files to their origins, to be able to refetch them at any given time when the file reference expires. + +Another new module I've implemented is the **PasswordCalculator**, that is used to calculate the new password hashes for the 2FA login, __really cool__. + +I've also rewritten the **APIFactory**, the abstraction layer that stands between you and MadelineProto when you do `$MadelineProto->method()`: it is now fully async, and MUCH faster thanks to a new cached method mapping system. +The same cached method mapping system is also used for the **event handler**, which means that now the **event handler is the fastest update management method**. -* Fully rewritten connection stack, with support for websockets, stuff -* updates -* simultaneous method calls -* new TL callback system -* added support for wallpapers +*** + +And now, on to the **API changes and improvements**: +* First of all, we've got several bucketloads of telegram API changes, that can be viewed in the first posts +* Added an [update_2fa](https://docs.madelineproto.xyz/update_2fa.html) method to update the login password +* Added a [get_full_dialogs](https://docs.madelineproto.xyz/docs/DIALOGS.html#get_full_dialogs-now-fully-async) method to get a full list of all chats you’re member of, including dialog info (such as the pinned/last message ID, unread count, tag count, notification settings and message drafts). +* [Added support for automatic file uploads by name in secret chats (as with normal chats); you can also now send secret chat messages using the sendMessage method as if it were a normal chat](https://github.com/danog/MadelineProto/blob/master/secret_bot.php) * Improved message splitting algorithm: performance improvements, and it will now notify you via the logs if there are too many entities in the logs, or if the entities are too long. -* Improved get_self method. -* reference database -* Rewritten proxy stack +* Improved the get_self method. + * magic sleep -* get_full_dialogs -* new APIfactory +* Simultaneous method calls * sendmessage with secret messages * automatic secret chat file upload -* 2fa+++++ * improved callfork -* split acks * new logging -* TL callabck * channel state -* logger * async construct * clean up repo, update dependencies and remove curl dependency * new phone call config @@ -99,12 +134,21 @@ Let's elaborate on this: when I say **thread**, I actually mean **green thread** * arrayaccess on args * increased flood wait +*** + +And now, let's elaborate on async: +With **MadelineProto 4.0**, each update is handled in **parallel** using a separate **thread**, and everything is done in **parallel** (even on restricted webhosts!). + +When I say **thread**, I actually mean **green thread** ([wikipedia](https://en.wikipedia.org/wiki/Green_threads)), often called **strand**. +**Strands** are behave exactly like normal **threads**, except that they're implemented in user-space, and they're much **faster**, **more reliable**, and **do not suffer** from synchronization issues present with normal threads. + +In + + Things to expect in the next releases: docs for get mime funcs +docs for HTML parser (div to avoid escaping) docs for update_2fa -docs for ResponseException -docs for PTSException -Document async apis optional max_id and min_id async iterators Method name changes @@ -123,5 +167,10 @@ do not use manual tell about madeline.php loading in the same dire arrayaccess on promises get sponsor of +ton +video calls +group calls +native calls +dnssec telegram passport \ No newline at end of file diff --git a/build_docs.php b/build_docs.php index 124004b8..bdad2ea0 100755 --- a/build_docs.php +++ b/build_docs.php @@ -27,14 +27,14 @@ image: https://docs.madelineproto.xyz/favicons/android-chrome-256x256.png '.str_replace(' ['td' => __DIR__.'/src/danog/MadelineProto/TL_td.tl'], 'title' => 'MadelineProto API documentation (td-lib)', 'description' => 'MadelineProto API documentation (td-lib)', 'output_dir' => __DIR__.'/docs/docs/TD_docs', 'readme' => false, 'td' => true, - ], + ],*/ [ 'tl_schema' => ['mtproto' => __DIR__.'/src/danog/MadelineProto/TL_mtproto_v1.json'], 'title' => 'MadelineProto API documentation (mtproto)', diff --git a/docs b/docs index ae6f992a..6fe646f4 160000 --- a/docs +++ b/docs @@ -1 +1 @@ -Subproject commit ae6f992aef745fecfa989ad01f959a467d94c7ae +Subproject commit 6fe646f4a22e8e0ccf5d8a8c562c6d7499a8a17c diff --git a/phar.php b/phar.php index 005571c7..a5ea78ec 100644 --- a/phar.php +++ b/phar.php @@ -1,29 +1,36 @@ '.$newline; - } - echo "**********************************************************************$newline"; - echo "**********************************************************************$newline$newline"; - echo "YOU ARE USING AN OLD AND BUGGED VERSION OF PHP, PLEASE UPDATE TO PHP 7$newline"; - echo "PHP 5 USERS WILL NOT RECEIVE MADELINEPROTO UPDATES AND BUGFIXES$newline$newline"; - echo "SUPPORTED VERSIONS: PHP 7.0, 7.1, 7.2, 7.3+$newline"; - echo "RECOMMENDED VERSION: PHP 7.3$newline$newline"; - echo "**********************************************************************$newline"; - echo "**********************************************************************$newline"; - unset($newline); -} - function ___install_madeline() { if (count(debug_backtrace(0)) === 1) { die('You must include this file in another PHP script'.PHP_EOL); } + $old = false; + if (PHP_MAJOR_VERSION === 5) { + if (PHP_MINOR_VERSION < 6) { + throw new \Exception('MadelineProto requires at least PHP 7.1 to run'); + } + $old = true; + } + if (PHP_MAJOR_VERSION === 7 && PHP_MINOR_VERSION === 0) { + $old = true; + } + if ($old) { + $newline = PHP_EOL; + if (php_sapi_name() !== 'cli') { + $newline = '
'.$newline; + } + echo "**********************************************************************************$newline"; + echo "**********************************************************************************$newline$newline"; + echo "YOU ARE USING AN OLD AND BUGGED VERSION OF PHP, PLEASE UPDATE TO PHP 7.3$newline"; + echo "PHP 5/7.0 USERS WILL NOT RECEIVE PHP UPDATES AND BUGFIXES: https://www.php.net/eol.php$newline"; + echo "PHP 5/7.0 USERS WILL NOT RECEIVE MADELINEPROTO UPDATES AND BUGFIXES$newline$newline"; + echo "SUPPORTED VERSIONS: PHP 7.1, 7.2, 7.3+$newline"; + echo "RECOMMENDED VERSION: PHP 7.3$newline$newline"; + echo "**********************************************************************************$newline"; + echo "**********************************************************************************$newline"; + unset($newline); + } // MTProxy update $file = debug_backtrace(0, 1)[0]['file']; diff --git a/src/danog/MadelineProto/DocsBuilder/Methods.php b/src/danog/MadelineProto/DocsBuilder/Methods.php index a17f1ccf..c87138a4 100644 --- a/src/danog/MadelineProto/DocsBuilder/Methods.php +++ b/src/danog/MadelineProto/DocsBuilder/Methods.php @@ -367,6 +367,8 @@ image: https://docs.madelineproto.xyz/favicons/android-chrome-256x256.png * [Login](https://docs.madelineproto.xyz/docs/LOGIN.html) +* [Change 2FA password](https://docs.madelineproto.xyz/update_2fa.html) + * [Get all chats, broadcast a message to all chats](https://docs.madelineproto.xyz/docs/DIALOGS.html) * [Get the full participant list of a channel/group/supergroup](https://docs.madelineproto.xyz/get_pwr_chat.html) diff --git a/src/danog/MadelineProto/MTProto.php b/src/danog/MadelineProto/MTProto.php index 5aea908d..f8e0eeaf 100644 --- a/src/danog/MadelineProto/MTProto.php +++ b/src/danog/MadelineProto/MTProto.php @@ -249,6 +249,11 @@ class MTProto extends AsyncConstruct implements TLCallback return $this->datacenter->getHTTPClient(); } + public function fileGetContents($url): \Generator + { + return $this->datacenter->fileGetContents($url); + } + public function __wakeup() { $backtrace = debug_backtrace(0, 3); @@ -595,9 +600,9 @@ class MTProto extends AsyncConstruct implements TLCallback 2 => [ // The rest will be fetched using help.getConfig 'ip_address' => '149.154.167.40', - 'port' => 443, + 'port' => 443, 'media_only' => false, - 'tcpo_only' => false, + 'tcpo_only' => false, ], ], 'ipv6' => [ @@ -605,9 +610,9 @@ class MTProto extends AsyncConstruct implements TLCallback 2 => [ // The rest will be fetched using help.getConfig 'ip_address' => '2001:067c:04e8:f002:0000:0000:0000:000e', - 'port' => 443, + 'port' => 443, 'media_only' => false, - 'tcpo_only' => false, + 'tcpo_only' => false, ], ], ], @@ -618,9 +623,9 @@ class MTProto extends AsyncConstruct implements TLCallback 2 => [ // The rest will be fetched using help.getConfig 'ip_address' => '149.154.167.51', - 'port' => 443, + 'port' => 443, 'media_only' => false, - 'tcpo_only' => false, + 'tcpo_only' => false, ], ], 'ipv6' => [ @@ -628,9 +633,9 @@ class MTProto extends AsyncConstruct implements TLCallback 2 => [ // The rest will be fetched using help.getConfig 'ip_address' => '2001:067c:04e8:f002:0000:0000:0000:000a', - 'port' => 443, + 'port' => 443, 'media_only' => false, - 'tcpo_only' => false, + 'tcpo_only' => false, ], ], ], @@ -651,17 +656,17 @@ class MTProto extends AsyncConstruct implements TLCallback 'proxy_extra' => Magic::$altervista ? ['address' => 'localhost', 'port' => 80] : [], // Extra parameters to pass to the proxy class using setExtra 'obfuscated' => false, - 'transport' => 'tcp', - 'pfs' => extension_loaded('gmp'), + 'transport' => 'tcp', + 'pfs' => extension_loaded('gmp'), ], 'default_dc' => 2, ], 'app_info' => [ // obtained in https://my.telegram.org //'api_id' => you should put an API id in the settings array you provide //'api_hash' => you should put an API hash in the settings array you provide - 'device_model' => $device_model, + 'device_model' => $device_model, 'system_version' => $system_version, - 'app_version' => $app_version, + 'app_version' => $app_version, // 🌚 // 'app_version' => self::V, 'lang_code' => $lang_code, @@ -695,10 +700,10 @@ class MTProto extends AsyncConstruct implements TLCallback */ // write to 'logger_param' => Magic::$script_cwd.'/MadelineProto.log', - 'logger' => php_sapi_name() === 'cli' ? 3 : 2, + 'logger' => php_sapi_name() === 'cli' ? 3 : 2, // overwrite previous setting and echo logs 'logger_level' => Logger::VERBOSE, - 'max_size' => 100 * 1024 * 1024, + 'max_size' => 100 * 1024 * 1024, // Logging level, available logging levels are: ULTRA_VERBOSE, VERBOSE, NOTICE, WARNING, ERROR, FATAL_ERROR. Can be provided as last parameter to the logging function. 'rollbar_token' => '', ], 'max_tries' => [ @@ -707,10 +712,10 @@ class MTProto extends AsyncConstruct implements TLCallback 'authorization' => 5, // How many times should I try to generate an authorization key before throwing an exception 'response' => 5, - ], 'flood_timeout' => ['wait_if_lt' => 10*60], 'msg_array_limit' => [ + ], 'flood_timeout' => ['wait_if_lt' => 10 * 60], 'msg_array_limit' => [ // How big should be the arrays containing the incoming and outgoing messages? - 'incoming' => 100, - 'outgoing' => 100, + 'incoming' => 100, + 'outgoing' => 100, 'call_queue' => 200, ], 'peer' => [ 'full_info_cache_time' => 3600, @@ -734,10 +739,10 @@ class MTProto extends AsyncConstruct implements TLCallback 'handler_workers' => 10, ], 'upload' => [ 'allow_automatic_upload' => true, - 'part_size' => 512 * 1024, + 'part_size' => 512 * 1024, ], 'download' => [ 'report_broken_media' => true, - 'part_size' => 1024 * 1024, + 'part_size' => 1024 * 1024, ], 'pwr' => [ 'pwr' => false, // Need info ? diff --git a/src/danog/MadelineProto/Wrappers/Login.php b/src/danog/MadelineProto/Wrappers/Login.php index bf1ef842..cbfa4b22 100644 --- a/src/danog/MadelineProto/Wrappers/Login.php +++ b/src/danog/MadelineProto/Wrappers/Login.php @@ -213,6 +213,14 @@ trait Login return $this->authorization; } + /** + * Update the 2FA password + * + * The params can contain password, new_password, email and hint params. + * + * @param array $params The params + * @return void + */ public function update_2fa_async(array $params) { $hasher = new PasswordCalculator();