Almost done

CHANGELOG.md

@@ -21,7 +21,7 @@ That's it!
 **No need** to set up thread pools (that don't even work in PHP), use synchronization primitives, and so on...
 Just `yield $MadelineProto->messages->sendMessage` instead of `$MadelineProto->messages->sendMessage`.  
 
-~~~~
+***
 
 Naturally, async is not the only feature present in MadelineProto 4.0: to implement async in MadelineProto, I rewrote the **entire codebase two times** over the course of the last six months, as shown in the diff:  
 ```
@@ -40,7 +40,9 @@ The simplified async buffered implementation of AMPHP sockets I created allowed
 Obfuscation is now handled via a separate setting, and can be enabled for **all** transports, not just `tcp_abridged` (it can be used to prevent ISP blocks in combination with the padded protocol, for example).  
 
 I also added support for different **MTProto transports**, including the brand new **websocket transport** and **secure websocket transport (HTTPS)**, until now only implemented in tdlib!  
-If you want all communcations with telegram to be **double** or **triple**-encrypted using HTTPS+MTProto+obfuscation, you can enable the `wss` transport.  
+If you want all communcations with telegram to be **triple**-encrypted using HTTPS+MTProto+obfuscation, you can enable the `wss` transport (the third layer of encryption w/ obfuscation will be enabled automatically).  
+Note: the **websocket HTTPS MTProto transport** is different from the **HTTPS MTProto protocol** (both are supported by MadelineProto).  
+The **websocket HTTPS MTProto transport** is more reliable and faster than the **HTTPS MTProto protocol**, since MadelineProto does not have to handle HTTP long polls.  
 
 (the websocket transport may or may not also be used in future to implement MadelineProto in the browser with webassembly ;)  
 
@@ -48,37 +50,70 @@ The new [proxy API](https://docs.madelineproto.xyz/docs/PROXY.html) is also base
 You can still use the socks5 and HTTP transports if you want.  
 [Writing your custom proxies](https://docs.madelineproto.xyz/docs/PROXY.html#build-your-proxy) now is now a LOT easier, thanks to the neat structure of the stream APIs and the abundant PHPDOC comments.  
 
-~~~
+***
 
-Streams and proxies aren't the only things that have changed in this release:  
+Streams and proxies aren't the only things that have been completely rewritten in this release:  
-I have written a **unique** socket message handler API based on [MadelineProto's loop API](https://docs.madelineproto.xyz/docs/ASYNC.html#async-loop-apis):  and greatly simplifies 
+I have written a **unique** socket message handler API based on [MadelineProto's loop API](https://docs.madelineproto.xyz/docs/ASYNC.html#async-loop-apis), and guarantees **maximum reliability** while communicating with telegram.  
+By the way, the new async background loop API can also be used by **you**! It is perfect for repeating tasks in a cron-like manner, running multiple resumable event loops and much more! More on that later.  
+
+The new **message handler loops** run in the background, and guarantee that every single method call you make gets delivered to telegram.  
+**Two signal loops** running in two separate green threads take care of writing and reading data from the socket asynchronously.  
+A **third signal loop** uses state request messages to make sure that the method calls were received by telegram, and queries replies/resend method calls if something's wrong.  
+A **fourth signal loop** takes care of HTTP long polling.  
+
+This guarantees maximum stability even if telegram's having server issues.  
+The write loop also greatly reduces overhead, increasing performances by automatically wrapping in containers multiple method calls: this is especially useful when making multiple method calls simultaneously with async (more on that later).  
+
+***
+
+Possibly the most __exciting__ thing to work on in this version of MadelineProto was the new **update management system**: I whipped it up in merely two days a few weeks ago, and it has **absolutely improved** the overall reliability of MadelineProto.  
+Huge thanks to Aliaksei Levin, the developer of tdlib, for explaining to me how exactly does the MTProto update API work: he saved me a lot of time, and was really nice <3<3<3.  
+
+While thinking of an easy way I could implement the new update system, I had an inspiration:  
+
+I created a **unique**, **reliable**, **fast** and **extremely simple** update handling system based on [MadelineProto's loop API](https://docs.madelineproto.xyz/docs/ASYNC.html#async-loop-apis), **not present in any** other MTProto client, **not even tdlib**.  
+
+Instead of messing around with various synchronization locks, checks and cluttered update receivers, **I implemented three simple feed loops**.  
+Updates are fed to the first update loop, where some simple duplicate/hole checks are done.
+Updates are then fed to a second update loop, where secondary duplicate/hole checks are done.  
+A third loop type takes care of eventual holes and periodically fetches updates for all supergroups, channels.  
+
+All of this is implemented without **any** kind of additional synchronization or locking due to the nature of the **MadelineProto loop APIs**, with **1%** of the complexity of the official clients (which means less bugs and more pony time for me).  
 
 
-Let's elaborate on this: when I say **thread**, I actually mean **green thread**, often called **strand**.  
+***
-**Strands** are behave exactly like normal **threads**, except that they're implemented in user-space, and they're much **faster**, **more reliable**, and **do not suffer** from synchronization issues present with normal threads.  
+
+Even if most of MadelineProto's logic is now concentrated in the new **loop** and **stream** APIs, some parts like the TL parser are still there, albeit with many changes.  
+For example: now the TL parser is fully asynchronous (that may sound a bit weird to some of you, but for dynamic TL parsers, this greatly increases performances and allows parallelism).  
+
+The TL deserializer now uses yet another well-structured API called the **TLCallback API** to automatically populate internal databases directly on deserialization: again, this paradigm greatly **reduces complexity**, **increases performance** and is **absolutely unique** to MadelineProto; no other MTProto client has it, not even official clients.  
+
+One of the **completely new** modules that I created for MadelineProto async is the **file reference database**: a **very complex** module, required to handle files in the newer versions of the Telegram API.  
+It makes use of the **TLCallback** API to map files to their origins, to be able to refetch them at any given time when the file reference expires.  
+
+Another new module I've implemented is the **PasswordCalculator**, that is used to calculate the new password hashes for the 2FA login, __really cool__.  
+
+I've also rewritten the **APIFactory**, the abstraction layer that stands between you and MadelineProto when you do `$MadelineProto->method()`: it is now fully async, and MUCH faster thanks to a new cached method mapping system.  
+The same cached method mapping system is also used for the **event handler**, which means that now the **event handler is the fastest update management method**.  
 
 
-* Fully rewritten connection stack, with support for websockets, stuff
+***
-* updates
+
-* simultaneous method calls
+And now, on to the **API changes and improvements**:  
-* new TL callback system
+* First of all, we've got several bucketloads of telegram API changes, that can be viewed in the first posts
-* added support for wallpapers
+* Added an [update_2fa](https://docs.madelineproto.xyz/update_2fa.html) method to update the login password
+* Added a [get_full_dialogs](https://docs.madelineproto.xyz/docs/DIALOGS.html#get_full_dialogs-now-fully-async) method to get a full list of all chats you’re member of, including dialog info (such as the pinned/last message ID, unread count, tag count, notification settings and message drafts).  
+* [Added support for automatic file uploads by name in secret chats (as with normal chats); you can also now send secret chat messages using the sendMessage method as if it were a normal chat](https://github.com/danog/MadelineProto/blob/master/secret_bot.php)
 * Improved message splitting algorithm: performance improvements, and it will now notify you via the logs if there are too many entities in the logs, or if the entities are too long.  
-* Improved get_self method.  
+* Improved the get_self method.  
-* reference database
+
-* Rewritten proxy stack
 * magic sleep
-* get_full_dialogs
+* Simultaneous method calls
-* new APIfactory
 * sendmessage with secret messages
 * automatic secret chat file upload
-* 2fa+++++
 * improved callfork
-* split acks
 * new logging
-* TL callabck
 * channel state
-* logger
 * async construct
 * clean up repo, update dependencies and remove curl dependency
 * new phone call config
@@ -99,12 +134,21 @@ Let's elaborate on this: when I say **thread**, I actually mean **green thread**
 * arrayaccess on args
 * increased flood wait
 
+***
+
+And now, let's elaborate on async:  
+With **MadelineProto 4.0**, each update is handled in **parallel** using a separate **thread**, and everything is done in **parallel** (even on restricted webhosts!).  
+
+When I say **thread**, I actually mean **green thread** ([wikipedia](https://en.wikipedia.org/wiki/Green_threads)), often called **strand**.  
+**Strands** are behave exactly like normal **threads**, except that they're implemented in user-space, and they're much **faster**, **more reliable**, and **do not suffer** from synchronization issues present with normal threads.  
+
+In
+
+
 Things to expect in the next releases:
 docs for get mime funcs
+docs for HTML parser (div to avoid escaping)
 docs for update_2fa
-docs for ResponseException
-docs for PTSException
-Document async apis
 optional max_id and min_id
 async iterators
 Method name changes
@@ -123,5 +167,10 @@ do not use manual
 tell about madeline.php loading in the same dire
 arrayaccess on promises
 get sponsor of 
+ton
+video calls
+group calls
+native calls
+dnssec
 
 telegram passport

build_docs.php

@@ -27,14 +27,14 @@ image: https://docs.madelineproto.xyz/favicons/android-chrome-256x256.png
 '.str_replace('<img', '<amp-img', file_get_contents('README.md')));
 
 $docs = [
-    [
+/*    [
         'tl_schema'   => ['td' => __DIR__.'/src/danog/MadelineProto/TL_td.tl'],
         'title'       => 'MadelineProto API documentation (td-lib)',
         'description' => 'MadelineProto API documentation (td-lib)',
         'output_dir'  => __DIR__.'/docs/docs/TD_docs',
         'readme'      => false,
         'td'          => true,
-    ],
+    ],*/
     [
         'tl_schema'   => ['mtproto' => __DIR__.'/src/danog/MadelineProto/TL_mtproto_v1.json'],
         'title'       => 'MadelineProto API documentation (mtproto)',

docs

@@ -1 +1 @@
-Subproject commit ae6f992aef745fecfa989ad01f959a467d94c7ae
+Subproject commit 6fe646f4a22e8e0ccf5d8a8c562c6d7499a8a17c

phar.php

@@ -1,29 +1,36 @@
 <?php
 
-if (PHP_MAJOR_VERSION === 5) {
-    if (PHP_MINOR_VERSION < 6) {
-        throw new \Exception('MadelineProto requires at least PHP 5.6 to run');
-    }
-    $newline = PHP_EOL;
-    if (php_sapi_name() !== 'cli') {
-        $newline = '<br>'.$newline;
-    }
-    echo "**********************************************************************$newline";
-    echo "**********************************************************************$newline$newline";
-    echo "YOU ARE USING AN OLD AND BUGGED VERSION OF PHP, PLEASE UPDATE TO PHP 7$newline";
-    echo "PHP 5 USERS WILL NOT RECEIVE MADELINEPROTO UPDATES AND BUGFIXES$newline$newline";
-    echo "SUPPORTED VERSIONS: PHP 7.0, 7.1, 7.2, 7.3+$newline";
-    echo "RECOMMENDED VERSION: PHP 7.3$newline$newline";
-    echo "**********************************************************************$newline";
-    echo "**********************************************************************$newline";
-    unset($newline);
-}
-
 function ___install_madeline()
 {
     if (count(debug_backtrace(0)) === 1) {
         die('You must include this file in another PHP script'.PHP_EOL);
     }
+    $old = false;
+    if (PHP_MAJOR_VERSION === 5) {
+        if (PHP_MINOR_VERSION < 6) {
+            throw new \Exception('MadelineProto requires at least PHP 7.1 to run');
+        }
+        $old = true;
+    }
+    if (PHP_MAJOR_VERSION === 7 && PHP_MINOR_VERSION === 0) {
+        $old = true;
+    }
+    if ($old) {
+        $newline = PHP_EOL;
+        if (php_sapi_name() !== 'cli') {
+            $newline = '<br>'.$newline;
+        }
+        echo "**********************************************************************************$newline";
+        echo "**********************************************************************************$newline$newline";
+        echo "YOU ARE USING AN OLD AND BUGGED VERSION OF PHP, PLEASE UPDATE TO PHP 7.3$newline";
+        echo "PHP 5/7.0 USERS WILL NOT RECEIVE PHP UPDATES AND BUGFIXES: https://www.php.net/eol.php$newline";
+        echo "PHP 5/7.0 USERS WILL NOT RECEIVE MADELINEPROTO UPDATES AND BUGFIXES$newline$newline";
+        echo "SUPPORTED VERSIONS: PHP 7.1, 7.2, 7.3+$newline";
+        echo "RECOMMENDED VERSION: PHP 7.3$newline$newline";
+        echo "**********************************************************************************$newline";
+        echo "**********************************************************************************$newline";
+        unset($newline);
+    }
 
     // MTProxy update
     $file = debug_backtrace(0, 1)[0]['file'];

src/danog/MadelineProto/DocsBuilder/Methods.php

@@ -367,6 +367,8 @@ image: https://docs.madelineproto.xyz/favicons/android-chrome-256x256.png
 
 * [Login](https://docs.madelineproto.xyz/docs/LOGIN.html)
 
+* [Change 2FA password](https://docs.madelineproto.xyz/update_2fa.html)
+
 * [Get all chats, broadcast a message to all chats](https://docs.madelineproto.xyz/docs/DIALOGS.html)
 
 * [Get the full participant list of a channel/group/supergroup](https://docs.madelineproto.xyz/get_pwr_chat.html)

src/danog/MadelineProto/MTProto.php

@@ -249,6 +249,11 @@ class MTProto extends AsyncConstruct implements TLCallback
         return $this->datacenter->getHTTPClient();
     }
 
+    public function fileGetContents($url): \Generator
+    {
+        return $this->datacenter->fileGetContents($url);
+    }
+
     public function __wakeup()
     {
         $backtrace = debug_backtrace(0, 3);
@@ -595,9 +600,9 @@ class MTProto extends AsyncConstruct implements TLCallback
                     2 => [
                         // The rest will be fetched using help.getConfig
                         'ip_address' => '149.154.167.40',
-                        'port'       => 443,
+                        'port' => 443,
                         'media_only' => false,
-                        'tcpo_only'  => false,
+                        'tcpo_only' => false,
                     ],
                 ],
                 'ipv6' => [
@@ -605,9 +610,9 @@ class MTProto extends AsyncConstruct implements TLCallback
                     2 => [
                         // The rest will be fetched using help.getConfig
                         'ip_address' => '2001:067c:04e8:f002:0000:0000:0000:000e',
-                        'port'       => 443,
+                        'port' => 443,
                         'media_only' => false,
-                        'tcpo_only'  => false,
+                        'tcpo_only' => false,
                     ],
                 ],
             ],
@@ -618,9 +623,9 @@ class MTProto extends AsyncConstruct implements TLCallback
                     2 => [
                         // The rest will be fetched using help.getConfig
                         'ip_address' => '149.154.167.51',
-                        'port'       => 443,
+                        'port' => 443,
                         'media_only' => false,
-                        'tcpo_only'  => false,
+                        'tcpo_only' => false,
                     ],
                 ],
                 'ipv6' => [
@@ -628,9 +633,9 @@ class MTProto extends AsyncConstruct implements TLCallback
                     2 => [
                         // The rest will be fetched using help.getConfig
                         'ip_address' => '2001:067c:04e8:f002:0000:0000:0000:000a',
-                        'port'       => 443,
+                        'port' => 443,
                         'media_only' => false,
-                        'tcpo_only'  => false,
+                        'tcpo_only' => false,
                     ],
                 ],
             ],
@@ -651,17 +656,17 @@ class MTProto extends AsyncConstruct implements TLCallback
                 'proxy_extra' => Magic::$altervista ? ['address' => 'localhost', 'port' => 80] : [],
                 // Extra parameters to pass to the proxy class using setExtra
                 'obfuscated' => false,
-                'transport'  => 'tcp',
+                'transport' => 'tcp',
-                'pfs'        => extension_loaded('gmp'),
+                'pfs' => extension_loaded('gmp'),
             ],
             'default_dc' => 2,
         ], 'app_info' => [
             // obtained in https://my.telegram.org
             //'api_id'          => you should put an API id in the settings array you provide
             //'api_hash'        => you should put an API hash in the settings array you provide
-            'device_model'   => $device_model,
+            'device_model' => $device_model,
             'system_version' => $system_version,
-            'app_version'    => $app_version,
+            'app_version' => $app_version,
             // 🌚
             //                'app_version'     => self::V,
             'lang_code' => $lang_code,
@@ -695,10 +700,10 @@ class MTProto extends AsyncConstruct implements TLCallback
              */
             // write to
             'logger_param' => Magic::$script_cwd.'/MadelineProto.log',
-            'logger'       => php_sapi_name() === 'cli' ? 3 : 2,
+            'logger' => php_sapi_name() === 'cli' ? 3 : 2,
             // overwrite previous setting and echo logs
             'logger_level' => Logger::VERBOSE,
-            'max_size'     => 100 * 1024 * 1024,
+            'max_size' => 100 * 1024 * 1024,
             // Logging level, available logging levels are: ULTRA_VERBOSE, VERBOSE, NOTICE, WARNING, ERROR, FATAL_ERROR. Can be provided as last parameter to the logging function.
             'rollbar_token' => '',
         ], 'max_tries' => [
@@ -707,10 +712,10 @@ class MTProto extends AsyncConstruct implements TLCallback
             'authorization' => 5,
             // How many times should I try to generate an authorization key before throwing an exception
             'response' => 5,
-        ], 'flood_timeout' => ['wait_if_lt' => 10*60], 'msg_array_limit' => [
+        ], 'flood_timeout' => ['wait_if_lt' => 10 * 60], 'msg_array_limit' => [
             // How big should be the arrays containing the incoming and outgoing messages?
-            'incoming'   => 100,
+            'incoming' => 100,
-            'outgoing'   => 100,
+            'outgoing' => 100,
             'call_queue' => 200,
         ], 'peer' => [
             'full_info_cache_time' => 3600,
@@ -734,10 +739,10 @@ class MTProto extends AsyncConstruct implements TLCallback
             'handler_workers' => 10,
         ], 'upload' => [
             'allow_automatic_upload' => true,
-            'part_size'              => 512 * 1024,
+            'part_size' => 512 * 1024,
         ], 'download' => [
             'report_broken_media' => true,
-            'part_size'           => 1024 * 1024,
+            'part_size' => 1024 * 1024,
         ], 'pwr' => [
             'pwr' => false,
             // Need info ?

src/danog/MadelineProto/Wrappers/Login.php

@@ -213,6 +213,14 @@ trait Login
         return $this->authorization;
     }
 
+    /**
+     * Update the 2FA password
+     *
+     * The params can contain password, new_password, email and hint params.
+     * 
+     * @param array $params The params
+     * @return void
+     */
     public function update_2fa_async(array $params)
     {
         $hasher = new PasswordCalculator();