Almost done

This commit is contained in:
Daniil Gentili 2019-06-04 23:55:02 +02:00
parent 7509e81d6f
commit 0f6cccf941
7 changed files with 136 additions and 65 deletions

View File

@ -21,7 +21,7 @@ That's it!
**No need** to set up thread pools (that don't even work in PHP), use synchronization primitives, and so on... **No need** to set up thread pools (that don't even work in PHP), use synchronization primitives, and so on...
Just `yield $MadelineProto->messages->sendMessage` instead of `$MadelineProto->messages->sendMessage`. Just `yield $MadelineProto->messages->sendMessage` instead of `$MadelineProto->messages->sendMessage`.
~~~~ ***
Naturally, async is not the only feature present in MadelineProto 4.0: to implement async in MadelineProto, I rewrote the **entire codebase two times** over the course of the last six months, as shown in the diff: Naturally, async is not the only feature present in MadelineProto 4.0: to implement async in MadelineProto, I rewrote the **entire codebase two times** over the course of the last six months, as shown in the diff:
``` ```
@ -40,7 +40,9 @@ The simplified async buffered implementation of AMPHP sockets I created allowed
Obfuscation is now handled via a separate setting, and can be enabled for **all** transports, not just `tcp_abridged` (it can be used to prevent ISP blocks in combination with the padded protocol, for example). Obfuscation is now handled via a separate setting, and can be enabled for **all** transports, not just `tcp_abridged` (it can be used to prevent ISP blocks in combination with the padded protocol, for example).
I also added support for different **MTProto transports**, including the brand new **websocket transport** and **secure websocket transport (HTTPS)**, until now only implemented in tdlib! I also added support for different **MTProto transports**, including the brand new **websocket transport** and **secure websocket transport (HTTPS)**, until now only implemented in tdlib!
If you want all communcations with telegram to be **double** or **triple**-encrypted using HTTPS+MTProto+obfuscation, you can enable the `wss` transport. If you want all communcations with telegram to be **triple**-encrypted using HTTPS+MTProto+obfuscation, you can enable the `wss` transport (the third layer of encryption w/ obfuscation will be enabled automatically).
Note: the **websocket HTTPS MTProto transport** is different from the **HTTPS MTProto protocol** (both are supported by MadelineProto).
The **websocket HTTPS MTProto transport** is more reliable and faster than the **HTTPS MTProto protocol**, since MadelineProto does not have to handle HTTP long polls.
(the websocket transport may or may not also be used in future to implement MadelineProto in the browser with webassembly ;) (the websocket transport may or may not also be used in future to implement MadelineProto in the browser with webassembly ;)
@ -48,37 +50,70 @@ The new [proxy API](https://docs.madelineproto.xyz/docs/PROXY.html) is also base
You can still use the socks5 and HTTP transports if you want. You can still use the socks5 and HTTP transports if you want.
[Writing your custom proxies](https://docs.madelineproto.xyz/docs/PROXY.html#build-your-proxy) now is now a LOT easier, thanks to the neat structure of the stream APIs and the abundant PHPDOC comments. [Writing your custom proxies](https://docs.madelineproto.xyz/docs/PROXY.html#build-your-proxy) now is now a LOT easier, thanks to the neat structure of the stream APIs and the abundant PHPDOC comments.
~~~ ***
Streams and proxies aren't the only things that have changed in this release: Streams and proxies aren't the only things that have been completely rewritten in this release:
I have written a **unique** socket message handler API based on [MadelineProto's loop API](https://docs.madelineproto.xyz/docs/ASYNC.html#async-loop-apis): and greatly simplifies I have written a **unique** socket message handler API based on [MadelineProto's loop API](https://docs.madelineproto.xyz/docs/ASYNC.html#async-loop-apis), and guarantees **maximum reliability** while communicating with telegram.
By the way, the new async background loop API can also be used by **you**! It is perfect for repeating tasks in a cron-like manner, running multiple resumable event loops and much more! More on that later.
The new **message handler loops** run in the background, and guarantee that every single method call you make gets delivered to telegram.
**Two signal loops** running in two separate green threads take care of writing and reading data from the socket asynchronously.
A **third signal loop** uses state request messages to make sure that the method calls were received by telegram, and queries replies/resend method calls if something's wrong.
A **fourth signal loop** takes care of HTTP long polling.
This guarantees maximum stability even if telegram's having server issues.
The write loop also greatly reduces overhead, increasing performances by automatically wrapping in containers multiple method calls: this is especially useful when making multiple method calls simultaneously with async (more on that later).
***
Possibly the most __exciting__ thing to work on in this version of MadelineProto was the new **update management system**: I whipped it up in merely two days a few weeks ago, and it has **absolutely improved** the overall reliability of MadelineProto.
Huge thanks to Aliaksei Levin, the developer of tdlib, for explaining to me how exactly does the MTProto update API work: he saved me a lot of time, and was really nice <3<3<3.
While thinking of an easy way I could implement the new update system, I had an inspiration:
I created a **unique**, **reliable**, **fast** and **extremely simple** update handling system based on [MadelineProto's loop API](https://docs.madelineproto.xyz/docs/ASYNC.html#async-loop-apis), **not present in any** other MTProto client, **not even tdlib**.
Instead of messing around with various synchronization locks, checks and cluttered update receivers, **I implemented three simple feed loops**.
Updates are fed to the first update loop, where some simple duplicate/hole checks are done.
Updates are then fed to a second update loop, where secondary duplicate/hole checks are done.
A third loop type takes care of eventual holes and periodically fetches updates for all supergroups, channels.
All of this is implemented without **any** kind of additional synchronization or locking due to the nature of the **MadelineProto loop APIs**, with **1%** of the complexity of the official clients (which means less bugs and more pony time for me).
Let's elaborate on this: when I say **thread**, I actually mean **green thread**, often called **strand**. ***
**Strands** are behave exactly like normal **threads**, except that they're implemented in user-space, and they're much **faster**, **more reliable**, and **do not suffer** from synchronization issues present with normal threads.
Even if most of MadelineProto's logic is now concentrated in the new **loop** and **stream** APIs, some parts like the TL parser are still there, albeit with many changes.
For example: now the TL parser is fully asynchronous (that may sound a bit weird to some of you, but for dynamic TL parsers, this greatly increases performances and allows parallelism).
The TL deserializer now uses yet another well-structured API called the **TLCallback API** to automatically populate internal databases directly on deserialization: again, this paradigm greatly **reduces complexity**, **increases performance** and is **absolutely unique** to MadelineProto; no other MTProto client has it, not even official clients.
One of the **completely new** modules that I created for MadelineProto async is the **file reference database**: a **very complex** module, required to handle files in the newer versions of the Telegram API.
It makes use of the **TLCallback** API to map files to their origins, to be able to refetch them at any given time when the file reference expires.
Another new module I've implemented is the **PasswordCalculator**, that is used to calculate the new password hashes for the 2FA login, __really cool__.
I've also rewritten the **APIFactory**, the abstraction layer that stands between you and MadelineProto when you do `$MadelineProto->method()`: it is now fully async, and MUCH faster thanks to a new cached method mapping system.
The same cached method mapping system is also used for the **event handler**, which means that now the **event handler is the fastest update management method**.
* Fully rewritten connection stack, with support for websockets, stuff ***
* updates
* simultaneous method calls And now, on to the **API changes and improvements**:
* new TL callback system * First of all, we've got several bucketloads of telegram API changes, that can be viewed in the first posts
* added support for wallpapers * Added an [update_2fa](https://docs.madelineproto.xyz/update_2fa.html) method to update the login password
* Added a [get_full_dialogs](https://docs.madelineproto.xyz/docs/DIALOGS.html#get_full_dialogs-now-fully-async) method to get a full list of all chats youre member of, including dialog info (such as the pinned/last message ID, unread count, tag count, notification settings and message drafts).
* [Added support for automatic file uploads by name in secret chats (as with normal chats); you can also now send secret chat messages using the sendMessage method as if it were a normal chat](https://github.com/danog/MadelineProto/blob/master/secret_bot.php)
* Improved message splitting algorithm: performance improvements, and it will now notify you via the logs if there are too many entities in the logs, or if the entities are too long. * Improved message splitting algorithm: performance improvements, and it will now notify you via the logs if there are too many entities in the logs, or if the entities are too long.
* Improved get_self method. * Improved the get_self method.
* reference database
* Rewritten proxy stack
* magic sleep * magic sleep
* get_full_dialogs * Simultaneous method calls
* new APIfactory
* sendmessage with secret messages * sendmessage with secret messages
* automatic secret chat file upload * automatic secret chat file upload
* 2fa+++++
* improved callfork * improved callfork
* split acks
* new logging * new logging
* TL callabck
* channel state * channel state
* logger
* async construct * async construct
* clean up repo, update dependencies and remove curl dependency * clean up repo, update dependencies and remove curl dependency
* new phone call config * new phone call config
@ -99,12 +134,21 @@ Let's elaborate on this: when I say **thread**, I actually mean **green thread**
* arrayaccess on args * arrayaccess on args
* increased flood wait * increased flood wait
***
And now, let's elaborate on async:
With **MadelineProto 4.0**, each update is handled in **parallel** using a separate **thread**, and everything is done in **parallel** (even on restricted webhosts!).
When I say **thread**, I actually mean **green thread** ([wikipedia](https://en.wikipedia.org/wiki/Green_threads)), often called **strand**.
**Strands** are behave exactly like normal **threads**, except that they're implemented in user-space, and they're much **faster**, **more reliable**, and **do not suffer** from synchronization issues present with normal threads.
In
Things to expect in the next releases: Things to expect in the next releases:
docs for get mime funcs docs for get mime funcs
docs for HTML parser (div to avoid escaping)
docs for update_2fa docs for update_2fa
docs for ResponseException
docs for PTSException
Document async apis
optional max_id and min_id optional max_id and min_id
async iterators async iterators
Method name changes Method name changes
@ -123,5 +167,10 @@ do not use manual
tell about madeline.php loading in the same dire tell about madeline.php loading in the same dire
arrayaccess on promises arrayaccess on promises
get sponsor of get sponsor of
ton
video calls
group calls
native calls
dnssec
telegram passport telegram passport

View File

@ -27,14 +27,14 @@ image: https://docs.madelineproto.xyz/favicons/android-chrome-256x256.png
'.str_replace('<img', '<amp-img', file_get_contents('README.md'))); '.str_replace('<img', '<amp-img', file_get_contents('README.md')));
$docs = [ $docs = [
[ /* [
'tl_schema' => ['td' => __DIR__.'/src/danog/MadelineProto/TL_td.tl'], 'tl_schema' => ['td' => __DIR__.'/src/danog/MadelineProto/TL_td.tl'],
'title' => 'MadelineProto API documentation (td-lib)', 'title' => 'MadelineProto API documentation (td-lib)',
'description' => 'MadelineProto API documentation (td-lib)', 'description' => 'MadelineProto API documentation (td-lib)',
'output_dir' => __DIR__.'/docs/docs/TD_docs', 'output_dir' => __DIR__.'/docs/docs/TD_docs',
'readme' => false, 'readme' => false,
'td' => true, 'td' => true,
], ],*/
[ [
'tl_schema' => ['mtproto' => __DIR__.'/src/danog/MadelineProto/TL_mtproto_v1.json'], 'tl_schema' => ['mtproto' => __DIR__.'/src/danog/MadelineProto/TL_mtproto_v1.json'],
'title' => 'MadelineProto API documentation (mtproto)', 'title' => 'MadelineProto API documentation (mtproto)',

2
docs

@ -1 +1 @@
Subproject commit ae6f992aef745fecfa989ad01f959a467d94c7ae Subproject commit 6fe646f4a22e8e0ccf5d8a8c562c6d7499a8a17c

View File

@ -1,29 +1,36 @@
<?php <?php
if (PHP_MAJOR_VERSION === 5) {
if (PHP_MINOR_VERSION < 6) {
throw new \Exception('MadelineProto requires at least PHP 5.6 to run');
}
$newline = PHP_EOL;
if (php_sapi_name() !== 'cli') {
$newline = '<br>'.$newline;
}
echo "**********************************************************************$newline";
echo "**********************************************************************$newline$newline";
echo "YOU ARE USING AN OLD AND BUGGED VERSION OF PHP, PLEASE UPDATE TO PHP 7$newline";
echo "PHP 5 USERS WILL NOT RECEIVE MADELINEPROTO UPDATES AND BUGFIXES$newline$newline";
echo "SUPPORTED VERSIONS: PHP 7.0, 7.1, 7.2, 7.3+$newline";
echo "RECOMMENDED VERSION: PHP 7.3$newline$newline";
echo "**********************************************************************$newline";
echo "**********************************************************************$newline";
unset($newline);
}
function ___install_madeline() function ___install_madeline()
{ {
if (count(debug_backtrace(0)) === 1) { if (count(debug_backtrace(0)) === 1) {
die('You must include this file in another PHP script'.PHP_EOL); die('You must include this file in another PHP script'.PHP_EOL);
} }
$old = false;
if (PHP_MAJOR_VERSION === 5) {
if (PHP_MINOR_VERSION < 6) {
throw new \Exception('MadelineProto requires at least PHP 7.1 to run');
}
$old = true;
}
if (PHP_MAJOR_VERSION === 7 && PHP_MINOR_VERSION === 0) {
$old = true;
}
if ($old) {
$newline = PHP_EOL;
if (php_sapi_name() !== 'cli') {
$newline = '<br>'.$newline;
}
echo "**********************************************************************************$newline";
echo "**********************************************************************************$newline$newline";
echo "YOU ARE USING AN OLD AND BUGGED VERSION OF PHP, PLEASE UPDATE TO PHP 7.3$newline";
echo "PHP 5/7.0 USERS WILL NOT RECEIVE PHP UPDATES AND BUGFIXES: https://www.php.net/eol.php$newline";
echo "PHP 5/7.0 USERS WILL NOT RECEIVE MADELINEPROTO UPDATES AND BUGFIXES$newline$newline";
echo "SUPPORTED VERSIONS: PHP 7.1, 7.2, 7.3+$newline";
echo "RECOMMENDED VERSION: PHP 7.3$newline$newline";
echo "**********************************************************************************$newline";
echo "**********************************************************************************$newline";
unset($newline);
}
// MTProxy update // MTProxy update
$file = debug_backtrace(0, 1)[0]['file']; $file = debug_backtrace(0, 1)[0]['file'];

View File

@ -367,6 +367,8 @@ image: https://docs.madelineproto.xyz/favicons/android-chrome-256x256.png
* [Login](https://docs.madelineproto.xyz/docs/LOGIN.html) * [Login](https://docs.madelineproto.xyz/docs/LOGIN.html)
* [Change 2FA password](https://docs.madelineproto.xyz/update_2fa.html)
* [Get all chats, broadcast a message to all chats](https://docs.madelineproto.xyz/docs/DIALOGS.html) * [Get all chats, broadcast a message to all chats](https://docs.madelineproto.xyz/docs/DIALOGS.html)
* [Get the full participant list of a channel/group/supergroup](https://docs.madelineproto.xyz/get_pwr_chat.html) * [Get the full participant list of a channel/group/supergroup](https://docs.madelineproto.xyz/get_pwr_chat.html)

View File

@ -249,6 +249,11 @@ class MTProto extends AsyncConstruct implements TLCallback
return $this->datacenter->getHTTPClient(); return $this->datacenter->getHTTPClient();
} }
public function fileGetContents($url): \Generator
{
return $this->datacenter->fileGetContents($url);
}
public function __wakeup() public function __wakeup()
{ {
$backtrace = debug_backtrace(0, 3); $backtrace = debug_backtrace(0, 3);
@ -595,9 +600,9 @@ class MTProto extends AsyncConstruct implements TLCallback
2 => [ 2 => [
// The rest will be fetched using help.getConfig // The rest will be fetched using help.getConfig
'ip_address' => '149.154.167.40', 'ip_address' => '149.154.167.40',
'port' => 443, 'port' => 443,
'media_only' => false, 'media_only' => false,
'tcpo_only' => false, 'tcpo_only' => false,
], ],
], ],
'ipv6' => [ 'ipv6' => [
@ -605,9 +610,9 @@ class MTProto extends AsyncConstruct implements TLCallback
2 => [ 2 => [
// The rest will be fetched using help.getConfig // The rest will be fetched using help.getConfig
'ip_address' => '2001:067c:04e8:f002:0000:0000:0000:000e', 'ip_address' => '2001:067c:04e8:f002:0000:0000:0000:000e',
'port' => 443, 'port' => 443,
'media_only' => false, 'media_only' => false,
'tcpo_only' => false, 'tcpo_only' => false,
], ],
], ],
], ],
@ -618,9 +623,9 @@ class MTProto extends AsyncConstruct implements TLCallback
2 => [ 2 => [
// The rest will be fetched using help.getConfig // The rest will be fetched using help.getConfig
'ip_address' => '149.154.167.51', 'ip_address' => '149.154.167.51',
'port' => 443, 'port' => 443,
'media_only' => false, 'media_only' => false,
'tcpo_only' => false, 'tcpo_only' => false,
], ],
], ],
'ipv6' => [ 'ipv6' => [
@ -628,9 +633,9 @@ class MTProto extends AsyncConstruct implements TLCallback
2 => [ 2 => [
// The rest will be fetched using help.getConfig // The rest will be fetched using help.getConfig
'ip_address' => '2001:067c:04e8:f002:0000:0000:0000:000a', 'ip_address' => '2001:067c:04e8:f002:0000:0000:0000:000a',
'port' => 443, 'port' => 443,
'media_only' => false, 'media_only' => false,
'tcpo_only' => false, 'tcpo_only' => false,
], ],
], ],
], ],
@ -651,17 +656,17 @@ class MTProto extends AsyncConstruct implements TLCallback
'proxy_extra' => Magic::$altervista ? ['address' => 'localhost', 'port' => 80] : [], 'proxy_extra' => Magic::$altervista ? ['address' => 'localhost', 'port' => 80] : [],
// Extra parameters to pass to the proxy class using setExtra // Extra parameters to pass to the proxy class using setExtra
'obfuscated' => false, 'obfuscated' => false,
'transport' => 'tcp', 'transport' => 'tcp',
'pfs' => extension_loaded('gmp'), 'pfs' => extension_loaded('gmp'),
], ],
'default_dc' => 2, 'default_dc' => 2,
], 'app_info' => [ ], 'app_info' => [
// obtained in https://my.telegram.org // obtained in https://my.telegram.org
//'api_id' => you should put an API id in the settings array you provide //'api_id' => you should put an API id in the settings array you provide
//'api_hash' => you should put an API hash in the settings array you provide //'api_hash' => you should put an API hash in the settings array you provide
'device_model' => $device_model, 'device_model' => $device_model,
'system_version' => $system_version, 'system_version' => $system_version,
'app_version' => $app_version, 'app_version' => $app_version,
// 🌚 // 🌚
// 'app_version' => self::V, // 'app_version' => self::V,
'lang_code' => $lang_code, 'lang_code' => $lang_code,
@ -695,10 +700,10 @@ class MTProto extends AsyncConstruct implements TLCallback
*/ */
// write to // write to
'logger_param' => Magic::$script_cwd.'/MadelineProto.log', 'logger_param' => Magic::$script_cwd.'/MadelineProto.log',
'logger' => php_sapi_name() === 'cli' ? 3 : 2, 'logger' => php_sapi_name() === 'cli' ? 3 : 2,
// overwrite previous setting and echo logs // overwrite previous setting and echo logs
'logger_level' => Logger::VERBOSE, 'logger_level' => Logger::VERBOSE,
'max_size' => 100 * 1024 * 1024, 'max_size' => 100 * 1024 * 1024,
// Logging level, available logging levels are: ULTRA_VERBOSE, VERBOSE, NOTICE, WARNING, ERROR, FATAL_ERROR. Can be provided as last parameter to the logging function. // Logging level, available logging levels are: ULTRA_VERBOSE, VERBOSE, NOTICE, WARNING, ERROR, FATAL_ERROR. Can be provided as last parameter to the logging function.
'rollbar_token' => '', 'rollbar_token' => '',
], 'max_tries' => [ ], 'max_tries' => [
@ -707,10 +712,10 @@ class MTProto extends AsyncConstruct implements TLCallback
'authorization' => 5, 'authorization' => 5,
// How many times should I try to generate an authorization key before throwing an exception // How many times should I try to generate an authorization key before throwing an exception
'response' => 5, 'response' => 5,
], 'flood_timeout' => ['wait_if_lt' => 10*60], 'msg_array_limit' => [ ], 'flood_timeout' => ['wait_if_lt' => 10 * 60], 'msg_array_limit' => [
// How big should be the arrays containing the incoming and outgoing messages? // How big should be the arrays containing the incoming and outgoing messages?
'incoming' => 100, 'incoming' => 100,
'outgoing' => 100, 'outgoing' => 100,
'call_queue' => 200, 'call_queue' => 200,
], 'peer' => [ ], 'peer' => [
'full_info_cache_time' => 3600, 'full_info_cache_time' => 3600,
@ -734,10 +739,10 @@ class MTProto extends AsyncConstruct implements TLCallback
'handler_workers' => 10, 'handler_workers' => 10,
], 'upload' => [ ], 'upload' => [
'allow_automatic_upload' => true, 'allow_automatic_upload' => true,
'part_size' => 512 * 1024, 'part_size' => 512 * 1024,
], 'download' => [ ], 'download' => [
'report_broken_media' => true, 'report_broken_media' => true,
'part_size' => 1024 * 1024, 'part_size' => 1024 * 1024,
], 'pwr' => [ ], 'pwr' => [
'pwr' => false, 'pwr' => false,
// Need info ? // Need info ?

View File

@ -213,6 +213,14 @@ trait Login
return $this->authorization; return $this->authorization;
} }
/**
* Update the 2FA password
*
* The params can contain password, new_password, email and hint params.
*
* @param array $params The params
* @return void
*/
public function update_2fa_async(array $params) public function update_2fa_async(array $params)
{ {
$hasher = new PasswordCalculator(); $hasher = new PasswordCalculator();