danog

14 KiB

Raw Blame History

title	description	image
Async	MadelineProto now features async, for incredible speed improvements, and parallel processing without buggy and slow threading/multiprocessing.	https://docs.madelineproto.xyz/favicons/android-chrome-256x256.png

Async

MadelineProto now features async, for incredible speed improvements, and parallel processing.
Powered by amphp, MadelineProto wraps the AMPHP APIs to provide a simpler generator-based async API.

Usage

Usage

What exactly is async, you may ask, and how is it better than threading or multiprocessing?
Async is a relatively new programming pattern that allows you to easily write non-blocking code as if you were using standard blocking functions, all without the need for complex message exchange systems and synchronization handling for threaded programs, that only add overhead and complexity to your programs, making everything slower and error-prone.

That's very cool and all, you might think, but how exactly does this async stuff work? Well, as it turns out, it's very easy.

Instead of writing code like this:

$file = $MadelineProto->download_to_dir($bigfile, '/tmp/');

Write it like this:

$file = yield $MadelineProto->download_to_dir($bigfile, '/tmp/');

That's it.

It's really that easy, you just have to add a yield before calling MadelineProto methods.
The yield will automatically suspend the execution of the function, letting the program do other stuff while the file is being downloaded.
Once the file is downloaded, execution is automatically resumed from that exact point in the function.

This means that you can handle multiple updates, download/upload multiple files all together in one process, as if you were writing normal synchronous code + making everything a lot faster.

If your code still relies on the old synchronous behaviour, don't worry, there is backward compatibility.
However, I highly recommend you switch to async, due to the huge performance and parallelism benefits.

Loading the latest version of MadelineProto

In order to use the yield operator in MadelineProto, you have to load the latest version of MadelineProto from the master branch (alpha) by loading it through composer (dev-master) or with madeline.php:

<?php

if (!file_exists('madeline.php')) {
    copy('https://phar.madelineproto.xyz/madeline.php', 'madeline.php');
}
define('MADELINE_BRANCH', '');
include 'madeline.php';

The MADELINE_BRANCH constant you defines which branch of MadelineProto madeline.php should load.
When the constant is not set, the old stable branch is loaded; if the value is an empty string, the master branch is loaded; otherwise, the selected branch name is loaded.
WARNING: MadelineProto async is not compatible with pthreads or pcntl, so please uninstall pthreads and do not use pcntl_fork in your bot.

Enabling the MadelineProto async API

The yield operator can only be used within functions, once MadelineProto's async mode is enabled.
To do that, simply run the async function, or pass the async enabler flag separately to each method call, if you want to make only some calls async.

This will enable async mode for all MadelineProto functions:

$MadelineProto->async(true);
// ...
yield $MadelineProto->messages->sendMessage(...);

This will enable async mode for only one specific call of a MadelineProto function (by adding a new array parameter after all the required parameters):

yield $MadelineProto->messages->sendMessage(..., ['async' => true]);

Using the MadelineProto async API

As mentioned earlier, you can only use the yield operator within functions, but not just any function, for example (WILL NOT WORK):

$sm = function ($chatID, $message) use ($MadelineProto) {
    $id = (yield $MadelineProto->get_info($chatID, ['async' => true]))['bot_api_id'];
    $res = yield $MadelineProto->messages->sendMesssage(['peer' => $chatID, 'message' => "Message from: $id\n$message"], ['async' => true]);
    return $res;
};
$result = $sm('@danogentili', 'hi');

This will not work, because the result of a function that uses yield is not the returned value, but a generator, which is what the async AMPHP API is based on.
If the generator is not passed to the AMPHP event loop, execution of the function will not be resumed: when MadelineProto asynchronously obtains the result of the get_info, execution of the function is never resumed, and the line with sendMessage is never called.
To avoid this problem, only call asynchronous functions in the event/callback update handler, or in functions called by the event/callback update handler, or inside a function passed to loop.
You can also call asynchronous functions created by you, within other asynchronous functions.
Generators in MadelineProto are equivalent to promises, which is a paradigm you may have used in other languages.

Async in event handler:

$MadelineProto->async(true);
class EventHandler extends \danog\MadelineProto\EventHandler
{
    public function onAny($update)
    {
        if (isset($update['message']['out']) && $update['message']['out']) {
            return;
        }
        if (isset($update['message']['media']) && $update['message']['media']['_'] !== 'messageMediaGame') {
            yield $this->download_to_dir($update, '/tmp');
            yield $this->messages->sendMedia(['peer' => $update, 'message' => $update['message']['message'], 'media' => $update]);
        }

        $res = json_encode($update, JSON_PRETTY_PRINT);

        yield $this->sleep(3);

        try {
            yield $this->sm($update, "<code>$res</code>\nAsynchronously, after 3 seconds");
        } catch (\danog\MadelineProto\RPCErrorException $e) {
            \danog\MadelineProto\Logger::log((string) $e, \danog\MadelineProto\Logger::FATAL_ERROR);
        } catch (\danog\MadelineProto\Exception $e) {
            \danog\MadelineProto\Logger::log((string) $e, \danog\MadelineProto\Logger::FATAL_ERROR);
        }
    }
    public function sm($peer, $message)
    {
        yield $this->messages->sendMessage(['peer' => $peer, 'message' => $message, 'reply_to_msg_id' => isset($update['message']['id']) ? $update['message']['id'] : null, 'parse_mode' => 'HTML']);
    }
}

Async in callback handler:

$MadelineProto->async(true);
$MadelineProto->setCallback(function ($update) use ($MadelineProto) {
    if (isset($update['message']['out']) && $update['message']['out']) {
        return;
    }
    yield $MadelineProto->sleep(3);
    yield $MadelineProto->messages->sendMessage(['peer' => $update, 'message' => 'Hi after 3 seconds']);
});

Wrapped async

$MadelineProto->async(true);
$MadelineProto->loop(function () use ($MadelineProto) {
    yield $MadelineProto->messages->sendMessage(['peer' => '@danogentili', 'message' => 'hi']);
    // You can also have an asynchronous get_updates (deprecated) loop in here, if you want to; just don't forget to use yield for all MadelineProto functions.
});

Ignored async

$MadelineProto->messages->sendMessage(['peer' => '@danogentili', 'message' => 'a'], ['async' => true]);
$MadelineProto->messages->sendMessage(['peer' => '@danogentili', 'message' => 'b'], ['async' => true]);

You can use the async version of MadelineProto functions without yield if you don't want the request to block, and you don't need the result of the function.
This is allowed, but the order of the function calls will not be guaranteed: you can use call queues if you want to make sure the order of the calls remains the same. See async forking.

Blocking async

$result = blocking_function();

Sometimes, you have to call non-async functions in your code: that is allowed in async MadelineProto, you just have to call your functions normally without yield.
However, you shouldn't do (or need to do) this, because this renders async completely useless.
AMPHP already provides async versions of curl, file_get_contents, MySQL, redis, postgres, and many more native PHP functions:

MadelineProto and AMPHP async APIs

MadelineProto and AMPHP both provide a lot of async functions: all of MadelineProto's functions are async, for example; and AMPHP provides multiple packages to work asynchronously with HTTP requests, websockets, databases (MySQL, redis, postgres, DNS, sockets and much more!
When using AMPHP libraries, you just have to use them with yield, no need to start the event loop, as long as you're running the functions inside MadelineProto's update handler/loop.

Also, you should read the AMPHP docs, especially the event loop docs: AMPHP provides multiple helper methods for executing actions repeatedly every N seconds in a non-blocking manner, or to defer execution of certain actions (aka async cron).

Helper methods

MadelineProto also provides a few generic async helper methods: when possible, always use MadelineProto's wrapped versions of the amphp combinators and amphp helpers instead of original amphp methods (all, any, some, first, ...).

Async sleep (does not block the main thread)

yield $MadelineProto->sleep(3);

Async forking (does async single-thread forking)

Useful if you need to start a process in the background and you want throwed exceptions to surface up.
These exceptions will exit the event loop, turning off the script unless you wrap $MadelineProto->loop() with a try-catch.
Use it when you do not need the result of a method (see ignored async), but you want eventual errors to crash the script.
Otherwise, just use the method without yield.

// Exceptions will surface out of the event loop()
$MadelineProto->callFork($MadelineProto->messages->sendMessage([...]));
// Exceptions will be ignored
$MadelineProto->messages->sendMessage([...]);

// Like the first one, but the call will be deferred to the next event loop tick
$MadelineProto->callForkDefer($MadelineProto->messages->sendMessage([...]));

Ignoring exceptions is usually not good practice, so it's best to wrap the method you're calling in a closure with a try-catch with some error handling code inside of it, calling it right after that and passing it to callFork:

$MadelineProto->callFork((function () use ($MadelineProto) {
    try {
        $MadelineProto->messages->sendMessage([...])
    } catch (\Exception $e) {
        // Handle by logging and stuff
    }
})());

Combining async operations

These methods can be used to execute multiple async operations simultaneously and wait for the result of all of them.
Each method has different error handling techniques, see the amphp docs.
Note that if you just take the result of these methods without yielding it, you can use it as a normal promise/generator.

$promise1 = $MadelineProto->messages->sendMessage(...);
$promise2 = $MadelineProto->messages->sendMessage(...);
// $promise3 = ...;

// Equivalent to Amp\Promise\all(), but works with generators, too
$results = yield $MadelineProto->all([$promise1, $promise2, $generator3]);

// Equivalent to Amp\Promise\first(), but works with generators, too
$results = yield $MadelineProto->first([$promise1, $promise2, $generator3]);

// Equivalent to Amp\Promise\any(), but works with generators, too
$results = yield $MadelineProto->any([$promise1, $promise2, $generator3]);

// Equivalent to Amp\Promise\some(), but works with generators, too
$results = yield $MadelineProto->some([$promise1, $promise2, $generator3]);

Handling timeouts

These methods can be used to wait for a certain amount of time for a result, and then throw an Amp\TimeoutException or simply continue execution if no result was obtained.

// Waits for the result for 2 seconds and then throws an \Amp\TimeoutException
$result = yield $MadelineProto->timeout($promise, 2)

// Waits for the result for 2 seconds, returns the result or null (which is the result of sleep())
$result = yield $MadelineProto->first([$promise, $MadelineProto->sleep(2)]);

Async loop APIs

MadelineProto provides a very useful async loop APIs, for executing operations periodically or on demand.
It's a more flexible and powerful alternative to AMPHP's repeat, allowing dynamically changeable repeat periods, resumes and signaling.

Next section

14 KiB Raw Blame History