OpenAPI, changelog, cache package support

This commit is contained in:
Sys 2021-08-23 13:12:50 +02:00
parent 137960dea2
commit 4d87d922f6
No known key found for this signature in database
GPG Key ID: 3CD2C29F8AB39BFD
19 changed files with 1005 additions and 249 deletions

View File

@ -1,9 +1,7 @@
name: Build
on:
push:
branches:
- master
pull_request:
release:
types: [ published ]
jobs:
build:

34
.github/workflows/pages.yml vendored Normal file
View File

@ -0,0 +1,34 @@
name: GH Pages
on:
schedule:
- cron: '0 0 * * *'
workflow_dispatch:
jobs:
build:
name: Build files for GH Pages
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout@v2
- name: Generate schemas
uses: addnab/docker-run-action@v3
with:
registry: ghcr.io
image: sysbot-org/tgscraper:latest
options: -v ${{ github.workspace }}:/out
run: |
'app:export-schema --readable botapi.json'
'app:export-schema --yaml --readable botapi.yaml'
'app:export-schema --postman --readable botapi_postman.json'
'app:export-schema --openapi --readable botapi_openapi.json'
'app:export-schema --yaml --openapi --readable botapi_openapi.yaml'
- name: Deploy
uses: peaceiris/actions-gh-pages@v3
with:
github_token: ${{ secrets.GITHUB_TOKEN }}
publish_dir: ${{ github.workspace }}
destination_dir: schemas
publish_branch: gh-pages
cname: tgscraper.sys001.ml
enable_jekyll: true

139
CHANGELOG.md Normal file
View File

@ -0,0 +1,139 @@
# Changelog
All notable changes to this project will be documented in this file.
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
## [Unreleased]
### Added
- Support for OpenAPI schema: you can now generate code in any language!
- Support for the new [sysbot/tgscraper-cache](https://github.com/Sysbot-org/tgscraper-cache) package: if installed, TGScraper will be much faster (there is no need to always fetch the live webpages)!
- You can now validate a schema by using the `validateSchema` method, provided by the `TgScraper` class.
- New `Versions::STABLE` constant: it will automatically return the latest stable version instead of the live version (useful for the cache package).
- Added the JSON schema specification for the custom format provided by TGScraper.
- Added this changelog.
### Changed
- The `Generator` class has now been renamed `TgScraper`.
- The `required` property in method fields has been replaced by the new `optional` property, for the sake of consistency.
- You now need a schema in order to instantiate the `TgScraper` class (don't worry, you can use the new methods `TgScraper::fromUrl` and `TgScraper::fromVersion`).
- The `Versions` class constants have been replaced with an actual version string. If you still need the URLs, use the new class constant `Versions::URLS`.
- TGScraper will now only return arrays. If you still need JSON or YAML encoding, please use the new `Encoder` class.
### Fixed
- Minor improvements to `StubCreator`.
- Fixed an issue with the CLI where `autoload.php` couldn't be found.
- When exporting the schema, the CLI will now make sure that the destination directory exists.
### Security
- When a custom schema is used, only use `version`, `types` and `methods` fields.
## [2.1.0] - 2021-07-31
### Added
- New repo workflows: automatic package build (and push to the GitHub registry), and automatic notifications via Telegram.
- New `version` field for schemas: it contains the bot API version (if possible).
- New `extended_by` field for types: if the current type is a parent one, it will contain its child types.
### Changed
- Now all type stubs implement the base `TypeInterface` interface.
- Children type stubs now extend their parent.
- Optional fields are now actually optional in the Postman collection (previously, you had to manually disable optional ones).
### Fixed
- Minor improvements to the schema extractor.
## [2.0.1] - 2021-07-24
### Changed
- The README now includes many CLI examples.
### Fixed
- The link for the Bot API 5.2.0 snapshot was broken, it's working now.
## [2.0.0] - 2021-07-24
### Added
- Support for Postman collection: now you can generate a JSON to use in Postman!
- New class for the URLs of various bot API snapshots: `TgScraper\Constants\Versions`.
### Changed
- Moved `TgScraper\StubCreator` to the new namespace `TgScraper\Common\StubCreator`.
- Moved scraping logic from `TgScraper\Generator` to the new class `TgScraper\Common\SchemaExtractor`.
- CLI has been completely reworked: it now uses the Symfony Console and it's much more reliable!
## [1.4.0] - 2021-06-23
### Added
- YAML format is now supported.
- Docker support! The package is published on the GitHub registry.
## [1.3.0] - 2021-06-22
### Added
- Badges in the README! They contain a lot of useful information about the project (such as the minimum PHP version, latest stable version, etc).
### Fixed
- CLI now catches exceptions more reliably.
### Security
- Dependency `paquettg/php-html-parser` has been upgraded to `^3.1`.
## [1.2.2] - 2021-06-20
### Fixed
- Fixed a typo in a property name of the `Response` class stub.
## [1.2.1] - 2021-06-19
### Removed
- The abstract constructor for the `API` trait stub has now been removed.
## [1.2.0] - 2021-06-19
### Changed
- The `API` class stub has been converted to a trait, and the constructor and the `sendRequest` methods are now abstract.
### Fixed
- Minor improvements to the CLI.
## [1.1.0] - 2021-06-18
### Added
- New field for types: `optional`. It tells whether a value is always present or not.
- Class stubs now have typed properties (with related PHPDoc comments).
### Changed
- Variable names have been changed to `camelCase`.
## [1.0.2] - 2021-06-18
### Fixed
- Improved argument parsing for the CLI.
## [1.0.1] - 2021-06-17
### Changed
- Project license is now the GNU Lesser GPL.
## [1.0.0] - 2021-06-17
### Added
- New CLI to easily generate JSON schema or class stubs!
- It's now possible to parse old bot API webpages! Pass the URL and it should work just fine.
- New API class stub, it implements all bot API methods (it's incomplete though, so you must add your custom logic).
### Changed
- Renamed project to `tgscraper`.
- Class namespace is now `TgScraper`, for the sake of consistency.
- `StubProvider` class is now named `StubCreator`.
- Reworked syntax for array of objects in field types: `Object[]` has been replaced by `Array<Object>`.
### Removed
- Method stubs will no longer be generated.
### Fixed
- The parser is now more reliable, it no longer needs to be updated at every bot API release!
[Unreleased]: https://github.com/Sysbot-org/tgscraper/compare/2.1...HEAD
[2.1.0]: https://github.com/Sysbot-org/tgscraper/compare/2.0.1...2.1
[2.0.1]: https://github.com/Sysbot-org/tgscraper/compare/2.0...2.0.1
[2.0.0]: https://github.com/Sysbot-org/tgscraper/compare/1.4...2.0
[1.4.0]: https://github.com/Sysbot-org/tgscraper/compare/1.3...1.4
[1.3.0]: https://github.com/Sysbot-org/tgscraper/compare/1.2.1...1.3
[1.2.1]: https://github.com/Sysbot-org/tgscraper/compare/1.2...1.2.1
[1.2.0]: https://github.com/Sysbot-org/tgscraper/compare/1.1...1.2
[1.1.0]: https://github.com/Sysbot-org/tgscraper/compare/1.0.2...1.1
[1.0.2]: https://github.com/Sysbot-org/tgscraper/compare/1.0.1...1.0.2
[1.0.1]: https://github.com/Sysbot-org/tgscraper/compare/1.0...1.0.1
[1.0.0]: https://github.com/Sysbot-org/tgscraper/releases/tag/1.0

View File

@ -3,9 +3,8 @@ FROM composer:latest AS tgscraper
MAINTAINER Sys <sys@sys001.ml>
WORKDIR /app
COPY . .
RUN composer install
WORKDIR /artifacts
VOLUME /artifacts
RUN composer require sysbot/tgscraper sysbot/tgscraper-cache --no-progress --no-interaction --no-ansi --prefer-stable --optimize-autoloader
WORKDIR /out
VOLUME /out
ENTRYPOINT ["php", "/app/bin/tgscraper"]
ENTRYPOINT ["php", "/app/vendor/bin/tgscraper"]

View File

@ -9,8 +9,10 @@
A PHP library used to extract JSON data (and auto-generate PHP classes)
from [Telegram bot API documentation page](https://core.telegram.org/bots/api).
**Note: the scraper is, obviously, based on a hack and you shouldn't rely on automagically generated files from it,
since they are prone to errors. I'll try to fix them ASAP, but manual review is always required (at least for now).**
## Changelog
Interested in recent changes? Have a look [here](CHANGELOG.md)!
## Installation
@ -52,12 +54,23 @@ Extract the latest schema in YAML format:
$ vendor/bin/tgscraper app:export-schema --yaml botapi.yaml
```
### OpenAPI
Work in progress.
### Stubs
_Note: since Telegram may change the page format at any time, do **NOT** rely on the automagically generated
stubs from this library, **ALWAYS** review the code!_
TGScraper can also generate class stubs that you can use in your library. A sample implementation is available in the [Sysbot Telegram module](https://github.com/Sysbot-org/Sysbot-tg).
Create stubs in the `out/` directory using `Sysbot\Api` as namespace prefix:
Create stubs in the `out/` directory using `Sysbot\Telegram` as namespace prefix:
```bash
$ vendor/bin/tgscraper app:create-stubs --namespace-prefix "Sysbot\Api" out
```
$ vendor/bin/tgscraper app:create-stubs --namespace-prefix "Sysbot\Telegram" out
```
## Custom format
If you're interested in the custom format generated by TGScraper, you can find its schema [here](docs/schema.json).

View File

@ -6,7 +6,17 @@ use Symfony\Component\Console\Application;
use TgScraper\Commands\CreateStubsCommand;
use TgScraper\Commands\ExportSchemaCommand;
require_once __DIR__ . '/../vendor/autoload.php';
$autoloadFiles = [
__DIR__ . '/../vendor/autoload.php',
__DIR__ . '/../../../autoload.php'
];
foreach ($autoloadFiles as $autoloadFile) {
if (file_exists($autoloadFile)) {
require_once $autoloadFile;
break;
}
}
$application = new Application('TGScraper', InstalledVersions::getVersion('sysbot/tgscraper'));

View File

@ -12,6 +12,9 @@
"symfony/console": "^5.3",
"symfony/yaml": "^5.3"
},
"suggest": {
"sysbot/tgscraper-cache": "To speed up schema fetching and generation."
},
"autoload": {
"psr-4": {
"TgScraper\\": "src/"

79
composer.lock generated
View File

@ -369,16 +369,16 @@
},
{
"name": "nette/utils",
"version": "v3.2.2",
"version": "v3.2.3",
"source": {
"type": "git",
"url": "https://github.com/nette/utils.git",
"reference": "967cfc4f9a1acd5f1058d76715a424c53343c20c"
"reference": "5c36cc1ba9bb6abb8a9e425cf054e0c3fd5b9822"
},
"dist": {
"type": "zip",
"url": "https://api.github.com/repos/nette/utils/zipball/967cfc4f9a1acd5f1058d76715a424c53343c20c",
"reference": "967cfc4f9a1acd5f1058d76715a424c53343c20c",
"url": "https://api.github.com/repos/nette/utils/zipball/5c36cc1ba9bb6abb8a9e425cf054e0c3fd5b9822",
"reference": "5c36cc1ba9bb6abb8a9e425cf054e0c3fd5b9822",
"shasum": ""
},
"require": {
@ -448,9 +448,9 @@
],
"support": {
"issues": "https://github.com/nette/utils/issues",
"source": "https://github.com/nette/utils/tree/v3.2.2"
"source": "https://github.com/nette/utils/tree/v3.2.3"
},
"time": "2021-03-03T22:53:25+00:00"
"time": "2021-08-16T21:05:00+00:00"
},
{
"name": "paquettg/php-html-parser",
@ -938,16 +938,16 @@
},
{
"name": "symfony/console",
"version": "v5.3.2",
"version": "v5.3.6",
"source": {
"type": "git",
"url": "https://github.com/symfony/console.git",
"reference": "649730483885ff2ca99ca0560ef0e5f6b03f2ac1"
"reference": "51b71afd6d2dc8f5063199357b9880cea8d8bfe2"
},
"dist": {
"type": "zip",
"url": "https://api.github.com/repos/symfony/console/zipball/649730483885ff2ca99ca0560ef0e5f6b03f2ac1",
"reference": "649730483885ff2ca99ca0560ef0e5f6b03f2ac1",
"url": "https://api.github.com/repos/symfony/console/zipball/51b71afd6d2dc8f5063199357b9880cea8d8bfe2",
"reference": "51b71afd6d2dc8f5063199357b9880cea8d8bfe2",
"shasum": ""
},
"require": {
@ -955,11 +955,12 @@
"symfony/deprecation-contracts": "^2.1",
"symfony/polyfill-mbstring": "~1.0",
"symfony/polyfill-php73": "^1.8",
"symfony/polyfill-php80": "^1.15",
"symfony/polyfill-php80": "^1.16",
"symfony/service-contracts": "^1.1|^2",
"symfony/string": "^5.1"
},
"conflict": {
"psr/log": ">=3",
"symfony/dependency-injection": "<4.4",
"symfony/dotenv": "<5.1",
"symfony/event-dispatcher": "<4.4",
@ -967,10 +968,10 @@
"symfony/process": "<4.4"
},
"provide": {
"psr/log-implementation": "1.0"
"psr/log-implementation": "1.0|2.0"
},
"require-dev": {
"psr/log": "~1.0",
"psr/log": "^1|^2",
"symfony/config": "^4.4|^5.0",
"symfony/dependency-injection": "^4.4|^5.0",
"symfony/event-dispatcher": "^4.4|^5.0",
@ -1016,7 +1017,7 @@
"terminal"
],
"support": {
"source": "https://github.com/symfony/console/tree/v5.3.2"
"source": "https://github.com/symfony/console/tree/v5.3.6"
},
"funding": [
{
@ -1032,7 +1033,7 @@
"type": "tidelift"
}
],
"time": "2021-06-12T09:42:48+00:00"
"time": "2021-07-27T19:10:22+00:00"
},
{
"name": "symfony/deprecation-contracts",
@ -1182,16 +1183,16 @@
},
{
"name": "symfony/polyfill-intl-grapheme",
"version": "v1.23.0",
"version": "v1.23.1",
"source": {
"type": "git",
"url": "https://github.com/symfony/polyfill-intl-grapheme.git",
"reference": "24b72c6baa32c746a4d0840147c9715e42bb68ab"
"reference": "16880ba9c5ebe3642d1995ab866db29270b36535"
},
"dist": {
"type": "zip",
"url": "https://api.github.com/repos/symfony/polyfill-intl-grapheme/zipball/24b72c6baa32c746a4d0840147c9715e42bb68ab",
"reference": "24b72c6baa32c746a4d0840147c9715e42bb68ab",
"url": "https://api.github.com/repos/symfony/polyfill-intl-grapheme/zipball/16880ba9c5ebe3642d1995ab866db29270b36535",
"reference": "16880ba9c5ebe3642d1995ab866db29270b36535",
"shasum": ""
},
"require": {
@ -1243,7 +1244,7 @@
"shim"
],
"support": {
"source": "https://github.com/symfony/polyfill-intl-grapheme/tree/v1.23.0"
"source": "https://github.com/symfony/polyfill-intl-grapheme/tree/v1.23.1"
},
"funding": [
{
@ -1259,7 +1260,7 @@
"type": "tidelift"
}
],
"time": "2021-05-27T09:17:38+00:00"
"time": "2021-05-27T12:26:48+00:00"
},
{
"name": "symfony/polyfill-intl-normalizer",
@ -1347,16 +1348,16 @@
},
{
"name": "symfony/polyfill-mbstring",
"version": "v1.23.0",
"version": "v1.23.1",
"source": {
"type": "git",
"url": "https://github.com/symfony/polyfill-mbstring.git",
"reference": "2df51500adbaebdc4c38dea4c89a2e131c45c8a1"
"reference": "9174a3d80210dca8daa7f31fec659150bbeabfc6"
},
"dist": {
"type": "zip",
"url": "https://api.github.com/repos/symfony/polyfill-mbstring/zipball/2df51500adbaebdc4c38dea4c89a2e131c45c8a1",
"reference": "2df51500adbaebdc4c38dea4c89a2e131c45c8a1",
"url": "https://api.github.com/repos/symfony/polyfill-mbstring/zipball/9174a3d80210dca8daa7f31fec659150bbeabfc6",
"reference": "9174a3d80210dca8daa7f31fec659150bbeabfc6",
"shasum": ""
},
"require": {
@ -1407,7 +1408,7 @@
"shim"
],
"support": {
"source": "https://github.com/symfony/polyfill-mbstring/tree/v1.23.0"
"source": "https://github.com/symfony/polyfill-mbstring/tree/v1.23.1"
},
"funding": [
{
@ -1423,7 +1424,7 @@
"type": "tidelift"
}
],
"time": "2021-05-27T09:27:20+00:00"
"time": "2021-05-27T12:26:48+00:00"
},
{
"name": "symfony/polyfill-php73",
@ -1506,16 +1507,16 @@
},
{
"name": "symfony/polyfill-php80",
"version": "v1.23.0",
"version": "v1.23.1",
"source": {
"type": "git",
"url": "https://github.com/symfony/polyfill-php80.git",
"reference": "eca0bf41ed421bed1b57c4958bab16aa86b757d0"
"reference": "1100343ed1a92e3a38f9ae122fc0eb21602547be"
},
"dist": {
"type": "zip",
"url": "https://api.github.com/repos/symfony/polyfill-php80/zipball/eca0bf41ed421bed1b57c4958bab16aa86b757d0",
"reference": "eca0bf41ed421bed1b57c4958bab16aa86b757d0",
"url": "https://api.github.com/repos/symfony/polyfill-php80/zipball/1100343ed1a92e3a38f9ae122fc0eb21602547be",
"reference": "1100343ed1a92e3a38f9ae122fc0eb21602547be",
"shasum": ""
},
"require": {
@ -1569,7 +1570,7 @@
"shim"
],
"support": {
"source": "https://github.com/symfony/polyfill-php80/tree/v1.23.0"
"source": "https://github.com/symfony/polyfill-php80/tree/v1.23.1"
},
"funding": [
{
@ -1585,7 +1586,7 @@
"type": "tidelift"
}
],
"time": "2021-02-19T12:13:01+00:00"
"time": "2021-07-28T13:41:28+00:00"
},
{
"name": "symfony/service-contracts",
@ -1751,16 +1752,16 @@
},
{
"name": "symfony/yaml",
"version": "v5.3.3",
"version": "v5.3.6",
"source": {
"type": "git",
"url": "https://github.com/symfony/yaml.git",
"reference": "485c83a2fb5893e2ff21bf4bfc7fdf48b4967229"
"reference": "4500fe63dc9c6ffc32d3b1cb0448c329f9c814b7"
},
"dist": {
"type": "zip",
"url": "https://api.github.com/repos/symfony/yaml/zipball/485c83a2fb5893e2ff21bf4bfc7fdf48b4967229",
"reference": "485c83a2fb5893e2ff21bf4bfc7fdf48b4967229",
"url": "https://api.github.com/repos/symfony/yaml/zipball/4500fe63dc9c6ffc32d3b1cb0448c329f9c814b7",
"reference": "4500fe63dc9c6ffc32d3b1cb0448c329f9c814b7",
"shasum": ""
},
"require": {
@ -1806,7 +1807,7 @@
"description": "Loads and dumps YAML files",
"homepage": "https://symfony.com",
"support": {
"source": "https://github.com/symfony/yaml/tree/v5.3.3"
"source": "https://github.com/symfony/yaml/tree/v5.3.6"
},
"funding": [
{
@ -1822,7 +1823,7 @@
"type": "tidelift"
}
],
"time": "2021-06-24T08:13:00+00:00"
"time": "2021-07-29T06:20:01+00:00"
}
],
"packages-dev": [],

114
docs/schema.json Normal file
View File

@ -0,0 +1,114 @@
{
"$schema": "http://json-schema.org/draft-07/schema#",
"title": "Schema for TGScraper custom Bot API format",
"description": "This schema should help you understanding the custom format used by TGScraper.",
"type": "object",
"required": [
"version",
"methods",
"types"
],
"properties": {
"version": {
"type": "string"
},
"methods": {
"type": "array",
"items": {
"$ref": "#/definitions/Method"
}
},
"types": {
"type": "array",
"items": {
"$ref": "#/definitions/Type"
}
}
},
"definitions": {
"Method": {
"type": "object",
"required": [
"name",
"description",
"fields",
"return_types"
],
"properties": {
"name": {
"type": "string"
},
"description": {
"type": "string"
},
"fields": {
"type": "array",
"items": {
"$ref": "#/definitions/Field"
}
},
"return_types": {
"type": "array",
"items": {
"type": "string"
}
}
}
},
"Field": {
"type": "object",
"required": [
"name",
"types",
"optional",
"description"
],
"properties": {
"name": {
"type": "string"
},
"types": {
"type": "array",
"items": {
"type": "string"
}
},
"required": {
"type": "boolean"
},
"description": {
"type": "string"
}
}
},
"Type": {
"type": "object",
"required": [
"name",
"description",
"fields",
"extended_by"
],
"properties": {
"name": {
"type": "string"
},
"description": {
"type": "string"
},
"fields": {
"type": "array",
"items": {
"$ref": "#/definitions/Field"
}
},
"extended_by": {
"type": "array",
"items": {
"type": "string"
}
}
}
}
}
}

View File

@ -12,7 +12,7 @@ use Symfony\Component\Console\Input\InputOption;
use Symfony\Component\Console\Logger\ConsoleLogger;
use Symfony\Component\Console\Output\OutputInterface;
use TgScraper\Constants\Versions;
use TgScraper\Generator;
use TgScraper\TgScraper;
use Throwable;
class CreateStubsCommand extends Command
@ -44,21 +44,30 @@ class CreateStubsCommand extends Command
InputOption::VALUE_REQUIRED,
'Path to YAML file to use instead of fetching from URL (this option takes precedence over "--layer" and "--json")'
)
->addOption('layer', 'l', InputOption::VALUE_REQUIRED, 'Bot API version to use', 'latest');
->addOption('layer', 'l', InputOption::VALUE_REQUIRED, 'Bot API version to use', 'latest')
->addOption(
'prefer-stable',
null,
InputOption::VALUE_NONE,
'Prefer latest stable version (takes precedence over "--layer")'
);
}
protected function execute(InputInterface $input, OutputInterface $output): int
{
$logger = new ConsoleLogger($output);
$url = Versions::getVersionFromText($input->getOption('layer'));
$version = Versions::getVersionFromText($input->getOption('layer'));
if ($input->getOption('prefer-stable')) {
$version = Versions::STABLE;
}
$yamlPath = $input->getOption('yaml');
if (empty($yamlPath)) {
$jsonPath = $input->getOption('json');
if (empty($jsonPath)) {
$logger->info('Using URL: ' . $url);
$logger->info('Using version: ' . $version);
try {
$output->writeln('Fetching data from URL...');
$generator = new Generator($logger, $url);
$output->writeln('Fetching data for version...');
$generator = TgScraper::fromVersion($logger, $version);
} catch (Throwable) {
return Command::FAILURE;
}
@ -70,7 +79,7 @@ class CreateStubsCommand extends Command
}
$logger->info('Using JSON schema: ' . $jsonPath);
/** @noinspection PhpUnhandledExceptionInspection */
$generator = Generator::fromJson($logger, $data);
$generator = TgScraper::fromJson($logger, $data);
}
} else {
$data = file_get_contents($yamlPath);
@ -80,7 +89,7 @@ class CreateStubsCommand extends Command
}
$logger->info('Using YAML schema: ' . $yamlPath);
/** @noinspection PhpUnhandledExceptionInspection */
$generator = Generator::fromYaml($logger, $data);
$generator = TgScraper::fromYaml($logger, $data);
}
try {
$output->writeln('Creating stubs...');

View File

@ -4,14 +4,17 @@
namespace TgScraper\Commands;
use Exception;
use Psr\Log\LoggerInterface;
use Symfony\Component\Console\Command\Command;
use Symfony\Component\Console\Input\InputArgument;
use Symfony\Component\Console\Input\InputInterface;
use Symfony\Component\Console\Input\InputOption;
use Symfony\Component\Console\Logger\ConsoleLogger;
use Symfony\Component\Console\Output\OutputInterface;
use TgScraper\Common\Encoder;
use TgScraper\Constants\Versions;
use TgScraper\Generator;
use TgScraper\TgScraper;
use Throwable;
class ExportSchemaCommand extends Command
@ -29,39 +32,40 @@ class ExportSchemaCommand extends Command
'yaml',
null,
InputOption::VALUE_NONE,
'Export schema as YAML instead of JSON (this option takes precedence over "--postman")'
'Export schema as YAML instead of JSON (does not affect "--postman")'
)
->addOption(
'postman',
null,
InputOption::VALUE_NONE,
'Export schema as a Postman-compatible JSON'
)
->addOption(
'openapi',
null,
InputOption::VALUE_NONE,
'Export schema as a OpenAPI-compatible file (takes precedence over "--postman")'
)
->addOption('postman', null, InputOption::VALUE_NONE, 'Export schema as a Postman-compatible JSON')
->addOption('options', 'o', InputOption::VALUE_REQUIRED, 'Encoder options', 0)
->addOption('readable', 'r', InputOption::VALUE_NONE, '(JSON only) Generate a human-readable JSON')
->addOption('inline', null, InputOption::VALUE_REQUIRED, '(YAML only) Inline level', 6)
->addOption(
'readable',
'r',
InputOption::VALUE_NONE,
'Generate a human-readable file (overrides "--inline" and "--indent")'
)
->addOption('inline', null, InputOption::VALUE_REQUIRED, '(YAML only) Inline level', 12)
->addOption('indent', null, InputOption::VALUE_REQUIRED, '(YAML only) Indent level', 4)
->addOption('layer', 'l', InputOption::VALUE_REQUIRED, 'Bot API version to use', 'latest');
->addOption('layer', 'l', InputOption::VALUE_REQUIRED, 'Bot API version to use', Versions::LATEST)
->addOption(
'prefer-stable',
null,
InputOption::VALUE_NONE,
'Prefer latest stable version (takes precedence over "--layer")'
);
}
protected function execute(InputInterface $input, OutputInterface $output): int
private function saveFile(ConsoleLogger $logger, OutputInterface $output, string $destination, string $data): int
{
$logger = new ConsoleLogger($output);
$url = Versions::getVersionFromText($input->getOption('layer'));
$logger->info('Using URL: ' . $url);
try {
$output->writeln('Fetching data from URL...');
$generator = new Generator($logger, $url);
} catch (Throwable) {
return Command::FAILURE;
}
$output->writeln('Exporting schema from data...');
$options = $input->getOption('options');
$useYaml = $input->getOption('yaml');
if ($useYaml) {
$data = $generator->toYaml($input->getOption('inline'), $input->getOption('indent'), $options);
}
$destination = $input->getArgument('destination');
if ($input->getOption('readable')) {
$options |= JSON_PRETTY_PRINT | JSON_UNESCAPED_SLASHES | JSON_UNESCAPED_UNICODE;
}
$data ??= $input->getOption('postman') ? $generator->toPostman($options) : $generator->toJson($options);
$output->writeln('Saving scheme to file...');
$result = file_put_contents($destination, $data);
if (false === $result) {
$logger->critical('Unable to save file to ' . $destination);
@ -71,4 +75,49 @@ class ExportSchemaCommand extends Command
return Command::SUCCESS;
}
protected function execute(InputInterface $input, OutputInterface $output): int
{
$logger = new ConsoleLogger($output);
$version = Versions::getVersionFromText($input->getOption('layer'));
if ($input->getOption('prefer-stable')) {
$version = Versions::STABLE;
}
$logger->info('Using version: ' . $version);
try {
$output->writeln('Fetching data for version...');
$generator = TgScraper::fromVersion($logger, $version);
} catch (Throwable) {
return Command::FAILURE;
}
$output->writeln('Exporting schema from data...');
$destination = $input->getArgument('destination');
try {
TgScraper::getTargetDirectory(pathinfo($destination)['dirname']);
} catch (Exception) {
return Command::FAILURE;
}
$readable = $input->getOption('readable');
$options = $input->getOption('options');
$useYaml = $input->getOption('yaml');
$inline = $readable ? 12 : $input->getOption('inline');
$indent = $readable ? 4 : $input->getOption('indent');
$output->writeln('Saving schema to file...');
if ($input->getOption('openapi')) {
$data = $generator->toOpenApi();
if ($useYaml) {
return $this->saveFile($logger, $output, $destination, Encoder::toYaml($data, $inline, $indent, $options));
}
return $this->saveFile($logger, $output, $destination, Encoder::toJson($data, $options, $readable));
}
if ($input->getOption('postman')) {
$data = $generator->toPostman();
return $this->saveFile($logger, $output, $destination, Encoder::toJson($data, $options, $readable));
}
$data = $generator->toArray();
if ($useYaml) {
return $this->saveFile($logger, $output, $destination, Encoder::toYaml($data, $inline, $indent, $options));
}
return $this->saveFile($logger, $output, $destination, Encoder::toJson($data, $options, $readable));
}
}

34
src/Common/Encoder.php Normal file
View File

@ -0,0 +1,34 @@
<?php
namespace TgScraper\Common;
use Symfony\Component\Yaml\Yaml;
class Encoder
{
/**
* @param mixed $data
* @param int $options
* @param bool $readable
* @return string
*/
public static function toJson(mixed $data, int $options = 0, bool $readable = false): string
{
if ($readable) {
$options |= JSON_PRETTY_PRINT | JSON_UNESCAPED_SLASHES | JSON_UNESCAPED_UNICODE;
}
return json_encode($data, $options);
}
/**
* @param mixed $data
* @param int $inline
* @param int $indent
* @param int $flags
* @return string
*/
public static function toYaml(mixed $data, int $inline = 12, int $indent = 4, int $flags = 0): string
{
return Yaml::dump($data, $inline, $indent, $flags);
}
}

View File

@ -0,0 +1,132 @@
<?php
namespace TgScraper\Common;
class OpenApiGenerator
{
public function __construct(private array $defaultResponses, private array $data, array $types, array $methods)
{
$this->addTypes($types);
$this->addMethods($methods);
}
public function setVersion($version = '1.0.0'): self
{
$this->data['info']['version'] = $version;
return $this;
}
public function addMethods(array $methods): self
{
foreach ($methods as $method) {
$this->addMethod($method['name'], $method['description'], $method['fields'], $method['return_types']);
}
return $this;
}
public function addMethod(string $name, string $description, array $fields, array $returnTypes): self
{
$method = '/' . $name;
$this->data['paths'][$method] = ['description' => $description];
$path = [];
$fields = self::addFields(['type' => 'object'], $fields);
$content = ['schema' => $fields];
if (!empty($fields['required'])) {
$path['requestBody']['required'] = true;
}
$path['requestBody']['content'] = [
'application/json' => $content,
'application/x-www-form-urlencoded' => $content,
'multipart/form-data' => $content
];
$path['responses'] = $this->defaultResponses;
$path['responses']['200']['content']['application/json']['schema']
['allOf'][1]['properties']['result'] = self::parsePropertyTypes($returnTypes);
$this->data['paths'][$method]['post'] = $path;
return $this;
}
public function addTypes(array $types): self
{
foreach ($types as $type) {
$this->addType($type['name'], $type['description'], $type['fields'], $type['extended_by']);
}
return $this;
}
public function addType(string $name, string $description, array $fields, array $extendedBy): self
{
$schema = ['description' => $description];
$schema = self::addFields($schema, $fields);
$this->data['components']['schemas'][$name] = $schema;
if (!empty($extendedBy)) {
foreach ($extendedBy as $extendedType) {
$this->data['components']['schemas'][$name]['anyOf'][] = self::parsePropertyType($extendedType);
}
return $this;
}
$this->data['components']['schemas'][$name]['type'] = 'object';
return $this;
}
private static function addFields(array $schema, array $fields): array
{
foreach ($fields as $field) {
$name = $field['name'];
$required = !$field['optional'];
if ($required) {
$schema['required'][] = $name;
}
$schema['properties'][$name] = self::parsePropertyTypes($field['types']);
}
return $schema;
}
private static function parsePropertyTypes(array $types): array
{
$result = [];
$hasMultipleTypes = count($types) > 1;
foreach ($types as $type) {
$type = self::parsePropertyType(trim($type));
if ($hasMultipleTypes) {
$result['anyOf'][] = $type;
continue;
}
$result = $type;
}
return $result;
}
private static function parsePropertyType(string $type): array
{
if (str_starts_with($type, 'Array')) {
return self::parsePropertyArray($type);
}
if (lcfirst($type) == $type) {
$type = str_replace(['int', 'float', 'bool'], ['integer', 'number', 'boolean'], $type);
return ['type' => $type];
}
return ['$ref' => '#/components/schemas/' . $type];
}
private static function parsePropertyArray(string $type): array
{
if (preg_match('/Array<(.+)>/', $type, $matches) === 1) {
return [
'type' => 'array',
'items' => self::parsePropertyTypes(explode(',', $matches[1]))
];
}
return [];
}
/**
* @return array
*/
public function getData(): array
{
return $this->data;
}
}

View File

@ -4,7 +4,10 @@
namespace TgScraper\Common;
use Composer\InstalledVersions;
use InvalidArgumentException;
use JetBrains\PhpStorm\ArrayShape;
use OutOfBoundsException;
use PHPHtmlParser\Dom;
use PHPHtmlParser\Exceptions\ChildNotFoundException;
use PHPHtmlParser\Exceptions\CircularException;
@ -33,33 +36,96 @@ class SchemaExtractor
'answerPreCheckoutQuery'
];
private Dom $dom;
/**
* @var string
*/
private string $version;
/**
* SchemaExtractor constructor.
* @param LoggerInterface $logger
* @param Dom $dom
* @throws ChildNotFoundException
* @throws NotLoadedException
*/
public function __construct(private LoggerInterface $logger, private Dom $dom)
{
$this->version = $this->parseVersion();
$this->logger->info('Bot API version: ' . $this->version);
}
/**
* @param LoggerInterface $logger
* @param string $version
* @return SchemaExtractor
* @throws OutOfBoundsException
* @throws Throwable
*/
public static function fromVersion(LoggerInterface $logger, string $version = Versions::LATEST): SchemaExtractor
{
if (InstalledVersions::isInstalled('sysbot/tgscraper-cache') and class_exists('\TgScraper\Cache\CacheLoader')) {
$logger->info('Cache package detected, searching for a cached version.');
try {
$path = \TgScraper\Cache\CacheLoader::getCachedVersion($version);
$logger->info('Cached version found.');
return self::fromFile($logger, $path);
} catch (OutOfBoundsException) {
$logger->info('Cached version not found, continuing with URL.');
}
}
$url = Versions::getUrlFromText($version);
$logger->info(sprintf('Using URL: %s', $url));
return self::fromUrl($logger, $url);
}
/**
* @param LoggerInterface $logger
* @param string $path
* @return SchemaExtractor
* @throws Throwable
*/
public static function fromFile(LoggerInterface $logger, string $path): SchemaExtractor
{
$dom = new Dom;
if (!file_exists($path)) {
throw new InvalidArgumentException('File not found');
}
$path = realpath($path);
try {
$logger->info(sprintf('Loading data from file "%s".', $path));
$dom->loadFromFile($path);
$logger->info('Data loaded.');
} catch (Throwable $e) {
$logger->critical(sprintf('Unable to load data from "%s": %s', $path, $e->getMessage()));
throw $e;
}
return new self($logger, $dom);
}
/**
* @param LoggerInterface $logger
* @param string $url
* @return SchemaExtractor
* @throws ChildNotFoundException
* @throws CircularException
* @throws ClientExceptionInterface
* @throws ContentLengthException
* @throws LogicalException
* @throws StrictException
* @throws Throwable
* @throws NotLoadedException
*/
public function __construct(private LoggerInterface $logger, private string $url = Versions::LATEST)
public static function fromUrl(LoggerInterface $logger, string $url): SchemaExtractor
{
$this->dom = new Dom();
$dom = new Dom;
try {
$this->dom->loadFromURL($this->url);
$dom->loadFromURL($url);
} catch (Throwable $e) {
$this->logger->critical(sprintf('Unable to load data from URL "%s": %s', $this->url, $e->getMessage()));
$logger->critical(sprintf('Unable to load data from URL "%s": %s', $url, $e->getMessage()));
throw $e;
}
$this->version = $this->parseVersion();
$this->logger->info('Bot API version: ' . $this->version);
$logger->info(sprintf('Data loaded from "%s".', $url));
return new self($logger, $dom);
}
/**
@ -96,6 +162,10 @@ class SchemaExtractor
return ['description' => $description, 'table' => $table, 'extended_by' => $extendedBy];
}
/**
* @throws ChildNotFoundException
* @throws NotLoadedException
*/
private function parseVersion(): string
{
/** @var Dom\Node\AbstractNode $element */
@ -120,14 +190,6 @@ class SchemaExtractor
/**
* @return array
* @throws ChildNotFoundException
* @throws CircularException
* @throws ContentLengthException
* @throws LogicalException
* @throws NotLoadedException
* @throws ParentNotFoundException
* @throws StrictException
* @throws ClientExceptionInterface
* @throws Throwable
*/
#[ArrayShape(['version' => "string", 'methods' => "array", 'types' => "array"])]
@ -136,7 +198,7 @@ class SchemaExtractor
try {
$elements = $this->dom->find('h4');
} catch (Throwable $e) {
$this->logger->critical(sprintf('Unable to load data from URL "%s": %s', $this->url, $e->getMessage()));
$this->logger->critical(sprintf('Unable to parse data: %s', $e->getMessage()));
throw $e;
}
$data = ['version' => $this->version];
@ -185,8 +247,7 @@ class SchemaExtractor
$result = [
'name' => $name,
'description' => htmlspecialchars_decode(strip_tags($description), ENT_QUOTES),
'fields' => $fields,
'extended_by' => $extendedBy
'fields' => $fields
];
if ($isMethod) {
$returnTypes = self::parseReturnTypes($description);
@ -196,6 +257,7 @@ class SchemaExtractor
$result['return_types'] = $returnTypes;
return $result;
}
$result['extended_by'] = $extendedBy;
return $result;
}
@ -203,15 +265,13 @@ class SchemaExtractor
* @param Dom\Node\Collection|null $fields
* @param bool $isMethod
* @return array
* @throws ChildNotFoundException
* @throws NotLoadedException
*/
private static function parseFields(?Dom\Node\Collection $fields, bool $isMethod): array
{
$parsedFields = [];
$fields = $fields ?? [];
foreach ($fields as $field) {
/* @var Dom $field */
/* @var Dom\Node\AbstractNode $fieldData */
$fieldData = $field->find('td');
$name = $fieldData[0]->text;
if (empty($name)) {
@ -224,7 +284,7 @@ class SchemaExtractor
$parsedData['types'] = self::parseFieldTypes($parsedData['type']);
unset($parsedData['type']);
if ($isMethod) {
$parsedData['required'] = $fieldData[2]->text == 'Yes';
$parsedData['optional'] = $fieldData[2]->text != 'Yes';
$parsedData['description'] = htmlspecialchars_decode(
strip_tags($fieldData[3]->innerHtml ?? $fieldData[3]->text ?? ''),
ENT_QUOTES

View File

@ -11,6 +11,7 @@ use Nette\PhpGenerator\Helpers;
use Nette\PhpGenerator\PhpFile;
use Nette\PhpGenerator\PhpNamespace;
use Nette\PhpGenerator\Type;
use TgScraper\TgScraper;
/**
* Class StubCreator
@ -48,9 +49,18 @@ class StubCreator
throw new InvalidArgumentException('Namespace invalid');
}
}
if (!is_array($this->schema['methods']) or !is_array($this->schema['types'])) {
if (!TgScraper::validateSchema($this->schema)) {
throw new InvalidArgumentException('Schema invalid');
}
$this->getExtendedTypes();
$this->namespace = $namespace;
}
/**
* Builds the abstract and the extended class lists.
*/
private function getExtendedTypes()
{
foreach ($this->schema['types'] as $type) {
if (!empty($type['extended_by'])) {
$this->abstractClasses[] = $type['name'];
@ -59,11 +69,12 @@ class StubCreator
}
}
}
print_r($this->extendedClasses);
print_r($this->abstractClasses);
$this->namespace = $namespace;
}
/**
* @param string $str
* @return string
*/
private static function toCamelCase(string $str): string
{
return lcfirst(str_replace(' ', '', ucwords(str_replace('_', ' ', $str))));
@ -74,11 +85,12 @@ class StubCreator
* @param PhpNamespace $phpNamespace
* @return array
*/
#[ArrayShape(['types' => "string", 'comments' => "string"])]
private function parseFieldTypes(
array $fieldTypes,
PhpNamespace $phpNamespace
): array {
#[ArrayShape([
'types' => "string",
'comments' => "string"
])]
private function parseFieldTypes(array $fieldTypes, PhpNamespace $phpNamespace): array
{
$types = [];
$comments = [];
foreach ($fieldTypes as $fieldType) {
@ -104,11 +116,12 @@ class StubCreator
* @param PhpNamespace $phpNamespace
* @return array
*/
#[ArrayShape(['types' => "string", 'comments' => "string"])]
private function parseApiFieldTypes(
array $apiTypes,
PhpNamespace $phpNamespace
): array {
#[ArrayShape([
'types' => "string",
'comments' => "string"
])]
private function parseApiFieldTypes(array $apiTypes, PhpNamespace $phpNamespace): array
{
$types = [];
$comments = [];
foreach ($apiTypes as $apiType) {
@ -138,9 +151,8 @@ class StubCreator
'Response' => "\Nette\PhpGenerator\PhpFile",
'TypeInterface' => "\Nette\PhpGenerator\ClassType"
])]
private function generateDefaultTypes(
string $namespace
): array {
private function generateDefaultTypes(string $namespace): array
{
$interfaceFile = new PhpFile;
$interfaceNamespace = $interfaceFile->addNamespace($namespace);
$interfaceNamespace->addInterface('TypeInterface');
@ -244,7 +256,7 @@ class StubCreator
usort(
$fields,
function ($a, $b) {
return $b['required'] - $a['required'];
return $a['optional'] - $b['optional'];
}
);
foreach ($fields as $field) {
@ -252,7 +264,7 @@ class StubCreator
$fieldName = self::toCamelCase($field['name']);
$parameter = $function->addParameter($fieldName)
->setType($types);
if (!$field['required']) {
if ($field['optional']) {
$parameter->setNullable()
->setDefaultValue(null);
}
@ -266,7 +278,10 @@ class StubCreator
/**
* @return array
*/
#[ArrayShape(['types' => "\Nette\PhpGenerator\PhpFile[]", 'api' => "string"])]
#[ArrayShape([
'types' => "\Nette\PhpGenerator\PhpFile[]",
'api' => "string"
])]
public function generateCode(): array
{
return [

View File

@ -7,46 +7,84 @@ namespace TgScraper\Constants;
class Versions
{
public const V100 = 'https://web.archive.org/web/20150714025308/https://core.telegram.org/bots/api/';
public const V110 = 'https://web.archive.org/web/20150812125616/https://core.telegram.org/bots/api';
public const V140 = 'https://web.archive.org/web/20150909214252/https://core.telegram.org/bots/api';
public const V150 = 'https://web.archive.org/web/20150921091215/https://core.telegram.org/bots/api/';
public const V160 = 'https://web.archive.org/web/20151023071257/https://core.telegram.org/bots/api';
public const V180 = 'https://web.archive.org/web/20160112101045/https://core.telegram.org/bots/api';
public const V182 = 'https://web.archive.org/web/20160126005312/https://core.telegram.org/bots/api';
public const V183 = 'https://web.archive.org/web/20160305132243/https://core.telegram.org/bots/api';
public const V200 = 'https://web.archive.org/web/20160413101342/https://core.telegram.org/bots/api';
public const V210 = 'https://web.archive.org/web/20160912130321/https://core.telegram.org/bots/api';
public const V211 = self::V210;
public const V220 = 'https://web.archive.org/web/20161004150232/https://core.telegram.org/bots/api';
public const V230 = 'https://web.archive.org/web/20161124162115/https://core.telegram.org/bots/api';
public const V231 = 'https://web.archive.org/web/20161204181811/https://core.telegram.org/bots/api';
public const V300 = 'https://web.archive.org/web/20170612094628/https://core.telegram.org/bots/api';
public const V310 = 'https://web.archive.org/web/20170703123052/https://core.telegram.org/bots/api';
public const V320 = 'https://web.archive.org/web/20170819054238/https://core.telegram.org/bots/api';
public const V330 = 'https://web.archive.org/web/20170914060628/https://core.telegram.org/bots/api';
public const V350 = 'https://web.archive.org/web/20171201065426/https://core.telegram.org/bots/api';
public const V360 = 'https://web.archive.org/web/20180217001114/https://core.telegram.org/bots/api';
public const V400 = 'https://web.archive.org/web/20180728174553/https://core.telegram.org/bots/api';
public const V410 = 'https://web.archive.org/web/20180828155646/https://core.telegram.org/bots/api';
public const V420 = 'https://web.archive.org/web/20190417160652/https://core.telegram.org/bots/api';
public const V430 = 'https://web.archive.org/web/20190601122107/https://core.telegram.org/bots/api';
public const V440 = 'https://web.archive.org/web/20190731114703/https://core.telegram.org/bots/api';
public const V450 = 'https://web.archive.org/web/20200107090812/https://core.telegram.org/bots/api';
public const V460 = 'https://web.archive.org/web/20200208225346/https://core.telegram.org/bots/api';
public const V470 = 'https://web.archive.org/web/20200401052001/https://core.telegram.org/bots/api';
public const V480 = 'https://web.archive.org/web/20200429054924/https://core.telegram.org/bots/api';
public const V490 = 'https://web.archive.org/web/20200611131321/https://core.telegram.org/bots/api';
public const V500 = 'https://web.archive.org/web/20201104151640/https://core.telegram.org/bots/api';
public const V510 = 'https://web.archive.org/web/20210315055600/https://core.telegram.org/bots/api';
public const V520 = 'https://web.archive.org/web/20210428195636/https://core.telegram.org/bots/api';
public const V530 = 'https://web.archive.org/web/20210626142851/https://core.telegram.org/bots/api';
public const LATEST = 'https://core.telegram.org/bots/api';
public const V100 = '1.0.0';
public const V110 = '1.1.0';
public const V140 = '1.4.0';
public const V150 = '1.5.0';
public const V160 = '1.6.0';
public const V180 = '1.8.0';
public const V182 = '1.8.2';
public const V183 = '1.8.3';
public const V200 = '2.0.0';
public const V210 = '2.1.0';
public const V211 = '2.1.1';
public const V220 = '2.2.0';
public const V230 = '2.3.0';
public const V231 = '2.3.1';
public const V300 = '3.0.0';
public const V310 = '3.1.0';
public const V320 = '3.2.0';
public const V330 = '3.3.0';
public const V350 = '3.5.0';
public const V360 = '3.6.0';
public const V400 = '4.0.0';
public const V410 = '4.1.0';
public const V420 = '4.2.0';
public const V430 = '4.3.0';
public const V440 = '4.4.0';
public const V450 = '4.5.0';
public const V460 = '4.6.0';
public const V470 = '4.7.0';
public const V480 = '4.8.0';
public const V490 = '4.9.0';
public const V500 = '5.0.0';
public const V510 = '5.1.0';
public const V520 = '5.2.0';
public const V530 = '5.3.0';
public const LATEST = 'latest';
public const STABLE = self::V530;
public const URLS = [
self::V100 => 'https://web.archive.org/web/20150714025308id_/https://core.telegram.org/bots/api/',
self::V110 => 'https://web.archive.org/web/20150812125616id_/https://core.telegram.org/bots/api',
self::V140 => 'https://web.archive.org/web/20150909214252id_/https://core.telegram.org/bots/api',
self::V150 => 'https://web.archive.org/web/20150921091215id_/https://core.telegram.org/bots/api/',
self::V160 => 'https://web.archive.org/web/20151023071257id_/https://core.telegram.org/bots/api',
self::V180 => 'https://web.archive.org/web/20160112101045id_/https://core.telegram.org/bots/api',
self::V182 => 'https://web.archive.org/web/20160126005312id_/https://core.telegram.org/bots/api',
self::V183 => 'https://web.archive.org/web/20160305132243id_/https://core.telegram.org/bots/api',
self::V200 => 'https://web.archive.org/web/20160413101342id_/https://core.telegram.org/bots/api',
self::V210 => 'https://web.archive.org/web/20160912130321id_/https://core.telegram.org/bots/api',
self::V211 => 'https://web.archive.org/web/20160912130321id_/https://core.telegram.org/bots/api',
self::V220 => 'https://web.archive.org/web/20161004150232id_/https://core.telegram.org/bots/api',
self::V230 => 'https://web.archive.org/web/20161124162115id_/https://core.telegram.org/bots/api',
self::V231 => 'https://web.archive.org/web/20161204181811id_/https://core.telegram.org/bots/api',
self::V300 => 'https://web.archive.org/web/20170612094628id_/https://core.telegram.org/bots/api',
self::V310 => 'https://web.archive.org/web/20170703123052id_/https://core.telegram.org/bots/api',
self::V320 => 'https://web.archive.org/web/20170819054238id_/https://core.telegram.org/bots/api',
self::V330 => 'https://web.archive.org/web/20170914060628id_/https://core.telegram.org/bots/api',
self::V350 => 'https://web.archive.org/web/20171201065426id_/https://core.telegram.org/bots/api',
self::V360 => 'https://web.archive.org/web/20180217001114id_/https://core.telegram.org/bots/api',
self::V400 => 'https://web.archive.org/web/20180728174553id_/https://core.telegram.org/bots/api',
self::V410 => 'https://web.archive.org/web/20180828155646id_/https://core.telegram.org/bots/api',
self::V420 => 'https://web.archive.org/web/20190417160652id_/https://core.telegram.org/bots/api',
self::V430 => 'https://web.archive.org/web/20190601122107id_/https://core.telegram.org/bots/api',
self::V440 => 'https://web.archive.org/web/20190731114703id_/https://core.telegram.org/bots/api',
self::V450 => 'https://web.archive.org/web/20200107090812id_/https://core.telegram.org/bots/api',
self::V460 => 'https://web.archive.org/web/20200208225346id_/https://core.telegram.org/bots/api',
self::V470 => 'https://web.archive.org/web/20200401052001id_/https://core.telegram.org/bots/api',
self::V480 => 'https://web.archive.org/web/20200429054924id_/https://core.telegram.org/bots/api',
self::V490 => 'https://web.archive.org/web/20200611131321id_/https://core.telegram.org/bots/api',
self::V500 => 'https://web.archive.org/web/20201104151640id_/https://core.telegram.org/bots/api',
self::V510 => 'https://web.archive.org/web/20210315055600id_/https://core.telegram.org/bots/api',
self::V520 => 'https://web.archive.org/web/20210428195636id_/https://core.telegram.org/bots/api',
self::V530 => 'https://web.archive.org/web/20210626142851id_/https://core.telegram.org/bots/api',
self::LATEST => 'https://core.telegram.org/bots/api'
];
public static function getVersionFromText(string $text): string
{
$text = str_replace('.', '', $text);
$text = str_replace(['.', 'v'], ['', ''], strtolower($text));
$const = sprintf('%s::V%s', self::class, $text);
if (defined($const)) {
return constant($const);
@ -54,4 +92,10 @@ class Versions
return self::LATEST;
}
public static function getUrlFromText(string $text): string
{
$version = self::getVersionFromText($text);
return self::URLS[$version] ?? self::URLS[self::LATEST];
}
}

View File

@ -2,10 +2,10 @@
namespace TgScraper;
use BadMethodCallException;
use Exception;
use InvalidArgumentException;
use JetBrains\PhpStorm\ArrayShape;
use JsonException;
use PHPHtmlParser\Exceptions\ChildNotFoundException;
use PHPHtmlParser\Exceptions\CircularException;
use PHPHtmlParser\Exceptions\ContentLengthException;
@ -16,16 +16,17 @@ use PHPHtmlParser\Exceptions\StrictException;
use Psr\Http\Client\ClientExceptionInterface;
use Psr\Log\LoggerInterface;
use Symfony\Component\Yaml\Yaml;
use TgScraper\Common\OpenApiGenerator;
use TgScraper\Common\SchemaExtractor;
use TgScraper\Common\StubCreator;
use TgScraper\Constants\Versions;
use Throwable;
/**
* Class Generator
* Class TgScraper
* @package TgScraper
*/
class Generator
class TgScraper
{
/**
@ -33,58 +34,92 @@ class Generator
*/
public const TEMPLATES_DIRECTORY = __DIR__ . '/../templates';
/**
* @var string
*/
private string $version;
/**
* @var array
*/
private array $schema;
private array $types;
/**
* @var array
*/
private array $methods;
/**
* Generator constructor.
* TgScraper constructor.
* @param LoggerInterface $logger
* @param string $url
* @param array|null $schema
* @throws ChildNotFoundException
* @throws CircularException
* @throws ClientExceptionInterface
* @throws ContentLengthException
* @throws LogicalException
* @throws NotLoadedException
* @throws ParentNotFoundException
* @throws StrictException
* @throws Throwable
* @param array $schema
*/
public function __construct(
private LoggerInterface $logger,
private string $url = Versions::LATEST,
?array $schema = null
) {
if (empty($schema)) {
try {
$extractor = new SchemaExtractor($this->logger, $this->url);
$this->logger->info('Schema not provided, extracting from URL.');
$schema = $extractor->extract();
} catch (Throwable $e) {
$this->logger->critical(
'An exception occurred while trying to extract the schema: ' . $e->getMessage()
);
throw $e;
}
public function __construct(private LoggerInterface $logger, array $schema)
{
if (!self::validateSchema($schema)) {
throw new InvalidArgumentException('Invalid schema provided');
}
/** @var array $schema */
$this->schema = $schema;
$this->version = $schema['version'] ?? '1.0.0';
$this->types = $schema['types'];
$this->methods = $schema['methods'];
}
/**
* @param LoggerInterface $logger
* @param string $url
* @return static
* @throws ChildNotFoundException
* @throws CircularException
* @throws ParentNotFoundException
* @throws StrictException
* @throws ClientExceptionInterface
* @throws NotLoadedException
* @throws ContentLengthException
* @throws LogicalException
* @throws NotLoadedException
* @throws ParentNotFoundException
* @throws StrictException
* @throws Throwable
*/
public static function fromUrl(LoggerInterface $logger, string $url): self
{
$extractor = SchemaExtractor::fromUrl($logger, $url);
$schema = $extractor->extract();
return new self($logger, $schema);
}
/**
* @param LoggerInterface $logger
* @param string $version
* @return static
* @throws ChildNotFoundException
* @throws CircularException
* @throws ClientExceptionInterface
* @throws ContentLengthException
* @throws LogicalException
* @throws NotLoadedException
* @throws ParentNotFoundException
* @throws StrictException
* @throws Throwable
*/
public static function fromVersion(LoggerInterface $logger, string $version = Versions::LATEST): self
{
$extractor = SchemaExtractor::fromVersion($logger, $version);
$schema = $extractor->extract();
return new self($logger, $schema);
}
/**
* @param array $schema
* @return bool
*/
public static function validateSchema(array $schema): bool
{
return array_key_exists('version', $schema) and is_string($schema['version']) and
array_key_exists('types', $schema) and is_array($schema['types']) and
array_key_exists('methods', $schema) and is_array($schema['methods']);
}
/**
* @param LoggerInterface $logger
* @param string $yaml
* @return TgScraper
*/
public static function fromYaml(LoggerInterface $logger, string $yaml): self
{
$data = Yaml::parse($yaml);
@ -92,15 +127,10 @@ class Generator
}
/**
* @throws ChildNotFoundException
* @throws ParentNotFoundException
* @throws CircularException
* @throws StrictException
* @throws ClientExceptionInterface
* @throws NotLoadedException
* @throws ContentLengthException
* @throws LogicalException
* @throws Throwable
* @param LoggerInterface $logger
* @param string $json
* @return TgScraper
* @throws JsonException
*/
public static function fromJson(LoggerInterface $logger, string $json): self
{
@ -124,8 +154,12 @@ class Generator
);
throw $e;
}
$typesDir = $directory . '/Types';
if (!file_exists($typesDir)) {
mkdir($typesDir, 0755);
}
try {
$creator = new StubCreator($this->schema, $namespace);
$creator = new StubCreator($this->toArray(), $namespace);
} catch (InvalidArgumentException $e) {
$this->logger->critical(
'An exception occurred while trying to parse the schema: ' . $e->getMessage()
@ -146,12 +180,12 @@ class Generator
* @return string
* @throws Exception
*/
private static function getTargetDirectory(string $path): string
public static function getTargetDirectory(string $path): string
{
$result = realpath($path);
if (false === $result) {
if (!mkdir($path)) {
$path = __DIR__ . '/../generated';
$path = getcwd() . '/gen';
if (!file_exists($path)) {
mkdir($path, 0755);
}
@ -161,61 +195,60 @@ class Generator
if (false === $result) {
throw new Exception('Could not create target directory');
}
@mkdir($result . '/Types', 0755);
return $result;
}
/**
* @param int $options
* @return string
* @return array
*/
public function toJson(int $options = 0): string
#[ArrayShape([
'version' => "string",
'types' => "array",
'methods' => "array"
])] public function toArray(): array
{
return json_encode($this->schema, $options);
return [
'version' => $this->version,
'types' => $this->types,
'methods' => $this->methods
];
}
/**
* @param int $inline
* @param int $indent
* @param int $flags
* @return string
* @return array
*/
public function toYaml(int $inline = 6, int $indent = 4, int $flags = 0): string
public function toOpenApi(): array
{
return Yaml::dump($this->schema, $inline, $indent, $flags);
}
/**
* @return string
*/
public function toOpenApi(): string
{
throw new BadMethodCallException('Not implemented');
$openapiTemplate = file_get_contents(self::TEMPLATES_DIRECTORY . '/openapi.json');
$openapiData = json_decode($openapiTemplate, true);
$responsesTemplate = file_get_contents(self::TEMPLATES_DIRECTORY . '/responses.json');
$responses = json_decode($responsesTemplate, true);
$openapi = new OpenApiGenerator($responses, $openapiData, $this->types, $this->methods);
$openapi->setVersion($this->version);
return $openapi->getData();
}
/**
* Thanks to davtur19 (https://github.com/davtur19/TuriBotGen/blob/master/postman.php)
* @param int $options
* @return string
* @return array
*/
#[ArrayShape(['info' => "string[]", 'variable' => "string[]", 'item' => "array[]"])]
public function toPostman(
int $options = 0
): string {
public function toPostman(): array
{
$template = file_get_contents(self::TEMPLATES_DIRECTORY . '/postman.json');
$result = json_decode($template, true);
$result['info']['version'] = $this->schema['version'];
foreach ($this->schema['methods'] as $method) {
$result['info']['version'] = $this->version;
foreach ($this->methods as $method) {
$formData = [];
if (!empty($method['fields'])) {
foreach ($method['fields'] as $field) {
$formData[] = [
'key' => $field['name'],
'disabled' => !$field['required'],
'disabled' => $field['optional'],
'description' => sprintf(
'%s. %s',
$field['required'] ? 'Required' : 'Optional',
$field['optional'] ? 'Optional' : 'Required',
$field['description']
),
'type' => 'text'
@ -247,7 +280,7 @@ class Generator
]
];
}
return json_encode($result, $options);
return $result;
}
}

View File

@ -17,7 +17,7 @@
}
],
"externalDocs": {
"description": "Official Telegram Bot API documentation",
"description": "Official Telegram Bot API documentation.",
"url": "https://core.telegram.org/bots/api"
},
"components": {
@ -81,11 +81,32 @@
}
}
}
},
"ServerError": {
"description": "The bot API is experiencing some issues, try again later.",
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/Error"
}
}
}
},
"UnknownError": {
"description": "An unknown error occurred.",
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/Error"
}
}
}
}
},
"schemas": {
"Response": {
"type": "object",
"description": "Represents the default response object.",
"required": [
"ok"
],
@ -96,6 +117,7 @@
}
},
"Success": {
"description": "Request was successful, the result is returned.",
"allOf": [
{
"$ref": "#/components/schemas/Response"
@ -114,6 +136,7 @@
]
},
"Error": {
"description": "Request was unsuccessful, so an error occurred.",
"allOf": [
{
"$ref": "#/components/schemas/Response"

46
templates/responses.json Normal file
View File

@ -0,0 +1,46 @@
{
"200": {
"description": "Request was successful, the result is returned.",
"content": {
"application/json": {
"schema": {
"allOf": [
{
"$ref": "#/components/schemas/Success"
},
{
"type": "object",
"properties": {
"result": {}
}
}
]
}
}
}
},
"400": {
"$ref": "#/components/responses/BadRequest"
},
"401": {
"$ref": "#/components/responses/Unauthorized"
},
"403": {
"$ref": "#/components/responses/Forbidden"
},
"404": {
"$ref": "#/components/responses/NotFound"
},
"409": {
"$ref": "#/components/responses/Conflict"
},
"429": {
"$ref": "#/components/responses/TooManyRequests"
},
"5XX": {
"$ref": "#/components/responses/ServerError"
},
"default": {
"$ref": "#/components/responses/UnknownError"
}
}