OpenAPI, changelog, cache package support

.github/workflows/build.yml

@@ -1,9 +1,7 @@
 name: Build
 on:
-  push:
+  release:
-    branches:
+    types: [ published ]
-      - master
-  pull_request:
 
 jobs:
   build:

.github/workflows/pages.yml

@@ -0,0 +1,34 @@
+name: GH Pages
+on:
+  schedule:
+    - cron: '0 0 * * *'
+  workflow_dispatch:
+
+jobs:
+  build:
+    name: Build files for GH Pages
+    runs-on: ubuntu-latest
+    steps:
+      - name: Checkout
+        uses: actions/checkout@v2
+      - name: Generate schemas
+        uses: addnab/docker-run-action@v3
+        with:
+          registry: ghcr.io
+          image: sysbot-org/tgscraper:latest
+          options: -v ${{ github.workspace }}:/out
+          run: |
+          'app:export-schema --readable botapi.json'
+          'app:export-schema --yaml --readable botapi.yaml'
+          'app:export-schema --postman --readable botapi_postman.json'
+          'app:export-schema --openapi --readable botapi_openapi.json'
+          'app:export-schema --yaml --openapi --readable botapi_openapi.yaml'
+      - name: Deploy
+        uses: peaceiris/actions-gh-pages@v3
+        with:
+          github_token: ${{ secrets.GITHUB_TOKEN }}
+          publish_dir: ${{ github.workspace }}
+          destination_dir: schemas
+          publish_branch: gh-pages
+          cname: tgscraper.sys001.ml
+          enable_jekyll: true

CHANGELOG.md

@@ -0,0 +1,139 @@
+# Changelog
+
+All notable changes to this project will be documented in this file.
+
+The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
+and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
+
+
+## [Unreleased]
+### Added
+- Support for OpenAPI schema: you can now generate code in any language!
+- Support for the new [sysbot/tgscraper-cache](https://github.com/Sysbot-org/tgscraper-cache) package: if installed, TGScraper will be much faster (there is no need to always fetch the live webpages)!
+- You can now validate a schema by using the `validateSchema` method, provided by the `TgScraper` class.
+- New `Versions::STABLE` constant: it will automatically return the latest stable version instead of the live version (useful for the cache package).
+- Added the JSON schema specification for the custom format provided by TGScraper.
+- Added this changelog.
+
+### Changed
+- The `Generator` class has now been renamed `TgScraper`.
+- The `required` property in method fields has been replaced by the new `optional` property, for the sake of consistency.
+- You now need a schema in order to instantiate the `TgScraper` class (don't worry, you can use the new methods `TgScraper::fromUrl` and `TgScraper::fromVersion`).
+- The `Versions` class constants have been replaced with an actual version string. If you still need the URLs, use the new class constant `Versions::URLS`.
+- TGScraper will now only return arrays. If you still need JSON or YAML encoding, please use the new `Encoder` class.  
+
+### Fixed
+- Minor improvements to `StubCreator`.
+- Fixed an issue with the CLI where `autoload.php` couldn't be found.
+- When exporting the schema, the CLI will now make sure that the destination directory exists.
+
+### Security
+- When a custom schema is used, only use `version`, `types` and `methods` fields.
+
+## [2.1.0] - 2021-07-31
+### Added
+- New repo workflows: automatic package build (and push to the GitHub registry), and automatic notifications via Telegram.
+- New `version` field for schemas: it contains the bot API version (if possible).
+- New `extended_by` field for types: if the current type is a parent one, it will contain its child types.
+
+### Changed
+- Now all type stubs implement the base `TypeInterface` interface.
+- Children type stubs now extend their parent.
+- Optional fields are now actually optional in the Postman collection (previously, you had to manually disable optional ones).
+
+### Fixed
+- Minor improvements to the schema extractor.
+
+## [2.0.1] - 2021-07-24
+### Changed
+- The README now includes many CLI examples.
+
+### Fixed
+- The link for the Bot API 5.2.0 snapshot was broken, it's working now.
+
+## [2.0.0] - 2021-07-24
+### Added
+- Support for Postman collection: now you can generate a JSON to use in Postman!
+- New class for the URLs of various bot API snapshots: `TgScraper\Constants\Versions`. 
+
+### Changed
+- Moved `TgScraper\StubCreator` to the new namespace `TgScraper\Common\StubCreator`.
+- Moved scraping logic from `TgScraper\Generator` to the new class `TgScraper\Common\SchemaExtractor`.
+- CLI has been completely reworked: it now uses the Symfony Console and it's much more reliable!
+
+## [1.4.0] - 2021-06-23
+### Added
+- YAML format is now supported.
+- Docker support! The package is published on the GitHub registry.
+
+## [1.3.0] - 2021-06-22
+### Added
+- Badges in the README! They contain a lot of useful information about the project (such as the minimum PHP version, latest stable version, etc).
+
+### Fixed
+- CLI now catches exceptions more reliably.
+
+### Security
+- Dependency `paquettg/php-html-parser` has been upgraded to `^3.1`.
+
+## [1.2.2] - 2021-06-20
+### Fixed
+- Fixed a typo in a property name of the `Response` class stub.
+
+## [1.2.1] - 2021-06-19
+### Removed
+- The abstract constructor for the `API` trait stub has now been removed.
+
+## [1.2.0] - 2021-06-19
+### Changed
+- The `API` class stub has been converted to a trait, and the constructor and the `sendRequest` methods are now abstract.
+
+### Fixed
+- Minor improvements to the CLI.
+
+## [1.1.0] - 2021-06-18
+### Added
+- New field for types: `optional`. It tells whether a value is always present or not.
+- Class stubs now have typed properties (with related PHPDoc comments).
+
+### Changed
+- Variable names have been changed to `camelCase`.
+
+## [1.0.2] - 2021-06-18
+### Fixed
+- Improved argument parsing for the CLI.
+
+## [1.0.1] - 2021-06-17
+### Changed
+- Project license is now the GNU Lesser GPL.
+
+## [1.0.0] - 2021-06-17
+### Added
+- New CLI to easily generate JSON schema or class stubs!
+- It's now possible to parse old bot API webpages! Pass the URL and it should work just fine.
+- New API class stub, it implements all bot API methods (it's incomplete though, so you must add your custom logic).
+
+### Changed
+- Renamed project to `tgscraper`.
+- Class namespace is now `TgScraper`, for the sake of consistency.
+- `StubProvider` class is now named `StubCreator`.
+- Reworked syntax for array of objects in field types: `Object[]` has been replaced by `Array<Object>`. 
+
+### Removed
+- Method stubs will no longer be generated.
+
+### Fixed
+- The parser is now more reliable, it no longer needs to be updated at every bot API release!
+
+[Unreleased]: https://github.com/Sysbot-org/tgscraper/compare/2.1...HEAD
+[2.1.0]: https://github.com/Sysbot-org/tgscraper/compare/2.0.1...2.1
+[2.0.1]: https://github.com/Sysbot-org/tgscraper/compare/2.0...2.0.1
+[2.0.0]: https://github.com/Sysbot-org/tgscraper/compare/1.4...2.0
+[1.4.0]: https://github.com/Sysbot-org/tgscraper/compare/1.3...1.4
+[1.3.0]: https://github.com/Sysbot-org/tgscraper/compare/1.2.1...1.3
+[1.2.1]: https://github.com/Sysbot-org/tgscraper/compare/1.2...1.2.1
+[1.2.0]: https://github.com/Sysbot-org/tgscraper/compare/1.1...1.2
+[1.1.0]: https://github.com/Sysbot-org/tgscraper/compare/1.0.2...1.1
+[1.0.2]: https://github.com/Sysbot-org/tgscraper/compare/1.0.1...1.0.2
+[1.0.1]: https://github.com/Sysbot-org/tgscraper/compare/1.0...1.0.1
+[1.0.0]: https://github.com/Sysbot-org/tgscraper/releases/tag/1.0

Dockerfile

@@ -3,9 +3,8 @@ FROM composer:latest AS tgscraper
 MAINTAINER Sys <sys@sys001.ml>
 
 WORKDIR /app
-COPY . .
+RUN composer require sysbot/tgscraper sysbot/tgscraper-cache --no-progress --no-interaction --no-ansi --prefer-stable --optimize-autoloader
-RUN composer install
+WORKDIR /out
-WORKDIR /artifacts
+VOLUME /out
-VOLUME /artifacts
 
-ENTRYPOINT ["php", "/app/bin/tgscraper"]
+ENTRYPOINT ["php", "/app/vendor/bin/tgscraper"]

README.md

@@ -9,8 +9,10 @@
 A PHP library used to extract JSON data (and auto-generate PHP classes)
 from [Telegram bot API documentation page](https://core.telegram.org/bots/api).
 
-**Note: the scraper is, obviously, based on a hack and you shouldn't rely on automagically generated files from it,
+## Changelog
-since they are prone to errors. I'll try to fix them ASAP, but manual review is always required (at least for now).**
+
+Interested in recent changes? Have a look [here](CHANGELOG.md)!
+
 
 ## Installation
 
@@ -52,12 +54,23 @@ Extract the latest schema in YAML format:
   $ vendor/bin/tgscraper app:export-schema --yaml botapi.yaml
 ```
 
+### OpenAPI
+
+Work in progress.
+
 ### Stubs
 
+_Note: since Telegram may change the page format at any time, do **NOT** rely on the automagically generated
+stubs from this library, **ALWAYS** review the code!_
+
 TGScraper can also generate class stubs that you can use in your library. A sample implementation is available in the [Sysbot Telegram module](https://github.com/Sysbot-org/Sysbot-tg).
 
-Create stubs in the `out/` directory using `Sysbot\Api` as namespace prefix:
+Create stubs in the `out/` directory using `Sysbot\Telegram` as namespace prefix:
 
 ```bash 
-  $ vendor/bin/tgscraper app:create-stubs --namespace-prefix "Sysbot\Api" out
+  $ vendor/bin/tgscraper app:create-stubs --namespace-prefix "Sysbot\Telegram" out
-```
+```
+
+## Custom format
+
+If you're interested in the custom format generated by TGScraper, you can find its schema [here](docs/schema.json).

bin/tgscraper

@@ -6,7 +6,17 @@ use Symfony\Component\Console\Application;
 use TgScraper\Commands\CreateStubsCommand;
 use TgScraper\Commands\ExportSchemaCommand;
 
-require_once __DIR__ . '/../vendor/autoload.php';
+$autoloadFiles = [
+    __DIR__ . '/../vendor/autoload.php',
+    __DIR__ . '/../../../autoload.php'
+];
+
+foreach ($autoloadFiles as $autoloadFile) {
+    if (file_exists($autoloadFile)) {
+        require_once $autoloadFile;
+        break;
+    }
+}
 
 $application = new Application('TGScraper', InstalledVersions::getVersion('sysbot/tgscraper'));
 

composer.json

@@ -12,6 +12,9 @@
     "symfony/console": "^5.3",
     "symfony/yaml": "^5.3"
   },
+  "suggest": {
+    "sysbot/tgscraper-cache": "To speed up schema fetching and generation."
+  },
   "autoload": {
     "psr-4": {
       "TgScraper\\": "src/"

composer.lock

@@ -369,16 +369,16 @@
         },
         {
             "name": "nette/utils",
-            "version": "v3.2.2",
+            "version": "v3.2.3",
             "source": {
                 "type": "git",
                 "url": "https://github.com/nette/utils.git",
-                "reference": "967cfc4f9a1acd5f1058d76715a424c53343c20c"
+                "reference": "5c36cc1ba9bb6abb8a9e425cf054e0c3fd5b9822"
             },
             "dist": {
                 "type": "zip",
-                "url": "https://api.github.com/repos/nette/utils/zipball/967cfc4f9a1acd5f1058d76715a424c53343c20c",
+                "url": "https://api.github.com/repos/nette/utils/zipball/5c36cc1ba9bb6abb8a9e425cf054e0c3fd5b9822",
-                "reference": "967cfc4f9a1acd5f1058d76715a424c53343c20c",
+                "reference": "5c36cc1ba9bb6abb8a9e425cf054e0c3fd5b9822",
                 "shasum": ""
             },
             "require": {
@@ -448,9 +448,9 @@
             ],
             "support": {
                 "issues": "https://github.com/nette/utils/issues",
-                "source": "https://github.com/nette/utils/tree/v3.2.2"
+                "source": "https://github.com/nette/utils/tree/v3.2.3"
             },
-            "time": "2021-03-03T22:53:25+00:00"
+            "time": "2021-08-16T21:05:00+00:00"
         },
         {
             "name": "paquettg/php-html-parser",
@@ -938,16 +938,16 @@
         },
         {
             "name": "symfony/console",
-            "version": "v5.3.2",
+            "version": "v5.3.6",
             "source": {
                 "type": "git",
                 "url": "https://github.com/symfony/console.git",
-                "reference": "649730483885ff2ca99ca0560ef0e5f6b03f2ac1"
+                "reference": "51b71afd6d2dc8f5063199357b9880cea8d8bfe2"
             },
             "dist": {
                 "type": "zip",
-                "url": "https://api.github.com/repos/symfony/console/zipball/649730483885ff2ca99ca0560ef0e5f6b03f2ac1",
+                "url": "https://api.github.com/repos/symfony/console/zipball/51b71afd6d2dc8f5063199357b9880cea8d8bfe2",
-                "reference": "649730483885ff2ca99ca0560ef0e5f6b03f2ac1",
+                "reference": "51b71afd6d2dc8f5063199357b9880cea8d8bfe2",
                 "shasum": ""
             },
             "require": {
@@ -955,11 +955,12 @@
                 "symfony/deprecation-contracts": "^2.1",
                 "symfony/polyfill-mbstring": "~1.0",
                 "symfony/polyfill-php73": "^1.8",
-                "symfony/polyfill-php80": "^1.15",
+                "symfony/polyfill-php80": "^1.16",
                 "symfony/service-contracts": "^1.1|^2",
                 "symfony/string": "^5.1"
             },
             "conflict": {
+                "psr/log": ">=3",
                 "symfony/dependency-injection": "<4.4",
                 "symfony/dotenv": "<5.1",
                 "symfony/event-dispatcher": "<4.4",
@@ -967,10 +968,10 @@
                 "symfony/process": "<4.4"
             },
             "provide": {
-                "psr/log-implementation": "1.0"
+                "psr/log-implementation": "1.0|2.0"
             },
             "require-dev": {
-                "psr/log": "~1.0",
+                "psr/log": "^1|^2",
                 "symfony/config": "^4.4|^5.0",
                 "symfony/dependency-injection": "^4.4|^5.0",
                 "symfony/event-dispatcher": "^4.4|^5.0",
@@ -1016,7 +1017,7 @@
                 "terminal"
             ],
             "support": {
-                "source": "https://github.com/symfony/console/tree/v5.3.2"
+                "source": "https://github.com/symfony/console/tree/v5.3.6"
             },
             "funding": [
                 {
@@ -1032,7 +1033,7 @@
                     "type": "tidelift"
                 }
             ],
-            "time": "2021-06-12T09:42:48+00:00"
+            "time": "2021-07-27T19:10:22+00:00"
         },
         {
             "name": "symfony/deprecation-contracts",
@@ -1182,16 +1183,16 @@
         },
         {
             "name": "symfony/polyfill-intl-grapheme",
-            "version": "v1.23.0",
+            "version": "v1.23.1",
             "source": {
                 "type": "git",
                 "url": "https://github.com/symfony/polyfill-intl-grapheme.git",
-                "reference": "24b72c6baa32c746a4d0840147c9715e42bb68ab"
+                "reference": "16880ba9c5ebe3642d1995ab866db29270b36535"
             },
             "dist": {
                 "type": "zip",
-                "url": "https://api.github.com/repos/symfony/polyfill-intl-grapheme/zipball/24b72c6baa32c746a4d0840147c9715e42bb68ab",
+                "url": "https://api.github.com/repos/symfony/polyfill-intl-grapheme/zipball/16880ba9c5ebe3642d1995ab866db29270b36535",
-                "reference": "24b72c6baa32c746a4d0840147c9715e42bb68ab",
+                "reference": "16880ba9c5ebe3642d1995ab866db29270b36535",
                 "shasum": ""
             },
             "require": {
@@ -1243,7 +1244,7 @@
                 "shim"
             ],
             "support": {
-                "source": "https://github.com/symfony/polyfill-intl-grapheme/tree/v1.23.0"
+                "source": "https://github.com/symfony/polyfill-intl-grapheme/tree/v1.23.1"
             },
             "funding": [
                 {
@@ -1259,7 +1260,7 @@
                     "type": "tidelift"
                 }
             ],
-            "time": "2021-05-27T09:17:38+00:00"
+            "time": "2021-05-27T12:26:48+00:00"
         },
         {
             "name": "symfony/polyfill-intl-normalizer",
@@ -1347,16 +1348,16 @@
         },
         {
             "name": "symfony/polyfill-mbstring",
-            "version": "v1.23.0",
+            "version": "v1.23.1",
             "source": {
                 "type": "git",
                 "url": "https://github.com/symfony/polyfill-mbstring.git",
-                "reference": "2df51500adbaebdc4c38dea4c89a2e131c45c8a1"
+                "reference": "9174a3d80210dca8daa7f31fec659150bbeabfc6"
             },
             "dist": {
                 "type": "zip",
-                "url": "https://api.github.com/repos/symfony/polyfill-mbstring/zipball/2df51500adbaebdc4c38dea4c89a2e131c45c8a1",
+                "url": "https://api.github.com/repos/symfony/polyfill-mbstring/zipball/9174a3d80210dca8daa7f31fec659150bbeabfc6",
-                "reference": "2df51500adbaebdc4c38dea4c89a2e131c45c8a1",
+                "reference": "9174a3d80210dca8daa7f31fec659150bbeabfc6",
                 "shasum": ""
             },
             "require": {
@@ -1407,7 +1408,7 @@
                 "shim"
             ],
             "support": {
-                "source": "https://github.com/symfony/polyfill-mbstring/tree/v1.23.0"
+                "source": "https://github.com/symfony/polyfill-mbstring/tree/v1.23.1"
             },
             "funding": [
                 {
@@ -1423,7 +1424,7 @@
                     "type": "tidelift"
                 }
             ],
-            "time": "2021-05-27T09:27:20+00:00"
+            "time": "2021-05-27T12:26:48+00:00"
         },
         {
             "name": "symfony/polyfill-php73",
@@ -1506,16 +1507,16 @@
         },
         {
             "name": "symfony/polyfill-php80",
-            "version": "v1.23.0",
+            "version": "v1.23.1",
             "source": {
                 "type": "git",
                 "url": "https://github.com/symfony/polyfill-php80.git",
-                "reference": "eca0bf41ed421bed1b57c4958bab16aa86b757d0"
+                "reference": "1100343ed1a92e3a38f9ae122fc0eb21602547be"
             },
             "dist": {
                 "type": "zip",
-                "url": "https://api.github.com/repos/symfony/polyfill-php80/zipball/eca0bf41ed421bed1b57c4958bab16aa86b757d0",
+                "url": "https://api.github.com/repos/symfony/polyfill-php80/zipball/1100343ed1a92e3a38f9ae122fc0eb21602547be",
-                "reference": "eca0bf41ed421bed1b57c4958bab16aa86b757d0",
+                "reference": "1100343ed1a92e3a38f9ae122fc0eb21602547be",
                 "shasum": ""
             },
             "require": {
@@ -1569,7 +1570,7 @@
                 "shim"
             ],
             "support": {
-                "source": "https://github.com/symfony/polyfill-php80/tree/v1.23.0"
+                "source": "https://github.com/symfony/polyfill-php80/tree/v1.23.1"
             },
             "funding": [
                 {
@@ -1585,7 +1586,7 @@
                     "type": "tidelift"
                 }
             ],
-            "time": "2021-02-19T12:13:01+00:00"
+            "time": "2021-07-28T13:41:28+00:00"
         },
         {
             "name": "symfony/service-contracts",
@@ -1751,16 +1752,16 @@
         },
         {
             "name": "symfony/yaml",
-            "version": "v5.3.3",
+            "version": "v5.3.6",
             "source": {
                 "type": "git",
                 "url": "https://github.com/symfony/yaml.git",
-                "reference": "485c83a2fb5893e2ff21bf4bfc7fdf48b4967229"
+                "reference": "4500fe63dc9c6ffc32d3b1cb0448c329f9c814b7"
             },
             "dist": {
                 "type": "zip",
-                "url": "https://api.github.com/repos/symfony/yaml/zipball/485c83a2fb5893e2ff21bf4bfc7fdf48b4967229",
+                "url": "https://api.github.com/repos/symfony/yaml/zipball/4500fe63dc9c6ffc32d3b1cb0448c329f9c814b7",
-                "reference": "485c83a2fb5893e2ff21bf4bfc7fdf48b4967229",
+                "reference": "4500fe63dc9c6ffc32d3b1cb0448c329f9c814b7",
                 "shasum": ""
             },
             "require": {
@@ -1806,7 +1807,7 @@
             "description": "Loads and dumps YAML files",
             "homepage": "https://symfony.com",
             "support": {
-                "source": "https://github.com/symfony/yaml/tree/v5.3.3"
+                "source": "https://github.com/symfony/yaml/tree/v5.3.6"
             },
             "funding": [
                 {
@@ -1822,7 +1823,7 @@
                     "type": "tidelift"
                 }
             ],
-            "time": "2021-06-24T08:13:00+00:00"
+            "time": "2021-07-29T06:20:01+00:00"
         }
     ],
     "packages-dev": [],

docs/schema.json

@@ -0,0 +1,114 @@
+{
+  "$schema": "http://json-schema.org/draft-07/schema#",
+  "title": "Schema for TGScraper custom Bot API format",
+  "description": "This schema should help you understanding the custom format used by TGScraper.",
+  "type": "object",
+  "required": [
+    "version",
+    "methods",
+    "types"
+  ],
+  "properties": {
+    "version": {
+      "type": "string"
+    },
+    "methods": {
+      "type": "array",
+      "items": {
+        "$ref": "#/definitions/Method"
+      }
+    },
+    "types": {
+      "type": "array",
+      "items": {
+        "$ref": "#/definitions/Type"
+      }
+    }
+  },
+  "definitions": {
+    "Method": {
+      "type": "object",
+      "required": [
+        "name",
+        "description",
+        "fields",
+        "return_types"
+      ],
+      "properties": {
+        "name": {
+          "type": "string"
+        },
+        "description": {
+          "type": "string"
+        },
+        "fields": {
+          "type": "array",
+          "items": {
+            "$ref": "#/definitions/Field"
+          }
+        },
+        "return_types": {
+          "type": "array",
+          "items": {
+            "type": "string"
+          }
+        }
+      }
+    },
+    "Field": {
+      "type": "object",
+      "required": [
+        "name",
+        "types",
+        "optional",
+        "description"
+      ],
+      "properties": {
+        "name": {
+          "type": "string"
+        },
+        "types": {
+          "type": "array",
+          "items": {
+            "type": "string"
+          }
+        },
+        "required": {
+          "type": "boolean"
+        },
+        "description": {
+          "type": "string"
+        }
+      }
+    },
+    "Type": {
+      "type": "object",
+      "required": [
+        "name",
+        "description",
+        "fields",
+        "extended_by"
+      ],
+      "properties": {
+        "name": {
+          "type": "string"
+        },
+        "description": {
+          "type": "string"
+        },
+        "fields": {
+          "type": "array",
+          "items": {
+            "$ref": "#/definitions/Field"
+          }
+        },
+        "extended_by": {
+          "type": "array",
+          "items": {
+            "type": "string"
+          }
+        }
+      }
+    }
+  }
+}

src/Commands/CreateStubsCommand.php

@@ -12,7 +12,7 @@ use Symfony\Component\Console\Input\InputOption;
 use Symfony\Component\Console\Logger\ConsoleLogger;
 use Symfony\Component\Console\Output\OutputInterface;
 use TgScraper\Constants\Versions;
-use TgScraper\Generator;
+use TgScraper\TgScraper;
 use Throwable;
 
 class CreateStubsCommand extends Command
@@ -44,21 +44,30 @@ class CreateStubsCommand extends Command
                 InputOption::VALUE_REQUIRED,
                 'Path to YAML file to use instead of fetching from URL (this option takes precedence over "--layer" and "--json")'
             )
-            ->addOption('layer', 'l', InputOption::VALUE_REQUIRED, 'Bot API version to use', 'latest');
+            ->addOption('layer', 'l', InputOption::VALUE_REQUIRED, 'Bot API version to use', 'latest')
+            ->addOption(
+                'prefer-stable',
+                null,
+                InputOption::VALUE_NONE,
+                'Prefer latest stable version (takes precedence over "--layer")'
+            );
     }
 
     protected function execute(InputInterface $input, OutputInterface $output): int
     {
         $logger = new ConsoleLogger($output);
-        $url = Versions::getVersionFromText($input->getOption('layer'));
+        $version = Versions::getVersionFromText($input->getOption('layer'));
+        if ($input->getOption('prefer-stable')) {
+            $version = Versions::STABLE;
+        }
         $yamlPath = $input->getOption('yaml');
         if (empty($yamlPath)) {
             $jsonPath = $input->getOption('json');
             if (empty($jsonPath)) {
-                $logger->info('Using URL: ' . $url);
+                $logger->info('Using version: ' . $version);
                 try {
-                    $output->writeln('Fetching data from URL...');
+                    $output->writeln('Fetching data for version...');
-                    $generator = new Generator($logger, $url);
+                    $generator = TgScraper::fromVersion($logger, $version);
                 } catch (Throwable) {
                     return Command::FAILURE;
                 }
@@ -70,7 +79,7 @@ class CreateStubsCommand extends Command
                 }
                 $logger->info('Using JSON schema: ' . $jsonPath);
                 /** @noinspection PhpUnhandledExceptionInspection */
-                $generator = Generator::fromJson($logger, $data);
+                $generator = TgScraper::fromJson($logger, $data);
             }
         } else {
             $data = file_get_contents($yamlPath);
@@ -80,7 +89,7 @@ class CreateStubsCommand extends Command
             }
             $logger->info('Using YAML schema: ' . $yamlPath);
             /** @noinspection PhpUnhandledExceptionInspection */
-            $generator = Generator::fromYaml($logger, $data);
+            $generator = TgScraper::fromYaml($logger, $data);
         }
         try {
             $output->writeln('Creating stubs...');

src/Commands/ExportSchemaCommand.php

@@ -4,14 +4,17 @@
 namespace TgScraper\Commands;
 
 
+use Exception;
+use Psr\Log\LoggerInterface;
 use Symfony\Component\Console\Command\Command;
 use Symfony\Component\Console\Input\InputArgument;
 use Symfony\Component\Console\Input\InputInterface;
 use Symfony\Component\Console\Input\InputOption;
 use Symfony\Component\Console\Logger\ConsoleLogger;
 use Symfony\Component\Console\Output\OutputInterface;
+use TgScraper\Common\Encoder;
 use TgScraper\Constants\Versions;
-use TgScraper\Generator;
+use TgScraper\TgScraper;
 use Throwable;
 
 class ExportSchemaCommand extends Command
@@ -29,39 +32,40 @@ class ExportSchemaCommand extends Command
                 'yaml',
                 null,
                 InputOption::VALUE_NONE,
-                'Export schema as YAML instead of JSON (this option takes precedence over "--postman")'
+                'Export schema as YAML instead of JSON (does not affect "--postman")'
+            )
+            ->addOption(
+                'postman',
+                null,
+                InputOption::VALUE_NONE,
+                'Export schema as a Postman-compatible JSON'
+            )
+            ->addOption(
+                'openapi',
+                null,
+                InputOption::VALUE_NONE,
+                'Export schema as a OpenAPI-compatible file (takes precedence over "--postman")'
             )
-            ->addOption('postman', null, InputOption::VALUE_NONE, 'Export schema as a Postman-compatible JSON')
             ->addOption('options', 'o', InputOption::VALUE_REQUIRED, 'Encoder options', 0)
-            ->addOption('readable', 'r', InputOption::VALUE_NONE, '(JSON only) Generate a human-readable JSON')
+            ->addOption(
-            ->addOption('inline', null, InputOption::VALUE_REQUIRED, '(YAML only) Inline level', 6)
+                'readable',
+                'r',
+                InputOption::VALUE_NONE,
+                'Generate a human-readable file (overrides "--inline" and "--indent")'
+            )
+            ->addOption('inline', null, InputOption::VALUE_REQUIRED, '(YAML only) Inline level', 12)
             ->addOption('indent', null, InputOption::VALUE_REQUIRED, '(YAML only) Indent level', 4)
-            ->addOption('layer', 'l', InputOption::VALUE_REQUIRED, 'Bot API version to use', 'latest');
+            ->addOption('layer', 'l', InputOption::VALUE_REQUIRED, 'Bot API version to use', Versions::LATEST)
+            ->addOption(
+                'prefer-stable',
+                null,
+                InputOption::VALUE_NONE,
+                'Prefer latest stable version (takes precedence over "--layer")'
+            );
     }
 
-    protected function execute(InputInterface $input, OutputInterface $output): int
+    private function saveFile(ConsoleLogger $logger, OutputInterface $output, string $destination, string $data): int
     {
-        $logger = new ConsoleLogger($output);
-        $url = Versions::getVersionFromText($input->getOption('layer'));
-        $logger->info('Using URL: ' . $url);
-        try {
-            $output->writeln('Fetching data from URL...');
-            $generator = new Generator($logger, $url);
-        } catch (Throwable) {
-            return Command::FAILURE;
-        }
-        $output->writeln('Exporting schema from data...');
-        $options = $input->getOption('options');
-        $useYaml = $input->getOption('yaml');
-        if ($useYaml) {
-            $data = $generator->toYaml($input->getOption('inline'), $input->getOption('indent'), $options);
-        }
-        $destination = $input->getArgument('destination');
-        if ($input->getOption('readable')) {
-            $options |= JSON_PRETTY_PRINT | JSON_UNESCAPED_SLASHES | JSON_UNESCAPED_UNICODE;
-        }
-        $data ??= $input->getOption('postman') ? $generator->toPostman($options) : $generator->toJson($options);
-        $output->writeln('Saving scheme to file...');
         $result = file_put_contents($destination, $data);
         if (false === $result) {
             $logger->critical('Unable to save file to ' . $destination);
@@ -71,4 +75,49 @@ class ExportSchemaCommand extends Command
         return Command::SUCCESS;
     }
 
+    protected function execute(InputInterface $input, OutputInterface $output): int
+    {
+        $logger = new ConsoleLogger($output);
+        $version = Versions::getVersionFromText($input->getOption('layer'));
+        if ($input->getOption('prefer-stable')) {
+            $version = Versions::STABLE;
+        }
+        $logger->info('Using version: ' . $version);
+        try {
+            $output->writeln('Fetching data for version...');
+            $generator = TgScraper::fromVersion($logger, $version);
+        } catch (Throwable) {
+            return Command::FAILURE;
+        }
+        $output->writeln('Exporting schema from data...');
+        $destination = $input->getArgument('destination');
+        try {
+            TgScraper::getTargetDirectory(pathinfo($destination)['dirname']);
+        } catch (Exception) {
+            return Command::FAILURE;
+        }
+        $readable = $input->getOption('readable');
+        $options = $input->getOption('options');
+        $useYaml = $input->getOption('yaml');
+        $inline = $readable ? 12 : $input->getOption('inline');
+        $indent = $readable ? 4 : $input->getOption('indent');
+        $output->writeln('Saving schema to file...');
+        if ($input->getOption('openapi')) {
+            $data = $generator->toOpenApi();
+            if ($useYaml) {
+                return $this->saveFile($logger, $output, $destination, Encoder::toYaml($data, $inline, $indent, $options));
+            }
+            return $this->saveFile($logger, $output, $destination, Encoder::toJson($data, $options, $readable));
+        }
+        if ($input->getOption('postman')) {
+            $data = $generator->toPostman();
+            return $this->saveFile($logger, $output, $destination, Encoder::toJson($data, $options, $readable));
+        }
+        $data = $generator->toArray();
+        if ($useYaml) {
+            return $this->saveFile($logger, $output, $destination, Encoder::toYaml($data, $inline, $indent, $options));
+        }
+        return $this->saveFile($logger, $output, $destination, Encoder::toJson($data, $options, $readable));
+    }
+
 }

src/Common/Encoder.php

@@ -0,0 +1,34 @@
+<?php
+
+namespace TgScraper\Common;
+
+use Symfony\Component\Yaml\Yaml;
+
+class Encoder
+{
+    /**
+     * @param mixed $data
+     * @param int $options
+     * @param bool $readable
+     * @return string
+     */
+    public static function toJson(mixed $data, int $options = 0, bool $readable = false): string
+    {
+        if ($readable) {
+            $options |= JSON_PRETTY_PRINT | JSON_UNESCAPED_SLASHES | JSON_UNESCAPED_UNICODE;
+        }
+        return json_encode($data, $options);
+    }
+
+    /**
+     * @param mixed $data
+     * @param int $inline
+     * @param int $indent
+     * @param int $flags
+     * @return string
+     */
+    public static function toYaml(mixed $data, int $inline = 12, int $indent = 4, int $flags = 0): string
+    {
+        return Yaml::dump($data, $inline, $indent, $flags);
+    }
+}

src/Common/OpenApiGenerator.php

@@ -0,0 +1,132 @@
+<?php
+
+namespace TgScraper\Common;
+
+class OpenApiGenerator
+{
+
+
+    public function __construct(private array $defaultResponses, private array $data, array $types, array $methods)
+    {
+        $this->addTypes($types);
+        $this->addMethods($methods);
+    }
+
+    public function setVersion($version = '1.0.0'): self
+    {
+        $this->data['info']['version'] = $version;
+        return $this;
+    }
+
+    public function addMethods(array $methods): self
+    {
+        foreach ($methods as $method) {
+            $this->addMethod($method['name'], $method['description'], $method['fields'], $method['return_types']);
+        }
+        return $this;
+    }
+
+    public function addMethod(string $name, string $description, array $fields, array $returnTypes): self
+    {
+        $method = '/' . $name;
+        $this->data['paths'][$method] = ['description' => $description];
+        $path = [];
+        $fields = self::addFields(['type' => 'object'], $fields);
+        $content = ['schema' => $fields];
+        if (!empty($fields['required'])) {
+            $path['requestBody']['required'] = true;
+        }
+        $path['requestBody']['content'] = [
+            'application/json' => $content,
+            'application/x-www-form-urlencoded' => $content,
+            'multipart/form-data' => $content
+        ];
+        $path['responses'] = $this->defaultResponses;
+        $path['responses']['200']['content']['application/json']['schema']
+        ['allOf'][1]['properties']['result'] = self::parsePropertyTypes($returnTypes);
+        $this->data['paths'][$method]['post'] = $path;
+        return $this;
+    }
+
+    public function addTypes(array $types): self
+    {
+        foreach ($types as $type) {
+            $this->addType($type['name'], $type['description'], $type['fields'], $type['extended_by']);
+        }
+        return $this;
+    }
+
+    public function addType(string $name, string $description, array $fields, array $extendedBy): self
+    {
+        $schema = ['description' => $description];
+        $schema = self::addFields($schema, $fields);
+        $this->data['components']['schemas'][$name] = $schema;
+        if (!empty($extendedBy)) {
+            foreach ($extendedBy as $extendedType) {
+                $this->data['components']['schemas'][$name]['anyOf'][] = self::parsePropertyType($extendedType);
+            }
+            return $this;
+        }
+        $this->data['components']['schemas'][$name]['type'] = 'object';
+        return $this;
+    }
+
+    private static function addFields(array $schema, array $fields): array
+    {
+        foreach ($fields as $field) {
+            $name = $field['name'];
+            $required = !$field['optional'];
+            if ($required) {
+                $schema['required'][] = $name;
+            }
+            $schema['properties'][$name] = self::parsePropertyTypes($field['types']);
+        }
+        return $schema;
+    }
+
+    private static function parsePropertyTypes(array $types): array
+    {
+        $result = [];
+        $hasMultipleTypes = count($types) > 1;
+        foreach ($types as $type) {
+            $type = self::parsePropertyType(trim($type));
+            if ($hasMultipleTypes) {
+                $result['anyOf'][] = $type;
+                continue;
+            }
+            $result = $type;
+        }
+        return $result;
+    }
+
+    private static function parsePropertyType(string $type): array
+    {
+        if (str_starts_with($type, 'Array')) {
+            return self::parsePropertyArray($type);
+        }
+        if (lcfirst($type) == $type) {
+            $type = str_replace(['int', 'float', 'bool'], ['integer', 'number', 'boolean'], $type);
+            return ['type' => $type];
+        }
+        return ['$ref' => '#/components/schemas/' . $type];
+    }
+
+    private static function parsePropertyArray(string $type): array
+    {
+        if (preg_match('/Array<(.+)>/', $type, $matches) === 1) {
+            return [
+                'type' => 'array',
+                'items' => self::parsePropertyTypes(explode(',', $matches[1]))
+            ];
+        }
+        return [];
+    }
+
+    /**
+     * @return array
+     */
+    public function getData(): array
+    {
+        return $this->data;
+    }
+}

src/Common/SchemaExtractor.php

@@ -4,7 +4,10 @@
 namespace TgScraper\Common;
 
 
+use Composer\InstalledVersions;
+use InvalidArgumentException;
 use JetBrains\PhpStorm\ArrayShape;
+use OutOfBoundsException;
 use PHPHtmlParser\Dom;
 use PHPHtmlParser\Exceptions\ChildNotFoundException;
 use PHPHtmlParser\Exceptions\CircularException;
@@ -33,33 +36,96 @@ class SchemaExtractor
         'answerPreCheckoutQuery'
     ];
 
-    private Dom $dom;
+    /**
-
+     * @var string
+     */
     private string $version;
 
     /**
      * SchemaExtractor constructor.
+     * @param LoggerInterface $logger
+     * @param Dom $dom
+     * @throws ChildNotFoundException
+     * @throws NotLoadedException
+     */
+    public function __construct(private LoggerInterface $logger, private Dom $dom)
+    {
+        $this->version = $this->parseVersion();
+        $this->logger->info('Bot API version: ' . $this->version);
+    }
+
+
+    /**
+     * @param LoggerInterface $logger
+     * @param string $version
+     * @return SchemaExtractor
+     * @throws OutOfBoundsException
+     * @throws Throwable
+     */
+    public static function fromVersion(LoggerInterface $logger, string $version = Versions::LATEST): SchemaExtractor
+    {
+        if (InstalledVersions::isInstalled('sysbot/tgscraper-cache') and class_exists('\TgScraper\Cache\CacheLoader')) {
+            $logger->info('Cache package detected, searching for a cached version.');
+            try {
+                $path = \TgScraper\Cache\CacheLoader::getCachedVersion($version);
+                $logger->info('Cached version found.');
+                return self::fromFile($logger, $path);
+            } catch (OutOfBoundsException) {
+                $logger->info('Cached version not found, continuing with URL.');
+            }
+        }
+        $url = Versions::getUrlFromText($version);
+        $logger->info(sprintf('Using URL: %s', $url));
+        return self::fromUrl($logger, $url);
+    }
+
+    /**
+     * @param LoggerInterface $logger
+     * @param string $path
+     * @return SchemaExtractor
+     * @throws Throwable
+     */
+    public static function fromFile(LoggerInterface $logger, string $path): SchemaExtractor
+    {
+        $dom = new Dom;
+        if (!file_exists($path)) {
+            throw new InvalidArgumentException('File not found');
+        }
+        $path = realpath($path);
+        try {
+            $logger->info(sprintf('Loading data from file "%s".', $path));
+            $dom->loadFromFile($path);
+            $logger->info('Data loaded.');
+        } catch (Throwable $e) {
+            $logger->critical(sprintf('Unable to load data from "%s": %s', $path, $e->getMessage()));
+            throw $e;
+        }
+        return new self($logger, $dom);
+    }
+
+    /**
      * @param LoggerInterface $logger
      * @param string $url
+     * @return SchemaExtractor
      * @throws ChildNotFoundException
      * @throws CircularException
      * @throws ClientExceptionInterface
      * @throws ContentLengthException
      * @throws LogicalException
      * @throws StrictException
-     * @throws Throwable
+     * @throws NotLoadedException
      */
-    public function __construct(private LoggerInterface $logger, private string $url = Versions::LATEST)
+    public static function fromUrl(LoggerInterface $logger, string $url): SchemaExtractor
     {
-        $this->dom = new Dom();
+        $dom = new Dom;
         try {
-            $this->dom->loadFromURL($this->url);
+            $dom->loadFromURL($url);
         } catch (Throwable $e) {
-            $this->logger->critical(sprintf('Unable to load data from URL "%s": %s', $this->url, $e->getMessage()));
+            $logger->critical(sprintf('Unable to load data from URL "%s": %s', $url, $e->getMessage()));
             throw $e;
         }
-        $this->version = $this->parseVersion();
+        $logger->info(sprintf('Data loaded from "%s".', $url));
-        $this->logger->info('Bot API version: ' . $this->version);
+        return new self($logger, $dom);
     }
 
     /**
@@ -96,6 +162,10 @@ class SchemaExtractor
         return ['description' => $description, 'table' => $table, 'extended_by' => $extendedBy];
     }
 
+    /**
+     * @throws ChildNotFoundException
+     * @throws NotLoadedException
+     */
     private function parseVersion(): string
     {
         /** @var Dom\Node\AbstractNode $element */
@@ -120,14 +190,6 @@ class SchemaExtractor
 
     /**
      * @return array
-     * @throws ChildNotFoundException
-     * @throws CircularException
-     * @throws ContentLengthException
-     * @throws LogicalException
-     * @throws NotLoadedException
-     * @throws ParentNotFoundException
-     * @throws StrictException
-     * @throws ClientExceptionInterface
      * @throws Throwable
      */
     #[ArrayShape(['version' => "string", 'methods' => "array", 'types' => "array"])]
@@ -136,7 +198,7 @@ class SchemaExtractor
         try {
             $elements = $this->dom->find('h4');
         } catch (Throwable $e) {
-            $this->logger->critical(sprintf('Unable to load data from URL "%s": %s', $this->url, $e->getMessage()));
+            $this->logger->critical(sprintf('Unable to parse data: %s', $e->getMessage()));
             throw $e;
         }
         $data = ['version' => $this->version];
@@ -185,8 +247,7 @@ class SchemaExtractor
         $result = [
             'name' => $name,
             'description' => htmlspecialchars_decode(strip_tags($description), ENT_QUOTES),
-            'fields' => $fields,
+            'fields' => $fields
-            'extended_by' => $extendedBy
         ];
         if ($isMethod) {
             $returnTypes = self::parseReturnTypes($description);
@@ -196,6 +257,7 @@ class SchemaExtractor
             $result['return_types'] = $returnTypes;
             return $result;
         }
+        $result['extended_by'] = $extendedBy;
         return $result;
     }
 
@@ -203,15 +265,13 @@ class SchemaExtractor
      * @param Dom\Node\Collection|null $fields
      * @param bool $isMethod
      * @return array
-     * @throws ChildNotFoundException
-     * @throws NotLoadedException
      */
     private static function parseFields(?Dom\Node\Collection $fields, bool $isMethod): array
     {
         $parsedFields = [];
         $fields = $fields ?? [];
         foreach ($fields as $field) {
-            /* @var Dom $field */
+            /* @var Dom\Node\AbstractNode $fieldData */
             $fieldData = $field->find('td');
             $name = $fieldData[0]->text;
             if (empty($name)) {
@@ -224,7 +284,7 @@ class SchemaExtractor
             $parsedData['types'] = self::parseFieldTypes($parsedData['type']);
             unset($parsedData['type']);
             if ($isMethod) {
-                $parsedData['required'] = $fieldData[2]->text == 'Yes';
+                $parsedData['optional'] = $fieldData[2]->text != 'Yes';
                 $parsedData['description'] = htmlspecialchars_decode(
                     strip_tags($fieldData[3]->innerHtml ?? $fieldData[3]->text ?? ''),
                     ENT_QUOTES

src/Common/StubCreator.php

@@ -11,6 +11,7 @@ use Nette\PhpGenerator\Helpers;
 use Nette\PhpGenerator\PhpFile;
 use Nette\PhpGenerator\PhpNamespace;
 use Nette\PhpGenerator\Type;
+use TgScraper\TgScraper;
 
 /**
  * Class StubCreator
@@ -48,9 +49,18 @@ class StubCreator
                 throw new InvalidArgumentException('Namespace invalid');
             }
         }
-        if (!is_array($this->schema['methods']) or !is_array($this->schema['types'])) {
+        if (!TgScraper::validateSchema($this->schema)) {
             throw new InvalidArgumentException('Schema invalid');
         }
+        $this->getExtendedTypes();
+        $this->namespace = $namespace;
+    }
+
+    /**
+     * Builds the abstract and the extended class lists.
+     */
+    private function getExtendedTypes()
+    {
         foreach ($this->schema['types'] as $type) {
             if (!empty($type['extended_by'])) {
                 $this->abstractClasses[] = $type['name'];
@@ -59,11 +69,12 @@ class StubCreator
                 }
             }
         }
-        print_r($this->extendedClasses);
-        print_r($this->abstractClasses);
-        $this->namespace = $namespace;
     }
 
+    /**
+     * @param string $str
+     * @return string
+     */
     private static function toCamelCase(string $str): string
     {
         return lcfirst(str_replace(' ', '', ucwords(str_replace('_', ' ', $str))));
@@ -74,11 +85,12 @@ class StubCreator
      * @param PhpNamespace $phpNamespace
      * @return array
      */
-    #[ArrayShape(['types' => "string", 'comments' => "string"])]
+    #[ArrayShape([
-    private function parseFieldTypes(
+        'types' => "string",
-        array $fieldTypes,
+        'comments' => "string"
-        PhpNamespace $phpNamespace
+    ])]
-    ): array {
+    private function parseFieldTypes(array $fieldTypes, PhpNamespace $phpNamespace): array
+    {
         $types = [];
         $comments = [];
         foreach ($fieldTypes as $fieldType) {
@@ -104,11 +116,12 @@ class StubCreator
      * @param PhpNamespace $phpNamespace
      * @return array
      */
-    #[ArrayShape(['types' => "string", 'comments' => "string"])]
+    #[ArrayShape([
-    private function parseApiFieldTypes(
+        'types' => "string",
-        array $apiTypes,
+        'comments' => "string"
-        PhpNamespace $phpNamespace
+    ])]
-    ): array {
+    private function parseApiFieldTypes(array $apiTypes, PhpNamespace $phpNamespace): array
+    {
         $types = [];
         $comments = [];
         foreach ($apiTypes as $apiType) {
@@ -138,9 +151,8 @@ class StubCreator
         'Response' => "\Nette\PhpGenerator\PhpFile",
         'TypeInterface' => "\Nette\PhpGenerator\ClassType"
     ])]
-    private function generateDefaultTypes(
+    private function generateDefaultTypes(string $namespace): array
-        string $namespace
+    {
-    ): array {
         $interfaceFile = new PhpFile;
         $interfaceNamespace = $interfaceFile->addNamespace($namespace);
         $interfaceNamespace->addInterface('TypeInterface');
@@ -244,7 +256,7 @@ class StubCreator
             usort(
                 $fields,
                 function ($a, $b) {
-                    return $b['required'] - $a['required'];
+                    return $a['optional'] - $b['optional'];
                 }
             );
             foreach ($fields as $field) {
@@ -252,7 +264,7 @@ class StubCreator
                 $fieldName = self::toCamelCase($field['name']);
                 $parameter = $function->addParameter($fieldName)
                     ->setType($types);
-                if (!$field['required']) {
+                if ($field['optional']) {
                     $parameter->setNullable()
                         ->setDefaultValue(null);
                 }
@@ -266,7 +278,10 @@ class StubCreator
     /**
      * @return array
      */
-    #[ArrayShape(['types' => "\Nette\PhpGenerator\PhpFile[]", 'api' => "string"])]
+    #[ArrayShape([
+        'types' => "\Nette\PhpGenerator\PhpFile[]",
+        'api' => "string"
+    ])]
     public function generateCode(): array
     {
         return [

src/Constants/Versions.php

@@ -7,46 +7,84 @@ namespace TgScraper\Constants;
 class Versions
 {
 
-    public const V100 = 'https://web.archive.org/web/20150714025308/https://core.telegram.org/bots/api/';
+    public const V100 = '1.0.0';
-    public const V110 = 'https://web.archive.org/web/20150812125616/https://core.telegram.org/bots/api';
+    public const V110 = '1.1.0';
-    public const V140 = 'https://web.archive.org/web/20150909214252/https://core.telegram.org/bots/api';
+    public const V140 = '1.4.0';
-    public const V150 = 'https://web.archive.org/web/20150921091215/https://core.telegram.org/bots/api/';
+    public const V150 = '1.5.0';
-    public const V160 = 'https://web.archive.org/web/20151023071257/https://core.telegram.org/bots/api';
+    public const V160 = '1.6.0';
-    public const V180 = 'https://web.archive.org/web/20160112101045/https://core.telegram.org/bots/api';
+    public const V180 = '1.8.0';
-    public const V182 = 'https://web.archive.org/web/20160126005312/https://core.telegram.org/bots/api';
+    public const V182 = '1.8.2';
-    public const V183 = 'https://web.archive.org/web/20160305132243/https://core.telegram.org/bots/api';
+    public const V183 = '1.8.3';
-    public const V200 = 'https://web.archive.org/web/20160413101342/https://core.telegram.org/bots/api';
+    public const V200 = '2.0.0';
-    public const V210 = 'https://web.archive.org/web/20160912130321/https://core.telegram.org/bots/api';
+    public const V210 = '2.1.0';
-    public const V211 = self::V210;
+    public const V211 = '2.1.1';
-    public const V220 = 'https://web.archive.org/web/20161004150232/https://core.telegram.org/bots/api';
+    public const V220 = '2.2.0';
-    public const V230 = 'https://web.archive.org/web/20161124162115/https://core.telegram.org/bots/api';
+    public const V230 = '2.3.0';
-    public const V231 = 'https://web.archive.org/web/20161204181811/https://core.telegram.org/bots/api';
+    public const V231 = '2.3.1';
-    public const V300 = 'https://web.archive.org/web/20170612094628/https://core.telegram.org/bots/api';
+    public const V300 = '3.0.0';
-    public const V310 = 'https://web.archive.org/web/20170703123052/https://core.telegram.org/bots/api';
+    public const V310 = '3.1.0';
-    public const V320 = 'https://web.archive.org/web/20170819054238/https://core.telegram.org/bots/api';
+    public const V320 = '3.2.0';
-    public const V330 = 'https://web.archive.org/web/20170914060628/https://core.telegram.org/bots/api';
+    public const V330 = '3.3.0';
-    public const V350 = 'https://web.archive.org/web/20171201065426/https://core.telegram.org/bots/api';
+    public const V350 = '3.5.0';
-    public const V360 = 'https://web.archive.org/web/20180217001114/https://core.telegram.org/bots/api';
+    public const V360 = '3.6.0';
-    public const V400 = 'https://web.archive.org/web/20180728174553/https://core.telegram.org/bots/api';
+    public const V400 = '4.0.0';
-    public const V410 = 'https://web.archive.org/web/20180828155646/https://core.telegram.org/bots/api';
+    public const V410 = '4.1.0';
-    public const V420 = 'https://web.archive.org/web/20190417160652/https://core.telegram.org/bots/api';
+    public const V420 = '4.2.0';
-    public const V430 = 'https://web.archive.org/web/20190601122107/https://core.telegram.org/bots/api';
+    public const V430 = '4.3.0';
-    public const V440 = 'https://web.archive.org/web/20190731114703/https://core.telegram.org/bots/api';
+    public const V440 = '4.4.0';
-    public const V450 = 'https://web.archive.org/web/20200107090812/https://core.telegram.org/bots/api';
+    public const V450 = '4.5.0';
-    public const V460 = 'https://web.archive.org/web/20200208225346/https://core.telegram.org/bots/api';
+    public const V460 = '4.6.0';
-    public const V470 = 'https://web.archive.org/web/20200401052001/https://core.telegram.org/bots/api';
+    public const V470 = '4.7.0';
-    public const V480 = 'https://web.archive.org/web/20200429054924/https://core.telegram.org/bots/api';
+    public const V480 = '4.8.0';
-    public const V490 = 'https://web.archive.org/web/20200611131321/https://core.telegram.org/bots/api';
+    public const V490 = '4.9.0';
-    public const V500 = 'https://web.archive.org/web/20201104151640/https://core.telegram.org/bots/api';
+    public const V500 = '5.0.0';
-    public const V510 = 'https://web.archive.org/web/20210315055600/https://core.telegram.org/bots/api';
+    public const V510 = '5.1.0';
-    public const V520 = 'https://web.archive.org/web/20210428195636/https://core.telegram.org/bots/api';
+    public const V520 = '5.2.0';
-    public const V530 = 'https://web.archive.org/web/20210626142851/https://core.telegram.org/bots/api';
+    public const V530 = '5.3.0';
-    public const LATEST = 'https://core.telegram.org/bots/api';
+    public const LATEST = 'latest';
+    public const STABLE = self::V530;
 
+    public const URLS = [
+        self::V100 => 'https://web.archive.org/web/20150714025308id_/https://core.telegram.org/bots/api/',
+        self::V110 => 'https://web.archive.org/web/20150812125616id_/https://core.telegram.org/bots/api',
+        self::V140 => 'https://web.archive.org/web/20150909214252id_/https://core.telegram.org/bots/api',
+        self::V150 => 'https://web.archive.org/web/20150921091215id_/https://core.telegram.org/bots/api/',
+        self::V160 => 'https://web.archive.org/web/20151023071257id_/https://core.telegram.org/bots/api',
+        self::V180 => 'https://web.archive.org/web/20160112101045id_/https://core.telegram.org/bots/api',
+        self::V182 => 'https://web.archive.org/web/20160126005312id_/https://core.telegram.org/bots/api',
+        self::V183 => 'https://web.archive.org/web/20160305132243id_/https://core.telegram.org/bots/api',
+        self::V200 => 'https://web.archive.org/web/20160413101342id_/https://core.telegram.org/bots/api',
+        self::V210 => 'https://web.archive.org/web/20160912130321id_/https://core.telegram.org/bots/api',
+        self::V211 => 'https://web.archive.org/web/20160912130321id_/https://core.telegram.org/bots/api',
+        self::V220 => 'https://web.archive.org/web/20161004150232id_/https://core.telegram.org/bots/api',
+        self::V230 => 'https://web.archive.org/web/20161124162115id_/https://core.telegram.org/bots/api',
+        self::V231 => 'https://web.archive.org/web/20161204181811id_/https://core.telegram.org/bots/api',
+        self::V300 => 'https://web.archive.org/web/20170612094628id_/https://core.telegram.org/bots/api',
+        self::V310 => 'https://web.archive.org/web/20170703123052id_/https://core.telegram.org/bots/api',
+        self::V320 => 'https://web.archive.org/web/20170819054238id_/https://core.telegram.org/bots/api',
+        self::V330 => 'https://web.archive.org/web/20170914060628id_/https://core.telegram.org/bots/api',
+        self::V350 => 'https://web.archive.org/web/20171201065426id_/https://core.telegram.org/bots/api',
+        self::V360 => 'https://web.archive.org/web/20180217001114id_/https://core.telegram.org/bots/api',
+        self::V400 => 'https://web.archive.org/web/20180728174553id_/https://core.telegram.org/bots/api',
+        self::V410 => 'https://web.archive.org/web/20180828155646id_/https://core.telegram.org/bots/api',
+        self::V420 => 'https://web.archive.org/web/20190417160652id_/https://core.telegram.org/bots/api',
+        self::V430 => 'https://web.archive.org/web/20190601122107id_/https://core.telegram.org/bots/api',
+        self::V440 => 'https://web.archive.org/web/20190731114703id_/https://core.telegram.org/bots/api',
+        self::V450 => 'https://web.archive.org/web/20200107090812id_/https://core.telegram.org/bots/api',
+        self::V460 => 'https://web.archive.org/web/20200208225346id_/https://core.telegram.org/bots/api',
+        self::V470 => 'https://web.archive.org/web/20200401052001id_/https://core.telegram.org/bots/api',
+        self::V480 => 'https://web.archive.org/web/20200429054924id_/https://core.telegram.org/bots/api',
+        self::V490 => 'https://web.archive.org/web/20200611131321id_/https://core.telegram.org/bots/api',
+        self::V500 => 'https://web.archive.org/web/20201104151640id_/https://core.telegram.org/bots/api',
+        self::V510 => 'https://web.archive.org/web/20210315055600id_/https://core.telegram.org/bots/api',
+        self::V520 => 'https://web.archive.org/web/20210428195636id_/https://core.telegram.org/bots/api',
+        self::V530 => 'https://web.archive.org/web/20210626142851id_/https://core.telegram.org/bots/api',
+        self::LATEST => 'https://core.telegram.org/bots/api'
+    ];
 
     public static function getVersionFromText(string $text): string
     {
-        $text = str_replace('.', '', $text);
+        $text = str_replace(['.', 'v'], ['', ''], strtolower($text));
         $const = sprintf('%s::V%s', self::class, $text);
         if (defined($const)) {
             return constant($const);
@@ -54,4 +92,10 @@ class Versions
         return self::LATEST;
     }
 
+    public static function getUrlFromText(string $text): string
+    {
+        $version = self::getVersionFromText($text);
+        return self::URLS[$version] ?? self::URLS[self::LATEST];
+    }
+
 }

src/TgScraper.php

@@ -2,10 +2,10 @@
 
 namespace TgScraper;
 
-use BadMethodCallException;
 use Exception;
 use InvalidArgumentException;
 use JetBrains\PhpStorm\ArrayShape;
+use JsonException;
 use PHPHtmlParser\Exceptions\ChildNotFoundException;
 use PHPHtmlParser\Exceptions\CircularException;
 use PHPHtmlParser\Exceptions\ContentLengthException;
@@ -16,16 +16,17 @@ use PHPHtmlParser\Exceptions\StrictException;
 use Psr\Http\Client\ClientExceptionInterface;
 use Psr\Log\LoggerInterface;
 use Symfony\Component\Yaml\Yaml;
+use TgScraper\Common\OpenApiGenerator;
 use TgScraper\Common\SchemaExtractor;
 use TgScraper\Common\StubCreator;
 use TgScraper\Constants\Versions;
 use Throwable;
 
 /**
- * Class Generator
+ * Class TgScraper
  * @package TgScraper
  */
-class Generator
+class TgScraper
 {
 
     /**
@@ -33,58 +34,92 @@ class Generator
      */
     public const TEMPLATES_DIRECTORY = __DIR__ . '/../templates';
 
+    /**
+     * @var string
+     */
+    private string $version;
     /**
      * @var array
      */
-    private array $schema;
+    private array $types;
+    /**
+     * @var array
+     */
+    private array $methods;
 
     /**
-     * Generator constructor.
+     * TgScraper constructor.
      * @param LoggerInterface $logger
-     * @param string $url
+     * @param array $schema
-     * @param array|null $schema
-     * @throws ChildNotFoundException
-     * @throws CircularException
-     * @throws ClientExceptionInterface
-     * @throws ContentLengthException
-     * @throws LogicalException
-     * @throws NotLoadedException
-     * @throws ParentNotFoundException
-     * @throws StrictException
-     * @throws Throwable
      */
-    public function __construct(
+    public function __construct(private LoggerInterface $logger, array $schema)
-        private LoggerInterface $logger,
+    {
-        private string $url = Versions::LATEST,
+        if (!self::validateSchema($schema)) {
-        ?array $schema = null
+            throw new InvalidArgumentException('Invalid schema provided');
-    ) {
-        if (empty($schema)) {
-            try {
-                $extractor = new SchemaExtractor($this->logger, $this->url);
-                $this->logger->info('Schema not provided, extracting from URL.');
-                $schema = $extractor->extract();
-            } catch (Throwable $e) {
-                $this->logger->critical(
-                    'An exception occurred while trying to extract the schema: ' . $e->getMessage()
-                );
-                throw $e;
-            }
         }
-        /** @var array $schema */
+        $this->version = $schema['version'] ?? '1.0.0';
-        $this->schema = $schema;
+        $this->types = $schema['types'];
+        $this->methods = $schema['methods'];
     }
 
     /**
+     * @param LoggerInterface $logger
+     * @param string $url
+     * @return static
      * @throws ChildNotFoundException
      * @throws CircularException
-     * @throws ParentNotFoundException
-     * @throws StrictException
      * @throws ClientExceptionInterface
-     * @throws NotLoadedException
      * @throws ContentLengthException
      * @throws LogicalException
+     * @throws NotLoadedException
+     * @throws ParentNotFoundException
+     * @throws StrictException
      * @throws Throwable
      */
+    public static function fromUrl(LoggerInterface $logger, string $url): self
+    {
+        $extractor = SchemaExtractor::fromUrl($logger, $url);
+        $schema = $extractor->extract();
+        return new self($logger, $schema);
+    }
+
+    /**
+     * @param LoggerInterface $logger
+     * @param string $version
+     * @return static
+     * @throws ChildNotFoundException
+     * @throws CircularException
+     * @throws ClientExceptionInterface
+     * @throws ContentLengthException
+     * @throws LogicalException
+     * @throws NotLoadedException
+     * @throws ParentNotFoundException
+     * @throws StrictException
+     * @throws Throwable
+     */
+    public static function fromVersion(LoggerInterface $logger, string $version = Versions::LATEST): self
+    {
+        $extractor = SchemaExtractor::fromVersion($logger, $version);
+        $schema = $extractor->extract();
+        return new self($logger, $schema);
+    }
+
+    /**
+     * @param array $schema
+     * @return bool
+     */
+    public static function validateSchema(array $schema): bool
+    {
+        return array_key_exists('version', $schema) and is_string($schema['version']) and
+            array_key_exists('types', $schema) and is_array($schema['types']) and
+            array_key_exists('methods', $schema) and is_array($schema['methods']);
+    }
+
+    /**
+     * @param LoggerInterface $logger
+     * @param string $yaml
+     * @return TgScraper
+     */
     public static function fromYaml(LoggerInterface $logger, string $yaml): self
     {
         $data = Yaml::parse($yaml);
@@ -92,15 +127,10 @@ class Generator
     }
 
     /**
-     * @throws ChildNotFoundException
+     * @param LoggerInterface $logger
-     * @throws ParentNotFoundException
+     * @param string $json
-     * @throws CircularException
+     * @return TgScraper
-     * @throws StrictException
+     * @throws JsonException
-     * @throws ClientExceptionInterface
-     * @throws NotLoadedException
-     * @throws ContentLengthException
-     * @throws LogicalException
-     * @throws Throwable
      */
     public static function fromJson(LoggerInterface $logger, string $json): self
     {
@@ -124,8 +154,12 @@ class Generator
             );
             throw $e;
         }
+        $typesDir = $directory . '/Types';
+        if (!file_exists($typesDir)) {
+            mkdir($typesDir, 0755);
+        }
         try {
-            $creator = new StubCreator($this->schema, $namespace);
+            $creator = new StubCreator($this->toArray(), $namespace);
         } catch (InvalidArgumentException $e) {
             $this->logger->critical(
                 'An exception occurred while trying to parse the schema: ' . $e->getMessage()
@@ -146,12 +180,12 @@ class Generator
      * @return string
      * @throws Exception
      */
-    private static function getTargetDirectory(string $path): string
+    public static function getTargetDirectory(string $path): string
     {
         $result = realpath($path);
         if (false === $result) {
             if (!mkdir($path)) {
-                $path = __DIR__ . '/../generated';
+                $path = getcwd() . '/gen';
                 if (!file_exists($path)) {
                     mkdir($path, 0755);
                 }
@@ -161,61 +195,60 @@ class Generator
         if (false === $result) {
             throw new Exception('Could not create target directory');
         }
-        @mkdir($result . '/Types', 0755);
         return $result;
     }
 
     /**
-     * @param int $options
+     * @return array
-     * @return string
      */
-    public function toJson(int $options = 0): string
+    #[ArrayShape([
+        'version' => "string",
+        'types' => "array",
+        'methods' => "array"
+    ])] public function toArray(): array
     {
-        return json_encode($this->schema, $options);
+        return [
+            'version' => $this->version,
+            'types' => $this->types,
+            'methods' => $this->methods
+        ];
     }
 
     /**
-     * @param int $inline
+     * @return array
-     * @param int $indent
-     * @param int $flags
-     * @return string
      */
-    public function toYaml(int $inline = 6, int $indent = 4, int $flags = 0): string
+    public function toOpenApi(): array
     {
-        return Yaml::dump($this->schema, $inline, $indent, $flags);
+        $openapiTemplate = file_get_contents(self::TEMPLATES_DIRECTORY . '/openapi.json');
-    }
+        $openapiData = json_decode($openapiTemplate, true);
-
+        $responsesTemplate = file_get_contents(self::TEMPLATES_DIRECTORY . '/responses.json');
-    /**
+        $responses = json_decode($responsesTemplate, true);
-     * @return string
+        $openapi = new OpenApiGenerator($responses, $openapiData, $this->types, $this->methods);
-     */
+        $openapi->setVersion($this->version);
-    public function toOpenApi(): string
+        return $openapi->getData();
-    {
-        throw new BadMethodCallException('Not implemented');
     }
 
 
     /**
      * Thanks to davtur19 (https://github.com/davtur19/TuriBotGen/blob/master/postman.php)
-     * @param int $options
+     * @return array
-     * @return string
      */
     #[ArrayShape(['info' => "string[]", 'variable' => "string[]", 'item' => "array[]"])]
-    public function toPostman(
+    public function toPostman(): array
-        int $options = 0
+    {
-    ): string {
         $template = file_get_contents(self::TEMPLATES_DIRECTORY . '/postman.json');
         $result = json_decode($template, true);
-        $result['info']['version'] = $this->schema['version'];
+        $result['info']['version'] = $this->version;
-        foreach ($this->schema['methods'] as $method) {
+        foreach ($this->methods as $method) {
             $formData = [];
             if (!empty($method['fields'])) {
                 foreach ($method['fields'] as $field) {
                     $formData[] = [
                         'key' => $field['name'],
-                        'disabled' => !$field['required'],
+                        'disabled' => $field['optional'],
                         'description' => sprintf(
                             '%s. %s',
-                            $field['required'] ? 'Required' : 'Optional',
+                            $field['optional'] ? 'Optional' : 'Required',
                             $field['description']
                         ),
                         'type' => 'text'
@@ -247,7 +280,7 @@ class Generator
                 ]
             ];
         }
-        return json_encode($result, $options);
+        return $result;
     }
 
 }

templates/openapi.json

@@ -17,7 +17,7 @@
     }
   ],
   "externalDocs": {
-    "description": "Official Telegram Bot API documentation",
+    "description": "Official Telegram Bot API documentation.",
     "url": "https://core.telegram.org/bots/api"
   },
   "components": {
@@ -81,11 +81,32 @@
             }
           }
         }
+      },
+      "ServerError": {
+        "description": "The bot API is experiencing some issues, try again later.",
+        "content": {
+          "application/json": {
+            "schema": {
+              "$ref": "#/components/schemas/Error"
+            }
+          }
+        }
+      },
+      "UnknownError": {
+        "description": "An unknown error occurred.",
+        "content": {
+          "application/json": {
+            "schema": {
+              "$ref": "#/components/schemas/Error"
+            }
+          }
+        }
       }
     },
     "schemas": {
       "Response": {
         "type": "object",
+        "description": "Represents the default response object.",
         "required": [
           "ok"
         ],
@@ -96,6 +117,7 @@
         }
       },
       "Success": {
+        "description": "Request was successful, the result is returned.",
         "allOf": [
           {
             "$ref": "#/components/schemas/Response"
@@ -114,6 +136,7 @@
         ]
       },
       "Error": {
+        "description": "Request was unsuccessful, so an error occurred.",
         "allOf": [
           {
             "$ref": "#/components/schemas/Response"

templates/responses.json

@@ -0,0 +1,46 @@
+{
+  "200": {
+    "description": "Request was successful, the result is returned.",
+    "content": {
+      "application/json": {
+        "schema": {
+          "allOf": [
+            {
+              "$ref": "#/components/schemas/Success"
+            },
+            {
+              "type": "object",
+              "properties": {
+                "result": {}
+              }
+            }
+          ]
+        }
+      }
+    }
+  },
+  "400": {
+    "$ref": "#/components/responses/BadRequest"
+  },
+  "401": {
+    "$ref": "#/components/responses/Unauthorized"
+  },
+  "403": {
+    "$ref": "#/components/responses/Forbidden"
+  },
+  "404": {
+    "$ref": "#/components/responses/NotFound"
+  },
+  "409": {
+    "$ref": "#/components/responses/Conflict"
+  },
+  "429": {
+    "$ref": "#/components/responses/TooManyRequests"
+  },
+  "5XX": {
+    "$ref": "#/components/responses/ServerError"
+  },
+  "default": {
+    "$ref": "#/components/responses/UnknownError"
+  }
+}