mirror of
https://github.com/nexus-stc/hyperboria
synced 2024-12-21 00:57:45 +01:00
|
|
||
|---|---|---|
| .. | ||
| configs | ||
| installer | ||
| schema | ||
| web | ||
| __init__.py | ||
| BUILD.bazel | ||
| docker-compose.yml | ||
| README.md | ||
Nexus Cognitron
Prerequisite
Follow the root guide to install Docker, IPFS and Bazel (optionally)
Guide
1. Download data dumps
export COLLECTION=bafykbzacebzohi352bddfunaub5rgqv5b324nejk5v6fltjh45be5ykw5jsjg
export COLLECTION_PATH=$(realpath $COLLECTION)
ipfs get $COLLECTION
2. Launch Nexus Cognitron
Create docker-compose.yml file to set up Nexus Cognitron and then launch it:
docker-compose up
3. (Optional) Deploy data dumps into your database
There is a function work in traversing script
that you can reimplement to iterate over the whole dataset and insert it into your
own database or do whatever you want in parallel mode.
By default this script is just printing documents.
bazel run -c opt installer -- iterate \
--data-filepath $COLLECTION_PATH/index/scitech \
--schema-filepath schema/scitech.yaml