MarcoBuster/hyperboria
1
0
Fork 0
mirror of https://github.com/nexus-stc/hyperboria synced 2024-11-18 09:19:26 +01:00
Code Issues Packages Projects Releases Wiki Activity
a78f686f44
hyperboria/nexus/cognitron
History
the-superpirate 16a151ae19 No description 
GitOrigin-RevId: 55a38effd769ab15a3e0893cac84754ffb1de5a0
2021-03-29 18:01:30 +03:00
..
configs - fix: Various fixes for release 2021-01-29 11:26:51 +03:00
installer - fix: Fix importing documentation 2021-01-29 12:08:40 +03:00
schema No description 2021-03-29 18:01:30 +03:00
web - fix: Change name 2021-02-21 15:22:32 +03:00
__init__.py - fix: Various fixes for release 2021-01-29 11:26:51 +03:00
BUILD.bazel - fix: Various fixes for release 2021-01-29 11:26:51 +03:00
docker-compose.yml - fix: Fix importing documentation 2021-01-29 12:08:40 +03:00
README.md - fix: Add pinning to documentation 2021-01-29 14:09:25 +03:00

README.md

Nexus Cognitron

Prerequisite

Follow the root guide to install Docker, IPFS and Bazel (optionally)

Guide

1. Download data dumps

export COLLECTION=bafykbzacebzohi352bddfunaub5rgqv5b324nejk5v6fltjh45be5ykw5jsjg
export COLLECTION_PATH=$(realpath $COLLECTION)
ipfs get $COLLECTION && ipfs pin add $COLLECTION

2. Launch Nexus Cognitron

Create docker-compose.yml file to set up Nexus Cognitron and then launch it:

docker-compose pull && docker-compose up

then go to http://localhost:3000

3. (Optional) Deploy data dumps into your database

There is a function work in traversing script that you can reimplement to iterate over the whole dataset and insert it into your own database or do whatever you want in parallel mode.

By default this script is just printing documents.

bazel run -c opt installer -- iterate \
  --data-filepath $COLLECTION_PATH/index/scitech \
  --schema-filepath schema/scitech.yaml