MarcoBuster/hyperboria
1
0
Fork 0
mirror of https://github.com/nexus-stc/hyperboria synced 2024-10-19 20:09:57 +02:00
Code Issues Packages Projects Releases Wiki Activity
e877f3e724
hyperboria/nexus/cognitron
History
the-superpirate e877f3e724 - feat(nexus): Update dependencies, fix bugs 
- feat(idm): Added monitoring
  - feat(nexus): Refactor GRPC base client
  - fix(nexus): Logging fixes
  - feat(hub): Extend logging
  - feat(hub): Extend logging
2 internal commit(s)

GitOrigin-RevId: 01de4153dd9c0edc2e5954563733085249a4551c
2021-04-15 17:23:15 +03:00
..
configs - fix: Various fixes for release 2021-01-29 11:26:51 +03:00
installer - fix: Fix importing documentation 2021-01-29 12:08:40 +03:00
schema No description 2021-03-29 18:01:30 +03:00
web - feat(nexus): Update dependencies, fix bugs 2021-04-15 17:23:15 +03:00
__init__.py - fix: Various fixes for release 2021-01-29 11:26:51 +03:00
BUILD.bazel - fix: Various fixes for release 2021-01-29 11:26:51 +03:00
docker-compose.yml - fix: Fix importing documentation 2021-01-29 12:08:40 +03:00
README.md - fix: Add pinning to documentation 2021-01-29 14:09:25 +03:00

README.md

Nexus Cognitron

Prerequisite

Follow the root guide to install Docker, IPFS and Bazel (optionally)

Guide

1. Download data dumps

export COLLECTION=bafykbzacebzohi352bddfunaub5rgqv5b324nejk5v6fltjh45be5ykw5jsjg
export COLLECTION_PATH=$(realpath $COLLECTION)
ipfs get $COLLECTION && ipfs pin add $COLLECTION

2. Launch Nexus Cognitron

Create docker-compose.yml file to set up Nexus Cognitron and then launch it:

docker-compose pull && docker-compose up

then go to http://localhost:3000

3. (Optional) Deploy data dumps into your database

There is a function work in traversing script that you can reimplement to iterate over the whole dataset and insert it into your own database or do whatever you want in parallel mode.

By default this script is just printing documents.

bazel run -c opt installer -- iterate \
  --data-filepath $COLLECTION_PATH/index/scitech \
  --schema-filepath schema/scitech.yaml