Commit Graph

12 Commits

Author SHA1 Message Date
Lunny Xiao
dd08853b10
Dump github/gitlab/gitea repository data to a local directory and restore to gitea (#12244)
* Dump github/gitlab repository data to a local directory

* Fix lint

* Adjust directory structure

* Allow migration special units

* Allow migration ignore release assets

* Fix lint

* Add restore repository

* stage the changes

* Merge

* Fix lint

* Update the interface

* Add some restore methods

* Finish restore

* Add comments

* Fix restore

* Add a token flag

* Fix bug

* Fix test

* Fix test

* Fix bug

* Fix bug

* Fix lint

* Fix restore

* refactor downloader

* fmt

* Fix bug isEnd detection on getIssues

* Refactor maxPerPage

* Remove unused codes

* Remove unused codes

* Fix bug

* Fix restore

* Fix dump

* Uploader should not depend downloader

* use release attachment name but not id

* Fix restore bug

* Fix lint

* Fix restore bug

* Add a method of DownloadFunc for base.Release to make uploader not depend on downloader

* fix Release yml marshal

* Fix trace information

* Fix bug when dump & restore

* Save relative path on yml file

* Fix bug

* Use relative path

* Update docs

* Use git service string but not int

* Recognize clone addr to service type
2020-12-27 11:34:19 +08:00
6543
49b1948cb1
Gitea 2 Gitea migration (#12657)
* first draft

* update gitea sdk to 9e280adb4da

* adapt feat of updated sdk

* releases now works

* break the Reactions loop

* use convertGiteaLabel

* fix endless loop because paggination is not supported there !!!

* rename gitea local uploader files

* pagination can bite you in the ass

* Version Checks

* lint

* docs

* rename gitea sdk import to miss future conficts

* go-swagger: dont scan the sdk structs

* make sure gitea can shutdown gracefully

* make GetPullRequests and GetIssues similar

* rm useles

* Add Test: started ...

* ... add tests ...

* Add tests and Fixing things

* Workaround missing SHA

* Adapt: Ensure that all migration requests are cancellable
(714ab71ddc)

* LINT: fix misspells in test set

* adapt ListMergeRequestAwardEmoji

* update sdk

* Return error when creating giteadownloader failed

* update sdk

* adapt new sdk

* adopt new features

* check version before err

* adapt: 'migrate service type switch page'

* optimize

* Fix DefaultBranch

* impruve

* handle subPath

* fix test

* Fix ReviewCommentPosition

* test GetReviews

* add DefaultBranch int test set

* rm unused

* Update SDK to v0.13.0

* addopt sdk changes

* found better link

* format template

* Update Docs

* Update Gitea SDK (v0.13.1)
2020-10-14 07:06:00 +03:00
zeripath
714ab71ddc
Ensure that all migration requests are cancellable (#12669)
* Ensure that all migration requests are cancellable

Signed-off-by: Andrew Thornton <art27@cantab.net>

* Use WithContext as RequestWithContext is go 1.14

Signed-off-by: Andrew Thornton <art27@cantab.net>

Co-authored-by: techknowlogick <techknowlogick@gitea.io>
2020-09-02 13:49:25 -04:00
John Olheiser
211321fb93
Git migration UX (#12619)
* Initial work

Signed-off-by: jolheiser <john.olheiser@gmail.com>

* Implementation

Signed-off-by: jolheiser <john.olheiser@gmail.com>

* Fix gitlab and token cloning

Signed-off-by: jolheiser <john.olheiser@gmail.com>

* Imports and JS

Signed-off-by: jolheiser <john.olheiser@gmail.com>

* Fix test

Signed-off-by: jolheiser <john.olheiser@gmail.com>

* Linting

Signed-off-by: jolheiser <john.olheiser@gmail.com>

* Generate swagger

Signed-off-by: jolheiser <john.olheiser@gmail.com>

* Move mirror toggle and rename options

Signed-off-by: jolheiser <john.olheiser@gmail.com>

Co-authored-by: Lauris BH <lauris@nix.lv>
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
2020-08-28 09:36:37 +08:00
6543
e37e217602
[Fix] RetryDownloader (add GetReviews()) (#11093)
* add GetReviews() to RetryDownloader
2020-04-20 03:23:28 +01:00
Lunny Xiao
f6067a8465 Migrate reviews when migrating repository from github (#9463)
* fix typo

* Migrate reviews when migrating repository from github

* fix lint

* Added test and migration when external user login

* fix test

* fix commented state

* Some improvements

* fix bug when get pull request and ref original author on code comments

* Fix migrated line; Added comment for review

* Don't load all pull requests attributes

* Fix typo

* wrong change copy head

* fix tests

* fix reactions

* Fix test

* fix fmt

* fix review comment reactions
2020-01-23 19:28:15 +02:00
Lunny Xiao
ffc904b1e0 Sleep longer if request speed is over github limitation (#9335)
* Sleep longer if request speed is over github limitation

* improve code

* remove unused code

* fix lint

* Use github's rate limit remain value to determine how long to sleep

* Save reset time when finished github api request

* fix bug

* fix lint

* Add context.Context for sleep

* fix test

* improve code

* fix bug and lint

* fix import order
2019-12-16 23:16:54 -05:00
Lunny Xiao
1b7182e5ec
Add retry for migration http/https requests (#9019)
* Add retry for migration http/https requests

* give the more suitable name for retry configuraion items

* fix docs and lint

* Only use retryDownloader when setting > 1
2019-11-16 16:30:06 +08:00
Lunny Xiao
e3e44a59d0 Update migrated repositories' issues/comments/prs poster id if user has a github external user saved (#7751)
* update migrated issues/comments when login as github

* add get userid when migrating or login with github oauth2

* fix lint

* add migrations for repository service type

* fix build

* remove unnecessary dependencies on migrations

* add cron task to update migrations poster ids and fix posterid when migrating

* fix lint

* fix lint

* improve code

* fix lint

* improve code

* replace releases publish id to actual author id

* fix import

* fix bug

* fix lint

* fix rawdata definition

* fix some bugs

* fix error message
2019-10-14 09:10:42 +03:00
Antoine GIRARD
46d6b92ad0 Import topics during migration (#7851)
* add GetTopics interface

* CreateTopics

* remove un-needed comment
2019-08-14 14:16:12 +08:00
Lunny Xiao
7d12ec2abd improve github downloader on migrations (#7049)
* improve github downloader on migrations

* fix tests

* fix  uppercase function parameters
2019-05-30 16:26:57 -04:00
Lunny Xiao
08069dc465
Improve migrations to support migrating milestones/labels/issues/comments/pullrequests (#6290)
* add migrations

* fix package dependency

* fix lints

* implements migrations except pull requests

* add releases

* migrating releases

* fix bug

* fix lint

* fix migrate releases

* fix tests

* add rollback

* pull request migtations

* fix import

* fix go module vendor

* add tests for upload to gitea

* more migrate options

* fix swagger-check

* fix misspell

* add options on migration UI

* fix log error

* improve UI options on migrating

* add support for username password when migrating from github

* fix tests

* remove comments and fix migrate limitation

* improve error handles

* migrate API will also support migrate milestones/labels/issues/pulls/releases

* fix tests and remove unused codes

* add DownloaderFactory and docs about how to create a new Downloader

* fix misspell

* fix migration docs

* Add hints about migrate options on migration page

* fix tests
2019-05-07 09:12:51 +08:00