Commit Graph

625 Commits

Author SHA1 Message Date
zeripath
154424623a
Remove unnecessary loading of settings in update hook (#9496)
This PR simply makes update an empty command rather than needlessly load the settings for each reference.
2019-12-27 15:21:33 +00:00
zeripath
7bfb83e064 Batch hook pre- and post-receive calls (#8602)
* make notifyWatchers work on multiple actions

* more efficient multiple notifyWatchers

* Make CommitRepoAction take advantage of multiple actions

* Batch post and pre-receive results

* Set batch to 30

* Auto adjust timeout & add logging

* adjust processing message

* Add some messages to pre-receive

* Make any non-200 status code from pre-receive an error

* Add missing hookPrintResults

* Remove shortcut for single action

* mistaken merge fix

* oops

* Move master branch to the front

* If repo was empty and the master branch is pushed ensure that that is set as the default branch

* fixup

* fixup

* Missed HookOptions in setdefaultbranch

* Batch PushUpdateAddTag and PushUpdateDelTag

Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
2019-12-26 13:29:45 +02:00
zeripath
3e166bd055 Add debug option to serv to help debug problems (#9492)
* Add debug option to serv to help debug problems

* fixup! Add debug option to serv to help debug problems
2019-12-25 23:44:57 +08:00
Rychu
2b0e4e0cb8 Fixed errors logging in dump.go (#9218)
* Fixed errors logging in dump.go

* Replaced overlooked log.Fatal() with fatal()

* Run make fmt on dump.go

* Added missing new line to the fatal log

* Run make fmt on dump.go
2019-12-17 18:12:10 +02:00
zeripath
d1a49977b0 AuthorizedKeysCommand should not query db directly (#9371)
* AuthorizedKeysCommand should not query db directly

* Update routers/private/internal.go

* Fix import order
2019-12-16 20:49:07 -05:00
zeripath
e3c3b33ea7
Graceful: Xorm, RepoIndexer, Cron and Others (#9282)
* Change graceful to use a singleton obtained through GetManager instead of a global.
* Graceful: Make TestPullRequests shutdownable
* Graceful: Make the cron tasks graceful
* Graceful: AddTestPullRequest run in graceful ctx
* Graceful: SyncMirrors shutdown
* Graceful: SetDefaultContext for Xorm to be HammerContext
* Avoid starting graceful for migrate commands and checkout
* Graceful: DeliverHooks now can be shutdown
* Fix multiple syncing errors in modules/sync/UniqueQueue &  Make UniqueQueue closable
* Begin the process of making the repo indexer shutdown gracefully
2019-12-15 09:51:28 +00:00
Lunny Xiao
7b2ff82d20 Move some repository methods from models to modules/repository (#9353)
* Move some repository methods from models to modules/repository

* fix test
2019-12-14 18:30:01 +01:00
zeripath
2c83dac5d4 FCGI: Allow FCGI over unix sockets (#9298)
* FCGI: Allow FCGI over unix sockets

* fixup! FCGI: Allow FCGI over unix sockets
2019-12-10 13:23:26 +01:00
zeripath
60c5339042 Graceful: Cancel Process on monitor pages & HammerTime (#9213)
* Graceful: Create callbacks to with contexts

* Graceful: Say when Gitea is completely finished

* Graceful: Git and Process within HammerTime

Force all git commands to terminate at HammerTime
Force all process commands to terminate at HammerTime
Move almost all git processes to run as git Commands

* Graceful: Always Hammer after Shutdown

* ProcessManager: Add cancel functionality

* Fix tests

* Make sure that process.Manager.Kill() cancels

* Make threadsafe access to Processes and remove own unused Kill

* Remove cmd from the process manager as it is no longer used

* the default context is the correct context

* get rid of double till
2019-11-30 16:40:22 +02:00
zeripath
7523314ef8
Graceful: Allow graceful restart for fcgi (#9112)
* Graceful: Allow graceful restart for fcgi

My previous interpretation was incorrect - we do not handle sockets
being passed in over stdin

* Update web.go
2019-11-24 10:39:50 +00:00
zeripath
e3f22ad2cc Graceful: Allow graceful restart for unix sockets (#9113)
Previously we could not handle graceful restarts for http over unix
sockets. These can now be handled.
2019-11-24 03:11:24 +01:00
zeripath
cbaa1de9ec Add Graceful shutdown for Windows and hooks for shutdown of goroutines (#8964)
* Graceful Shutdown for windows and others

Restructures modules/graceful, adding shutdown for windows, removing and
replacing the old minwinsvc code.

Creates a new waitGroup - terminate which allows for goroutines to
finish up after the shutdown of the servers.

Shutdown and terminate hooks are added for goroutines.

* Remove unused functions - these can be added in a different PR

* Add startup timeout functionality

* Document STARTUP_TIMEOUT
2019-11-21 13:32:02 -05:00
zeripath
dd1beee2ef
Enforce Gitea environment for pushes (#8982)
* Enforce Gitea environment for pushes

* Update custom/conf/app.ini.sample

Co-Authored-By: Antoine GIRARD <sapk@users.noreply.github.com>
2019-11-14 22:39:48 +00:00
zeripath
722a7c902d
Add Close() method to gogitRepository (#8901)
In investigating #7947 it has become clear that the storage component of go-git repositories needs closing.

This PR adds this Close function and adds the Close functions as necessary.

In TransferOwnership the ctx.Repo.GitRepo is closed if it is open to help prevent the risk of multiple open files.

Fixes #7947
2019-11-13 07:01:19 +00:00
zeripath
b48cad6a2a
Adjust the must-change-password help (#8755) 2019-10-30 18:28:14 +00:00
zeripath
f067e12859
Graceful fixes (#8645)
* Only attempt to kill parent once

* Apply suggestions from code review

Co-Authored-By: guillep2k <18600385+guillep2k@users.noreply.github.com>

* Add waitgroup for running servers
2019-10-23 16:32:19 +01:00
zeripath
0bfe5eb10b
Allow Protected Branches to Whitelist Deploy Keys (#8483)
Add an option to protected branches to add writing deploy keys to the whitelist for pushing.

Please note this is technically a breaking change: previously if the owner of a repository was on the whitelist then any writing deploy key was effectively on the whitelist. This option will now need to be set if that is desired.

Closes #8472 

Details:
* Allow Protected Branches to Whitelist Deploy Keys
* Add migration
* Ensure that IsDeployKey is set to false on the http pushes
* add not null default false
2019-10-21 09:21:45 +01:00
zeripath
167e8f18da
Restore Graceful Restarting & Socket Activation (#7274)
* Prevent deadlock in indexer initialisation during graceful restart

* Move from gracehttp to our own service to add graceful ssh

* Add timeout for start of indexers and make hammer time configurable

* Fix issue with re-initialization in indexer during tests

* move the code to detect use of closed to graceful

* Handle logs gracefully - add a pid suffix just before restart

* Move to using a cond and a holder for indexers

* use time.Since

* Add some comments and attribution

* update modules.txt

* Use zero to disable timeout

* Move RestartProcess to its own file

* Add cleanup routine
2019-10-15 14:39:51 +01:00
Maxim Tkachenko
db657192d0 Password Complexity Checks (#6230)
Add password complexity checks. The default settings require a lowercase, uppercase, number and a special character within passwords.

Co-Authored-By: T-M-A <maxim.tkachenko@gmail.com>
Co-Authored-By: Lanre Adelowo <adelowomailbox@gmail.com>
Co-Authored-By: guillep2k <18600385+guillep2k@users.noreply.github.com>
Co-Authored-By: Lauris BH <lauris@nix.lv>
2019-10-14 16:24:26 +01:00
Lunny Xiao
f83db078f0 Move database settings from models to setting (#7806)
* move database settings from models to setting

* update docs

* fix checkout pr

* fix tests

* fix lint

* remove unsupported tidb options

* correct wrong variable name

* remove tidb totally
2019-08-24 11:24:45 +02:00
Tamal Saha
171b359877 Use gitea forked macaron (#7933)
Signed-off-by: Tamal Saha <tamal@appscode.com>
2019-08-23 12:40:29 -04:00
zeripath
def84840db Attempt to fix hook problem (#7854) 2019-08-14 12:25:05 +03:00
clavinet
8101d803e6 Update serv.go (#7822)
small semantics fix
"Hi there, user!" looks better than "Hi there: user!"
2019-08-11 20:15:58 +08:00
Allen Wild
2186a99c76 cmd/serv: actually exit after fatal errors (#7458)
Regression in 356854fc5f, where
a log.Fatal call was removed. log.Fatal calls os.Exit(1) as intended,
but without it the fail() function returns normally and execution
continues past the fatal error, causing a panic.

This is visible as a go panic log and stack trace returned to the SSH
client, which is not only ugly, it leaks server and build system
information.

Fix by removing the stray return statement so that the fail() function
always calls os.Exit(1).

Fixes: https://github.com/go-gitea/gitea/issues/7457
Signed-off-by: Allen Wild <allenwild93@gmail.com>
2019-07-13 21:38:18 +01:00
zeripath
3563650bdb #6946 Run hooks on merge/edit and cope with protected branches (#6961)
* Fix #6946 by checking PullRequest ID on pushing

* Ensure we have the owner name, the pr attributes and the the issue

* Fix TestSearchRepo by waiting till indexing is done

* Update integrations/repo_search_test.go

* changes as per @mrsdizzie

* missing comma

* Spelling mistake

* Fix full pushing environment
2019-07-01 09:18:13 +08:00
Lunny Xiao
8ec659722d Only check and config git on web subcommand but not others (#7236)
* only check and config git on web subcommand but not others

* add Init in git tests
2019-06-19 19:53:37 +03:00
ngourdon
a618df8d84 Add CLI commands to manage LDAP authentication source (#6681)
* add CLI commands to manage LDAP authentication source

* delete Gogs copyright

* remove unused return value of func parseLoginSource

* fix comment

Co-Authored-By: ngourdon <31291059+ngourdon@users.noreply.github.com>

* remove config flag already present in global flags

* remove config flag from ldap commands in docs

* remove config flag handling
2019-06-17 14:32:20 -04:00
zeripath
eaa3a1db4e Use certmanager provided TLSConfig for LetsEncrypt (#7229) 2019-06-17 13:33:55 -04:00
kolaente
f9ec2f89f2 Add golangci (#6418) 2019-06-12 15:41:28 -04:00
Lunny Xiao
74690f6451
fix pusher name via ssh push (#7167) 2019-06-11 09:13:24 +08:00
Lunny Xiao
23a2ee3510
Add command to convert mysql database from utf8 to utf8mb4 (#7144)
* add command to convert mysql database from utf8 to utf8mb4

* Update cmd/convert.go

Co-Authored-By: John Olheiser <42128690+jolheiser@users.noreply.github.com>

* Update cmd/convert.go

Co-Authored-By: John Olheiser <42128690+jolheiser@users.noreply.github.com>

* Update cmd/convert.go

Co-Authored-By: John Olheiser <42128690+jolheiser@users.noreply.github.com>

* Update models/convert.go

Co-Authored-By: John Olheiser <42128690+jolheiser@users.noreply.github.com>

* Update models/convert.go

Co-Authored-By: John Olheiser <42128690+jolheiser@users.noreply.github.com>

* Update cmd/convert.go

Co-Authored-By: John Olheiser <42128690+jolheiser@users.noreply.github.com>

* Update cmd/convert.go

Co-Authored-By: John Olheiser <42128690+jolheiser@users.noreply.github.com>
2019-06-08 21:53:45 +08:00
Sandro Santilli
45f588e85d Avoid arbitrary format strings upon calling fail() function (#7112) 2019-06-03 09:07:03 +01:00
zeripath
356854fc5f Move serv hook functionality & drop GitLogger (#6993)
* Move hook functionality internally

* Internalise serv logic

* Remove old internal paths

* finally remove the gitlogger

* Disallow push on archived repositories

* fix lint error

* Update modules/private/key.go

* Update routers/private/hook.go

* Update routers/private/hook.go

* Update routers/private/hook.go

* Updated routers/private/serv.go

* Fix LFS Locks over SSH

* rev-list needs to be run by the hook process

* fixup

* Improve git test

* Ensure that the lfs files are created with a different prefix

* Reduce the replication in git_test.go

* slight refactor

* Remove unnecessary "/"

* Restore ensureAnonymousClone

* Restore ensureAnonymousClone

* Run rev-list on server side

* Try passing in the alternative directories instead

* Mark test as skipped

* Improve git test

* Ensure that the lfs files are created with a different prefix
* Reduce the replication in git_test.go
* Remove unnecessary "/"
2019-06-01 23:00:21 +08:00
Lunny Xiao
355ab0c62c Fix wrong init dependency on markup extensions (#7038)
* fix wrong init dependency on markup extensions
2019-05-25 18:15:39 +01:00
James E. Blair
488d34691a Ignore non-standard refs in git push (#6758)
When replicating to gitea from a remote system which makes use of
git refs to store extra data (for example, gerrit), pushing a lot
of refs to gitea can cause problems due to the extra processing
that the pre and post receive hooks perform.  But it's still
useful for gitea to be able to serve those refs.  This change
skips unecessary processing of refs other than branches or tags.

We don't need to check any ref that isn't a branch for branch
protection (protection will never be enabled).  So in the
pre-receive hook, we wrap that check in a test for whether the
ref is a branch.

We also don't need to add information to the activity stream about
pushes to non-standard refs, so we skip that step in the
post-receive hook for refs which are not branches or tags.

For some concrete examples, gerrit maintains a ref for every
patchset of every change in the form refs/changes/XX/YYYY/Z.
Many systems use refs/notes to store additonal data about commits.
This change allows these and other schemes to be used without
affecting gitea.
2019-05-14 10:40:27 -04:00
Lanre Adelowo
e0dde8173f Generate access token in admin cli (#6847)
* add cli flag for access token while creating a user
2019-05-04 23:03:09 +01:00
techknowlogick
897927690f
Rename LFS_JWT_SECRET to include OAUTH2 as well (#6826) 2019-05-01 22:32:06 -04:00
John Olheiser
e255df83a6 Change verbose flag in dump command to avoid colliding with global version flag (#6822)
* Change verbose flag to avoid colliding with version flag
* Update docs

Signed-off-by: jolheiser <john.olheiser@gmail.com>
2019-05-01 21:36:09 +01:00
zeripath
8d0d7bc28d Make CustomPath, CustomConf and AppWorkPath configurable at build (#6631) 2019-04-29 14:08:21 -04:00
ngourdon
2b9b3310f6 Add username flag in create-user command (#6534)
* Add username flag in create-user command

* Fix the error message

* Change name to username on flag of create-user command

* Include create-user flags of both versions

* delete deprecated example of create-user command
2019-04-09 11:21:55 -04:00
mrsdizzie
e14cad79c9 Include custom configuration file in dump (#6516)
When running the dump command, include the specific configuration file
provided as part of the backup. Currently we only zip up custom/ and this
misses the config file if it is somewhere else, like /etc/gitea/app.ini

Fixes #6034
2019-04-05 09:24:28 -04:00
zeripath
704da08fdc
Better logging (#6038) (#6095)
* Panic don't fatal on create new logger

Fixes #5854

Signed-off-by: Andrew Thornton <art27@cantab.net>

* partial broken

* Update the logging infrastrcture

Signed-off-by: Andrew Thornton <art27@cantab.net>

* Reset the skip levels for Fatal and Error

Signed-off-by: Andrew Thornton <art27@cantab.net>

* broken ncsa

* More log.Error fixes

Signed-off-by: Andrew Thornton <art27@cantab.net>

* Remove nal

* set log-levels to lowercase

* Make console_test test all levels

* switch to lowercased levels

* OK now working

* Fix vetting issues

* Fix lint

* Fix tests

* change default logging to match current gitea

* Improve log testing

Signed-off-by: Andrew Thornton <art27@cantab.net>

* reset error skip levels to 0

* Update documentation and access logger configuration

* Redirect the router log back to gitea if redirect macaron log but also allow setting the log level - i.e. TRACE

* Fix broken level caching

* Refactor the router log

* Add Router logger

* Add colorizing options

* Adjust router colors

* Only create logger if they will be used

* update app.ini.sample

* rename Attribute ColorAttribute

* Change from white to green for function

* Set fatal/error levels

* Restore initial trace logger

* Fix Trace arguments in modules/auth/auth.go

* Properly handle XORMLogger

* Improve admin/config page

* fix fmt

* Add auto-compression of old logs

* Update error log levels

* Remove the unnecessary skip argument from Error, Fatal and Critical

* Add stacktrace support

* Fix tests

* Remove x/sync from vendors?

* Add stderr option to console logger

* Use filepath.ToSlash to protect against Windows in tests

* Remove prefixed underscores from names in colors.go

* Remove not implemented database logger

This was removed from Gogs on 4 Mar 2016 but left in the configuration
since then.

* Ensure that log paths are relative to ROOT_PATH

* use path.Join

* rename jsonConfig to logConfig

* Rename "config" to "jsonConfig" to make it clearer

* Requested changes

* Requested changes: XormLogger

* Try to color the windows terminal

If successful default to colorizing the console logs

* fixup

* Colorize initially too

* update vendor

* Colorize logs on default and remove if this is not a colorizing logger

* Fix documentation

* fix test

* Use go-isatty to detect if on windows we are on msys or cygwin

* Fix spelling mistake

* Add missing vendors

* More changes

* Rationalise the ANSI writer protection

* Adjust colors on advice from @0x5c

* Make Flags a comma separated list

* Move to use the windows constant for ENABLE_VIRTUAL_TERMINAL_PROCESSING

* Ensure matching is done on the non-colored message - to simpify EXPRESSION
2019-04-02 08:48:31 +01:00
glaszig
dbba46c216 support custom file name in gitea dump command (#6474)
* support custom file name in `gitea dump` command

* simpler approach to handle default dump file name
in `gitea dump` command
2019-04-01 00:31:37 -04:00
Lunny Xiao
d578b71d61
move code.gitea.io/git to code.gitea.io/gitea/modules/git (#6364)
* move code.gitea.io/git to code.gitea.io/gitea/modules/git

* fix imports

* fix fmt

* fix misspell

* remove wrong tests data

* fix unit tests

* fix tests

* fix tests

* fix tests

* fix tests

* fix tests

* enable Debug to trace the failure tests

* fix tests

* fix tests

* fix tests

* fix tests

* fix tests

* comment commit count tests since git clone depth is 50

* fix tests

* update from code.gitea.io/git

* revert change to makefile
2019-03-27 17:33:00 +08:00
mrsdizzie
ca46385637 Clean up various use of escape/unescape functions for URL generation (#6334)
* Use PathUnescape instead of QueryUnescape when working with branch names

Currently branch names with a '+' fail in certain situations because
QueryUnescape replaces the + character with a blank space.

Using PathUnescape should be better since it is defined as:

// PathUnescape is identical to QueryUnescape except that it does not
// unescape '+' to ' ' (space).

Fixes #6333

* Change error to match new function name

* Add new util function PathEscapeSegments

This function simply runs PathEscape on each segment of a path without
touching the forward slash itself. We want to use this instead of
PathEscape/QueryEscape in most cases because a forward slash is a valid name for a
branch etc... and we don't want that escaped in a URL.

Putting this in new file url.go and also moving a couple similar
functions into that file as well.

* Use EscapePathSegments where appropriate

Replace various uses of EscapePath/EscapeQuery with new
EscapePathSegments. Also remove uncessary uses of various
escape/unescape functions when the text had already been escaped or was
not escaped.

* Reformat comment to make drone build happy

* Remove no longer used url library

* Requested code changes
2019-03-18 10:00:23 -04:00
Jonas Franz
e777c6bdc6 Integrate OAuth2 Provider (#5378) 2019-03-08 11:42:50 -05:00
zeripath
01c10a951b
Fix ssh deploy and user key constraints (#1357) (#5939)
1. A key can either be an ssh user key or a deploy key. It cannot be both.
2. If a key is a user key - it can only be associated with one user.
3. If a key is a deploy key - it can be used in multiple repositories and the permissions it has on those repositories can be different.
4. If a repository is deleted, its deploy keys must be deleted too.

We currently don't enforce any of this and multiple repositories access with different permissions doesn't work at all. This PR enforces the following constraints:

- [x] You should not be able to add the same user key as another user
- [x] You should not be able to add a ssh user key which is being used as a deploy key
- [x] You should not be able to add a ssh deploy key which is being used as a user key
- [x] If you add an ssh deploy key to another repository you should be able to use it in different modes without losing the ability to use it in the other mode.
- [x] If you delete a repository you must delete all its deploy keys.

Fix #1357
2019-02-03 23:56:53 +00:00
zeripath
13c0f7dfff
Fix notifications on pushing with deploy keys by setting hook environment variables (#5935)
The gitea prerecieve and postrecieve hooks and the gitea PushUpdate function require that the PusherID and PusherName are real users. Previously, these environment variables were not being set when using a deploy key - the main result being that pushing to empty repositories meant that is_empty status was not changed.

I've also added an integration test to ensure that the is_empty status is updated on pushing with a deploy key.

There is a slight issue in that the deploy key is now considered a proxy for the owner - we don't have a way of separating out the deploy key from the owner at present. This can be fixed in another PR.

Fix #3795 

Signed-off-by: Andrew Thornton art27@cantab.net
2019-02-03 14:00:48 +00:00
zeripath
2902b3a68c
Fix #5866: Silence console logger in gitea serv (#5887)
By default, if `setting.NewContext()` prints out any warning logs, these are printed to the stdout breaking `git receive-pack` etc. meaning that even if there is a warning because of a minor problem in your app.ini but gitea starts despite this - you **CANNOT** push or pull over SSH.

This PR disables the console logger whilst in `serv.go`

Signed-off-by: Andrew Thornton <art27@cantab.net>
2019-02-03 11:19:28 +00:00
Lanre Adelowo
e1fcd6b742 Disallow empty titles (#5785)
* add util method and tests

* make sure the title of an issue cannot be empty

* wiki title cannot be empty

* pull request title cannot be empty

* update to make use of the new util methof
2019-01-21 13:45:32 +02:00