shutil.rmtree(ignore_errors=True) was for ignores error when directory not exist,
but it also ignores permission error, so we shouldn't use that.
Run os.path.exists() before shutil.rmtree() instead.
See scylladb/scylla#7337
On older distribution such as CentOS7, it does not support systemd user mode.
On such distribution nonroot mode does not work, show warning message and
skip running systemctl --user.
See scylladb/scylla#7071
On some environment, systemctl enable <service> fails when we use symlink.
So just directly copy systemd units to ~/.config/systemd/user, instead of
creating symlink.
See scylladb/scylla#7288
Instead of using the baked-in values from SCYLLA-VERSION-GEN,
allow passing an override. This will be used by the supermodule
to have an identical product-version-release (especially release,
which contains the git hash) across all packages.
"
We do not want to install dependencies on package building time,
we want to install it on dbuild container.
So drop package installation from the scripts.
scylladb/scylla#7219
"
Closes#138.
* 'dont_install_builddep' of https://github.com/syuu1228/scylla-jmx:
reloc: simplified .deb build process
reloc: simplified .rpm build process
dist: do not install build dependencies on build script
dist/debian: Remove conflict tag for Java 11
Fix the path of debuild, similar to what commit
f57fbb77b0a3f8d240c9924f3fa4529f5b5c8122 ("dist: debian: fix detection
of debuild") did in scylla-tools.
Message-Id: <20200911164153.34699-1-penberg@scylladb.com>
We do not want to install dependencies on package building time,
we want to install it on dbuild container.
So drop package installation from the scripts.
scylladb/scylla#7219
"
Fixes#133Fixes#134
Refs #135
Makes CF mbean refresh code synchronized and tries to remove reductant
calls if we contend. Adds background reaping of dead objects to reduce
memory load in (test) scenarios where we manage to refresh to add, but
not cause removal (i.e. no wildcard queries).
TableMetricsObjectName serialization is fixed in the series because
without it we see loads of exceptions when refreshing the mbean set.
"
* elcallio-jmx-fixes:
scylla-jmx: Use registration checker objects
scylla-jmx: Introduce a registration check object
scylla-jmx: Fix TableMetricObjectName serialization
Fixes#134
Refs #135
Replaces previous refresh calls with ones bound to registration
check objects, which provides some sync between threads doing
refresh, and reduced redundant calls.
Also adds repeated reaping of dead objects, i.e. every 5 minutes
we try to remove dead CF:s (not adding new ones), to reduce
idle footprint.
"Currently after offline installation, the scylla-jmx fails to start.
This pull request fixes issues with openjdk version detection and
working directory configuration to make scylla-jmx start.
Fixes: scylladb/scylla#7098 by [PATCH] install.sh: check both openjdk-8 and openjdk-11
Fixes: #129 by [PATCH] nonroot.conf: set WorkingDirectory to empty"
Reviewed-by: Takuya ASADA <syuu@scylladb.com>
* 'openjdk' of git://github.com/amoskong/scylla-jmx:
install.sh: check both openjdk-8 and openjdk-11
nonroot.conf: set WorkingDirectory to empty
Fixes#133
TableMetricObjectName is not serializable as such, since
it depends on a lexicon object etc.
Use writeReplace to put a regular ObjectName in
the stream instead.
On Debian10, only Openjdk-11 is available, the install.sh fails in java
checking. Openjdk-8 and Openjdk-11 all work well, we should check both of them.
This patch also fixed the error message.
Signed-off-by: Amos Kong <amos@scylladb.com>
After offline installation, scylla-jmx fails to be started for a chdir
error. WorkingDirectory is set to /var/lib/scylla in scylla-jmx.service,
it doesn't exist in nonroot install. This patch solved the problem by
setting WorkingDirectory to empty in nonroot.conf.
$ systemctl --user status scylla-jmx
● scylla-jmx.service - Scylla JMX
Loaded: loaded (/home/scylla-test/.config/systemd/user/../../../scylladb/etc/systemd/scylla-jmx.service; linked; vendor preset: enabled)
Drop-In: /home/scylla-test/.config/systemd/user/scylla-jmx.service.d
└─nonroot.conf
Active: failed (Result: exit-code) since Wed 2020-08-26 15:19:56 UTC; 2s ago
Process: 66955 ExecStart=/home/scylla-test/install_root/jmx/scylla-jmx $SCYLLA_JMX_PORT $SCYLLA_API_PORT $SCYLLA_API_ADDR $SCYLLA_JMX_ADDR $SCYLLA_JMX_FILE $SCYLLA_JMX_LOCAL $SCYLLA_JMX_REMOTE $SCYLLA_JMX_DEBUG (code=exited, status=200/CHDIR)
Main PID: 66955 (code=exited, status=200/CHDIR)
systemd[5654]: Started Scylla JMX.
systemd[66955]: scylla-jmx.service: Changing to the requested working directory failed: No such file or directory
systemd[66955]: scylla-jmx.service: Failed at step CHDIR spawning /home/scylla-test/scylladb/jmx/scylla-jmx: No such file or directory
systemd[5654]: scylla-jmx.service: Main process exited, code=exited, status=200/CHDIR
systemd[5654]: scylla-jmx.service: Failed with result 'exit-code'.
Signed-off-by: Amos Kong <amos@scylladb.com>
We current require Java 8 to install the scylla-jmx package on Debian.
As Debian 10 defaults to Java 11, let's remove the conflict flag and add
Java 11 to the dependencies list.
We current require Java 8 to install the scylla-jmx package on Debian.
As Debian 10 defaults to Java 11, let's remove the conflict flag and add
Java 11 to the dependencies list.
Currently scylla-jmx.service's PWD is "/", we get following error when
JVM trying to write heap dump on current directory:
Aug 17 05:52:15 localhost.localdomain scylla-jmx[3469]: Starting the JMX server
Aug 17 05:52:16 localhost.localdomain scylla-jmx[3469]: java.lang.OutOfMemoryError: Java heap space
Aug 17 05:52:16 localhost.localdomain scylla-jmx[3469]: Dumping heap to java_pid3469.hprof ...
Aug 17 05:52:16 localhost.localdomain scylla-jmx[3469]: Unable to create java_pid3469.hprof: Permission denied
To fix this, we need to specify WorkingDirectory on systemd unit.
Debian package builds provide a root environment for the installation
scripts, since that's what typical installation scripts expect. To
avoid providing actual root, a "fakeroot" system is used where syscalls
are intercepted and any effect that requires root (like chown) is emulated.
However, fakeroot sporadically fails for us, aborting the package build.
Since our install scripts don't really require root (when operating in
the --packaging mode), we can just tell dpkg-buildpackage that we don't
need fakeroot. This ought to fix the sporadic failures.
As a side effect, package builds are faster.
Follows scylla.git's b608af870b0a1ad88b91a72bddeff0c321877f9e.
Refs scylladb/scylla#6655.
This directory is created by a build and shouldn't be commited so
it's best for git to just ignore it.
Signed-off-by: Piotr Jastrzebski <piotr@scylladb.com>
Message-Id: <b20dba12fb726aebd51b2ab9494e7c52f8058feb.1595259605.git.piotr@scylladb.com>
We need the ability to control build directory in scylla.git build
system. Let's add support for the "--builddir" option like in other
variants of the same scripts.
Message-Id: <20200717085723.701209-1-penberg@scylladb.com>
rpmbuild defaults to logging the name of every file it unpacks from
the archive. This is quite a lot for Java applications.
Make it quiet with the %setup -q flag.
When building scylla-jmx in place, `scylla-apiclient/target/` is left
behind and should be ignored by `.gitignore` otherwise the scylla
submodule directory appears to be dirty.
* bhalevy-gitignore:
gitignore: do not track scylla-apiclient/target/
gitignore: fix typo in dependency-reduced-pom.xml
Same as scylladb/scylla#6752,
we currently does not able to apply version number fixup for .orig.tar.gz file,
even we applied correct fixup on debian/changelog, becuase it just reading
SCYLLA-VERSION-FILE.
We should parse debian/{changelog,control} instead.
Fixes#120
On 52bd496, we stopped to rm -rf debian/ on build_deb.sh, since now we have
prebuilt debian/ directory.
However, it might cause .deb build error when we modified debian package source,
since it never cleanup.
To prevent build error, we need to cleanup build/debian on reloc/build_deb.sh,
before extracting contents from relocatable package.
To install scylla-jmx using install.sh easily, we need:
- run dependency check before install
- run postinst script after install
But we don't want to run them when we build .rpm/.deb package,
we also need to add --packaging option to skip them.
See scylladb/scylla#5830
Now we generate dist/changelog on relocatable package generation time,
we cannot run '.rc' fixup on .deb package building time, need to do it
in debian_files_gen.py.
To make unified relocatable package easily, we may want to merge tarballs to single tarball like this:
zcat .tar.gz | gzip -c > scylla-unified.tar.xz
But it's not possible with current relocatable package format, since there are multiple files conflicts, install.sh, SCYLLA--FILE, dist/, README.md, etc..
To support this, we need to archive everything in the directory when building relocatable package.
See: scylladb/scylla#6315
The scrub command was not supported from node_tool, but now when we want
to enable it the current API is not compatible with the 3.11 MBean
definition.
This patch adds the definition to the MBean and the implementation to
StorageService.
It also address two problems with the old scrub implementation, just
in case someone will use them.
1. Implementation didn't pass the parameters to the API.
2. A stub implementation called itself instead of calling an actual
implementation.
This patch will enable to test the command from nodetool additional
changes may come on top of it if more command line options will be
supported.
Signed-off-by: Amnon Heiman <amnon@scylladb.com>
Drop dependency on pystache since it nolong present in Fedora 32.
To implement it, simplified debian package build process.
It will be generate debian/ directory when building relocatable package,
we just need to run debuild using the package.
To generate debian/ directory this commit added debian_files_gen.py,
it construct whole directory including control and changelog files
from template files.
Since we need to stop pystache, these template files swiched to
string.Template class which is included python3 standard library.
see: https://github.com/scylladb/scylla/pull/6313
On the commit 4c8660d, we dropped /usr/lib/scylla/jmx since it likely no user
script invoke scripts under the directory.
However, we found there are possibility scylla-jmx.service tries to load .jar
file from /usr/lib/scylla/jmx, when user upgraded from older version of scylla.
Because /etc/sysconfig/scylla-jmx is marked as 'noreplace' on our rpm,
yum upgrade may keep old sysconfig file when it modified by user, that may
causes to load .jar from /usr/lib/scylla/jmx since we specify the path in the
sysconfig file.
To avoid the issue it's better to have symlinks on /usr/lib/scylla/jmx for
safety.
See #90
This is scylla-jmx part of https://github.com/scylladb/scylla/pull/5530
After we stopped replacing /usr/lib/scylla with symlink,
creating symlink on /opt/scylladb/scripts/jmx become meaningless,
so we can drop it now.
We able to introduce new symlink on /usr/lib/scylla, but it likely no user
directly invoke scripts under /usr/lib/scylla/jmx, so we don't have to
do that.
Since Debian 10 dropped OpenJDK 8, so we need to switch to other
JVM distribution which still maintain OpenJDK 8 for Debian.
We can use AdoptOpenJDK for such environment.
See scylladb/scylla#5463
Signed-off-by: Takuya ASADA <syuu@scylladb.com>
Message-Id: <20191218124838.35017-1-syuu@scylladb.com>
