Compare commits

...

157 Commits

Author SHA1 Message Date
Andrea Cavalli
69951d5a2d Remove debug code 2023-01-24 16:14:19 +01:00
Andrea Cavalli
12cd93f7c1 Fix exports 2023-01-24 16:11:00 +01:00
Andrea Cavalli
c43992ba08 Change version 2023-01-24 16:06:39 +01:00
Andrea Cavalli
cd30267633 Update to java 17 2023-01-24 14:58:50 +01:00
Takuya ASADA
88d9bdc5b2 install.sh: add --without-systemd option
Since we fail to write files to $USER/.config on Jenkins jobs, we need
an option to skip installing systemd units.
Let's add --without-systemd to do that.

Also, to detect the option availability, we need to increment
relocatable package version.

See scylladb/scylla-dtest#2819
2022-09-12 13:00:59 +03:00
Takuya ASADA
06f27357b4 build_reloc.sh: rename relocatable packages
Currently, we use following naming convention for relocatable package
filename:
  ${package_name}-${arch}-package-${version}.${release}.tar.gz
But this is very different with Linux standard packaging system such as
.rpm and .deb.
Let's align the convention to .rpm style, so new convention should be:
  ${package_name}-${version}-${release}.${arch}.tar.gz

See scylladb/scylla#9799

Closes #185
2022-07-19 15:32:05 +03:00
Piotr Grabowski
fe351e8491 Update jackson dependency
Update jackson dependency to a newer version without any known
vulnerabilities. I have checked changelogs of all versions between
2.12.1 and 2.12.6.1, and none of the changes were potentially
problematic (minor fixes, etc).

2.12.6.1 version of jackson-databind is compatible with 2.12.6 versions
of other jackson packages.
2022-05-31 13:46:06 +03:00
Nadav Har'El
53f7f55e8c pom.xml: drop unneeded logging dependencies
pom.xml specifies a dependency on slf4j (the Simple Logging Facade for
Java) and its ancient log4j backend (slf4j-log4j12), but we don't
actually use it the scylla-jmx project - we use the standard
java.util.logging.

So let's drop the unnecessary (and these log4shell days, scary) dependencies.

Signed-off-by: Nadav Har'El <nyh@scylladb.com>
Message-Id: <20211213083055.1383507-1-nyh@scylladb.com>
2021-12-16 11:39:40 +02:00
Benny Halevy
2c43d99aa5 removeNode: support ignoreNodes options
Refs scylladb/scylla-tools-java#225

Signed-off-by: Benny Halevy <bhalevy@scylladb.com>

Closes #178
2021-11-15 15:27:07 +02:00
Avi Kivity
26a6919714 build: replace yum with dnf
dnf has replaced yum on Fedora and CentOS. On modern versions of Fedora,
you have to install an extra package to get the old name working, so
avoid that inconvenience and use dnf directly.

Closes #181
2021-11-15 15:19:22 +02:00
Avi Kivity
d6225c5231 build: use utc for build datestamp
This helps keep packages built on different machines have the
same datestamp, if started on the same time.
2021-11-07 15:58:09 +02:00
Benny Halevy
48d37f3402 StorageService: scrub: fix scrubMode is empty condition
`!=` compares references not values.

Use !"".equals(scrubMode) instead, as it also covers
the null scrubMode case.

Signed-off-by: Benny Halevy <bhalevy@scylladb.com>

Closes #179
2021-11-02 15:21:08 +02:00
Takuya ASADA
5c383b641b reloc: stop removing entire $BUILDDIR
We found that user can mistakenly break system with --builddir option,
something like './reloc/build_deb.sh --builddir /'.
To avoid that we need to stop removing entire $BUILDDIR, remove
directories only we have to clean up before building deb package.

See: https://github.com/scylladb/scylla-python3/pull/23#discussion_r707088453

Closes #177
2021-09-19 10:01:40 +03:00
Juliusz Stasiewicz
658818b2d0 Support --load-and-stream option from nodetool refresh
This information is translated to {"load_and_stream", "true"} entry in the
POST request to Scylla's HTTP API at `storage_service/sstables/{keyspace}`
endpoint.

More about this feature: scylladb/scylla#7846

This change is a consequence of scylladb/scylla-tools-java#253.
2021-09-13 18:22:19 +03:00
Benny Halevy
70b19e6270 scrub: support scrubMode and deprecate skipCorrupted
Support new scrubMode option and deprecate skipCorrupted
that's equivalent to scrubMode="SKIP".

Signed-off-by: Benny Halevy <bhalevy@scylladb.com>

Closes #175
2021-08-24 14:51:05 +03:00
Benny Halevy
5311e9bae3 storage_service: takeSnapshot: support the skipFlush option
Fixes #167

Signed-off-by: Benny Halevy <bhalevy@scylladb.com>

Closes #168
2021-06-18 12:58:27 +03:00
dependabot[bot]
fbfbdaa298 build(deps): bump snakeyaml from 1.16 to 1.26 in /scylla-apiclient
Bumps [snakeyaml](https://bitbucket.org/asomov/snakeyaml) from 1.16 to 1.26.
- [Commits](https://bitbucket.org/asomov/snakeyaml/branches/compare/snakeyaml-1.26..v1.16)

---
updated-dependencies:
- dependency-name: org.yaml:snakeyaml
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>

Closes #169
2021-06-10 09:59:48 +03:00
Piotr Wojtczak
a7c4c39dd0 storage_service: Fix getToppartitions to always return both reads and writes
In line with the previous API, the getToppartitions function returned
results for one specified sampler (reads OR writes). This forced
the user to call the function once for each sampler, which is
suboptimal.
This commit changes the signature so that results for both samplers
are returned and the user can then pick whichever they need.
2021-05-10 18:07:07 +03:00
Piotr Wojtczak
440313eb72 storage_service: Add a generic toppartitions endpoint
As part of making the toppartitions API more generic
(i.e. being able to consider multiple tables
and keyspaces specified by the user) this commit adds
a JMX endpoint to call the generic Scylla REST API
introduced in #7864. It has been put inside
storage_service as being now able to query more than
one column family makes it no longer suitable for the
'column_family' group.

Fixes #4520
2021-03-25 12:35:18 +02:00
Takuya ASADA
9c687b562e dist/redhat: add support SLES
CentOS/RHEL and SLES has differnt package name of opejdk, use common
name of JRE.
Note that using common name of Java package is also useful when user want to
use differnt implementation of JRE for Scylla.

Also, disable AutoReqProv which is mistakenly enabled but required for
cross build rpm.
2021-03-15 17:23:14 +02:00
Amnon Heiman
15c1d4f43f StorageService: Add a method to return the uptime
Currently, the nodetool uses the jmx server for the uptime, this is
confusing is what we expect is Scylla uptime.

This patch exposes the API uptime using MBean.

Relates to #154

Signed-off-by: Amnon Heiman <amnon@scylladb.com>

Closes #155
2021-03-04 10:52:08 +02:00
dependabot[bot]
ffab41d714 Bump Jackson version in scylla-apiclient
Bumps `jackson.version` from 2.10.4 to 2.12.1.

Updates `jackson-annotations` from 2.10.4 to 2.12.1
- [Release notes](https://github.com/FasterXML/jackson/releases)
- [Commits](https://github.com/FasterXML/jackson/commits)

Updates `jackson-databind` from 2.10.4 to 2.12.1
- [Release notes](https://github.com/FasterXML/jackson/releases)
- [Commits](https://github.com/FasterXML/jackson/commits)

Updates `jackson-jaxrs-json-provider` from 2.10.4 to 2.12.1

Signed-off-by: dependabot[bot] <support@github.com>

Closes #159
2021-03-04 10:48:34 +02:00
Pekka Enberg
bac7d0b31e Merge 'Fix locking in APIBuilder.remove()' from Pekka Enberg
This pull request reverts the commit c2fc96b ("APIBuilder: Remove
RW-lock in JMX server repository wrapper") and fixes a missing unlock
from APIBuilder.remove().

Closes #163

* github.com:scylladb/scylla-jmx:
  APIBuilder: Unlock RW-lock in remove()
  Revert "APIBuilder: Remove RW-lock in JMX server repository wrapper"
2021-03-03 18:29:57 +02:00
Pekka Enberg
59fd4d2b03 APIBuilder: Unlock RW-lock in remove()
The remove() function accidentally calls lock() in the finally
block, leaving the RW-lock unlocked.

Refs: scylladb/scylla#7991
2021-03-03 18:23:41 +02:00
Pekka Enberg
9d7ee8af3c Revert "APIBuilder: Remove RW-lock in JMX server repository wrapper"
This reverts commit c2fc96be71. The
RW-lock usage had a bug, which will be fixed in a follow up patch.
2021-03-03 18:20:46 +02:00
Calle Wilund
c2fc96be71 APIBuilder: Remove RW-lock in JMX server repository wrapper
This is a seemingly pointless change. The RW-lock code is 100%
correct (afaict), yet we've seen repeated cases of test runs
hanging in JMX query because this lock is seemingly left held
by what seems to be the reaper task.

There is no explanation for this, no sign of exceptions/errors
that could explain the lock being broken. Nor any known JDK/JVM
bugs.

Yet, in tests, it seems that replacing the lock with a more
coarse, yet proven, synchronized, fixes the issue. So there.

I officially hate this patch, and it should not exist.
2021-03-03 15:40:33 +02:00
Amnon Heiman
8073af6e06 CompactionManager: add the compaction id when available
This patch adds the compaction id in getCompactions if it returns by the
API, if it's not the current behaviour will be used and it will return none.

After this patch a call to nodetool compactionstats -H

Will return:

id                                   compaction type keyspace  table     completed total unit progress
c942bd30-7a62-11eb-84bc-576502584f9a COMPACTION      keyspace1 standard1 1062      8576  keys 12.38%
c9429620-7a62-11eb-8afb-576402584f9a COMPACTION      keyspace1 standard1 972       8448  keys 11.51%
Active compaction remaining time :   0h00m00s

Fixes scylladb/scylla#7927

Signed-off-by: Amnon Heiman <amnon@scylladb.com>
2021-03-01 10:04:08 +02:00
Pekka Enberg
bf8bb16b52 Merge 'dist/debian: fix renaming debian/scylla-* files rule' from Takuya ASADA
Current renaming rule of debian/scylla-* files is buggy, it fails to
install some .service files when custom product name specified.

Introduce regex based rewriting instead of adhoc renaming, and fixed
wrong renaming rule.

Fixes scylladb/scylla#8113

Closes #158

* github.com:scylladb/scylla-jmx:
  dist/debian: fix renaming debian/scylla-* files rule
  dist/debian: sync packaging script with scylla main repo
2021-02-18 10:34:29 +02:00
Takuya ASADA
8f62d71e11 dist/debian: fix renaming debian/scylla-* files rule
Current renaming rule of debian/scylla-* files is buggy, it fails to
install some .service files when custom product name specified.

Introduce regex based rewriting instead of adhoc renaming, and fixed
wrong renaming rule.

Related scylladb/scylla#8113
2021-02-18 04:38:36 +09:00
Takuya ASADA
3618481e23 dist/debian: sync packaging script with scylla main repo 2021-02-18 04:25:48 +09:00
Takuya ASADA
949cefc251 dist/redhat: stop using systemd macros, call systemctl directly
Fedora version of systemd macros does not work correctly on CentOS7,
since CentOS7 does not support "file trigger" feature.
Even after 05d4378, scriptlets on old scylla .rpm and new scylla .rpm is
not completely same.

To fix the issue we need to stop using systemd macros, call systemctl
directly.

Fixes #94
2021-02-02 11:28:55 +02:00
Piotr Wojtczak
611d586981 Remove obsolete FIXME
The cardinality problem has already been fixed in #149.
2021-01-25 13:07:40 +02:00
Amos Kong
2c9565024f install.sh: set a valid WorkingDirectory for nonroot offline install
In commit 6311525, we set an empty value to WorkDirectory for nonroot.conf
of scylla-jmx.service. It works with ubuntu16, debian9, debian 10. But it
doesn't work with ubuntu 18.

This patch changed the WorkingDirectory of nonroot offline install to
default install directory (/home/scylla-test/scylladb).

Fixes #151

Signed-off-by: Amos Kong <amos@scylladb.com>
2020-12-28 21:18:35 +02:00
Piotr Wojtczak
20469bf749 column_family: Return proper cardinality for toppartitions requests
Right now, in the finishLocalSampling method of the ColumnFamilyStore
we return the size of the list of returned partitions. Instead, we should
be propagating the actual cardinality of the sampled set.
Let's just read the read_cardinality and write_cardinality properties
of the scylla's REST API response.

Fixes #148
2020-12-13 13:50:56 +02:00
Eliran Sinvani
6174a47924 Relocatable Package: create product prefixed relocatable archive
The build system was hardcoded to produce a package that is
prefixed with scylla instead of the product name. This is not
in line with out CI system requirements and can be also a source
for confusion.
This commit make the packaging system generate a package of
the format: {product}-jmx-package.tar.gz instead of
scylla-jmx-package.tar.gz

Closes #146
2020-10-15 17:10:21 +03:00
dependabot[bot]
91e7adffb1 build(deps-dev): bump junit from 4.8.2 to 4.13.1
Bumps [junit](https://github.com/junit-team/junit4) from 4.8.2 to 4.13.1.
- [Release notes](https://github.com/junit-team/junit4/releases)
- [Changelog](https://github.com/junit-team/junit4/blob/main/doc/ReleaseNotes4.13.1.md)
- [Commits](https://github.com/junit-team/junit4/compare/r4.8.2...r4.13.1)

Signed-off-by: dependabot[bot] <support@github.com>
2020-10-15 14:22:38 +03:00
Amnon Heiman
c51906ed01 StorageService.java: Use the endpoint for getRangeToEndpointMap
After implementing range_to_endpoint_map endpoint update the API call to
it.

Fixes #36

Signed-off-by: Amnon Heiman <amnon@scylladb.com>
2020-10-08 11:53:33 +03:00
Takuya ASADA
c55f3f292b dist/debian/debian_files_gen.py: don't ignore permission error on shutil.rmtree()
shutil.rmtree(ignore_errors=True) was for ignores error when directory not exist,
but it also ignores permission error, so we shouldn't use that.
Run os.path.exists() before shutil.rmtree() instead.

See scylladb/scylla#7337
2020-10-08 11:48:30 +03:00
Takuya ASADA
e3a381d5a1 install.sh: show warning nonroot mode when systemd does not support user mode
On older distribution such as CentOS7, it does not support systemd user mode.
On such distribution nonroot mode does not work, show warning message and
skip running systemctl --user.

See scylladb/scylla#7071
2020-10-07 11:33:22 +03:00
Takuya ASADA
25bcd76017 install.sh: stop using symlinks for systemd units on nonroot mode
On some environment, systemctl enable <service> fails when we use symlink.
So just directly copy systemd units to ~/.config/systemd/user, instead of
creating symlink.

See scylladb/scylla#7288
2020-09-29 13:31:30 +03:00
Avi Kivity
45e4f28766 build: support passing product-version-release as a parameter
Instead of using the baked-in values from SCYLLA-VERSION-GEN,
allow passing an override. This will be used by the supermodule
to have an identical product-version-release (especially release,
which contains the git hash) across all packages.
2020-09-23 12:57:50 +03:00
Avi Kivity
6795a22afe Merge "dist: do not install build dependencies on build script" from Takuya
"
We do not want to install dependencies on package building time,
we want to install it on dbuild container.
So drop package installation from the scripts.

scylladb/scylla#7219
"

Closes #138.

* 'dont_install_builddep' of https://github.com/syuu1228/scylla-jmx:
  reloc: simplified .deb build process
  reloc: simplified .rpm build process
  dist: do not install build dependencies on build script
  dist/debian: Remove conflict tag for Java 11
2020-09-16 10:29:23 +03:00
Takuya ASADA
5fa422c4ea reloc: simplified .deb build process
We don't really need to have two build_deb.sh, merge it to reloc.
2020-09-16 16:19:24 +09:00
Takuya ASADA
f1612ef508 reloc: simplified .rpm build process
We don't really need to have two build_rpm.sh, merge it to reloc.
2020-09-16 16:19:24 +09:00
Pekka Enberg
d3096f32e0 dist: debian: fix detection of debuild
Fix the path of debuild, similar to what commit
f57fbb77b0a3f8d240c9924f3fa4529f5b5c8122 ("dist: debian: fix detection
of debuild") did in scylla-tools.

Message-Id: <20200911164153.34699-1-penberg@scylladb.com>
2020-09-13 16:26:22 +03:00
Takuya ASADA
99e491df40 dist: do not install build dependencies on build script
We do not want to install dependencies on package building time,
we want to install it on dbuild container.
So drop package installation from the scripts.

scylladb/scylla#7219
2020-09-13 20:19:16 +09:00
Pekka Enberg
8d92e5450e Merge 'JMX footprint work' from Calle
"
Fixes #133
Fixes #134
Refs #135

Makes CF mbean refresh code synchronized and tries to remove reductant
calls if we contend. Adds background reaping of dead objects to reduce
memory load in (test) scenarios where we manage to refresh to add, but
not cause removal (i.e. no wildcard queries).

TableMetricsObjectName serialization is fixed in the series because
without it we see loads of exceptions when refreshing the mbean set.
"

* elcallio-jmx-fixes:
  scylla-jmx: Use registration checker objects
  scylla-jmx: Introduce a registration check object
  scylla-jmx: Fix TableMetricObjectName serialization
2020-09-07 13:54:56 +03:00
Calle Wilund
ba3f58c63c scylla-jmx: Use registration checker objects
Fixes #134
Refs #135

Replaces previous refresh calls with ones bound to registration
check objects, which provides some sync between threads doing
refresh, and reduced redundant calls.

Also adds repeated reaping of dead objects, i.e. every 5 minutes
we try to remove dead CF:s (not adding new ones), to reduce
idle footprint.
2020-09-07 11:00:42 +02:00
Calle Wilund
771fe3e360 scylla-jmx: Introduce a registration check object
Allows for shared code for synchronized and optionally
partial update checks.
2020-09-07 11:00:42 +02:00
Pekka Enberg
12ab6aaeb8 Merge "Fix JMX startup after offline installation" from Amos
"Currently after offline installation, the scylla-jmx fails to start.
 This pull request fixes issues with openjdk version detection and
 working directory configuration to make scylla-jmx start.

 Fixes: scylladb/scylla#7098 by [PATCH] install.sh: check both openjdk-8 and openjdk-11

 Fixes: #129 by [PATCH] nonroot.conf: set WorkingDirectory to empty"

Reviewed-by: Takuya ASADA <syuu@scylladb.com>
* 'openjdk' of git://github.com/amoskong/scylla-jmx:
  install.sh: check both openjdk-8 and openjdk-11
  nonroot.conf: set WorkingDirectory to empty
2020-09-07 09:38:28 +03:00
Calle Wilund
1219faf9f1 scylla-jmx: Fix TableMetricObjectName serialization
Fixes #133

TableMetricObjectName is not serializable as such, since
it depends on a lexicon object etc.

Use writeReplace to put a regular ObjectName in
the stream instead.
2020-09-01 15:46:18 +02:00
Amos Kong
d998ac2e1e install.sh: check both openjdk-8 and openjdk-11
On Debian10, only Openjdk-11 is available, the install.sh fails in java
checking. Openjdk-8 and Openjdk-11 all work well, we should check both of them.
This patch also fixed the error message.

Signed-off-by: Amos Kong <amos@scylladb.com>
2020-08-28 01:24:37 +08:00
Amos Kong
6311525346 nonroot.conf: set WorkingDirectory to empty
After offline installation, scylla-jmx fails to be started for a chdir
error. WorkingDirectory is set to /var/lib/scylla in scylla-jmx.service,
it doesn't exist in nonroot install. This patch solved the problem by
setting WorkingDirectory to empty in nonroot.conf.

$ systemctl --user status scylla-jmx
â—Ź scylla-jmx.service - Scylla JMX
   Loaded: loaded (/home/scylla-test/.config/systemd/user/../../../scylladb/etc/systemd/scylla-jmx.service; linked; vendor preset: enabled)
  Drop-In: /home/scylla-test/.config/systemd/user/scylla-jmx.service.d
           └─nonroot.conf
   Active: failed (Result: exit-code) since Wed 2020-08-26 15:19:56 UTC; 2s ago
  Process: 66955 ExecStart=/home/scylla-test/install_root/jmx/scylla-jmx $SCYLLA_JMX_PORT $SCYLLA_API_PORT $SCYLLA_API_ADDR $SCYLLA_JMX_ADDR $SCYLLA_JMX_FILE $SCYLLA_JMX_LOCAL $SCYLLA_JMX_REMOTE $SCYLLA_JMX_DEBUG (code=exited, status=200/CHDIR)
 Main PID: 66955 (code=exited, status=200/CHDIR)

systemd[5654]: Started Scylla JMX.
systemd[66955]: scylla-jmx.service: Changing to the requested working directory failed: No such file or directory
systemd[66955]: scylla-jmx.service: Failed at step CHDIR spawning /home/scylla-test/scylladb/jmx/scylla-jmx: No such file or directory
systemd[5654]: scylla-jmx.service: Main process exited, code=exited, status=200/CHDIR
systemd[5654]: scylla-jmx.service: Failed with result 'exit-code'.

Signed-off-by: Amos Kong <amos@scylladb.com>
2020-08-26 23:34:28 +08:00
Yaron Kaikov
d5d1efd188 dist/debian: Remove conflict tag for Java 11
We current require Java 8 to install the scylla-jmx package on Debian.
As Debian 10 defaults to Java 11, let's remove the conflict flag and add
Java 11 to the dependencies list.
2020-08-25 15:46:04 +03:00
Yaron Kaikov
23da40b559 dist/debian: Remove conflict tag for Java 11
We current require Java 8 to install the scylla-jmx package on Debian.
As Debian 10 defaults to Java 11, let's remove the conflict flag and add
Java 11 to the dependencies list.
2020-08-24 09:13:34 +03:00
Takuya ASADA
be8f1ac511 dist/common/systemd: set WorkingDirectory to get heap dump correctly
Currently scylla-jmx.service's PWD is "/", we get following error when
JVM trying to write heap dump on current directory:

Aug 17 05:52:15 localhost.localdomain scylla-jmx[3469]: Starting the JMX server
Aug 17 05:52:16 localhost.localdomain scylla-jmx[3469]: java.lang.OutOfMemoryError: Java heap space
Aug 17 05:52:16 localhost.localdomain scylla-jmx[3469]: Dumping heap to java_pid3469.hprof ...
Aug 17 05:52:16 localhost.localdomain scylla-jmx[3469]: Unable to create java_pid3469.hprof: Permission denied

To fix this, we need to specify WorkingDirectory on systemd unit.
2020-08-17 09:54:38 +03:00
Avi Kivity
c5ed83178a dist: debian: support non-x86
The package is already arch independent, so remove the artifical
restriction to x86.
2020-08-04 13:07:42 +03:00
Avi Kivity
626fd75173 dist: debian: do not require root during package build
Debian package builds provide a root environment for the installation
scripts, since that's what typical installation scripts expect. To
avoid providing actual root, a "fakeroot" system is used where syscalls
are intercepted and any effect that requires root (like chown) is emulated.

However, fakeroot sporadically fails for us, aborting the package build.
Since our install scripts don't really require root (when operating in
the --packaging mode), we can just tell dpkg-buildpackage that we don't
need fakeroot. This ought to fix the sporadic failures.

As a side effect, package builds are faster.

Follows scylla.git's b608af870b0a1ad88b91a72bddeff0c321877f9e.

Refs scylladb/scylla#6655.
2020-07-29 12:53:20 +03:00
Piotr Jastrzebski
c0d9d0f051 add build/ to gitignore
This directory is created by a build and shouldn't be commited so
it's best for git to just ignore it.

Signed-off-by: Piotr Jastrzebski <piotr@scylladb.com>
Message-Id: <b20dba12fb726aebd51b2ab9494e7c52f8058feb.1595259605.git.piotr@scylladb.com>
2020-07-21 09:10:25 +03:00
Pekka Enberg
7578d359af reloc: Add "--builddir" option to build_{rpm,deb}.sh
We need the ability to control build directory in scylla.git build
system. Let's add support for the "--builddir" option like in other
variants of the same scripts.

Message-Id: <20200717085723.701209-1-penberg@scylladb.com>
2020-07-18 12:16:31 +03:00
Avi Kivity
aa94fe53e0 dist: redhat: reduce log spam from unpacking sources when building rpm
rpmbuild defaults to logging the name of every file it unpacks from
the archive. This is quite a lot for Java applications.

Make it quiet with the %setup -q flag.
2020-07-15 12:27:43 +03:00
Pekka Enberg
4727910b5e Merge 'gitignore: fix typo and add scylla-apiclient/target/' from Benny
When building scylla-jmx in place, `scylla-apiclient/target/` is left
behind and should be ignored by `.gitignore` otherwise the scylla
submodule directory appears to be dirty.

* bhalevy-gitignore:
  gitignore: do not track scylla-apiclient/target/
  gitignore: fix typo in dependency-reduced-pom.xml
2020-07-14 10:21:44 +03:00
Pekka Enberg
15eb6adf92 apiclient: Bump Jackson version to 2.10.4
Jackson 2.9.x has various vulnerabilities that are fixed in 2.10 series:

https://github.com/FasterXML/jackson-databind/issues/2700#issuecomment-619590967

Let's update to the latest version of Jackson. This is a similar fix to
Github's Dependabot proposal, except we bump the version number across
all Jackson components:

https://github.com/scylladb/scylla-jmx/pull/116
2020-07-14 10:19:49 +03:00
Takuya ASADA
5820992a8e dist/debian: apply generated package version for .orig.tar.gz file
Same as scylladb/scylla#6752,
we currently does not able to apply version number fixup for .orig.tar.gz file,
even we applied correct fixup on debian/changelog, becuase it just reading
SCYLLA-VERSION-FILE.
We should parse debian/{changelog,control} instead.

Fixes #120
2020-07-06 12:49:15 +03:00
Benny Halevy
38eb871383 gitignore: do not track scylla-apiclient/target/
It is created when building jmx.

Signed-off-by: Benny Halevy <bhalevy@scylladb.com>
2020-06-24 13:59:34 +03:00
Benny Halevy
28fe33e588 gitignore: fix typo in dependency-reduced-pom.xml
Signed-off-by: Benny Halevy <bhalevy@scylladb.com>
2020-06-24 13:59:12 +03:00
Pekka Enberg
b2195734cc Upgrade to Guava 29.0
CVE-2018-10237 impacts Guava 24.1.0 and earlier, so let's upgrade to the latest version.

Reported-by: GitHub and Shlomi Livne
2020-06-16 10:04:48 +03:00
Juliusz Stasiewicz
b2e4796901 Added support for checkAndRepairCdcStreams command 2020-06-15 14:58:13 +03:00
Takuya ASADA
78c3b7627f dist/debian: cleanup build/debian before building .deb
On 52bd496, we stopped to rm -rf debian/ on build_deb.sh, since now we have
prebuilt debian/ directory.
However, it might cause .deb build error when we modified debian package source,
since it never cleanup.

To prevent build error, we need to cleanup build/debian on reloc/build_deb.sh,
before extracting contents from relocatable package.
2020-06-08 18:15:04 +03:00
Takuya ASADA
e0b21b9a19 dist: add --packaging for .rpm/.deb build
354df10 mistakenly does not contain dist/redhat & dist/debian change,
add --package option to them.
2020-06-08 17:45:52 +03:00
Takuya ASADA
2883a8dc63 dist/debian: don't install systemd unit by install.sh, use debian/*.service
Installing *.service by install.sh script causes the error on installing .deb
package, use debian/*.service instead.

Fixes scylladb/scylla#6010
Related scylladb/scylla#5640
Related 29285b28e2
2020-06-08 12:24:03 +03:00
Takuya ASADA
354df10ea9 install.sh: add dependency check and postinst script for manual install
To install scylla-jmx using install.sh easily, we need:
 - run dependency check before install
 - run postinst script after install

But we don't want to run them when we build .rpm/.deb package,
we also need to add --packaging option to skip them.

See scylladb/scylla#5830
2020-06-08 12:22:28 +03:00
Takuya ASADA
3fb777a8f0 dist/debian: support version number containing '_'
.deb packaging system does not support version number contains '_',
it should be replacedwith '-'
2020-06-04 05:27:04 +09:00
Takuya ASADA
f044c8988e dist/debian: move version number fixup to debian_files_gen.py
Now we generate dist/changelog on relocatable package generation time,
we cannot run '.rc' fixup on .deb package building time, need to do it
in debian_files_gen.py.
2020-06-04 05:27:02 +09:00
Takuya ASADA
ec2a830876 reloc-pkg: move all files under project name directory
To make unified relocatable package easily, we may want to merge tarballs to single tarball like this:
zcat .tar.gz | gzip -c > scylla-unified.tar.xz
But it's not possible with current relocatable package format, since there are multiple files conflicts, install.sh, SCYLLA--FILE, dist/, README.md, etc..

To support this, we need to archive everything in the directory when building relocatable package.

See: scylladb/scylla#6315
2020-06-03 09:53:11 +03:00
Amnon Heiman
9628cc0728 StorageService: Add the scrub 3.11 command implementation
The scrub command was not supported from node_tool, but now when we want
to enable it the current API is not compatible with the 3.11 MBean
definition.

This patch adds the definition to the MBean and the implementation to
StorageService.

It also address two problems with the old scrub implementation, just
in case someone will use them.

1. Implementation didn't pass the parameters to the API.
2. A stub implementation called itself instead of calling an actual
implementation.

This patch will enable to test the command from nodetool additional
changes may come on top of it if more command line options will be
supported.

Signed-off-by: Amnon Heiman <amnon@scylladb.com>
2020-05-29 14:12:09 +03:00
Ivan Prisyazhnyy
c7dcbd7f42 fix is auto compaction disabled
align API to the recent changes at https://github.com/scylladb/scylla/pull/6176

don't wrap API exceptions into IOException for enableAutoCompaction
2020-05-29 14:02:40 +03:00
dependabot[bot]
fc43c56369 build(deps): bump jackson-databind in /scylla-apiclient
Bumps [jackson-databind](https://github.com/FasterXML/jackson) from 2.9.10.1 to 2.9.10.4.
- [Release notes](https://github.com/FasterXML/jackson/releases)
- [Commits](https://github.com/FasterXML/jackson/commits)

Signed-off-by: dependabot[bot] <support@github.com>
2020-05-29 14:00:31 +03:00
Takuya ASADA
52bd496006 dist/debian: drop dependency on pystache
Drop dependency on pystache since it nolong present in Fedora 32.

To implement it, simplified debian package build process.
It will be generate debian/ directory when building relocatable package,
we just need to run debuild using the package.

To generate debian/ directory this commit added debian_files_gen.py,
it construct whole directory including control and changelog files
from template files.
Since we need to stop pystache, these template files swiched to
string.Template class which is included python3 standard library.

see: https://github.com/scylladb/scylla/pull/6313
2020-05-23 06:08:01 +03:00
Takuya ASADA
18f8acc60e dist/redhat: drop dependency on pystache
Same as https://github.com/scylladb/scylla/pull/6313, drop dependency on
pystache since it nolong present in Fedora 32.
2020-05-19 08:15:59 +03:00
Takuya ASADA
773a82d539 dist: allow specify JVM options from sysconfig (#93)
Add SCYLLA_JMX_JVM_OPTS on sysconfig to specify JVM options.

Reviewed-by: Ľuboš Koščo <lubos@scylladb.com>

Fixes #58
2020-01-28 12:43:03 +02:00
Takuya ASADA
46681753cd dist: add /usr/lib/scylla/jmx for compatibility (#91)
On the commit 4c8660d, we dropped /usr/lib/scylla/jmx since it likely no user
script invoke scripts under the directory.
However, we found there are possibility scylla-jmx.service tries to load .jar
file from /usr/lib/scylla/jmx, when user upgraded from older version of scylla.
Because /etc/sysconfig/scylla-jmx is marked as 'noreplace' on our rpm,
yum upgrade may keep old sysconfig file when it modified by user, that may
causes to load .jar from /usr/lib/scylla/jmx since we specify the path in the
sysconfig file.

To avoid the issue it's better to have symlinks on /usr/lib/scylla/jmx for
safety.

See #90
2020-01-16 15:51:39 +02:00
Takuya ASADA
29601254fc dist/redhat: call systemctl --daemon-reload when upgraded (#92)
Since %systemd_post does not call systemctl --daemon-reload, we need to call it
manually to apply changes.

Fixes #90
2020-01-08 13:35:38 +02:00
Takuya ASADA
4c8660d41a dist: drop symlink to scripts (#89)
This is scylla-jmx part of https://github.com/scylladb/scylla/pull/5530

After we stopped replacing /usr/lib/scylla with symlink,
creating symlink on /opt/scylladb/scripts/jmx become meaningless,
so we can drop it now.

We able to introduce new symlink on /usr/lib/scylla, but it likely no user
directly invoke scripts under /usr/lib/scylla/jmx, so we don't have to
do that.
2019-12-30 13:51:33 +02:00
Takuya ASADA
2f34a97c6e dist/debian: add AdoptOpenJDK for Debian 10
Since Debian 10 dropped OpenJDK 8, so we need to switch to other
JVM distribution which still maintain OpenJDK 8 for Debian.
We can use AdoptOpenJDK for such environment.

See scylladb/scylla#5463

Signed-off-by: Takuya ASADA <syuu@scylladb.com>
Message-Id: <20191218124838.35017-1-syuu@scylladb.com>
2019-12-18 15:32:04 +02:00
Takuya ASADA
236ffa6c98 dist/debian: add Conflicts with openjdk-11
Since Debian variants can install multiple JRE, "Depends: openjdk-8-jre" may
not mean default JRE is openjdk-8.
We should block openjdk-11 for now, until we support it.

Related with scylladb/scylla#5463

Signed-off-by: Takuya ASADA <syuu@scylladb.com>
Message-Id: <20191218111835.25618-1-syuu@scylladb.com>
2019-12-18 14:08:28 +02:00
Takuya ASADA
771cf6ea50 dist/redhat: force xz compression on rpm binary payload
Same as 301c835cbf,
Fedora 31 switched the default compression to zstd, which isn't readable
by some older rpm distributions (CentOS 7 in particular). Tell it to use
the older xz compression instead, so packages produced on Fedora 31 can
be installed on older distributions.

See: https://github.com/scylladb/scylla/pull/5310

Signed-off-by: Takuya ASADA <syuu@scylladb.com>
Message-Id: <20191210191441.108774-1-syuu@scylladb.com>
2019-12-10 22:18:29 +02:00
Takuya ASADA
31e6bcf9be dist/redhat: fix rpmbuild error on Fedora 31
Same as scylladb/scylla-ami#53, it seems like rpm macro %systemd_postun requires
 one argument starting from Fedora 31, otherwise it causes the error.
The solution is passing systemd unit name just like
%systemd_post and %systemd_preun.

see: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/thread/TU3T2ZYY67SMAJFR2TD4HY6SCPPDVS5V/

Fixes #87

Signed-off-by: Takuya ASADA <syuu@scylladb.com>
Message-Id: <20191205120514.9382-1-syuu@scylladb.com>
2019-12-05 17:03:10 +02:00
Alexandros Bantis
d8c47603d9 Create a HTTP client per instance (#86)
Create javax HTTP client once per instance instead of per request.

Fixes #82
2019-11-19 17:28:09 +02:00
Pekka Enberg
f45ae1833e Add Pekka as a "code owner" on GitHub (#85)
Add myself as a "code owner" so that I am assigned a review
automatically:

https://help.github.com/en/github/creating-cloning-and-archiving-repositories/about-code-owners

I also wanted to add Amnon and Calle, but apparently you need to have
write permissions in order to be a code owner.

The purpose of this automation is to ensure Scylla JMX pull requests
show up in my github.com/pulls page. Thanks Maciej Zimnoch for the tip!
2019-11-14 02:18:33 -08:00
dependabot[bot]
8e1beb11f4 Upgrade jackson-databind from 2.9.9 to 2.9.10.1 (#84)
This upgrades jackson-databind dependency from version 2.9.9 to 2.9.10.1, which fixes various security vulnerabilities:

https://www.cvedetails.com/vulnerability-list/vendor_id-15866/product_id-42991/Fasterxml-Jackson-databind.html
2019-11-13 19:57:55 +02:00
Glauber Costa
27fed6136a Run scylla-jmx in a systemd slice (#79)
Scylla now supports server-defined systemd slices that are used to provide
isolation between components.

This patch adds scylla-jmx to the helper slice. This will guarantee that
scylla-jmx does not use too much resources, influencing the server performance.

Signed-off-by: Glauber Costa <glauber@scylladb.com>
2019-11-13 18:28:14 +02:00
Amos Kong
fa00e84794 build_deb.sh: don't generate scylla-jmx.service from mustache template (#81)
Takuya had untemplataize scylla-jmx.service in commit e8355087ea
But the build_deb.sh still tries to generate service file from a deleted
mustache template file -- scylla-jmx.spec.mustache. It wrongly redirects
a path to service file, then scylla-jmx would fail to start.

Fixes #80
2019-10-01 18:35:50 +03:00
Calle Wilund
f915f8fc7a sstableinfo: Fix deserizalization of "properties"
Refs #76

Since the incoming json uses swagger "key", "value" syntax
we need to do explicit deserialization of this property
as well (not just extended props).

Message-Id: <20190930115432.27801-1-calle@scylladb.com>
2019-09-30 15:29:52 +03:00
Avi Kivity
dc7f37b901 Merge "nonroot installer" from Takuya
"This is nonroot installer patchset v4, for scylla-jmx."

* 'nonroot_v4' of https://github.com/syuu1228/scylla-jmx:
  install.sh: add --nonroot mode
  dist/common/systemd: untemplataize *.service, use drop-in units instead
  dist: move package build script to install.sh
2019-09-10 14:46:04 +03:00
Takuya ASADA
9ef12f4651 install.sh: add --nonroot mode
This implements the way to install Scylla without requires root privilege,
not distribution dependent, does not uses package manager.
2019-09-04 09:54:42 +09:00
Takuya ASADA
e8355087ea dist/common/systemd: untemplataize *.service, use drop-in units instead
Since systemd unit can override parameters using drop-in unit, we don't need
mustache template for them.

Also, drop --disttype option on install.sh since it does not required anymore,
introduce --sysconfdir instead for non-redhat distributions.
2019-09-04 08:54:05 +09:00
Takuya ASADA
a1044e3bd1 dist: move package build script to install.sh
Move package build script from .rpm/.deb to single script, install.sh.
We need this to support nonroot mode, and also to keep package build script
consistent between .rpm/.deb.
2019-09-04 08:30:37 +09:00
Pekka Enberg
04ea3ab7e0 Merge 'Implement sstable_info command' from Calle
"Fixes #76

Implements JMX level call for "sstable_info" REST api command.

Requires seastar patch:
json: Make date formatter use RFC8601/RFC3339 format

Requires scylla patch set "Implement sstable_info API command (info on sstables)"

Forwards call to REST sstable_info and packs the data
into CompositeData for JMX consumption."
* 'sstabledesc' of git://github.com/elcallio/scylla-jmx:
  storage_service: Add "getSSTableInfo" command/attribute
  service: Add objects for deserializing sstable_info json
  scylla-apiclient: Add Date json serializer helper
  APIClient: Add jackson JSON serializer support to client object
  apiclient/pom.xml: Add jackson JSON support libs for REST client
2019-08-13 14:40:25 +03:00
Calle Wilund
133b2e4728 storage_service: Add "getSSTableInfo" command/attribute
Fixes #76

Requires seastar patch:
 json: Make date formatter use RFC8601/RFC3339 format

Requires scylla patch set "Sstabledesc"

Forwards call to REST sstable_info and packs the data
into CompositeData for JMX consumption.
2019-08-06 08:12:14 +00:00
Amnon Heiman
71170f5713 CompactionMetrics: use the pending compaction API (#75)
The PendingTasksByTableName metric should use the pending_tasks_by_table
API to get the real value of the pending compaction.

Fixes #74

Signed-off-by: Amnon Heiman <amnon@scylladb.com>
2019-08-05 14:12:48 +03:00
Amnon Heiman
ff0723abc6 ColumnFamilyStore: Mbean API support the hex format param (#69)
Cassandra 3.0 version of the JMX added a parameter that allows accepting
the parameter as hex.

This breaks the current implementation with a NoSuchMethodException.

This patch adds the missing implementation.

For a full support, a follow up patch in Scylla is needed, but for the
current functionality it would work.

After this patch usage example:

nodetool getsstables keyspace1 standard1 39303138374b4d343830

Signed-off-by: Amnon Heiman <amnon@scylladb.com>
2019-07-29 10:09:04 +03:00
Calle Wilund
cb42205061 service: Add objects for deserializing sstable_info json
Objects + serial logic to automate the transform of
scylla REST json object for sstable_info into
compositedata that can be consumed by nodetool
2019-07-24 14:31:10 +00:00
Calle Wilund
b2f3eeee05 scylla-apiclient: Add Date json serializer helper
To handle RFC8601 formattedd dates in JAXB
2019-07-24 14:30:02 +00:00
Calle Wilund
d8efa60ab7 APIClient: Add jackson JSON serializer support to client object
Allows java ws to deserialize json objects directly.
2019-07-24 14:28:38 +00:00
Calle Wilund
bbc817013e apiclient/pom.xml: Add jackson JSON support libs for REST client 2019-07-24 14:27:56 +00:00
Calle Wilund
263735379e scylla-jmx: Remove depdendency-reduced-pom.xml from tracking
This file is (re-)generated by maven on occasions. It should
not be version controlled.

Add to .gitignore as well.
2019-07-23 11:05:34 +03:00
Amnon Heiman
f0d2df3d15 StorageProxy.java: Add view write metrics
nodetool proxyhistograms command look for the view write metric.

While we do not report that metric yet, we still want the command to
succeed.

After this patch:
$ nodetool proxyhistograms
proxy histograms
Percentile       Read Latency      Write Latency      Range Latency   CAS Read Latency  CAS Write Latency View Write Latency
                     (micros)           (micros)           (micros)           (micros)           (micros)           (micros)
50%                    326.00             110.00             424.50               0.00               0.00               0.00
75%                   1253.00             193.25             877.75               0.00               0.00               0.00
95%                   2935.90            1007.25            5182.55               0.00               0.00               0.00
98%                   3100.00            1040.60            5492.00               0.00               0.00               0.00
99%                   3100.00            1058.00            5492.00               0.00               0.00               0.00
Min                     34.00               9.00              36.00               0.00               0.00               0.00
Max                   3100.00            1058.00            5492.00               0.00               0.00               0.00

See scylladb/scylla#4470

Signed-off-by: Amnon Heiman <amnon@scylladb.com>
Message-Id: <20190514063316.28040-1-amnon@scylladb.com>
2019-07-21 19:20:37 +03:00
Amnon Heiman
c7bce65919 APIMBeanServer: Support both Table and Tables as metric name
Fixes #71

Signed-off-by: Amnon Heiman <amnon@scylladb.com>
2019-07-17 10:56:44 +03:00
Pekka Enberg
4303f06426 Merge "Make API client a separate module for reuse" from Ľuboš
There's an effort to implement a version of "nodetool" that uses
Scylla's REST API directly, Let's make the API client a separate module,
so nodetool can use it.

* 'scylla-apiclient' of https://github.com/tarzanek/scylla-jmx:
  fix README for building instructions
  trigger build from parent maven to have the local repo properly set up
  cleanup commented implicit steps in mvn
  make scylla-apiclient a separate module so the jar can be reused
2019-07-10 22:31:12 +03:00
Lubos Kosco
183eb6158a fix README for building instructions 2019-07-08 11:02:45 +02:00
Lubos Kosco
4296c7d3ae trigger build from parent maven to have the local repo properly set up 2019-07-08 10:54:04 +02:00
Lubos Kosco
222990d821 cleanup commented implicit steps in mvn 2019-07-08 10:04:24 +02:00
Lubos Kosco
91ae4ec8ee make scylla-apiclient a separate module so the jar can be reused 2019-07-01 17:33:08 +02:00
Amnon Heiman
9dae28e2f0 ColumnFamilyStore: finishLocalSampling should respect count limit
When calling nodetool toppartitions with size limit, finishLocalSampling
should respect that and limit the number of the results.

Example:
$ nodetool toppartitions -k 2 keyspace1 standard1 20
WRITES Sampler:
  Cardinality: ~2 (256 capacity)
  Top 2 partitions:
	Partition                Count       +/-
	38333032394d4f4d5030         4         3
	4e353937383137503330         4         3

READS Sampler:
  Cardinality: ~2 (256 capacity)
  Top 2 partitions:
	Nothing recorded during sampling period...

Fixes #66

Signed-off-by: Amnon Heiman <amnon@scylladb.com>
2019-06-23 15:31:18 +03:00
Amnon Heiman
2fac82434b APIClient: delete command should check for errors
delete commands do not return a value, still, it is possible that the
command will return a value different than OK.

In such a case, the error should be propagate to the caller via an
exception.

Fixes #65

Signed-off-by: Amnon Heiman <amnon@scylladb.com>
Message-Id: <20190618135312.2776-1-amnon@scylladb.com>
2019-06-18 18:56:30 +03:00
Takuya ASADA
eda6f4e1a8 dist/debian: run 'systemctl daemon-reload' automatically on package install/uninstall
Since we cannot use dh --with=systemd because we don't want to
automatically enabling systemd units, manage them by our setup scripts,
we have to do 'systemctl daemon-reload' manually.
(On dh --with=systemd, systemd helper automatically provides such
scirpts)

See scylladb/scylla-enterprise#825

Signed-off-by: Takuya ASADA <syuu@scylladb.com>
Message-Id: <20190618000414.29142-1-syuu@scylladb.com>
2019-06-18 15:46:29 +03:00
Takuya ASADA
f73da49f62 dist: merge /usr/lib/scylla to /opt/scylladb
Since scylla-jmx uses /usr/lib/scylla/jmx for program directory, we also
need to move them under /opt/scylladb.

Signed-off-by: Takuya ASADA <syuu@scylladb.com>
Message-Id: <20190618122451.27721-1-syuu@scylladb.com>
2019-06-18 15:43:53 +03:00
Calle Wilund
512638ed6e APIMBeanServer: Handle nodeprobe wildcard queries in CF refresh
Fixes #63
Message-Id: <20190311082942.3310-2-calle@scylladb.com>
2019-05-05 18:10:37 +03:00
Calle Wilund
5f974bc2bb ColumnFamilyStore: Propapgate exception cause in sampling wait
Message-Id: <20190311082942.3310-1-calle@scylladb.com>
2019-05-05 18:10:37 +03:00
Takuya ASADA
5e50090bfd dist: merge product name parameter on single place
When we add product name customization, we mistakenly defined the
parameter on each package build script.
Number of script is increasing since we recently added relocatable
python3 package, we should merge it in single place.

Also we should save the parameter on relocatable package, just like
version-release parameters.

So move the definition to SCYLLA-VERSION-GEN, save it to
build/SCYLLA-PRODUCT-FILE then archive it to relocatable package.

Signed-off-by: Takuya ASADA <syuu@scylladb.com>
Message-Id: <20190422105304.23454-1-syuu@scylladb.com>
2019-04-22 13:55:53 +03:00
Piotr Jastrzebski
cb1ac4a58c Capture heap dump on OutOfMemoryError
Signed-off-by: Piotr Jastrzebski <piotr@scylladb.com>
Message-Id: <ae90bb6e4b18f51376d2e7009078bf8c8e6ed7fd.1552407088.git.piotr@scylladb.com>
2019-03-26 16:24:35 +02:00
Calle Wilund
da21305989 StorageService: Include the arguments in "upgrade" call.
Message-Id: <20190219133431.29009-1-calle@scylladb.com>
2019-02-27 10:33:43 +02:00
Amnon Heiman
27313ee2c4 ColumnFamilyStore: Add an implementation for table sampling
This patch adds the implementation for begin and finish local sampling
of a column family.

There is a difference in the implementation of Cassandra API and Scylla.

In Cassandra and the JMX an external source start and stop the sampling.

In Scylla, a single API call start the sampling and return with the
result. In Scylla the API call always return sampling of the read and of
the writes.

To bridge the difference, the begin sampling command will use a Future
when calling the API. The finish method will wait for the future to end.

Because of the different implementation, it is possible that two
consecutive calls will be made to start sampling one for the read and
one for the write, similarly, two calls will be made to finish for read
and write.

The implementation would ignore the second call to start and will
store the result, so the second call to finish will be served from the
stored result.

Note, that the use of future is only for safety, the way we expect it to
work, the caller to the begin sampling will sleep anyhow while waiting
for the result.

To avoid breaking the MBean compatibility we piggyback the duration on
top of the sampler string.

If no duration is given, a default duration will be taken, this is also
just as a precaution, we will modify the nodetool implementation to
pass that information.

There is a known issue with cardinality, that will need to be addressed.
Also we return a value in the raw column to match what Cassandra JMX
returns, but it's a duplication of the partition key.

See scylladb/scylla#2811

Signed-off-by: Amnon Heiman <amnon@scylladb.com>
Message-Id: <20190128143505.5241-1-amnon@scylladb.com>
2019-02-03 12:40:04 +02:00
Takuya ASADA
d4493295ff dist/debian: skip running dh_strip_nondeterminism
On some Fedora environment dh build tries to run
dh_strip_nondeterminism, and fails sice Fedora does not provide such
command.

To prevent the build error we need to skip it.

Fixes #62
See 5bf9a03d65

Signed-off-by: Takuya ASADA <syuu@scylladb.com>
Message-Id: <20190125223351.20381-1-syuu@scylladb.com>
2019-01-28 09:12:50 +02:00
Calle Wilund
9eec9eabf6 scylla-jmx: Make scylla-jmx compatible with jdk9+
Adds explicit maven dependecies for libraries
removed from JDK.
Removes reflection calls forbidden in jdk9+.

Message-Id: <20181120142550.22852-1-calle@scylladb.com>
2018-11-21 13:00:24 +02:00
Takuya ASADA
854e6072a2 dist/redhat: prevent build error on older Fedora/CentOS
Current scylla.spec fails build on Fedora 27, since python2-pystache is
new package name that renamed on Fedora 28.
But Fedora 28's python2-pystache has tag "Provides: pystache",
so we can depends on old package name, this way we can build scylla.spec both
on Fedora 27/28.

Signed-off-by: Takuya ASADA <syuu@scylladb.com>
Message-Id: <20181028175757.32224-1-syuu@scylladb.com>
2018-10-29 11:37:25 +02:00
Takuya ASADA
21e22d4e1a dist/redhat: minor fixes for relocatable .rpm
- we don't use mock anymore, so drop mock directory
- build_rpm.sh usage need to update
- build_rpm.sh should install rpmbuild

Signed-off-by: Takuya ASADA <syuu@scylladb.com>
Message-Id: <20181024222136.3332-2-syuu@scylladb.com>
2018-10-25 10:29:30 +03:00
Takuya ASADA
9ed8a01519 dist/debian: minor fixes for relocatable .deb
- we don't use pbuilder anymore, so drop pbuilderrc
- on build_deb.sh is_debian / is_ubuntu functions are unused now, drop them

Signed-off-by: Takuya ASADA <syuu@scylladb.com>
Message-Id: <20181024222136.3332-1-syuu@scylladb.com>
2018-10-25 10:29:27 +03:00
Avi Kivity
df2dee2402 Merge "relocatable package support for jmx" from Takuya
"
This patchset adds relocatable package support for scylla-jmx, and also
support generating .rpm/.deb from relocatable package.

 - Scripts are based on relocatable .rpm/.deb support patchset for main repo
   (not merged https://github.com/syuu1228/scylla/tree/reloc_rpmdeb_v4)
 - Single .rpm package provided for CentOS7/Fedora(unofficial)
 - Single .deb package provided for Ubuntu 14/16/18, Debian 8/9
"

* 'reloc_v1' of https://github.com/syuu1228/scylla-jmx:
  dist/debian: use relocatable package to produce .deb
  dist/redhat: use relocatable package to produce .rpm
  reloc: add support relocatable package
2018-10-24 11:51:34 +03:00
Takuya ASADA
e9bfbedfed dist/debian: use relocatable package to produce .deb 2018-10-24 11:45:11 +09:00
Takuya ASADA
5edfedf642 dist/redhat: use relocatable package to produce .rpm 2018-10-24 02:04:49 +00:00
Takuya ASADA
92847e3381 reloc: add support relocatable package
To align build system with scylla main repo, adding relocatable package
support.

On scylla-jmx, we don't provide libraries and linker since it's Java
program, just contains .jar file and dist/ directory.
2018-10-24 02:02:25 +00:00
Calle Wilund
ca3fa8de20 scylla-jmx: Fix tablemetricsobjectname breakage
Fixes #57

The usage of TableMetricsObjectName-yada-yada relies on translating the
"fake" objectname to a canonical one on remote
publication/serialization. However, the implementation of
ObjectName.getInstance has changed in JDK (micro) updates so it no
longer applies overridable methods -> wrong name published.

Fix by doing explicit ObjectName instansiation.
Message-Id: <20181023132005.23099-1-calle@scylladb.com>
2018-10-23 16:30:29 +03:00
Takuya ASADA
74fa1a40ca dist/debian: install GPG key for cross-building
We found on some Debian environment Ubuntu .deb build fails with
gpg error because lack of Ubuntu GPG key, so we need to install it before
start pbuilder.
Same as on Ubuntu, it needs to install Debian GPG key.

See scylladb/scylla#3823

Signed-off-by: Takuya ASADA <syuu@scylladb.com>
Message-Id: <20181008110724.18335-1-syuu@scylladb.com>
2018-10-08 15:34:18 +03:00
Takuya ASADA
cd1c79f90f dist/debian: support package renaming on build script
To automatically rename packages on enterprise release, added package name
prefix as a variable on build_deb.sh.

Signed-off-by: Takuya ASADA <syuu@scylladb.com>
Message-Id: <20180828010445.11920-2-syuu@scylladb.com>
2018-08-28 09:26:02 +03:00
Takuya ASADA
a27d9601f5 dist/redhat: support package renaming on build script
To automatically rename packages on enterprise release, added package name
prefix as a variable on build_rpm.sh.

Signed-off-by: Takuya ASADA <syuu@scylladb.com>
Message-Id: <20180822072105.9420-2-syuu@scylladb.com>
2018-08-22 11:05:00 +03:00
Calle Wilund
c6aee9f63e scylla-jmx: Add "PendingTasksByTableName" gauge to CompactionMetrics
Required by origin 3.11 nodetool.

Message-Id: <20180801084545.23239-1-calle@scylladb.com>
2018-08-01 14:25:06 +03:00
Calle Wilund
9c3ac3e547 scylla-jmx: Update JMX interfaces to origin 3.11
Almost 100% null implementations, which is ok for most purposes
currently used by scylla. Some of these new calls (like dropped
mutations etc) should perhaps however be implemented.

Tested with the nodetool dtests. So sparsely.

Needed when/if scylla-tools-java is upgraded to origin 3.11,
otherwise noodtool breaks.

Message-Id: <20180730113741.14952-1-calle@scylladb.com>
2018-07-30 15:47:43 +03:00
Avi Kivity
b4d983b45a dist: redhat: fix up bad file ownership of rpms/srpms
mock outputs files owned by root. This causes attempts
by scripts that want to junk the working directory (typically
continuous integration) to fail on permission errors.

Fixup those permissions after the fact.
Message-Id: <20180719163258.4393-1-avi@scylladb.com>
2018-07-26 08:17:59 +03:00
Takuya ASADA
d6c408445e dist/debian/build_deb.sh: make build_deb.sh more simplified
Use is_debian()/is_ubuntu() to detect target distribution, also install
pystache by path since package name is different between Fedora and
CentOS.

Signed-off-by: Takuya ASADA <syuu@scylladb.com>
Message-Id: <20180703183211.3455-1-syuu@scylladb.com>
2018-07-04 10:12:31 +03:00
Takuya ASADA
2af17c1f53 dist: simplified build script templates
Currently we are using *.in files for templates, applying parameters by sed
command one-by-one.
This patch will replace them by Mustache, it's simple and easy syntax template
language.

Signed-off-by: Takuya ASADA <syuu@scylladb.com>
Message-Id: <20180606183113.25275-1-syuu@scylladb.com>
2018-06-10 19:37:26 +03:00
Takuya ASADA
b8394f677b dist/debian: run update-ca-certificates to avoid security exception on Ubuntu 18.04
On Ubuntu 18.04 build fails by java.security.InvalidAlgorithmParameterException,
while downloading .pom file from HTTPS URL.
Looks like it's ca-certs problem, can fix by running update-ca-certificates.

Fixed #52

Signed-off-by: Takuya ASADA <syuu@scylladb.com>
Message-Id: <20180601092954.28319-1-syuu@scylladb.com>
2018-06-03 10:51:11 +03:00
Piotr Jastrzebski
1ad2ba8507 TableRepository: wrap initial repository
Before we were discarding the initial repository while
overriding it with TableRepository. This was a mistake that
caused dtests to fail. Proper solution is to keep the initial
repository inside TableRepository. That way whatever was registered
at the time of JmxMBeanServer creation is still handled properly.

Signed-off-by: Piotr Jastrzebski <piotr@scylladb.com>
Message-Id: <22181859012fd20ddf37e049a145bc94a3a91a33.1527844328.git.piotr@scylladb.com>
2018-06-02 20:42:00 +03:00
Avi Kivity
71f857b1de Merge "Reduce memory usage and speed up start time" from Piotr
"
Use more memory efficient MBeans repository and remove quadratic behaviour on startup.

This reduces memory usage for 2000 tables from 127M to 82M and reduces start time
from 270 seconds to 2 seconds.

Changes since last version:
1. Fix registered map to handle multiple JMX servers and to properly deregister mbeans
2. Clean up TableRepository code.
"

* 'speedup_2' of https://github.com/haaawk/scylla-jmx:
  Use more efficient MBeans repository
  Remove unnecessary quadratic algorithm from MetricsMBean.register
2018-05-21 11:13:15 +03:00
Avi Kivity
e27312df10 Merge "Reduce memory usage" from Piotr
"
Introduce more memory efficient version of ObjectName and use it in JMX Server.
The original version stores the same data multiple times in different forms.
Big part of data is shared by multiple instances of ObjectName.
Original class keeps a separate copy for each instance.
The new version keeps only one copy that's shared by all instances.
"

* 'speedup_1' of https://github.com/haaawk/scylla-jmx:
  Introduce and use TableMetricObjectName
  Ensure regular ObjectName is returned to remote callers
  Use JmxMBeanServer instead of MBeanServer
2018-05-21 11:10:22 +03:00
Piotr Jastrzebski
862aea4a33 Use more efficient MBeans repository
Default implementation stores MBeans in the following map:

<domain name> -> (<properties as a single string> -> NamedObject)

This is problematic because NamedObject contains ObjectName that
has both domain and properties inside itself.

This means we're storing the same data twice.

For domain "" we want to store MBeans in a more compact way using map:

ObjectName -> DynamicMBean

which is equivalent to NamedObject.

Signed-off-by: Piotr Jastrzebski <piotr@scylladb.com>
2018-05-16 16:53:09 +02:00
Piotr Jastrzebski
5cba016962 Remove unnecessary quadratic algorithm from MetricsMBean.register
Before this change it was taking JMX Server 270 seconds to start
when Scylla had 2000 tables. After the change it takes only 2 seconds.

Signed-off-by: Piotr Jastrzebski <piotr@scylladb.com>
2018-05-16 16:21:21 +02:00
Piotr Jastrzebski
455f5717ea Introduce and use TableMetricObjectName
This is a new extention of ObjectName that uses less memory.

TableMetricNameFactory and AllTableMetricNameFactory can
create it instead of regular ObjectName to save memory.

It is possible to save memory because each name created by
TableMetricNameFactory (or AllTableMetricNameFactory) shares
most of its data with other names created by the same factory
and there's no need to create multiple copies.

Signed-off-by: Piotr Jastrzebski <piotr@scylladb.com>
2018-05-12 19:08:37 +02:00
Piotr Jastrzebski
48408dc6a3 Ensure regular ObjectName is returned to remote callers
Next patch will introduce new ObjectName implementation that
will use less memory. This new object won't be serializable.
This means it won't be possible to transport it to a remote
caller. We want to keep this new object local to JMX server as well.

This patch makes sure that every ObjectName returned
from APIBeanServer is transformed into a regular ObjectName.

It also makes sure that every ObjectInstance returned from
APIBeanServer has its ObjectName swapped with a regular ObjectName.

Signed-off-by: Piotr Jastrzebski <piotr@scylladb.com>
2018-05-12 18:54:38 +02:00
Piotr Jastrzebski
2c48bab91a Use JmxMBeanServer instead of MBeanServer
JmxMBeanServer is a concrete implementation of a MBeanServer.
We want to use it directly because we need to bypass calls to
JmxMBeanServer.registerMBean and JmxMBeanServer.unregisterMBean.
They take ObjectName as parameter, copy it using
ObjectName.getInstance(ObjectName) and pass it to registerMBean
and unregisterMBean of JmxMBeanServer.getMBeanServerInterceptor().
We want to avoid this copy and pass the ObjectName argument directly
to JmxMBeanServer.getMBeanServerInterceptor().

To do that this patch:
1. changes all MBeanServer variables to JmxMBeanServer
2. creates JmxMBeanServer in APIBuilder making sure accessing
   interceptors is allowed
3. makes sure that JmxMBeanServer.getMBeanServerInterceptor().registerMBean
   is called wherever JmxMBeanServer.registerMBean was called
4. makes sure that JmxMBeanServer.getMBeanServerInterceptor().unregisterMBean
   is called whenever JmxMBeanServer.unregisterMBean was called

Next patch will use different ObjectName implementation that will
use less memory and this patch is crucial because without it every ObjectName
is transformed with ObjectName.getInstance which turns the object into
a regular ObjectName.

Signed-off-by: Piotr Jastrzebski <piotr@scylladb.com>
2018-05-12 18:35:18 +02:00
Avi Kivity
dd8d5c87ed dist: recognize epel-7-x86_64 mock target and enable networking
The default epel-7-x86_64 wisely disables networking, however our
maven build accesses the maven repository during the build process.

Recognize the target name and redirect it to our networking-enabled
configuration.

Message-Id: <20180408122138.16672-1-avi@scylladb.com>
2018-04-09 11:18:34 +03:00
Duarte Nunes
55abaa1bc8 StorageService: Allow querying the view build status
Signed-off-by: Duarte Nunes <duarte@scylladb.com>
Message-Id: <20180327002342.11494-1-duarte@scylladb.com>
2018-04-03 14:43:27 +03:00
Amnon Heiman
4e4589ba6f FailureDetector: check that states is not null before use
When a node is part of a cluster but is down (like in the situation where
a cluster is taken down and up again but not all nodes are up). There is
no application_state information for that node.

This patch check that the information exists before using it to prevent
null pointer exception.

After this patch, a call to nodetool gossipinfo would return the
available information without failing.

See scylladb/scylla#3330

Signed-off-by: Amnon Heiman <amnon@scylladb.com>
Message-Id: <20180329115345.29357-1-amnon@scylladb.com>
2018-03-29 15:18:48 +03:00
Avi Kivity
8d499401f0 Revert "dist: Rename package to scylla-enterprise-jmx"
This reverts commit df65f40bcd. Committed on
wrong branch.
2018-03-27 22:17:43 +03:00
Pekka Enberg
df65f40bcd dist: Rename package to scylla-enterprise-jmx
This patch renames all the packages generated from this repository from
"scylla" prefix to "scylla-enterprise" prefix. The changes are ported
from the 2017.1 enterprise repository.

Message-Id: <20180326132049.32320-1-penberg@scylladb.com>
2018-03-26 16:26:07 +03:00
Takuya ASADA
3c3d7ba8a7 dist/debian: support Ubuntu 18.04
We supported Ubuntu 18.04 on scylla-server, need to support on jmx/tools too.

Signed-off-by: Takuya ASADA <syuu@scylladb.com>
Message-Id: <1521189263-17592-1-git-send-email-syuu@scylladb.com>
2018-03-19 11:28:12 +02:00
74 changed files with 2923 additions and 748 deletions

1
.github/CODEOWNERS vendored Normal file
View File

@ -0,0 +1 @@
* @penberg

9
.gitignore vendored Normal file
View File

@ -0,0 +1,9 @@
/target/
/bin/
dependency-reduced-pom.xml
scylla-apiclient/target/
.classpath
.project
.settings
build/
/.idea/

View File

@ -7,7 +7,7 @@ Scylla JMX server implements the Apache Cassandra JMX interface for compatibilit
To compile JMX server, run:
```console
$ mvn package
$ mvn --file scylla-jmx-parent/pom.xml package
```
## Running

View File

@ -1,5 +1,6 @@
#!/bin/sh
PRODUCT=scylla
VERSION=666.development
if test -f version
@ -7,13 +8,42 @@ then
SCYLLA_VERSION=$(cat version | awk -F'-' '{print $1}')
SCYLLA_RELEASE=$(cat version | awk -F'-' '{print $2}')
else
DATE=$(date +%Y%m%d)
DATE=$(date --utc +%Y%m%d)
GIT_COMMIT=$(git log --pretty=format:'%h' -n 1)
SCYLLA_VERSION=$VERSION
SCYLLA_RELEASE=$DATE.$GIT_COMMIT
fi
usage() {
echo "usage: $0"
echo " [--version product-version-release] # override p-v-r"
exit 1
}
OVERRIDE=
while [[ $# > 0 ]]; do
case "$1" in
--version)
OVERRIDE="$2"
shift 2
;;
*)
usage
;;
esac
done
if [[ -n "$OVERRIDE" ]]; then
# regular expression for p-v-r: alphabetic+dashes for product, trailing non-dashes
# for release, everything else for version
RE='^([-a-z]+)-(.+)-([^-]+)$'
PRODUCT="$(sed -E "s/$RE/\\1/" <<<"$OVERRIDE")"
SCYLLA_VERSION="$(sed -E "s/$RE/\\2/" <<<"$OVERRIDE")"
SCYLLA_RELEASE="$(sed -E "s/$RE/\\3/" <<<"$OVERRIDE")"
fi
echo "$SCYLLA_VERSION-$SCYLLA_RELEASE"
mkdir -p build
echo "$SCYLLA_VERSION" > build/SCYLLA-VERSION-FILE
echo "$SCYLLA_RELEASE" > build/SCYLLA-RELEASE-FILE
echo "$PRODUCT" > build/SCYLLA-PRODUCT-FILE

View File

@ -20,10 +20,13 @@ SCYLLA_CONF=/etc/scylla
#SCYLLA_JMX_FILE="-cf /etc/scylla.d/scylla-user.cfg"
# The location of the jmx proxy jar file
SCYLLA_JMX_LOCAL="-l /usr/lib/scylla/jmx"
SCYLLA_JMX_LOCAL="-l /opt/scylladb/jmx"
# allow to run remotely
#SCYLLA_JMX_REMOTE="-r"
# allow debug
#SCYLLA_JMX_DEBUG="-d"
# specify JVM options
JAVA_TOOL_OPTIONS=""

18
dist/common/systemd/scylla-jmx.service vendored Normal file
View File

@ -0,0 +1,18 @@
[Unit]
Description=Scylla JMX
Requires=scylla-server.service
After=scylla-server.service
[Service]
Type=simple
EnvironmentFile=/etc/sysconfig/scylla-jmx
User=scylla
Group=scylla
ExecStart=/opt/scylladb/jmx/scylla-jmx $SCYLLA_JMX_PORT $SCYLLA_API_PORT $SCYLLA_API_ADDR $SCYLLA_JMX_ADDR $SCYLLA_JMX_FILE $SCYLLA_JMX_LOCAL $SCYLLA_JMX_REMOTE $SCYLLA_JMX_DEBUG
KillMode=process
Restart=on-abnormal
Slice=scylla-helper.slice
WorkingDirectory=/var/lib/scylla
[Install]
WantedBy=multi-user.target

View File

@ -1,16 +0,0 @@
[Unit]
Description=Scylla JMX
Requires=scylla-server.service
After=scylla-server.service
[Service]
Type=simple
EnvironmentFile=@@SYSCONFDIR@@/scylla-jmx
User=scylla
Group=scylla
ExecStart=/usr/lib/scylla/jmx/scylla-jmx $SCYLLA_JMX_PORT $SCYLLA_API_PORT $SCYLLA_API_ADDR $SCYLLA_JMX_ADDR $SCYLLA_JMX_FILE $SCYLLA_JMX_LOCAL $SCYLLA_JMX_REMOTE $SCYLLA_JMX_DEBUG
KillMode=process
Restart=on-abnormal
[Install]
WantedBy=multi-user.target

View File

@ -1,126 +0,0 @@
#!/bin/bash -e
. /etc/os-release
print_usage() {
echo "build_deb.sh -target <codename>"
echo " --target target distribution codename"
exit 1
}
TARGET=
while [ $# -gt 0 ]; do
case "$1" in
"--target")
TARGET=$2
shift 2
;;
*)
print_usage
;;
esac
done
is_redhat_variant() {
[ -f /etc/redhat-release ]
}
is_debian_variant() {
[ -f /etc/debian_version ]
}
pkg_install() {
if is_redhat_variant; then
sudo yum install -y $1
elif is_debian_variant; then
sudo apt-get install -y $1
else
echo "Requires to install following command: $1"
exit 1
fi
}
if [ ! -e dist/debian/build_deb.sh ]; then
echo "run build_deb.sh in top of scylla dir"
exit 1
fi
if [ "$(arch)" != "x86_64" ]; then
echo "Unsupported architecture: $(arch)"
exit 1
fi
if [ -e debian ] || [ -e build/release ]; then
sudo rm -rf debian build
mkdir build
fi
if is_debian_variant; then
sudo apt-get -y update
fi
# this hack is needed since some environment installs 'git-core' package, it's
# subset of the git command and doesn't works for our git-archive-all script.
if is_redhat_variant && [ ! -f /usr/libexec/git-core/git-submodule ]; then
sudo yum install -y git
fi
if [ ! -f /usr/bin/git ]; then
pkg_install git
fi
if [ ! -f /usr/bin/python ]; then
pkg_install python
fi
if [ ! -f /usr/sbin/pbuilder ]; then
pkg_install pbuilder
fi
if [ ! -f /usr/bin/mvn ]; then
pkg_install maven
fi
if [ ! -f /usr/bin/dh_testdir ]; then
pkg_install debhelper
fi
if [ -z "$TARGET" ]; then
if is_debian_variant; then
if [ ! -f /usr/bin/lsb_release ]; then
pkg_install lsb-release
fi
TARGET=`lsb_release -c|awk '{print $2}'`
else
echo "Please specify target"
exit 1
fi
fi
VERSION=$(./SCYLLA-VERSION-GEN)
SCYLLA_VERSION=$(cat build/SCYLLA-VERSION-FILE | sed 's/\.rc/~rc/')
SCYLLA_RELEASE=$(cat build/SCYLLA-RELEASE-FILE)
echo $VERSION > version
./scripts/git-archive-all --extra version --force-submodules --prefix scylla-jmx ../scylla-jmx_$SCYLLA_VERSION-$SCYLLA_RELEASE.orig.tar.gz
cp -a dist/debian/debian debian
cp dist/debian/changelog.in debian/changelog
cp dist/debian/rules.in debian/rules
sed -i -e "s/@@VERSION@@/$SCYLLA_VERSION/g" debian/changelog
sed -i -e "s/@@RELEASE@@/$SCYLLA_RELEASE/g" debian/changelog
sed -i -e "s/@@CODENAME@@/$TARGET/g" debian/changelog
if [ "$TARGET" = "trusty" ] || [ "$TARGET" = "xenial" ] || [ "$TARGET" = "yakkety" ] || [ "$TARGET" = "zesty" ] || [ "$TARGET" = "artful" ]; then
sed -i -e "s/@@REVISION@@/0ubuntu1~$TARGET/g" debian/changelog
else
sed -i -e "s/@@REVISION@@/1~$TARGET/g" debian/changelog
fi
if [ "$TARGET" = "trusty" ]; then
sed -i -e "s/@@DH_INSTALLINIT@@/--upstart-only/g" debian/rules
else
sed -i -e "s/@@DH_INSTALLINIT@@//g" debian/rules
fi
cp dist/common/systemd/scylla-jmx.service.in debian/scylla-jmx.service
sed -i -e "s#@@SYSCONFDIR@@#/etc/default#g" debian/scylla-jmx.service
cp ./dist/debian/pbuilderrc ~/.pbuilderrc
sudo rm -fv /var/cache/pbuilder/scylla-jmx-$TARGET.tgz
sudo -E DIST=$TARGET /usr/sbin/pbuilder clean
sudo -E DIST=$TARGET /usr/sbin/pbuilder create
sudo -E DIST=$TARGET /usr/sbin/pbuilder update
if [ "$TARGET" = "jessie" ]; then
echo "apt-get install -y -t jessie-backports ca-certificates-java" > build/jessie-pkginst.sh
chmod a+rx build/jessie-pkginst.sh
sudo -E DIST=$TARGET /usr/sbin/pbuilder execute build/jessie-pkginst.sh
fi
sudo -E DIST=$TARGET pdebuild --buildresult build/debs

View File

@ -1,4 +1,4 @@
scylla-jmx (@@VERSION@@-@@RELEASE@@-@@REVISION@@) @@CODENAME@@; urgency=medium
%{product}-jmx (%{version}-%{release}-%{revision}) %{codename}; urgency=medium
* Initial release.

View File

@ -1,14 +1,14 @@
Source: scylla-jmx
Source: %{product}-jmx
Maintainer: Takuya ASADA <syuu@scylladb.com>
Homepage: http://scylladb.com
Section: database
Priority: optional
Standards-Version: 3.9.5
Build-Depends: debhelper (>= 9), maven, openjdk-8-jdk-headless
Rules-Requires-Root: no
Package: scylla-jmx
Package: %{product}-jmx
Architecture: all
Depends: ${shlibs:Depends}, ${misc:Depends}, openjdk-8-jre-headless | openjdk-8-jre | oracle-java8-set-default, scylla-server
Depends: ${shlibs:Depends}, ${misc:Depends}, openjdk-8-jre-headless | openjdk-8-jre | oracle-java8-set-default | adoptopenjdk-8-hotspot-jre | openjdk-11-jre-headless | openjdk-11-jre |oracle-java11-set-default , %{product}-server
Description: Scylla JMX server binaries
Scylla is a highly scalable, eventually consistent, distributed,
partitioned row DB.

23
dist/debian/debian/rules vendored Executable file
View File

@ -0,0 +1,23 @@
#!/usr/bin/make -f
include /usr/share/dpkg/pkg-info.mk
override_dh_auto_build:
override_dh_auto_clean:
override_dh_auto_install:
dh_auto_install
cd scylla-jmx; ./install.sh --packaging --root "$(CURDIR)/debian/tmp" --sysconfdir /etc/default
override_dh_installinit:
ifeq ($(DEB_SOURCE),scylla-jmx)
dh_installinit --no-start
else
dh_installinit --no-start --name scylla-jmx
endif
override_dh_strip_nondeterminism:
%:
dh $@

4
dist/debian/debian/scylla-jmx.install vendored Normal file
View File

@ -0,0 +1,4 @@
etc/default/scylla-jmx
etc/systemd/system/scylla-jmx.service.d/sysconfdir.conf
opt/scylladb/jmx/*
usr/lib/scylla/jmx/*

View File

@ -0,0 +1,7 @@
#!/bin/sh
if [ -d /run/systemd/system ]; then
systemctl --system daemon-reload >/dev/null || true
fi
#DEBHELPER#

7
dist/debian/debian/scylla-jmx.postrm vendored Normal file
View File

@ -0,0 +1,7 @@
#!/bin/sh
if [ -d /run/systemd/system ]; then
systemctl --system daemon-reload >/dev/null || true
fi
#DEBHELPER#

1
dist/debian/debian/scylla-jmx.service vendored Symbolic link
View File

@ -0,0 +1 @@
../../common/systemd/scylla-jmx.service

View File

@ -1,21 +0,0 @@
# scylla-jmx - ScyllaDB
#
# ScyllaDB
description "ScyllaDB jmx"
start on started scylla-server
stop on stopping scylla-server
umask 022
console log
setuid scylla
setgid scylla
script
. /etc/default/scylla-jmx
export SCYLLA_HOME SCYLLA_CONF
exec /usr/lib/scylla/jmx/scylla-jmx -l /usr/lib/scylla/jmx
end script

80
dist/debian/debian_files_gen.py vendored Executable file
View File

@ -0,0 +1,80 @@
#!/usr/bin/env python3
# -*- coding: utf-8 -*-
#
# Copyright (C) 2020 ScyllaDB
#
#
# This file is part of Scylla.
#
# Scylla is free software: you can redistribute it and/or modify
# it under the terms of the GNU Affero General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# Scylla is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with Scylla. If not, see <http://www.gnu.org/licenses/>.
#
import string
import os
import shutil
import re
from pathlib import Path
class DebianFilesTemplate(string.Template):
delimiter = '%'
scriptdir = os.path.dirname(__file__)
with open(os.path.join(scriptdir, 'changelog.template')) as f:
changelog_template = f.read()
with open(os.path.join(scriptdir, 'control.template')) as f:
control_template = f.read()
with open('build/SCYLLA-PRODUCT-FILE') as f:
product = f.read().strip()
with open('build/SCYLLA-VERSION-FILE') as f:
version = f.read().strip().replace('.rc', '~rc').replace('_', '-')
with open('build/SCYLLA-RELEASE-FILE') as f:
release = f.read().strip()
if os.path.exists('build/debian/debian'):
shutil.rmtree('build/debian/debian')
shutil.copytree('dist/debian/debian', 'build/debian/debian')
if product != 'scylla':
for p in Path('build/debian/debian').glob('scylla-*'):
# pat1: scylla-server.service
# -> scylla-enterprise-server.scylla-server.service
# pat2: scylla-server.scylla-fstrim.service
# -> scylla-enterprise-server.scylla-fstrim.service
# pat3: scylla-conf.install
# -> scylla-enterprise-conf.install
if m := re.match(r'^scylla(-[^.]+)\.service$', p.name):
p.rename(p.parent / f'{product}{m.group(1)}.{p.name}')
elif m := re.match(r'^scylla(-[^.]+\.scylla-[^.]+\.[^.]+)$', p.name):
p.rename(p.parent / f'{product}{m.group(1)}')
else:
p.rename(p.parent / p.name.replace('scylla', product, 1))
s = DebianFilesTemplate(changelog_template)
changelog_applied = s.substitute(product=product, version=version, release=release, revision='1', codename='stable')
s = DebianFilesTemplate(control_template)
control_applied = s.substitute(product=product)
with open('build/debian/debian/changelog', 'w') as f:
f.write(changelog_applied)
with open('build/debian/debian/control', 'w') as f:
f.write(control_applied)

View File

@ -1,26 +0,0 @@
USENETWORK=yes
BUILD_HOME=/tmp
BASETGZ="/var/cache/pbuilder/scylla-jmx-$DIST.tgz"
DISTRIBUTION="$DIST"
BUILDRESULT="/var/cache/pbuilder/scylla-jmx-$DIST/result/"
APTCACHE="/var/cache/pbuilder/scylla-jmx-$DIST/aptcache/"
ALLOWUNTRUSTED=yes
if [ "$DIST" = "trusty" ] || [ "$DIST" = "xenial" ] || [ "$DIST" = "yakkety" ] || [ "$DIST" = "zesty" ] || [ "$DIST" = "artful" ]; then
MIRRORSITE="http://archive.ubuntu.com/ubuntu/"
COMPONENTS="main restricted universe multiverse"
DEBOOTSTRAPOPTS="--keyring=/usr/share/keyrings/ubuntu-archive-keyring.gpg"
if [ "$DIST" = "trusty" ]; then
OTHERMIRROR="deb http://ppa.launchpad.net/openjdk-r/ppa/ubuntu trusty main"
fi
elif [ "$DIST" = "jessie" ] || [ "$DIST" = "stretch" ] || [ "$DIST" = "buster" ] || [ "$DIST" = "sid" ]; then
MIRRORSITE="http://deb.debian.org/debian/"
COMPONENTS="main contrib non-free"
DEBOOTSTRAPOPTS="--keyring=/usr/share/keyrings/debian-archive-keyring.gpg"
if [ "$DIST" = "jessie" ]; then
OTHERMIRROR="deb http://httpredir.debian.org/debian jessie-backports main"
fi
else
echo "Unknown distribution: $DIST"
exit 1
fi

32
dist/debian/rules.in vendored
View File

@ -1,32 +0,0 @@
#!/usr/bin/make -f
DOC = $(CURDIR)/debian/scylla-jmx/usr/share/doc/scylla-jmx
DEST = $(CURDIR)/debian/scylla-jmx/usr/lib/scylla/jmx
M2_REPO= $(CURDIR)/m2
override_dh_auto_build:
mvn -B -Dmaven.repo.local=$(M2_REPO) install
override_dh_auto_clean:
rm -rf target m2
override_dh_auto_install:
mkdir -p $(CURDIR)/debian/scylla-jmx/etc/default/ && \
cp $(CURDIR)/dist/common/sysconfig/scylla-jmx \
$(CURDIR)/debian/scylla-jmx/etc/default/
mkdir -p $(DOC) && \
cp $(CURDIR)/*.md $(DOC)
cp $(CURDIR)/NOTICE $(DOC)
mkdir -p $(DEST)
cp $(CURDIR)/scripts/scylla-jmx $(DEST)
cp $(CURDIR)/target/scylla-jmx-1.0.jar $(DEST)
mkdir $(DEST)/symlinks
ln -sf /usr/bin/java $(DEST)/symlinks/scylla-jmx
override_dh_installinit:
dh_installinit --no-start @@DH_INSTALLINIT@@
%:
dh $@

View File

@ -1,71 +0,0 @@
#!/bin/bash -e
. /etc/os-release
print_usage() {
echo "build_rpm.sh -target epel-7-x86_64 --configure-user"
echo " --target target distribution in mock cfg name"
exit 1
}
TARGET=
while [ $# -gt 0 ]; do
case "$1" in
"--target")
TARGET=$2
shift 2
;;
*)
print_usage
;;
esac
done
is_redhat_variant() {
[ -f /etc/redhat-release ]
}
pkg_install() {
if is_redhat_variant; then
sudo yum install -y $1
else
echo "Requires to install following command: $1"
exit 1
fi
}
if [ ! -e dist/redhat/build_rpm.sh ]; then
echo "run build_rpm.sh in top of scylla-jmx dir"
exit 1
fi
if [ "$(arch)" != "x86_64" ]; then
echo "Unsupported architecture: $(arch)"
exit 1
fi
if [ -z "$TARGET" ]; then
if [ "$ID" = "centos" -o "$ID" = "rhel" ] && [ "$VERSION_ID" = "7" ]; then
TARGET=./dist/redhat/mock/scylla-jmx-epel-7-x86_64.cfg
elif [ "$ID" = "fedora" ]; then
TARGET=$ID-$VERSION_ID-x86_64
else
echo "Please specify target"
exit 1
fi
fi
if [ ! -f /usr/bin/mock ]; then
pkg_install mock
fi
if [ ! -f /usr/bin/git ]; then
pkg_install git
fi
VERSION=$(./SCYLLA-VERSION-GEN)
SCYLLA_VERSION=$(cat build/SCYLLA-VERSION-FILE)
SCYLLA_RELEASE=$(cat build/SCYLLA-RELEASE-FILE)
git archive --format=tar --prefix=scylla-jmx-$SCYLLA_VERSION/ HEAD -o build/scylla-jmx-$VERSION.tar
cp dist/redhat/scylla-jmx.spec.in build/scylla-jmx.spec
sed -i -e "s/@@VERSION@@/$SCYLLA_VERSION/g" build/scylla-jmx.spec
sed -i -e "s/@@RELEASE@@/$SCYLLA_RELEASE/g" build/scylla-jmx.spec
sudo mock --buildsrpm --root=$TARGET --resultdir=`pwd`/build/srpms --spec=build/scylla-jmx.spec --sources=build/scylla-jmx-$VERSION.tar
sudo mock --rebuild --root=$TARGET --resultdir=`pwd`/build/rpms build/srpms/scylla-jmx-$VERSION*.src.rpm

View File

@ -1,53 +0,0 @@
config_opts['root'] = 'epel-7-x86_64'
config_opts['target_arch'] = 'x86_64'
config_opts['legal_host_arches'] = ('x86_64',)
config_opts['chroot_setup_cmd'] = 'install @buildsys-build'
config_opts['dist'] = 'el7' # only useful for --resultdir variable subst
config_opts['releasever'] = '7'
config_opts['rpmbuild_networking'] = True
config_opts['yum.conf'] = """
[main]
keepcache=1
debuglevel=2
reposdir=/dev/null
logfile=/var/log/yum.log
retries=20
obsoletes=1
gpgcheck=0
assumeyes=1
syslog_ident=mock
syslog_device=
mdpolicy=group:primary
best=1
# repos
[scylla-centos-base]
name=BaseOS
mirrorlist=http://mirrorlist.centos.org/?release=7&arch=x86_64&repo=os
failovermethod=priority
gpgkey=https://www.centos.org/keys/RPM-GPG-KEY-CentOS-7
gpgcheck=1
[scylla-centos-updates]
name=updates
enabled=1
mirrorlist=http://mirrorlist.centos.org/?release=7&arch=x86_64&repo=updates
failovermethod=priority
gpgkey=https://www.centos.org/keys/RPM-GPG-KEY-CentOS-7
gpgcheck=1
[scylla-centos-extras]
name=extras
mirrorlist=http://mirrorlist.centos.org/?release=7&arch=x86_64&repo=extras
failovermethod=priority
gpgkey=https://www.centos.org/keys/RPM-GPG-KEY-CentOS-7
gpgcheck=1
[scylla-epel]
name=epel
mirrorlist=http://mirrors.fedoraproject.org/mirrorlist?repo=epel-7&arch=x86_64
failovermethod=priority
gpgkey=https://dl.fedoraproject.org/pub/epel/RPM-GPG-KEY-EPEL-7
gpgcheck=1
"""

75
dist/redhat/scylla-jmx.spec vendored Normal file
View File

@ -0,0 +1,75 @@
Name: %{product}-jmx
Version: %{version}
Release: %{release}%{?dist}
Summary: Scylla JMX
Group: Applications/Databases
License: AGPLv3
URL: http://www.scylladb.com/
Source0: %{reloc_pkg}
BuildArch: noarch
BuildRequires: systemd-units
Requires: %{product}-server jre-1.8.0-headless
AutoReqProv: no
%description
%prep
%setup -q -n scylla-jmx
%build
%install
./install.sh --packaging --root "$RPM_BUILD_ROOT"
%pre
/usr/sbin/groupadd scylla 2> /dev/null || :
/usr/sbin/useradd -g scylla -s /sbin/nologin -r -d ${_sharedstatedir}/scylla scylla 2> /dev/null || :
ping -c1 `hostname` > /dev/null 2>&1
if [ $? -ne 0 ]; then
echo
echo "**************************************************************"
echo "* WARNING: You need to add hostname on /etc/hosts, otherwise *"
echo "* scylla-jmx will not able to start up. *"
echo "**************************************************************"
echo
fi
%post
if [ $1 -eq 1 ] ; then
/usr/bin/systemctl preset scylla-jmx.service ||:
fi
/usr/bin/systemctl daemon-reload ||:
%preun
if [ $1 -eq 0 ] ; then
/usr/bin/systemctl --no-reload disable scylla-jmx.service ||:
/usr/bin/systemctl stop scylla-jmx.service ||:
fi
%postun
/usr/bin/systemctl daemon-reload ||:
%clean
rm -rf $RPM_BUILD_ROOT
%files
%defattr(-,root,root)
%config(noreplace) %{_sysconfdir}/sysconfig/scylla-jmx
%{_unitdir}/scylla-jmx.service
/opt/scylladb/jmx/scylla-jmx
/opt/scylladb/jmx/scylla-jmx-1.1.jar
/opt/scylladb/jmx/symlinks/scylla-jmx
%{_prefix}/lib/scylla/jmx/scylla-jmx
%{_prefix}/lib/scylla/jmx/scylla-jmx-1.1.jar
%{_prefix}/lib/scylla/jmx/symlinks/scylla-jmx
%changelog
* Fri Aug 7 2015 Takuya ASADA Takuya ASADA <syuu@cloudius-systems.com>
- inital version of scylla-tools.spec

View File

@ -1,80 +0,0 @@
Name: scylla-jmx
Version: @@VERSION@@
Release: @@RELEASE@@%{?dist}
Summary: Scylla JMX
Group: Applications/Databases
License: AGPLv3
URL: http://www.scylladb.com/
Source0: %{name}-@@VERSION@@-@@RELEASE@@.tar
BuildArch: noarch
BuildRequires: maven systemd-units java-1.8.0-openjdk-devel
Requires: scylla-server java-1.8.0-openjdk-headless
%description
%prep
%setup -q
%build
mvn -B install
mkdir build
cp dist/common/systemd/scylla-jmx.service.in build/scylla-jmx.service
sed -i -e "s#@@SYSCONFDIR@@#/etc/sysconfig#g" build/scylla-jmx.service
%install
rm -rf $RPM_BUILD_ROOT
mkdir -p $RPM_BUILD_ROOT%{_sysconfdir}/sysconfig/
mkdir -p $RPM_BUILD_ROOT%{_unitdir}
mkdir -p $RPM_BUILD_ROOT%{_prefix}/lib/scylla/
install -m644 dist/common/sysconfig/scylla-jmx $RPM_BUILD_ROOT%{_sysconfdir}/sysconfig/
install -m644 build/*.service $RPM_BUILD_ROOT%{_unitdir}/
install -d -m755 $RPM_BUILD_ROOT%{_prefix}/lib/scylla
install -d -m755 $RPM_BUILD_ROOT%{_prefix}/lib/scylla/jmx
install -d -m755 $RPM_BUILD_ROOT%{_prefix}/lib/scylla/jmx/symlinks
install -m644 target/scylla-jmx-1.0.jar $RPM_BUILD_ROOT%{_prefix}/lib/scylla/jmx/
install -m755 scripts/scylla-jmx $RPM_BUILD_ROOT%{_prefix}/lib/scylla/jmx
ln -sf /usr/bin/java $RPM_BUILD_ROOT%{_prefix}/lib/scylla/jmx/symlinks/scylla-jmx
%pre
/usr/sbin/groupadd scylla 2> /dev/null || :
/usr/sbin/useradd -g scylla -s /sbin/nologin -r -d ${_sharedstatedir}/scylla scylla 2> /dev/null || :
ping -c1 `hostname` > /dev/null 2>&1
if [ $? -ne 0 ]; then
echo
echo "**************************************************************"
echo "* WARNING: You need to add hostname on /etc/hosts, otherwise *"
echo "* scylla-jmx will not able to start up. *"
echo "**************************************************************"
echo
fi
%post
%systemd_post scylla-jmx.service
%preun
%systemd_preun scylla-jmx.service
%postun
%systemd_postun
%clean
rm -rf $RPM_BUILD_ROOT
%files
%defattr(-,root,root)
%config(noreplace) %{_sysconfdir}/sysconfig/scylla-jmx
%{_unitdir}/scylla-jmx.service
%{_prefix}/lib/scylla/jmx/scylla-jmx
%{_prefix}/lib/scylla/jmx/scylla-jmx-1.0.jar
%{_prefix}/lib/scylla/jmx/symlinks/scylla-jmx
%changelog
* Fri Aug 7 2015 Takuya ASADA Takuya ASADA <syuu@cloudius-systems.com>
- inital version of scylla-tools.spec

26
install-dependencies.sh Executable file
View File

@ -0,0 +1,26 @@
#!/bin/bash
#
# This file is open source software, licensed to you under the terms
# of the Apache License, Version 2.0 (the "License"). See the NOTICE file
# distributed with this work for additional information regarding copyright
# ownership. You may not use this file except in compliance with the License.
#
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing,
# software distributed under the License is distributed on an
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
# KIND, either express or implied. See the License for the
# specific language governing permissions and limitations
# under the License.
#
. /etc/os-release
if [ "$ID" = "ubuntu" ] || [ "$ID" = "debian" ]; then
apt -y install maven openjdk-8-jdk-headless
elif [ "$ID" = "fedora" ] || [ "$ID" = "centos" ]; then
dnf install -y maven java-1.8.0-openjdk-devel
fi

173
install.sh Executable file
View File

@ -0,0 +1,173 @@
#!/bin/bash
#
# Copyright (C) 2019 ScyllaDB
#
#
# This file is part of Scylla.
#
# Scylla is free software: you can redistribute it and/or modify
# it under the terms of the GNU Affero General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# Scylla is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with Scylla. If not, see <http://www.gnu.org/licenses/>.
#
set -e
print_usage() {
cat <<EOF
Usage: install.sh [options]
Options:
--root /path/to/root alternative install root (default /)
--prefix /prefix directory prefix (default /usr)
--nonroot shortcut of '--disttype nonroot'
--sysconfdir /etc/sysconfig specify sysconfig directory name
--packaging use install.sh for packaging
--without-systemd skip installing systemd units
--help this helpful message
EOF
exit 1
}
root=/
sysconfdir=/etc/sysconfig
nonroot=false
packaging=false
without_systemd=false
while [ $# -gt 0 ]; do
case "$1" in
"--root")
root="$2"
shift 2
;;
"--prefix")
prefix="$2"
shift 2
;;
"--nonroot")
nonroot=true
shift 1
;;
"--sysconfdir")
sysconfdir="$2"
shift 2
;;
"--packaging")
packaging=true
shift 1
;;
"--without-systemd")
without_systemd=true
shift 1
;;
"--help")
shift 1
print_usage
;;
*)
print_usage
;;
esac
done
check_usermode_support() {
user=$(systemctl --help|grep -e '--user')
[ -n "$user" ]
}
if ! $packaging; then
has_java=false
if [ -x /usr/bin/java ]; then
javaver=$(/usr/bin/java -version 2>&1|head -n1|cut -f 3 -d " ")
has_java=true
fi
if ! $has_java; then
echo "Please install openjdk-8, openjdk-11, or openjdk-17 before running install.sh."
exit 1
fi
fi
if [ -z "$prefix" ]; then
if $nonroot; then
prefix=~/scylladb
else
prefix=/opt/scylladb
fi
fi
rprefix=$(realpath -m "$root/$prefix")
if ! $nonroot; then
retc="$root/etc"
rsysconfdir="$root/$sysconfdir"
rusr="$root/usr"
rsystemd="$rusr/lib/systemd/system"
else
retc="$rprefix/etc"
rsysconfdir="$rprefix/$sysconfdir"
rsystemd="$HOME/.config/systemd/user"
fi
install -d -m755 "$rsysconfdir"
if ! $without_systemd; then
install -d -m755 "$rsystemd"
fi
install -d -m755 "$rprefix/scripts" "$rprefix/jmx" "$rprefix/jmx/symlinks"
install -m644 dist/common/sysconfig/scylla-jmx -Dt "$rsysconfdir"
if ! $without_systemd; then
install -m644 dist/common/systemd/scylla-jmx.service -Dt "$rsystemd"
fi
if ! $nonroot && ! $without_systemd; then
if [ "$sysconfdir" != "/etc/sysconfig" ]; then
install -d -m755 "$retc"/systemd/system/scylla-jmx.service.d
cat << EOS > "$retc"/systemd/system/scylla-jmx.service.d/sysconfdir.conf
[Service]
EnvironmentFile=
EnvironmentFile=$sysconfdir/scylla-jmx
EOS
fi
elif ! $without_systemd; then
install -d -m755 "$rsystemd"/scylla-jmx.service.d
cat << EOS > "$rsystemd"/scylla-jmx.service.d/nonroot.conf
[Service]
EnvironmentFile=
EnvironmentFile=$retc/sysconfig/scylla-jmx
ExecStart=
ExecStart=$rprefix/jmx/scylla-jmx \$SCYLLA_JMX_PORT \$SCYLLA_API_PORT \$SCYLLA_API_ADDR \$SCYLLA_JMX_ADDR \$SCYLLA_JMX_FILE \$SCYLLA_JMX_LOCAL \$SCYLLA_JMX_REMOTE \$SCYLLA_JMX_DEBUG
User=
Group=
WorkingDirectory=$rprefix
EOS
fi
install -m644 scylla-jmx-1.1.jar "$rprefix/jmx"
install -m755 scylla-jmx "$rprefix/jmx"
ln -sf /usr/bin/java "$rprefix/jmx/symlinks/scylla-jmx"
if ! $nonroot; then
install -m755 -d "$rusr"/lib/scylla/jmx/symlinks
ln -srf "$rprefix"/jmx/scylla-jmx-1.1.jar "$rusr"/lib/scylla/jmx/
ln -srf "$rprefix"/jmx/scylla-jmx "$rusr"/lib/scylla/jmx/
ln -sf /usr/bin/java "$rusr"/lib/scylla/jmx/symlinks/scylla-jmx
fi
if $nonroot; then
sed -i -e "s#/var/lib/scylla#$rprefix#g" "$rsysconfdir"/scylla-jmx
sed -i -e "s#/etc/scylla#$rprefix/etc/scylla#g" "$rsysconfdir"/scylla-jmx
sed -i -e "s#/opt/scylladb/jmx#$rprefix/jmx#g" "$rsysconfdir"/scylla-jmx
if ! $without_systemd && check_usermode_support; then
systemctl --user daemon-reload
fi
echo "Scylla-JMX non-root install completed."
elif ! $without_systemd && ! $packaging; then
systemctl --system daemon-reload
fi

118
pom.xml
View File

@ -2,87 +2,81 @@
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>com.scylladb.jmx</groupId>
<artifactId>scylla-jmx</artifactId>
<version>1.0</version>
<version>1.1</version>
<packaging>jar</packaging>
<parent>
<groupId>it.cavallium.scylladb.jmx</groupId>
<artifactId>scylla-jmx-parent</artifactId>
<version>1.1</version>
<relativePath>./scylla-jmx-parent/pom.xml</relativePath>
</parent>
<name>Scylla JMX</name>
<properties>
<maven.compiler.target>1.8</maven.compiler.target>
<maven.compiler.source>1.8</maven.compiler.source>
</properties>
<dependencies>
<dependency>
<groupId>org.yaml</groupId>
<artifactId>snakeyaml</artifactId>
<version>1.16</version>
</dependency>
<dependency>
<groupId>org.glassfish.jersey.core</groupId>
<artifactId>jersey-common</artifactId>
<version>2.22.1</version>
</dependency>
<dependency>
<groupId>javax.ws.rs</groupId>
<artifactId>javax.ws.rs-api</artifactId>
<version>2.0.1</version>
</dependency>
<dependency>
<groupId>javax.ws.rs</groupId>
<artifactId>jsr311-api</artifactId>
<version>1.1.1</version>
</dependency>
<dependency>
<groupId>org.glassfish.jersey.core</groupId>
<artifactId>jersey-client</artifactId>
<version>2.22.1</version>
</dependency>
<dependency>
<groupId>org.slf4j</groupId>
<artifactId>slf4j-api</artifactId>
<version>1.7.5</version>
</dependency>
<dependency>
<groupId>org.slf4j</groupId>
<artifactId>slf4j-simple</artifactId>
<version>1.6.4</version>
</dependency>
<dependency>
<groupId>org.slf4j</groupId>
<artifactId>slf4j-log4j12</artifactId>
<version>1.7.5</version>
<groupId>it.cavallium.scylladb.jmx</groupId>
<artifactId>scylla-apiclient</artifactId>
<version>1.1</version>
</dependency>
<dependency>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
<version>4.8.2</version>
<version>4.13.1</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.glassfish</groupId>
<artifactId>javax.json</artifactId>
<version>1.0.4</version>
</dependency>
<dependency>
<groupId>com.google.guava</groupId>
<artifactId>guava</artifactId>
<version>18.0</version>
</dependency>
<dependency>
<groupId>com.google.collections</groupId>
<artifactId>google-collections</artifactId>
<version>1.0</version>
</dependency>
</dependencies>
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-compiler-plugin</artifactId>
<version>3.10.1</version>
<configuration>
<source>11</source>
<target>11</target>
<compilerArgs>
<arg>--add-exports</arg>
<arg>java.management/com.sun.jmx.mbeanserver=scylla.jmx</arg>
<arg>--add-exports</arg>
<arg>java.management/com.sun.jmx.interceptor=scylla.jmx</arg>
</compilerArgs>
</configuration>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-shade-plugin</artifactId>
<version>2.4.1</version>
<version>3.4.1</version>
<configuration>
<artifactSet>
<includes>
<include>*:*</include>
</includes>
<excludes>
<exclude>com.sun.activation:jakarta.activation</exclude>
</excludes>
</artifactSet>
<filters>
<filter>
<artifact>*:*</artifact>
<excludes>
<exclude>module-info.class</exclude>
<exclude>META-INF/versions/*/module-info.class</exclude>
<exclude>META-INF/*.SF</exclude>
<exclude>META-INF/*.DSA</exclude>
<exclude>META-INF/*.RSA</exclude>
<exclude>META-INF/MANIFEST.MF</exclude>
<exclude>META-INF/*.MD</exclude>
<exclude>META-INF/*.md</exclude>
<exclude>META-INF/LICENSE</exclude>
<exclude>META-INF/LICENSE.txt</exclude>
<exclude>META-INF/NOTICE</exclude>
</excludes>
</filter>
</filters>
</configuration>
<executions>
<execution>
<phase>package</phase>

42
reloc/build_deb.sh Executable file
View File

@ -0,0 +1,42 @@
#!/bin/bash -e
print_usage() {
echo "build_deb.sh --reloc-pkg build/scylla-jmx-package.tar.gz"
echo " --reloc-pkg specify relocatable package path"
echo " --builddir specify Debian package build path"
exit 1
}
RELOC_PKG=build/scylla-jmx-package.tar.gz
BUILDDIR=build/debian
while [ $# -gt 0 ]; do
case "$1" in
"--reloc-pkg")
RELOC_PKG=$2
shift 2
;;
"--builddir")
BUILDDIR="$2"
shift 2
;;
*)
print_usage
;;
esac
done
RELOC_PKG=$(readlink -f $RELOC_PKG)
rm -rf "$BUILDDIR"/scylla-package "$BUILDDIR"/scylla-package.orig "$BUILDDIR"/debian
mkdir -p "$BUILDDIR"/scylla-package
tar -C "$BUILDDIR"/scylla-package -xpf $RELOC_PKG
cd "$BUILDDIR"/scylla-package
RELOC_PKG=$(readlink -f $RELOC_PKG)
mv scylla-jmx/debian debian
PKG_NAME=$(dpkg-parsechangelog --show-field Source)
# XXX: Drop revision number from version string.
# Since it always '1', this should be okay for now.
PKG_VERSION=$(dpkg-parsechangelog --show-field Version |sed -e 's/-1$//')
ln -fv $RELOC_PKG ../"$PKG_NAME"_"$PKG_VERSION".orig.tar.gz
debuild -rfakeroot -us -uc

70
reloc/build_reloc.sh Executable file
View File

@ -0,0 +1,70 @@
#!/bin/bash -e
. /etc/os-release
print_usage() {
echo "build_reloc.sh --clean --nodeps"
echo " --clean clean build directory"
echo " --nodeps skip installing dependencies"
echo " --version V product-version-release string (overriding SCYLLA-VERSION-GEN)"
exit 1
}
CLEAN=
NODEPS=
VERSION_OVERRIDE=
while [ $# -gt 0 ]; do
case "$1" in
"--clean")
CLEAN=yes
shift 1
;;
"--nodeps")
NODEPS=yes
shift 1
;;
"--version")
VERSION_OVERRIDE="$2"
shift 2
;;
*)
print_usage
;;
esac
done
VERSION=$(./SCYLLA-VERSION-GEN ${VERSION_OVERRIDE:+ --version "$VERSION_OVERRIDE"})
# the former command should generate build/SCYLLA-PRODUCT-FILE and some other version
# related files
PRODUCT=`cat build/SCYLLA-PRODUCT-FILE`
DEST="build/$PRODUCT-jmx-$VERSION.noarch.tar.gz"
is_redhat_variant() {
[ -f /etc/redhat-release ]
}
is_debian_variant() {
[ -f /etc/debian_version ]
}
if [ ! -e reloc/build_reloc.sh ]; then
echo "run build_reloc.sh in top of scylla dir"
exit 1
fi
if [ "$CLEAN" = "yes" ]; then
rm -rf build target
fi
if [ -f "$DEST" ]; then
rm "$DEST"
fi
if [ -z "$NODEPS" ]; then
sudo ./install-dependencies.sh
fi
mvn -B --file scylla-jmx-parent/pom.xml install
./SCYLLA-VERSION-GEN ${VERSION_OVERRIDE:+ --version "$VERSION_OVERRIDE"}
./dist/debian/debian_files_gen.py
scripts/create-relocatable-package.py "$DEST"

52
reloc/build_rpm.sh Executable file
View File

@ -0,0 +1,52 @@
#!/bin/bash -e
print_usage() {
echo "build_rpm.sh --reloc-pkg build/scylla-jmx-package.tar.gz"
echo " --reloc-pkg specify relocatable package path"
echo " --builddir specify rpmbuild directory"
exit 1
}
RELOC_PKG=build/scylla-jmx-package.tar.gz
BUILDDIR=build/redhat
while [ $# -gt 0 ]; do
case "$1" in
"--reloc-pkg")
RELOC_PKG=$2
shift 2
;;
"--builddir")
BUILDDIR="$2"
shift 2
;;
*)
print_usage
;;
esac
done
RELOC_PKG=$(readlink -f $RELOC_PKG)
RPMBUILD=$(readlink -f $BUILDDIR)
mkdir -p "$BUILDDIR"
tar -C "$BUILDDIR" -xpf $RELOC_PKG scylla-jmx/SCYLLA-RELEASE-FILE scylla-jmx/SCYLLA-RELOCATABLE-FILE scylla-jmx/SCYLLA-VERSION-FILE scylla-jmx/SCYLLA-PRODUCT-FILE scylla-jmx/dist/redhat
cd "$BUILDDIR"/scylla-jmx
RELOC_PKG_BASENAME=$(basename "$RELOC_PKG")
SCYLLA_VERSION=$(cat SCYLLA-VERSION-FILE)
SCYLLA_RELEASE=$(cat SCYLLA-RELEASE-FILE)
VERSION=$SCYLLA_VERSION-$SCYLLA_RELEASE
PRODUCT=$(cat SCYLLA-PRODUCT-FILE)
mkdir -p $RPMBUILD/{BUILD,BUILDROOT,RPMS,SOURCES,SPECS,SRPMS}
ln -fv $RELOC_PKG $RPMBUILD/SOURCES/
parameters=(
-D"version $SCYLLA_VERSION"
-D"release $SCYLLA_RELEASE"
-D"product $PRODUCT"
-D"reloc_pkg $RELOC_PKG_BASENAME"
)
cp dist/redhat/scylla-jmx.spec $RPMBUILD/SPECS
# this rpm can be install on both fedora / centos7, so drop distribution name from the file name
rpmbuild -ba "${parameters[@]}" --define '_binary_payload w2.xzdio' --define "_topdir $RPMBUILD" --undefine "dist" $RPMBUILD/SPECS/scylla-jmx.spec

View File

@ -0,0 +1,64 @@
#!/usr/bin/python3
#
# Copyright (C) 2018 ScyllaDB
#
#
# This file is part of Scylla.
#
# Scylla is free software: you can redistribute it and/or modify
# it under the terms of the GNU Affero General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# Scylla is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with Scylla. If not, see <http://www.gnu.org/licenses/>.
#
import argparse
import io
import os
import tarfile
import pathlib
RELOC_PREFIX='scylla-jmx'
def reloc_add(self, name, arcname=None, recursive=True, *, filter=None):
if arcname:
return self.add(name, arcname="{}/{}".format(RELOC_PREFIX, arcname))
else:
return self.add(name, arcname="{}/{}".format(RELOC_PREFIX, name))
tarfile.TarFile.reloc_add = reloc_add
ap = argparse.ArgumentParser(description='Create a relocatable scylla package.')
ap.add_argument('dest',
help='Destination file (tar format)')
args = ap.parse_args()
output = args.dest
ar = tarfile.open(output, mode='w|gz')
# relocatable package format version = 2.2
with open('build/.relocatable_package_version', 'w') as f:
f.write('2.2\n')
ar.add('build/.relocatable_package_version', arcname='.relocatable_package_version')
pathlib.Path('build/SCYLLA-RELOCATABLE-FILE').touch()
ar.reloc_add('build/SCYLLA-RELOCATABLE-FILE', arcname='SCYLLA-RELOCATABLE-FILE')
ar.reloc_add('build/SCYLLA-RELEASE-FILE', arcname='SCYLLA-RELEASE-FILE')
ar.reloc_add('build/SCYLLA-VERSION-FILE', arcname='SCYLLA-VERSION-FILE')
ar.reloc_add('build/SCYLLA-PRODUCT-FILE', arcname='SCYLLA-PRODUCT-FILE')
ar.reloc_add('dist')
ar.reloc_add('install.sh')
ar.reloc_add('target/scylla-jmx-1.1.jar', arcname='scylla-jmx-1.1.jar')
ar.reloc_add('scripts/scylla-jmx', arcname='scylla-jmx')
ar.reloc_add('README.md')
ar.reloc_add('NOTICE')
ar.reloc_add('build/debian/debian', arcname='debian')

View File

@ -131,10 +131,13 @@ else
fi
fi
exec "$LOCATION_SCRIPTS"/symlinks/scylla-jmx $DEBUG \
"$LOCATION_SCRIPTS"/symlinks/scylla-jmx $DEBUG \
$API_PORT $API_ADDR $CONF_FILE -Xmx256m -XX:+UseSerialGC \
-XX:+HeapDumpOnOutOfMemoryError \
$JMX_AUTH $JMX_SSL $JMX_ADDR $JMX_LOCAL \
--add-exports java.management/com.sun.jmx.mbeanserver=ALL-UNNAMED \
--add-exports java.management/com.sun.jmx.interceptor=ALL-UNNAMED \
-Dcom.sun.management.jmxremote -Dcom.sun.management.jmxremote.port=$JMX_PORT \
-Djava.rmi.server.hostname=$HOSTNAME -Dcom.sun.management.jmxremote.rmi.port=$JMX_PORT \
-Djavax.management.builder.initial=com.scylladb.jmx.utils.APIBuilder \
$PROPERTIES -jar $LOCATION/scylla-jmx-1.0.jar
$PROPERTIES -jar $LOCATION/scylla-jmx-1.1.jar

99
scylla-apiclient/pom.xml Normal file
View File

@ -0,0 +1,99 @@
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<artifactId>scylla-apiclient</artifactId>
<packaging>jar</packaging>
<version>1.1</version>
<parent>
<relativePath>../scylla-jmx-parent/pom.xml</relativePath>
<groupId>it.cavallium.scylladb.jmx</groupId>
<artifactId>scylla-jmx-parent</artifactId>
<version>1.1</version>
</parent>
<name>Scylla REST API client</name>
<properties>
<jackson.version>2.14.0</jackson.version>
<jackson.databind.version>2.14.0</jackson.databind.version>
</properties>
<dependencies>
<dependency>
<groupId>org.eclipse.parsson</groupId>
<artifactId>parsson</artifactId>
<version>1.1.1</version>
</dependency>
<dependency>
<groupId>org.yaml</groupId>
<artifactId>snakeyaml</artifactId>
<version>1.33</version>
</dependency>
<dependency>
<groupId>org.glassfish.jersey.core</groupId>
<artifactId>jersey-common</artifactId>
<version>3.1.0</version>
</dependency>
<dependency>
<groupId>jakarta.ws.rs</groupId>
<artifactId>jakarta.ws.rs-api</artifactId>
<version>3.1.0</version>
</dependency>
<dependency>
<groupId>org.glassfish.jersey.core</groupId>
<artifactId>jersey-client</artifactId>
<version>3.1.0</version>
</dependency>
<dependency>
<groupId>org.glassfish.jersey.inject</groupId>
<artifactId>jersey-hk2</artifactId>
<version>3.1.0</version>
</dependency>
<dependency>
<groupId>jakarta.json</groupId>
<artifactId>jakarta.json-api</artifactId>
<version>2.1.1</version>
</dependency>
<dependency>
<groupId>com.google.guava</groupId>
<artifactId>guava</artifactId>
<version>31.1-jre</version>
</dependency>
<dependency>
<groupId>jakarta.activation</groupId>
<artifactId>jakarta.activation-api</artifactId>
<version>2.1.1</version>
</dependency>
<dependency>
<groupId>com.fasterxml.jackson.core</groupId>
<artifactId>jackson-annotations</artifactId>
<version>${jackson.version}</version>
</dependency>
<dependency>
<groupId>com.fasterxml.jackson.core</groupId>
<artifactId>jackson-databind</artifactId>
<version>${jackson.databind.version}</version>
</dependency>
<dependency>
<groupId>com.fasterxml.jackson.jakarta.rs</groupId>
<artifactId>jackson-jakarta-rs-json-provider</artifactId>
<version>2.14.1</version>
</dependency>
</dependencies>
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-compiler-plugin</artifactId>
<version>3.10.1</version>
<configuration>
<release>11</release>
</configuration>
</plugin>
</plugins>
</build>
</project>

View File

@ -3,7 +3,24 @@
*/
package com.scylladb.jmx.api;
import com.fasterxml.jackson.jakarta.rs.json.JacksonJsonProvider;
import jakarta.json.Json;
import jakarta.json.JsonArray;
import jakarta.json.JsonObject;
import jakarta.json.JsonReader;
import jakarta.json.JsonReaderFactory;
import jakarta.json.JsonString;
import jakarta.ws.rs.ProcessingException;
import jakarta.ws.rs.client.Client;
import jakarta.ws.rs.client.ClientBuilder;
import jakarta.ws.rs.client.Entity;
import jakarta.ws.rs.client.Invocation;
import jakarta.ws.rs.client.WebTarget;
import jakarta.ws.rs.core.MediaType;
import jakarta.ws.rs.core.MultivaluedMap;
import jakarta.ws.rs.core.Response;
import java.io.StringReader;
import java.lang.System.Logger.Level;
import java.net.InetAddress;
import java.net.UnknownHostException;
import java.util.ArrayList;
@ -18,30 +35,28 @@ import java.util.Set;
import java.util.function.BiFunction;
import java.util.logging.Logger;
import javax.json.Json;
import javax.json.JsonArray;
import javax.json.JsonObject;
import javax.json.JsonReader;
import javax.json.JsonReaderFactory;
import javax.json.JsonString;
import javax.management.openmbean.TabularData;
import javax.management.openmbean.TabularDataSupport;
import javax.ws.rs.ProcessingException;
import javax.ws.rs.client.Client;
import javax.ws.rs.client.ClientBuilder;
import javax.ws.rs.client.Entity;
import javax.ws.rs.client.Invocation;
import javax.ws.rs.client.WebTarget;
import javax.ws.rs.core.MediaType;
import javax.ws.rs.core.MultivaluedMap;
import javax.ws.rs.core.Response;
import org.glassfish.jersey.client.ClientConfig;
import com.scylladb.jmx.utils.SnapshotDetailsTabularData;
import com.scylladb.jmx.api.utils.SnapshotDetailsTabularData;
public class APIClient {
private Map<String, CacheEntry> cache = new HashMap<String, CacheEntry>();
private final APIConfig config;
private final ClientConfig clientConfig;
private final Client client;
private JsonReaderFactory factory = Json.createReaderFactory(null);
private static final Logger logger = Logger.getLogger(APIClient.class.getName());
public APIClient(APIConfig config) {
this.config = config;
this.clientConfig = new ClientConfig();
clientConfig.register(new JacksonJsonProvider());
this.client = ClientBuilder.newClient(clientConfig);
}
private String getCacheKey(String key, MultivaluedMap<String, String> param, long duration) {
if (duration <= 0) {
@ -74,21 +89,12 @@ public class APIClient {
return (value != null && value.valid(duration)) ? value.jsonObject() : null;
}
private JsonReaderFactory factory = Json.createReaderFactory(null);
private static final Logger logger = Logger.getLogger(APIClient.class.getName());
private final APIConfig config;
public APIClient(APIConfig config) {
this.config = config;
}
private String getBaseUrl() {
return config.getBaseUrl();
}
public Invocation.Builder get(String path, MultivaluedMap<String, String> queryParams) {
Client client = ClientBuilder.newClient(new ClientConfig());
WebTarget webTarget = client.target(getBaseUrl()).path(path);
if (queryParams != null) {
for (Entry<String, List<String>> qp : queryParams.entrySet()) {
@ -152,7 +158,11 @@ public class APIClient {
get(path, queryParams).delete();
return;
}
get(path).delete();
Response response = get(path).delete();
if (response.getStatus() != Response.Status.OK.getStatusCode()) {
throw getException("Scylla API server HTTP delete to URL '" + path + "' failed",
response.readEntity(String.class));
}
}
public void delete(String path) {

View File

@ -21,7 +21,8 @@
package com.scylladb.jmx.api;
import javax.json.JsonObject;
import jakarta.json.JsonObject;
class CacheEntry {
private long time;

View File

@ -22,7 +22,7 @@
* Modified by Cloudius Systems
*/
package com.scylladb.jmx.utils;
package com.scylladb.jmx.api.utils;
import java.io.File;
import java.text.DecimalFormat;

View File

@ -22,7 +22,7 @@
* Modified by Cloudius Systems
*/
package com.scylladb.jmx.utils;
package com.scylladb.jmx.api.utils;
import com.google.common.base.Objects;

View File

@ -20,7 +20,7 @@
*
* Modified by Cloudius Systems
*/
package com.scylladb.jmx.utils;
package com.scylladb.jmx.api.utils;
import java.util.Map;

View File

@ -0,0 +1,15 @@
module scylla.apiclient {
exports com.scylladb.jmx.api;
exports com.scylladb.jmx.api.utils;
requires org.eclipse.parsson;
requires jakarta.ws.rs;
requires com.fasterxml.jackson.jakarta.rs.json;
requires jersey.client;
requires java.logging;
requires jakarta.json;
requires java.management;
requires org.yaml.snakeyaml;
requires com.google.common;
requires jersey.common;
requires jersey.hk2;
}

29
scylla-jmx-parent/pom.xml Normal file
View File

@ -0,0 +1,29 @@
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>it.cavallium.scylladb.jmx</groupId>
<artifactId>scylla-jmx-parent</artifactId>
<version>1.1</version>
<packaging>pom</packaging>
<modules>
<module>../</module>
<module>../scylla-apiclient</module>
</modules>
<name>Scylla JMX Parent</name>
<distributionManagement>
<repository>
<id>mchv-release-distribution</id>
<name>MCHV Release Apache Maven Packages Distribution</name>
<url>https://mvn.mchv.eu/repository/mchv</url>
</repository>
<snapshotRepository>
<id>mchv-snapshot-distribution</id>
<name>MCHV Snapshot Apache Maven Packages Distribution</name>
<url>https://mvn.mchv.eu/repository/mchv-snapshot</url>
</snapshotRepository>
</distributionManagement>
</project>

View File

@ -27,12 +27,27 @@ import com.scylladb.jmx.api.APIConfig;
import com.scylladb.jmx.metrics.APIMBean;
public class Main {
// todo: command line options. Make us an agent class (also)
private static final APIConfig config = new APIConfig();
public static final APIClient client = new APIClient(config);
private static APIConfig config;
private static APIClient client;
public static synchronized APIConfig getApiConfig() {
if (config == null) {
config = new APIConfig();
}
return config;
}
public static synchronized APIClient getApiClient() {
if (client == null) {
client = new APIClient(getApiConfig());
}
return client;
}
public static void main(String[] args) throws Exception {
System.out.println("Connecting to " + config.getBaseUrl());
System.out.printf("Java %s%n", System.getProperty("java.version"));
System.out.printf("Connecting to %s%n", getApiConfig().getBaseUrl());
System.out.println("Starting the JMX server");
MBeanServer server = getPlatformMBeanServer();
@ -40,7 +55,7 @@ public class Main {
CommitLog.class, Gossiper.class, EndpointSnitchInfo.class, FailureDetector.class, CacheService.class,
CompactionManager.class, GCInspector.class, StreamManager.class)) {
Constructor<? extends APIMBean> c = clazz.getDeclaredConstructor(APIClient.class);
APIMBean m = c.newInstance(client);
APIMBean m = c.newInstance(getApiClient());
server.registerMBean(m, null);
}

View File

@ -1,6 +1,7 @@
package com.scylladb.jmx.metrics;
import java.lang.reflect.Field;
import java.util.EnumSet;
import java.util.Set;
import java.util.function.Function;
import java.util.function.Predicate;
@ -20,6 +21,7 @@ import javax.management.ObjectName;
import javax.management.QueryExp;
import com.scylladb.jmx.api.APIClient;
import com.sun.jmx.mbeanserver.JmxMBeanServer;
/**
* Base type for MBeans in scylla-jmx. Wraps auto naming and {@link APIClient}
@ -54,35 +56,39 @@ public class APIMBean implements MBeanRegistration {
* @param generator
* {@link Function} to create a new MBean instance for a given
* {@link ObjectName}
*
* @return
* @throws MalformedObjectNameException
*/
public static boolean checkRegistration(MBeanServer server, Set<ObjectName> all,
final Predicate<ObjectName> predicate, Function<ObjectName, Object> generator)
throws MalformedObjectNameException {
Set<ObjectName> registered = queryNames(server, predicate);
for (ObjectName name : registered) {
if (!all.contains(name)) {
try {
server.unregisterMBean(name);
} catch (MBeanRegistrationException | InstanceNotFoundException e) {
}
}
}
public static boolean checkRegistration(JmxMBeanServer server, Set<ObjectName> all,
EnumSet<RegistrationMode> mode, final Predicate<ObjectName> predicate,
Function<ObjectName, Object> generator) throws MalformedObjectNameException {
Set<ObjectName> registered = queryNames(server, predicate);
if (mode.contains(RegistrationMode.Remove)) {
for (ObjectName name : registered) {
if (!all.contains(name)) {
try {
server.getMBeanServerInterceptor().unregisterMBean(name);
} catch (MBeanRegistrationException | InstanceNotFoundException e) {
}
}
}
}
int added = 0;
for (ObjectName name : all) {
if (!registered.contains(name)) {
try {
server.registerMBean(generator.apply(name), name);
added++;
} catch (InstanceAlreadyExistsException | MBeanRegistrationException | NotCompliantMBeanException e) {
}
}
}
return added > 0;
}
int added = 0;
if (mode.contains(RegistrationMode.Add)) {
for (ObjectName name : all) {
if (!registered.contains(name)) {
try {
server.getMBeanServerInterceptor().registerMBean(generator.apply(name), name);
added++;
} catch (InstanceAlreadyExistsException | MBeanRegistrationException
| NotCompliantMBeanException e) {
}
}
}
}
return added > 0;
}
/**
* Helper method to query {@link ObjectName}s from an {@link MBeanServer}
@ -92,7 +98,7 @@ public class APIMBean implements MBeanRegistration {
* @param predicate
* @return
*/
public static Set<ObjectName> queryNames(MBeanServer server, final Predicate<ObjectName> predicate) {
public static Set<ObjectName> queryNames(JmxMBeanServer server, final Predicate<ObjectName> predicate) {
@SuppressWarnings("serial")
Set<ObjectName> registered = server.queryNames(null, new QueryExp() {
@Override
@ -108,7 +114,7 @@ public class APIMBean implements MBeanRegistration {
return registered;
}
MBeanServer server;
JmxMBeanServer server;
ObjectName name;
protected final ObjectName getBoundName() {
@ -162,7 +168,7 @@ public class APIMBean implements MBeanRegistration {
if (this.server != null) {
throw new IllegalStateException("Can only exist in a single MBeanServer");
}
this.server = server;
this.server = (JmxMBeanServer) server;
if (name == null) {
name = generateName();
}

View File

@ -3,6 +3,11 @@ package com.scylladb.jmx.metrics;
import static java.util.Arrays.asList;
import java.util.Collection;
import java.util.Collections;
import java.util.HashMap;
import java.util.HashSet;
import java.util.Map;
import java.util.Set;
import java.util.function.Predicate;
import java.util.function.Supplier;
@ -16,6 +21,7 @@ import org.apache.cassandra.metrics.Metrics;
import org.apache.cassandra.metrics.MetricsRegistry;
import com.scylladb.jmx.api.APIClient;
import com.sun.jmx.mbeanserver.JmxMBeanServer;
/**
* Base type for MBeans containing {@link Metrics}.
@ -24,6 +30,9 @@ import com.scylladb.jmx.api.APIClient;
*
*/
public abstract class MetricsMBean extends APIMBean {
private static final Map<JmxMBeanServer, Map<String, Integer>> registered = new HashMap<>();
private static final Object registrationLock = new Object();
private final Collection<Metrics> metrics;
public MetricsMBean(APIClient client, Metrics... metrics) {
@ -47,13 +56,50 @@ public abstract class MetricsMBean extends APIMBean {
};
}
private void register(MetricsRegistry registry, MBeanServer server) throws MalformedObjectNameException {
// Check if we're the first/last of our type bound/removed.
boolean empty = queryNames(server, getTypePredicate()).isEmpty();
for (Metrics m : metrics) {
if (empty) {
m.registerGlobals(registry);
// Has to be called with registrationLock hold
private static boolean shouldRegisterGlobals(JmxMBeanServer server, String domainAndType, boolean reversed) {
Map<String, Integer> serverMap = registered.get(server);
if (serverMap == null) {
assert !reversed;
serverMap = new HashMap<>();
serverMap.put(domainAndType, 1);
registered.put(server, serverMap);
return true;
}
Integer count = serverMap.get(domainAndType);
if (count == null) {
assert !reversed;
serverMap.put(domainAndType, 1);
return true;
}
if (reversed) {
--count;
if (count == 0) {
serverMap.remove(domainAndType);
if (serverMap.isEmpty()) {
registered.remove(server);
}
return true;
}
serverMap.put(domainAndType, count);
return false;
} else {
serverMap.put(domainAndType, count + 1);
}
return false;
}
private void register(MetricsRegistry registry, JmxMBeanServer server, boolean reversed) throws MalformedObjectNameException {
// Check if we're the first/last of our type bound/removed.
synchronized (registrationLock) {
boolean registerGlobals = shouldRegisterGlobals(server, name.getDomain() + ":" + name.getKeyProperty("type"), reversed);
if (registerGlobals) {
for (Metrics m : metrics) {
m.registerGlobals(registry);
}
}
}
for (Metrics m : metrics) {
m.register(registry);
}
}
@ -63,7 +109,7 @@ public abstract class MetricsMBean extends APIMBean {
// Get name etc.
name = super.preRegister(server, name);
// Register all metrics in server
register(new MetricsRegistry(client, server), server);
register(new MetricsRegistry(client, (JmxMBeanServer) server), (JmxMBeanServer) server, false);
return name;
}
@ -77,12 +123,12 @@ public abstract class MetricsMBean extends APIMBean {
public void register(Supplier<MetricMBean> s, ObjectName... objectNames) {
for (ObjectName name : objectNames) {
try {
server.unregisterMBean(name);
server.getMBeanServerInterceptor().unregisterMBean(name);
} catch (MBeanRegistrationException | InstanceNotFoundException e) {
}
}
}
}, server);
}, server, true);
} catch (MalformedObjectNameException e) {
// TODO : log?
}

View File

@ -0,0 +1,69 @@
package com.scylladb.jmx.metrics;
import static com.scylladb.jmx.metrics.RegistrationMode.Remove;
import static com.scylladb.jmx.metrics.RegistrationMode.Wait;
import static java.util.EnumSet.allOf;
import static java.util.EnumSet.of;
import java.net.UnknownHostException;
import java.util.EnumSet;
import java.util.concurrent.locks.Lock;
import java.util.concurrent.locks.ReentrantLock;
import javax.management.OperationsException;
import com.scylladb.jmx.api.APIClient;
import com.sun.jmx.mbeanserver.JmxMBeanServer;
/**
* Helper type to do optional locking for registration. Allows for
* per-bind-point locks and registration, instead of per-type or per-instance
* locks which may be misguiding, since for example one instance can be bound to
* many MBeanServers etc.
*
* Also allows for polled checks, i.e. try-lock and either wait or skip. Wait,
* because we probably should not repeat things hidden by this type too often,
* and skip because for example a periodic task checking can just skip if a
* user-initiated registration check is being done.
*
* @author calle
*
*/
@SuppressWarnings("restriction")
public abstract class RegistrationChecker {
private final Lock lock = new ReentrantLock();
public static final EnumSet<RegistrationMode> REMOVE_NO_WAIT = of(Remove);
public static final EnumSet<RegistrationMode> ADD_AND_REMOVE = allOf(RegistrationMode.class);
public final void reap(APIClient client, JmxMBeanServer server) throws OperationsException, UnknownHostException {
check(client, server, REMOVE_NO_WAIT);
}
public final void check(APIClient client, JmxMBeanServer server) throws OperationsException, UnknownHostException {
check(client, server, ADD_AND_REMOVE);
}
public final void check(APIClient client, JmxMBeanServer server, EnumSet<RegistrationMode> mode)
throws OperationsException, UnknownHostException {
if (!lock.tryLock()) {
if (mode.contains(Wait)) {
// someone is doing update.
// since this is jmx, and sloppy, we'll just
// assume that once he is done, things are
// good enough.
lock.lock();
lock.unlock();
}
return;
}
try {
doCheck(client, server, mode);
} finally {
lock.unlock();
}
}
protected abstract void doCheck(APIClient client, JmxMBeanServer server, EnumSet<RegistrationMode> mode)
throws OperationsException, UnknownHostException;
}

View File

@ -0,0 +1,5 @@
package com.scylladb.jmx.metrics;
public enum RegistrationMode {
Wait, Add, Remove,
}

View File

@ -3,7 +3,19 @@ package com.scylladb.jmx.utils;
* Copyright 2016 ScyllaDB
*/
import static com.scylladb.jmx.main.Main.client;
import static com.scylladb.jmx.main.Main.getApiClient;
import static com.sun.jmx.mbeanserver.Util.wildmatch;
import static java.util.logging.Level.SEVERE;
import static javax.management.MBeanServerDelegate.DELEGATE_NAME;
import java.security.AccessController;
import java.security.PrivilegedActionException;
import java.security.PrivilegedExceptionAction;
import java.util.HashMap;
import java.util.Map;
import java.util.Set;
import java.util.concurrent.locks.ReentrantReadWriteLock;
import java.util.logging.Logger;
/*
* This file is part of Scylla.
@ -22,14 +34,463 @@ import static com.scylladb.jmx.main.Main.client;
* along with Scylla. If not, see <http://www.gnu.org/licenses/>.
*/
import javax.management.DynamicMBean;
import javax.management.InstanceAlreadyExistsException;
import javax.management.InstanceNotFoundException;
import javax.management.MBeanServer;
import javax.management.MBeanServerBuilder;
import javax.management.MBeanServerDelegate;
import javax.management.MalformedObjectNameException;
import javax.management.ObjectName;
import javax.management.QueryExp;
import javax.management.RuntimeOperationsException;
import com.sun.jmx.interceptor.DefaultMBeanServerInterceptor;
import com.sun.jmx.mbeanserver.JmxMBeanServer;
import com.sun.jmx.mbeanserver.NamedObject;
import com.sun.jmx.mbeanserver.Repository;
/**
* This class purposly knows way to much of the inner workings
* of Oracle JDK MBeanServer workings, and pervert it for
* performance sakes. It is not portable to other MBean implementations.
*
*/
@SuppressWarnings("restriction")
public class APIBuilder extends MBeanServerBuilder {
private static final Logger logger = Logger.getLogger(APIBuilder.class.getName());
private static class TableRepository extends Repository {
private static final Logger logger = Logger.getLogger(TableRepository.class.getName());
private final Repository wrapped;
private final ReentrantReadWriteLock lock = new ReentrantReadWriteLock();
private final Map<TableMetricParams, DynamicMBean> tableMBeans = new HashMap<>();
private static boolean isTableMetricName(ObjectName name) {
return isTableMetricDomain(name.getDomain());
}
private static boolean isTableMetricDomain(String domain) {
return TableMetricParams.TABLE_METRICS_DOMAIN.equals(domain);
}
public TableRepository(String defaultDomain, final Repository repository) {
super(defaultDomain);
wrapped = repository;
}
@Override
public String getDefaultDomain() {
return wrapped.getDefaultDomain();
}
@Override
public boolean contains(final ObjectName name) {
if (!isTableMetricName(name)) {
return wrapped.contains(name);
} else {
lock.readLock().lock();
try {
return tableMBeans.containsKey(new TableMetricParams(name));
} finally {
lock.readLock().unlock();
}
}
}
@Override
public String[] getDomains() {
final String[] domains = wrapped.getDomains();
if (tableMBeans.isEmpty()) {
return domains;
}
final String[] res = new String[domains.length + 1];
System.arraycopy(domains, 0, res, 0, domains.length);
res[domains.length] = TableMetricParams.TABLE_METRICS_DOMAIN;
return res;
}
@Override
public Integer getCount() {
lock.readLock().lock();
try {
return wrapped.getCount() + tableMBeans.size();
} finally {
lock.readLock().unlock();
}
}
@Override
public void addMBean(final DynamicMBean bean, final ObjectName name, final RegistrationContext ctx)
throws InstanceAlreadyExistsException {
if (!isTableMetricName(name)) {
wrapped.addMBean(bean, name, ctx);
} else {
final TableMetricParams key = new TableMetricParams(name);
lock.writeLock().lock();
try {
if (tableMBeans.containsKey(key)) {
throw new InstanceAlreadyExistsException(name.toString());
}
tableMBeans.put(key, bean);
if (ctx == null) return;
try {
ctx.registering();
} catch (RuntimeOperationsException x) {
throw x;
} catch (RuntimeException x) {
throw new RuntimeOperationsException(x);
}
} finally {
lock.writeLock().unlock();
}
}
}
@Override
public void remove(final ObjectName name, final RegistrationContext ctx) throws InstanceNotFoundException {
if (!isTableMetricName(name)) {
wrapped.remove(name, ctx);
} else {
final TableMetricParams key = new TableMetricParams(name);
lock.writeLock().lock();
try {
if (tableMBeans.remove(key) == null) {
throw new InstanceNotFoundException(name.toString());
}
if (ctx == null) {
return;
}
try {
ctx.unregistered();
} catch (Exception x) {
logger.log(SEVERE, "Unexpected error.", x);
}
} finally {
lock.writeLock().unlock();
}
}
}
@Override
public DynamicMBean retrieve(final ObjectName name) {
if (!isTableMetricName(name)) {
return wrapped.retrieve(name);
} else {
lock.readLock().lock();
try {
return tableMBeans.get(new TableMetricParams(name));
} finally {
lock.readLock().unlock();
}
}
}
private void addAll(final Set<NamedObject> res) {
for (Map.Entry<TableMetricParams, DynamicMBean> e : tableMBeans.entrySet()) {
try {
res.add(new NamedObject(e.getKey().toName(), e.getValue()));
} catch (MalformedObjectNameException e1) {
// This should never happen
logger.log(SEVERE, "Unexpected error.", e1);
}
}
}
private void addAllMatching(final Set<NamedObject> res,
final ObjectNamePattern pattern) {
for (Map.Entry<TableMetricParams, DynamicMBean> e : tableMBeans.entrySet()) {
try {
ObjectName name = e.getKey().toName();
if (pattern.matchKeys(name)) {
res.add(new NamedObject(name, e.getValue()));
}
} catch (MalformedObjectNameException e1) {
// This should never happen
logger.log(SEVERE, "Unexpected error.", e1);
}
}
}
@Override
public Set<NamedObject> query(final ObjectName pattern, final QueryExp query) {
Set<NamedObject> res = wrapped.query(pattern, query);
ObjectName name;
if (pattern == null ||
pattern.getCanonicalName().length() == 0 ||
pattern.equals(ObjectName.WILDCARD)) {
name = ObjectName.WILDCARD;
} else {
name = pattern;
}
lock.readLock().lock();
try {
// If pattern is not a pattern, retrieve this mbean !
if (!name.isPattern() && isTableMetricName(name)) {
final DynamicMBean bean = tableMBeans.get(new TableMetricParams(name));
if (bean != null) {
res.add(new NamedObject(name, bean));
return res;
}
}
// All names in all domains
if (name == ObjectName.WILDCARD) {
addAll(res);
return res;
}
final String canonical_key_property_list_string =
name.getCanonicalKeyPropertyListString();
final boolean allNames =
(canonical_key_property_list_string.length()==0);
final ObjectNamePattern namePattern =
(allNames?null:new ObjectNamePattern(name));
// All names in default domain
if (name.getDomain().length() == 0) {
if (isTableMetricDomain(getDefaultDomain())) {
if (allNames) {
addAll(res);
} else {
addAllMatching(res, namePattern);
}
}
return res;
}
if (!name.isDomainPattern()) {
if (isTableMetricDomain(getDefaultDomain())) {
if (allNames) {
addAll(res);
} else {
addAllMatching(res, namePattern);
}
}
return res;
}
// Pattern matching in the domain name (*, ?)
final String dom2Match = name.getDomain();
if (wildmatch(TableMetricParams.TABLE_METRICS_DOMAIN, dom2Match)) {
if (allNames) {
addAll(res);
} else {
addAllMatching(res, namePattern);
}
}
} finally {
lock.readLock().unlock();
}
return res;
}
}
private final static class ObjectNamePattern {
private final String[] keys;
private final String[] values;
private final String properties;
private final boolean isPropertyListPattern;
private final boolean isPropertyValuePattern;
/**
* The ObjectName pattern against which ObjectNames are matched.
**/
public final ObjectName pattern;
/**
* Builds a new ObjectNamePattern object from an ObjectName pattern.
* @param pattern The ObjectName pattern under examination.
**/
public ObjectNamePattern(ObjectName pattern) {
this(pattern.isPropertyListPattern(),
pattern.isPropertyValuePattern(),
pattern.getCanonicalKeyPropertyListString(),
pattern.getKeyPropertyList(),
pattern);
}
/**
* Builds a new ObjectNamePattern object from an ObjectName pattern
* constituents.
* @param propertyListPattern pattern.isPropertyListPattern().
* @param propertyValuePattern pattern.isPropertyValuePattern().
* @param canonicalProps pattern.getCanonicalKeyPropertyListString().
* @param keyPropertyList pattern.getKeyPropertyList().
* @param pattern The ObjectName pattern under examination.
**/
ObjectNamePattern(boolean propertyListPattern,
boolean propertyValuePattern,
String canonicalProps,
Map<String,String> keyPropertyList,
ObjectName pattern) {
this.isPropertyListPattern = propertyListPattern;
this.isPropertyValuePattern = propertyValuePattern;
this.properties = canonicalProps;
final int len = keyPropertyList.size();
this.keys = new String[len];
this.values = new String[len];
int i = 0;
for (Map.Entry<String,String> entry : keyPropertyList.entrySet()) {
keys[i] = entry.getKey();
values[i] = entry.getValue();
i++;
}
this.pattern = pattern;
}
/**
* Return true if the given ObjectName matches the ObjectName pattern
* for which this object has been built.
* WARNING: domain name is not considered here because it is supposed
* not to be wildcard when called. PropertyList is also
* supposed not to be zero-length.
* @param name The ObjectName we want to match against the pattern.
* @return true if <code>name</code> matches the pattern.
**/
public boolean matchKeys(ObjectName name) {
// If key property value pattern but not key property list
// pattern, then the number of key properties must be equal
//
if (isPropertyValuePattern &&
!isPropertyListPattern &&
(name.getKeyPropertyList().size() != keys.length)) {
return false;
}
// If key property value pattern or key property list pattern,
// then every property inside pattern should exist in name
//
if (isPropertyValuePattern || isPropertyListPattern) {
for (int i = keys.length - 1; i >= 0 ; i--) {
// Find value in given object name for key at current
// index in receiver
//
String v = name.getKeyProperty(keys[i]);
// Did we find a value for this key ?
//
if (v == null) {
return false;
}
// If this property is ok (same key, same value), go to next
//
if (isPropertyValuePattern &&
pattern.isPropertyValuePattern(keys[i])) {
// wildmatch key property values
// values[i] is the pattern;
// v is the string
if (wildmatch(v,values[i])) {
continue;
} else {
return false;
}
}
if (v.equals(values[i])) {
continue;
}
return false;
}
return true;
}
// If no pattern, then canonical names must be equal
//
final String p1 = name.getCanonicalKeyPropertyListString();
final String p2 = properties;
return (p1.equals(p2));
}
}
public static class TableMetricParams {
public static final String TABLE_METRICS_DOMAIN = "org.apache.cassandra.metrics";
private final ObjectName name;
public TableMetricParams(ObjectName name) {
this.name = name;
}
public ObjectName toName() throws MalformedObjectNameException {
return name;
}
private static boolean equal(Object a, Object b) {
return (a == null) ? b == null : a.equals(b);
}
@Override
public boolean equals(Object o) {
if (this == o) return true;
if (!(o instanceof TableMetricParams)) {
return false;
}
TableMetricParams oo = (TableMetricParams) o;
return equal(name.getKeyProperty("keyspace"), oo.name.getKeyProperty("keyspace"))
&& equal(name.getKeyProperty("scope"), oo.name.getKeyProperty("scope"))
&& equal(name.getKeyProperty("name"), oo.name.getKeyProperty("name"))
&& equal(name.getKeyProperty("type"), oo.name.getKeyProperty("type"));
}
private static int hash(Object o) {
return o == null ? 0 : o.hashCode();
}
private static int safeAdd(int ... nums) {
long res = 0;
for (int n : nums) {
res = (res + n) % Integer.MAX_VALUE;
}
return (int)res;
}
@Override
public int hashCode() {
return safeAdd(hash(name.getKeyProperty("keyspace")),
hash(name.getKeyProperty("scope")),
hash(name.getKeyProperty("name")),
hash(name.getKeyProperty("type")));
}
}
@Override
public MBeanServer newMBeanServer(String defaultDomain, MBeanServer outer, MBeanServerDelegate delegate) {
MBeanServer nested = super.newMBeanServer(defaultDomain, outer, delegate);
return new APIMBeanServer(client, nested);
// It is important to set |interceptors| to true while creating the
// JmxMBeanSearver. It is required for calls to
// JmxMBeanServer.setMBeanServerInterceptor() to be allowed.
JmxMBeanServer nested = (JmxMBeanServer) JmxMBeanServer.newMBeanServer(defaultDomain, outer, delegate, true);
// This is not very clean, we depend on knowledge of how the Sun/Oracle
// MBean chain looks internally. But we need haxxor support, so
// lets replace the interceptor.
// Note: Removed reflection gunk to eliminate jdk9+ warnings on
// execution. Also, if we can get by without reflection, it is
// better.
final DefaultMBeanServerInterceptor interceptor = new DefaultMBeanServerInterceptor(outer != null ? outer : nested,
delegate, nested.getMBeanInstantiator(),
new TableRepository(defaultDomain, new Repository(defaultDomain)));
nested.setMBeanServerInterceptor(interceptor);
final MBeanServerDelegate d = nested.getMBeanServerDelegate();
try {
// Interceptor needs the delegate present. Normally done
// by inaccessible method in JmxMBeanServer
AccessController.doPrivileged(new PrivilegedExceptionAction<Object>() {
public Object run() throws Exception {
interceptor.registerMBean(d, DELEGATE_NAME);
return null;
}
});
} catch (PrivilegedActionException e) {
logger.log(SEVERE, "Unexpected error.", e);
throw new RuntimeException(e);
}
return new APIMBeanServer(getApiClient(), nested);
}
}
}

View File

@ -1,10 +1,16 @@
package com.scylladb.jmx.utils;
import static java.util.Arrays.asList;
import static java.util.concurrent.Executors.newScheduledThreadPool;
import static java.util.concurrent.TimeUnit.MINUTES;
import java.io.ObjectInputStream;
import java.net.UnknownHostException;
import java.util.Set;
import java.util.concurrent.ScheduledExecutorService;
import java.util.logging.Logger;
import java.util.regex.Pattern;
import java.util.stream.Collectors;
import javax.management.Attribute;
import javax.management.AttributeList;
@ -33,50 +39,84 @@ import org.apache.cassandra.db.ColumnFamilyStore;
import org.apache.cassandra.metrics.StreamingMetrics;
import com.scylladb.jmx.api.APIClient;
import com.scylladb.jmx.metrics.RegistrationChecker;
import com.sun.jmx.mbeanserver.JmxMBeanServer;
@SuppressWarnings("restriction")
public class APIMBeanServer implements MBeanServer {
@SuppressWarnings("unused")
private static final Logger logger = Logger.getLogger(APIMBeanServer.class.getName());
private static final ScheduledExecutorService executor = newScheduledThreadPool(1);
private final RegistrationChecker columnFamilyStoreChecker = ColumnFamilyStore.createRegistrationChecker();
private final RegistrationChecker streamingMetricsChecker = StreamingMetrics.createRegistrationChecker();
private final APIClient client;
private final MBeanServer server;
private final JmxMBeanServer server;
public APIMBeanServer(APIClient client, MBeanServer server) {
public APIMBeanServer(APIClient client, JmxMBeanServer server) {
this.client = client;
this.server = server;
executor.scheduleWithFixedDelay(() -> {
for (RegistrationChecker c : asList(columnFamilyStoreChecker, streamingMetricsChecker)) {
try {
c.reap(client, server);
} catch (OperationsException | UnknownHostException e) {
// TODO: log?
}
}
}, 1, 5, MINUTES);
}
private static ObjectInstance prepareForRemote(final ObjectInstance i) {
return new ObjectInstance(prepareForRemote(i.getObjectName()), i.getClassName());
}
private static ObjectName prepareForRemote(final ObjectName n) {
/*
* ObjectName.getInstance has changed in JDK (micro) updates so it no longer applies
* overridable methods -> wrong name published.
* Fix by doing explicit ObjectName instansiation.
*/
try {
return new ObjectName(n.getCanonicalName());
} catch (MalformedObjectNameException e) {
throw new IllegalArgumentException(n.toString());
}
}
@Override
public ObjectInstance createMBean(String className, ObjectName name) throws ReflectionException,
InstanceAlreadyExistsException, MBeanRegistrationException, MBeanException, NotCompliantMBeanException {
return server.createMBean(className, name);
return prepareForRemote(server.createMBean(className, name));
}
@Override
public ObjectInstance createMBean(String className, ObjectName name, ObjectName loaderName)
throws ReflectionException, InstanceAlreadyExistsException, MBeanRegistrationException, MBeanException,
NotCompliantMBeanException, InstanceNotFoundException {
return server.createMBean(className, name, loaderName);
return prepareForRemote(server.createMBean(className, name, loaderName));
}
@Override
public ObjectInstance createMBean(String className, ObjectName name, Object[] params, String[] signature)
throws ReflectionException, InstanceAlreadyExistsException, MBeanRegistrationException, MBeanException,
NotCompliantMBeanException {
return server.createMBean(className, name, params, signature);
return prepareForRemote(server.createMBean(className, name, params, signature));
}
@Override
public ObjectInstance createMBean(String className, ObjectName name, ObjectName loaderName, Object[] params,
String[] signature) throws ReflectionException, InstanceAlreadyExistsException, MBeanRegistrationException,
MBeanException, NotCompliantMBeanException, InstanceNotFoundException {
return server.createMBean(className, name, loaderName, params, signature);
return prepareForRemote(server.createMBean(className, name, loaderName, params, signature));
}
@Override
public ObjectInstance registerMBean(Object object, ObjectName name)
throws InstanceAlreadyExistsException, MBeanRegistrationException, NotCompliantMBeanException {
return server.registerMBean(object, name);
return prepareForRemote(server.registerMBean(object, name));
}
@Override
@ -87,19 +127,19 @@ public class APIMBeanServer implements MBeanServer {
@Override
public ObjectInstance getObjectInstance(ObjectName name) throws InstanceNotFoundException {
checkRegistrations(name);
return server.getObjectInstance(name);
return prepareForRemote(server.getObjectInstance(name));
}
@Override
public Set<ObjectName> queryNames(ObjectName name, QueryExp query) {
checkRegistrations(name);
return server.queryNames(name, query);
return server.queryNames(name, query).stream().map(n -> prepareForRemote(n)).collect(Collectors.toSet());
}
@Override
public Set<ObjectInstance> queryMBeans(ObjectName name, QueryExp query) {
checkRegistrations(name);
return server.queryMBeans(name, query);
return server.queryMBeans(name, query).stream().map(i -> prepareForRemote(i)).collect(Collectors.toSet());
}
@Override
@ -265,26 +305,23 @@ public class APIMBeanServer implements MBeanServer {
return server.getClassLoaderRepository();
}
static final Pattern tables = Pattern.compile("^(ColumnFamil(ies|y)|(Index)?Tables?)$");
private boolean checkRegistrations(ObjectName name) {
static final Pattern tables = Pattern.compile("^\\*?((Index)?ColumnFamil(ies|y)|(Index)?(Table(s)?)?)$");
private void checkRegistrations(ObjectName name) {
if (name != null && server.isRegistered(name)) {
return false;
return;
}
boolean result = false;
try {
String type = name != null ? name.getKeyProperty("type") : null;
if (type == null || tables.matcher(type).matches()) {
result |= ColumnFamilyStore.checkRegistration(client, server);
columnFamilyStoreChecker.check(client, server);
}
if (type == null || StreamingMetrics.TYPE_NAME.equals(type)) {
result |= StreamingMetrics.checkRegistration(client, server);
streamingMetricsChecker.check(client, server);
}
} catch (MalformedObjectNameException | UnknownHostException e) {
} catch (OperationsException | UnknownHostException e) {
// TODO: log
}
return result;
}
}

View File

@ -0,0 +1,18 @@
package com.scylladb.jmx.utils;
import jakarta.xml.bind.annotation.adapters.XmlAdapter;
import java.time.Instant;
import java.util.Date;
public class DateXmlAdapter extends XmlAdapter<String, Date> {
@Override
public String marshal(Date v) throws Exception {
return Instant.ofEpochMilli(v.getTime()).toString();
}
@Override
public Date unmarshal(String v) throws Exception {
return new Date(Instant.parse(v).toEpochMilli());
}
}

View File

@ -0,0 +1,16 @@
module scylla.jmx {
opens com.scylladb.jmx.utils;
exports com.scylladb.jmx.utils;
opens com.scylladb.jmx.main;
exports com.scylladb.jmx.main;
opens com.scylladb.jmx.metrics;
exports com.scylladb.jmx.metrics;
requires java.logging;
requires java.management;
requires scylla.apiclient;
requires jakarta.json;
requires jakarta.ws.rs;
requires com.google.common;
requires jakarta.xml.bind;
requires com.fasterxml.jackson.annotation;
}

View File

@ -23,37 +23,55 @@
*/
package org.apache.cassandra.db;
import static jakarta.json.Json.createObjectBuilder;
import static java.lang.String.valueOf;
import static java.util.Arrays.asList;
import static java.util.stream.Collectors.toMap;
import static javax.json.Json.createObjectBuilder;
import jakarta.json.Json;
import jakarta.json.JsonArray;
import jakarta.json.JsonObject;
import jakarta.json.JsonObjectBuilder;
import jakarta.json.JsonReader;
import jakarta.ws.rs.core.MultivaluedHashMap;
import jakarta.ws.rs.core.MultivaluedMap;
import java.io.StringReader;
import java.io.OutputStream;
import java.util.Collections;
import java.util.EnumSet;
import java.util.HashMap;
import java.util.HashSet;
import java.util.List;
import java.util.Map;
import java.util.Set;
import java.util.concurrent.ExecutionException;
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;
import java.util.concurrent.Future;
import java.util.concurrent.TimeUnit;
import java.util.logging.Logger;
import javax.json.Json;
import javax.json.JsonArray;
import javax.json.JsonObject;
import javax.json.JsonObjectBuilder;
import javax.json.JsonReader;
import javax.management.MBeanServer;
import javax.management.MalformedObjectNameException;
import javax.management.ObjectName;
import javax.management.OperationsException;
import javax.management.openmbean.CompositeData;
import javax.management.openmbean.CompositeDataSupport;
import javax.management.openmbean.CompositeType;
import javax.management.openmbean.OpenDataException;
import javax.ws.rs.core.MultivaluedHashMap;
import javax.ws.rs.core.MultivaluedMap;
import javax.management.openmbean.OpenType;
import javax.management.openmbean.SimpleType;
import javax.management.openmbean.TabularDataSupport;
import javax.management.openmbean.TabularType;
import org.apache.cassandra.metrics.TableMetrics;
import com.scylladb.jmx.api.APIClient;
import com.scylladb.jmx.metrics.MetricsMBean;
import com.scylladb.jmx.metrics.RegistrationChecker;
import com.scylladb.jmx.metrics.RegistrationMode;
import com.sun.jmx.mbeanserver.JmxMBeanServer;
import com.google.common.base.Throwables;
public class ColumnFamilyStore extends MetricsMBean implements ColumnFamilyStoreMBean {
private static final Logger logger = Logger.getLogger(ColumnFamilyStore.class.getName());
@ -61,6 +79,72 @@ public class ColumnFamilyStore extends MetricsMBean implements ColumnFamilyStore
private final String type;
private final String keyspace;
private final String name;
private static final String[] COUNTER_NAMES = new String[]{"raw", "count", "error", "string"};
private static final String[] COUNTER_DESCS = new String[]
{ "partition key in raw hex bytes", // Table name and comments match Cassandra, we will use the partition key
"value of this partition for given sampler",
"value is within the error bounds plus or minus of this",
"the partition key turned into a human readable format" };
private static final CompositeType COUNTER_COMPOSITE_TYPE;
private static final TabularType COUNTER_TYPE;
private static final String[] SAMPLER_NAMES = new String[]{"cardinality", "partitions"};
private static final String[] SAMPLER_DESCS = new String[]
{ "cardinality of partitions",
"list of counter results" };
private static final String SAMPLING_RESULTS_NAME = "SAMPLING_RESULTS";
private static final CompositeType SAMPLING_RESULT;
public static final String SNAPSHOT_TRUNCATE_PREFIX = "truncated";
public static final String SNAPSHOT_DROP_PREFIX = "dropped";
private JsonObject tableSamplerResult = null;
private Future<JsonObject> futureTableSamperResult = null;
private ExecutorService service = null;
static
{
try
{
OpenType<?>[] counterTypes = new OpenType[] { SimpleType.STRING, SimpleType.LONG, SimpleType.LONG, SimpleType.STRING };
COUNTER_COMPOSITE_TYPE = new CompositeType(SAMPLING_RESULTS_NAME, SAMPLING_RESULTS_NAME, COUNTER_NAMES, COUNTER_DESCS, counterTypes);
COUNTER_TYPE = new TabularType(SAMPLING_RESULTS_NAME, SAMPLING_RESULTS_NAME, COUNTER_COMPOSITE_TYPE, COUNTER_NAMES);
OpenType<?>[] samplerTypes = new OpenType[] { SimpleType.LONG, COUNTER_TYPE };
SAMPLING_RESULT = new CompositeType(SAMPLING_RESULTS_NAME, SAMPLING_RESULTS_NAME, SAMPLER_NAMES, SAMPLER_DESCS, samplerTypes);
} catch (OpenDataException e)
{
throw Throwables.propagate(e);
}
}
protected synchronized void startTableSampling(MultivaluedMap<String, String> queryParams) {
if (futureTableSamperResult != null) {
return;
}
futureTableSamperResult = service.submit(() -> {
tableSamplerResult = client.getJsonObj("column_family/toppartitions/" + getCFName(), queryParams);
return null;
});
}
/*
* Wait until the action is completed
* It is safe to call this method multiple times
*/
public synchronized void waitUntilSamplingCompleted() {
try {
if (futureTableSamperResult != null) {
futureTableSamperResult.get();
futureTableSamperResult = null;
}
} catch (InterruptedException | ExecutionException e) {
futureTableSamperResult = null;
throw new RuntimeException("Failed getting table statistics", e);
}
}
public static final Set<String> TYPE_NAMES = new HashSet<>(asList("ColumnFamilies", "IndexTables", "Tables"));
@ -74,6 +158,7 @@ public class ColumnFamilyStore extends MetricsMBean implements ColumnFamilyStore
this.type = type;
this.keyspace = keyspace;
this.name = name;
service = Executors.newSingleThreadExecutor();
}
public ColumnFamilyStore(APIClient client, ObjectName name) {
@ -102,15 +187,22 @@ public class ColumnFamilyStore extends MetricsMBean implements ColumnFamilyStore
"org.apache.cassandra.db:type=" + type + ",keyspace=" + keyspace + ",columnfamily=" + name);
}
public static boolean checkRegistration(APIClient client, MBeanServer server) throws MalformedObjectNameException {
JsonArray mbeans = client.getJsonArray("/column_family/");
Set<ObjectName> all = new HashSet<ObjectName>();
for (int i = 0; i < mbeans.size(); i++) {
JsonObject mbean = mbeans.getJsonObject(i);
all.add(getName(mbean.getString("type"), mbean.getString("ks"), mbean.getString("cf")));
}
return checkRegistration(server, all, n -> TYPE_NAMES.contains(n.getKeyProperty("type")), n -> new ColumnFamilyStore(client, n));
}
public static RegistrationChecker createRegistrationChecker() {
return new RegistrationChecker() {
@Override
protected void doCheck(APIClient client, JmxMBeanServer server, EnumSet<RegistrationMode> mode)
throws OperationsException {
JsonArray mbeans = client.getJsonArray("/column_family/");
Set<ObjectName> all = new HashSet<ObjectName>();
for (int i = 0; i < mbeans.size(); i++) {
JsonObject mbean = mbeans.getJsonObject(i);
all.add(getName(mbean.getString("type"), mbean.getString("ks"), mbean.getString("cf")));
}
checkRegistration(server, all, mode,
n -> TYPE_NAMES.contains(n.getKeyProperty("type")), n -> new ColumnFamilyStore(client, n));
}
};
}
/**
* @return the name of the column family
@ -240,7 +332,7 @@ public class ColumnFamilyStore extends MetricsMBean implements ColumnFamilyStore
@Override
public boolean isAutoCompactionDisabled() {
log(" isAutoCompactionDisabled()");
return client.getBooleanValue("column_family/autocompaction/" + getCFName());
return !client.getBooleanValue("column_family/autocompaction/" + getCFName());
}
/** Number of tombstoned cells retreived during the last slicequery */
@ -288,6 +380,23 @@ public class ColumnFamilyStore extends MetricsMBean implements ColumnFamilyStore
return client.getListStrValue("column_family/sstables/by_key/" + getCFName(), queryParams);
}
/**
* Returns a list of filenames that contain the given key on this node
* @param key
* @param hexFormat if key is in hex string format
* @return list of filenames containing the key
*/
@Override
public List<String> getSSTablesForKey(String key, boolean hexFormat)
{
log(" getSSTablesForKey(String key)");
MultivaluedMap<String, String> queryParams = new MultivaluedHashMap<String, String>();
queryParams.add("key", key);
if (hexFormat) {
queryParams.add("format", "hex");
}
return client.getListStrValue("column_family/sstables/by_key/" + getCFName(), queryParams);
}
/**
* Scan through Keyspace/ColumnFamily's data directory determine which
* SSTables should be loaded and load them
@ -415,16 +524,41 @@ public class ColumnFamilyStore extends MetricsMBean implements ColumnFamilyStore
}
@Override
public void beginLocalSampling(String sampler, int capacity) {
// TODO Auto-generated method stub
public void beginLocalSampling(String sampler_base, int capacity) {
MultivaluedMap<String, String> queryParams = new MultivaluedHashMap<String, String>();
queryParams.add("capacity", Integer.toString(capacity));
if (sampler_base.contains(":")) {
String[] parts = sampler_base.split(":");
queryParams.add("duration", parts[1]);
} else {
queryParams.add("duration", "10000");
}
startTableSampling(queryParams);
log(" beginLocalSampling()");
}
@Override
public CompositeData finishLocalSampling(String sampler, int count) throws OpenDataException {
// TODO Auto-generated method stub
public CompositeData finishLocalSampling(String samplerType, int count) throws OpenDataException {
log(" finishLocalSampling()");
return null;
waitUntilSamplingCompleted();
TabularDataSupport result = new TabularDataSupport(COUNTER_TYPE);
JsonArray counters = tableSamplerResult.getJsonArray((samplerType.equalsIgnoreCase("reads")) ? "read" : "write");
long cardinality = tableSamplerResult.getJsonNumber((samplerType.equalsIgnoreCase("reads")) ? "read_cardinality" : "write_cardinality").longValue();
long size = 0;
if (counters != null) {
size = (count > counters.size()) ? counters.size() : count;
for (int i = 0; i < size; i++) {
JsonObject counter = counters.getJsonObject(i);
result.put(new CompositeDataSupport(COUNTER_COMPOSITE_TYPE, COUNTER_NAMES,
new Object[] { counter.getString("partition"), // raw
counter.getJsonNumber("count").longValue(), // count
counter.getJsonNumber("error").longValue(), // error
counter.getString("partition") })); // string
}
}
return new CompositeDataSupport(SAMPLING_RESULT, SAMPLER_NAMES, new Object[] { cardinality, result });
}
}

View File

@ -17,6 +17,7 @@
*/
package org.apache.cassandra.db;
import java.util.Collection;
import java.util.List;
import java.util.Map;
import java.util.concurrent.ExecutionException;
@ -45,6 +46,11 @@ public interface ColumnFamilyStoreMBean {
*/
public void forceMajorCompaction(boolean splitOutput) throws ExecutionException, InterruptedException;
// NOT even default-throw implementing
// forceCompactionForTokenRange
// as this is clearly a misplaced method that should not be in the mbean interface
// (uses internal cassandra types)
/**
* Gets the minimum number of sstables in queue before compaction kicks off
*/
@ -135,6 +141,14 @@ public interface ColumnFamilyStoreMBean {
*/
public List<String> getSSTablesForKey(String key);
/**
* Returns a list of filenames that contain the given key on this node
* @param key
* @param hexFormat if key is in hex string format
* @return list of filenames containing the key
*/
public List<String> getSSTablesForKey(String key, boolean hexFormat);
/**
* Scan through Keyspace/ColumnFamily's data directory determine which
* SSTables should be loaded and load them
@ -154,6 +168,14 @@ public interface ColumnFamilyStoreMBean {
*/
public int[] getSSTableCountPerLevel();
/**
* @return sstable fanout size for level compaction strategy.
*/
default public int getLevelFanoutSize() {
// TODO: implement for real. This is sort of default.
return 10;
}
/**
* Get the ratio of droppable tombstones to real columns (and non-droppable
* tombstones)

View File

@ -22,8 +22,8 @@
*/
package org.apache.cassandra.db.compaction;
import javax.json.JsonArray;
import javax.json.JsonObject;
import jakarta.json.JsonArray;
import jakarta.json.JsonObject;
import javax.management.openmbean.CompositeDataSupport;
import javax.management.openmbean.CompositeType;
import javax.management.openmbean.OpenDataException;

View File

@ -17,18 +17,18 @@
*/
package org.apache.cassandra.db.compaction;
import jakarta.json.JsonArray;
import jakarta.json.JsonObject;
import jakarta.ws.rs.core.MultivaluedHashMap;
import jakarta.ws.rs.core.MultivaluedMap;
import java.util.ArrayList;
import java.util.HashMap;
import java.util.List;
import java.util.Map;
import java.util.logging.Logger;
import javax.json.JsonArray;
import javax.json.JsonObject;
import javax.management.openmbean.OpenDataException;
import javax.management.openmbean.TabularData;
import javax.ws.rs.core.MultivaluedHashMap;
import javax.ws.rs.core.MultivaluedMap;
import org.apache.cassandra.metrics.CompactionMetrics;
@ -75,7 +75,7 @@ public class CompactionManager extends MetricsMBean implements CompactionManager
result.put("keyspace", compaction.getString("ks"));
result.put("columnfamily", compaction.getString("cf"));
result.put("unit", compaction.getString("unit"));
result.put("compactionId", "<none>");
result.put("compactionId", (compaction.containsKey("id"))? compaction.getString("id") : "<none>");
results.add(result);
}
return results;

View File

@ -43,6 +43,18 @@ public interface CompactionManagerMBean {
*/
public void forceUserDefinedCompaction(String dataFiles);
/**
* Triggers the cleanup of user specified sstables.
* You can specify files from various keyspaces and columnfamilies.
* If you do so, cleanup is performed each file individually
*
* @param dataFiles a comma separated list of sstable file to cleanup.
* must contain keyspace and columnfamily name in path(for 2.1+) or file name itself.
*/
default public void forceUserDefinedCleanup(String dataFiles) {
throw new UnsupportedOperationException();
}
/**
* Stop all running compaction-like tasks having the provided {@code type}.
*

View File

@ -24,13 +24,13 @@
package org.apache.cassandra.gms;
import jakarta.json.JsonArray;
import jakarta.json.JsonObject;
import jakarta.json.JsonValue;
import java.net.UnknownHostException;
import java.util.HashMap;
import java.util.Map;
import javax.json.JsonArray;
import javax.json.JsonObject;
import javax.json.JsonValue;
import javax.management.openmbean.CompositeData;
import javax.management.openmbean.CompositeDataSupport;
import javax.management.openmbean.CompositeType;
@ -105,9 +105,11 @@ public class FailureDetector extends APIMBean implements FailureDetectorMBean {
ep.setAliave(obj.getBoolean("is_alive"));
ep.setUpdateTimestamp(obj.getJsonNumber("update_time").longValue());
JsonArray states = obj.getJsonArray("application_state");
for (int j = 0; j < states.size(); j++) {
JsonObject state = states.getJsonObject(j);
ep.addApplicationState(state.getInt("application_state"), state.getString("value"));
if (states != null) {
for (int j = 0; j < states.size(); j++) {
JsonObject state = states.getJsonObject(j);
ep.addApplicationState(state.getInt("application_state"), state.getString("value"));
}
}
res.put(obj.getString("addrs"), ep);
}

View File

@ -23,12 +23,11 @@
*/
package org.apache.cassandra.gms;
import jakarta.ws.rs.core.MultivaluedHashMap;
import jakarta.ws.rs.core.MultivaluedMap;
import java.net.UnknownHostException;
import java.util.logging.Logger;
import javax.ws.rs.core.MultivaluedHashMap;
import javax.ws.rs.core.MultivaluedMap;
import com.scylladb.jmx.api.APIClient;
import com.scylladb.jmx.metrics.APIMBean;

View File

@ -19,13 +19,12 @@ package org.apache.cassandra.locator;
import static java.util.Collections.singletonMap;
import jakarta.ws.rs.core.MultivaluedHashMap;
import jakarta.ws.rs.core.MultivaluedMap;
import java.net.InetAddress;
import java.net.UnknownHostException;
import java.util.logging.Logger;
import javax.ws.rs.core.MultivaluedHashMap;
import javax.ws.rs.core.MultivaluedMap;
import com.scylladb.jmx.api.APIClient;
import com.scylladb.jmx.metrics.APIMBean;

View File

@ -23,6 +23,11 @@
*/
package org.apache.cassandra.metrics;
import jakarta.json.JsonArray;
import jakarta.json.JsonObject;
import java.util.HashMap;
import java.util.Map;
import javax.management.MalformedObjectNameException;
/**
@ -47,6 +52,25 @@ public class CompactionMetrics implements Metrics {
/** Total number of bytes compacted since server [re]start */
registry.register(() -> registry.meter("/compaction_manager/metrics/bytes_compacted"),
factory.createMetricName("BytesCompacted"));
}
registry.register(() -> registry.gauge((client) -> {
Map<String, Map<String, Integer>> result = new HashMap<>();
JsonArray compactions = client.getJsonArray("compaction_manager/metrics/pending_tasks_by_table");
for (int i = 0; i < compactions.size(); i++) {
JsonObject c = compactions.getJsonObject(i);
String ks = c.getString("ks");
String cf = c.getString("cf");
if (!result.containsKey(ks)) {
result.put(ks, new HashMap<>());
}
Map<String, Integer> map = result.get(ks);
map.put(cf, (int)(c.getJsonNumber("task").longValue()));
}
return result;
}), factory.createMetricName("PendingTasksByTableName"));
}
}

View File

@ -21,6 +21,9 @@ import static com.scylladb.jmx.api.APIClient.getReader;
import static java.lang.Math.floor;
import static java.util.logging.Level.SEVERE;
import jakarta.json.JsonArray;
import jakarta.json.JsonNumber;
import jakarta.json.JsonObject;
import java.util.Arrays;
import java.util.Locale;
import java.util.concurrent.TimeUnit;
@ -29,9 +32,6 @@ import java.util.function.Function;
import java.util.function.Supplier;
import java.util.logging.Logger;
import javax.json.JsonArray;
import javax.json.JsonNumber;
import javax.json.JsonObject;
import javax.management.InstanceAlreadyExistsException;
import javax.management.MBeanRegistrationException;
import javax.management.MBeanServer;
@ -39,6 +39,7 @@ import javax.management.NotCompliantMBeanException;
import javax.management.ObjectName;
import com.scylladb.jmx.api.APIClient;
import com.sun.jmx.mbeanserver.JmxMBeanServer;
/**
* Makes integrating 3.0 metrics API with 2.0.
@ -53,9 +54,9 @@ public class MetricsRegistry {
private static final Logger logger = Logger.getLogger(MetricsRegistry.class.getName());
private final APIClient client;
private final MBeanServer mBeanServer;
private final JmxMBeanServer mBeanServer;
public MetricsRegistry(APIClient client, MBeanServer mBeanServer) {
public MetricsRegistry(APIClient client, JmxMBeanServer mBeanServer) {
this.client = client;
this.mBeanServer = mBeanServer;
}
@ -108,7 +109,7 @@ public class MetricsRegistry {
MetricMBean bean = f.get();
for (ObjectName name : objectNames) {
try {
mBeanServer.registerMBean(bean, name);
mBeanServer.getMBeanServerInterceptor().registerMBean(bean, name);
} catch (InstanceAlreadyExistsException | MBeanRegistrationException | NotCompliantMBeanException e) {
logger.log(SEVERE, "Could not register mbean", e);
}
@ -130,7 +131,15 @@ public class MetricsRegistry {
}
public MetricMBean counter(final String url) {
return new JmxCounter(url);
if (url != null) {
return new JmxCounter(url);
}
return new JmxCounter(url) {
@Override
public long getCount() {
return 0;
}
};
}
private abstract class IntermediatelyUpdated {

View File

@ -24,21 +24,24 @@
package org.apache.cassandra.metrics;
import static java.util.Arrays.asList;
import static java.util.Collections.emptySet;
import static org.apache.cassandra.metrics.DefaultNameFactory.createMetricName;
import jakarta.json.JsonArray;
import java.net.InetAddress;
import java.net.UnknownHostException;
import java.util.EnumSet;
import java.util.HashSet;
import java.util.Set;
import javax.json.JsonArray;
import javax.management.MBeanServer;
import javax.management.MalformedObjectNameException;
import javax.management.ObjectName;
import javax.management.OperationsException;
import com.scylladb.jmx.api.APIClient;
import com.scylladb.jmx.metrics.APIMBean;
import com.scylladb.jmx.metrics.RegistrationChecker;
import com.scylladb.jmx.metrics.RegistrationMode;
import com.sun.jmx.mbeanserver.JmxMBeanServer;
/**
* Metrics for streaming.
@ -64,46 +67,45 @@ public class StreamingMetrics {
private static boolean isStreamingName(ObjectName n) {
return TYPE_NAME.equals(n.getKeyProperty("type"));
}
public static RegistrationChecker createRegistrationChecker() {
return new RegistrationChecker() {
@Override
protected void doCheck(APIClient client, JmxMBeanServer server, EnumSet<RegistrationMode> mode) throws OperationsException, UnknownHostException {
Set<ObjectName> all = new HashSet<ObjectName>(globalNames);
JsonArray streams = client.getJsonArray("/stream_manager/");
for (int i = 0; i < streams.size(); i++) {
JsonArray sessions = streams.getJsonObject(i).getJsonArray("sessions");
for (int j = 0; j < sessions.size(); j++) {
String peer = sessions.getJsonObject(j).getString("peer");
String scope = InetAddress.getByName(peer).getHostAddress().replaceAll(":", ".");
all.add(createMetricName(TYPE_NAME, "IncomingBytes", scope));
all.add(createMetricName(TYPE_NAME, "OutgoingBytes", scope));
}
}
public static void unregister(APIClient client, MBeanServer server) throws MalformedObjectNameException {
APIMBean.checkRegistration(server, emptySet(), StreamingMetrics::isStreamingName, (n) -> null);
}
MetricsRegistry registry = new MetricsRegistry(client, server);
APIMBean.checkRegistration(server, all, mode, StreamingMetrics::isStreamingName, n -> {
String scope = n.getKeyProperty("scope");
String name = n.getKeyProperty("name");
public static boolean checkRegistration(APIClient client, MBeanServer server)
throws MalformedObjectNameException, UnknownHostException {
Set<ObjectName> all = new HashSet<ObjectName>(globalNames);
JsonArray streams = client.getJsonArray("/stream_manager/");
for (int i = 0; i < streams.size(); i++) {
JsonArray sessions = streams.getJsonObject(i).getJsonArray("sessions");
for (int j = 0; j < sessions.size(); j++) {
String peer = sessions.getJsonObject(j).getString("peer");
String scope = InetAddress.getByName(peer).getHostAddress().replaceAll(":", ".");
all.add(createMetricName(TYPE_NAME, "IncomingBytes", scope));
all.add(createMetricName(TYPE_NAME, "OutgoingBytes", scope));
}
}
MetricsRegistry registry = new MetricsRegistry(client, server);
return APIMBean.checkRegistration(server, all, StreamingMetrics::isStreamingName, n -> {
String scope = n.getKeyProperty("scope");
String name = n.getKeyProperty("name");
String url = null;
if ("ActiveOutboundStreams".equals(name)) {
url = "/stream_manager/metrics/outbound";
} else if ("IncomingBytes".equals(name) || "TotalIncomingBytes".equals(name)) {
url = "/stream_manager/metrics/incoming";
} else if ("OutgoingBytes".equals(name) || "TotalOutgoingBytes".equals(name)) {
url = "/stream_manager/metrics/outgoing";
}
if (url == null) {
throw new IllegalArgumentException();
}
if (scope != null) {
url = url + "/" + scope;
}
return registry.counter(url);
});
}
String url = null;
if ("ActiveOutboundStreams".equals(name)) {
url = "/stream_manager/metrics/outbound";
} else if ("IncomingBytes".equals(name) || "TotalIncomingBytes".equals(name)) {
url = "/stream_manager/metrics/incoming";
} else if ("OutgoingBytes".equals(name) || "TotalOutgoingBytes".equals(name)) {
url = "/stream_manager/metrics/outgoing";
}
if (url == null) {
throw new IllegalArgumentException();
}
if (scope != null) {
url = url + "/" + scope;
}
return registry.counter(url);
});
}
};
}
}

View File

@ -19,6 +19,9 @@ package org.apache.cassandra.metrics;
import static com.scylladb.jmx.api.APIClient.getReader;
import java.io.InvalidObjectException;
import java.io.ObjectStreamException;
import java.util.Hashtable;
import java.util.function.BiFunction;
import java.util.function.Function;
import java.util.function.Supplier;
@ -27,7 +30,6 @@ import javax.management.MalformedObjectNameException;
import javax.management.ObjectName;
import org.apache.cassandra.db.ColumnFamilyStore;
import org.apache.cassandra.metrics.MetricsRegistry.MetricMBean;
import com.scylladb.jmx.api.APIClient;
@ -155,6 +157,24 @@ public class TableMetrics implements Metrics {
aliasFactory.createMetricName(alias));
}
private static <T> BiFunction<APIClient, String, T> getDummy(Class<T> type) {
if (type == String.class) {
return (c, s) -> type.cast("");
} else if (type == Integer.class) {
return (c, s) -> type.cast(0);
} else if (type == Double.class) {
return (c, s) -> type.cast(0.0);
} else if (type == Long.class) {
return (c, s) -> type.cast(0L);
}
throw new IllegalArgumentException(type.getName());
}
public <T> void createDummyTableGauge(Class<T> c, String name) throws MalformedObjectNameException {
register(() -> gauge(newGauge(getDummy(c), null)), factory.createMetricName(name),
aliasFactory.createMetricName(name));
}
public <L, G> void createTableGauge(Class<L> c1, Class<G> c2, String name, String alias, String uri)
throws MalformedObjectNameException {
if (cfName != null) {
@ -173,6 +193,11 @@ public class TableMetrics implements Metrics {
aliasFactory.createMetricName(alias));
}
public void createDummyTableCounter(String name) throws MalformedObjectNameException {
register(() -> counter(null), factory.createMetricName(name),
aliasFactory.createMetricName(name));
}
public void createTableHistogram(String name, String uri, boolean considerZeros)
throws MalformedObjectNameException {
createTableHistogram(name, name, uri, considerZeros);
@ -205,6 +230,9 @@ public class TableMetrics implements Metrics {
for (LatencyMetrics l : latencyMetrics) {
l.register(registry);
}
// TODO: implement
registry.createDummyTableCounter("DroppedMutations");
}
private static void registerCommon(Registry registry) throws MalformedObjectNameException {
@ -264,9 +292,138 @@ public class TableMetrics implements Metrics {
registry.createTableCounter("RowCacheHitOutOfRange", "row_cache_hit_out_of_range");
registry.createTableCounter("RowCacheHit", "row_cache_hit");
registry.createTableCounter("RowCacheMiss", "row_cache_miss");
// TODO: implement
registry.createDummyTableGauge(Double.class, "PercentRepaired");
}
static class TableMetricNameFactory implements MetricNameFactory {
static class TableMetricObjectName extends javax.management.ObjectName {
private final TableMetricStringNameFactory factory;
private final String metricName;
public TableMetricObjectName(TableMetricStringNameFactory factory, String metricName) throws MalformedObjectNameException {
super("");
this.factory = factory;
this.metricName = metricName;
}
@Override
public boolean isPropertyValuePattern(String property) {
return false;
}
@Override
public String getCanonicalName() {
return factory.createMetricStringName(metricName);
}
@Override
public String getDomain() {
return factory.getDomain();
}
@Override
public String getKeyProperty(String property) {
if (property == "name") {
return metricName;
}
return factory.getKeyProperty(property);
}
@Override
public Hashtable<String,String> getKeyPropertyList() {
Hashtable<String, String> res = factory.getKeyPropertyList();
res.put("name", metricName);
return res;
}
@Override
public String getKeyPropertyListString() {
return factory.getKeyPropertyListString(metricName);
}
@Override
public String getCanonicalKeyPropertyListString() {
return getKeyPropertyListString();
}
@Override
public String toString() {
return getCanonicalName();
}
@Override
public boolean equals(Object o) {
if (this == o) return true;
return getCanonicalName().equals(((ObjectName) o).getCanonicalName());
}
@Override
public int hashCode() {
return getCanonicalName().hashCode();
}
@Override
public boolean apply(ObjectName name) {
if (name.isDomainPattern() || name.isPropertyListPattern() || name.isPropertyValuePattern()) {
return false;
}
return getCanonicalName().equals(name.getCanonicalName());
}
@Override
public boolean isPattern() {
return false;
}
@Override
public boolean isDomainPattern() {
return false;
}
@Override
public boolean isPropertyPattern() {
return false;
}
@Override
public boolean isPropertyListPattern() {
return false;
}
@Override
public boolean isPropertyValuePattern() {
return false;
}
/**
* This type is not really serializable.
* Replace it with vanilla objectname.
*/
private Object writeReplace() throws ObjectStreamException {
try {
return new ObjectName(getDomain(), getKeyPropertyList());
} catch (MalformedObjectNameException e) {
throw new InvalidObjectException(toString());
}
}
}
static interface TableMetricStringNameFactory {
String createMetricStringName(String metricName);
String getDomain();
String getKeyProperty(String property);
Hashtable<String,String> getKeyPropertyList();
String getKeyPropertyListString(String metricName);
}
static class TableMetricNameFactory implements MetricNameFactory, TableMetricStringNameFactory {
private final String keyspaceName;
private final String tableName;
private final boolean isIndex;
@ -279,37 +436,114 @@ public class TableMetrics implements Metrics {
this.type = type;
}
@Override
public ObjectName createMetricName(String metricName) throws MalformedObjectNameException {
String groupName = TableMetrics.class.getPackage().getName();
private void appendKeyPropertyListString(final StringBuilder sb, final String metricName) {
String type = isIndex ? "Index" + this.type : this.type;
// Order matters here - keys have to be sorted
sb.append("keyspace=").append(keyspaceName);
sb.append(",name=").append(metricName);
sb.append(",scope=").append(tableName);
sb.append(",type=").append(type);
}
@Override
public String createMetricStringName(String metricName) {
String groupName = TableMetrics.class.getPackage().getName();
StringBuilder mbeanName = new StringBuilder();
mbeanName.append(groupName).append(":");
mbeanName.append("type=").append(type);
mbeanName.append(",keyspace=").append(keyspaceName);
mbeanName.append(",scope=").append(tableName);
mbeanName.append(",name=").append(metricName);
appendKeyPropertyListString(mbeanName, metricName);
return mbeanName.toString();
}
return new ObjectName(mbeanName.toString());
@Override
public String getDomain() {
return TableMetrics.class.getPackage().getName();
}
@Override
public String getKeyProperty(String property) {
switch (property) {
case "keyspace": return keyspaceName;
case "scope": return tableName;
case "type": return type;
default: return null;
}
}
@Override
public Hashtable<String,String> getKeyPropertyList() {
Hashtable<String, String> res = new Hashtable<>();
res.put("keyspace", keyspaceName);
res.put("scope", tableName);
res.put("type", type);
return res;
}
@Override
public String getKeyPropertyListString(String metricName) {
final StringBuilder sb = new StringBuilder();
appendKeyPropertyListString(sb, metricName);
return sb.toString();
}
@Override
public ObjectName createMetricName(String metricName) throws MalformedObjectNameException {
return new TableMetricObjectName(this, metricName);
}
}
static class AllTableMetricNameFactory implements MetricNameFactory {
static class AllTableMetricNameFactory implements MetricNameFactory, TableMetricStringNameFactory {
private final String type;
public AllTableMetricNameFactory(String type) {
this.type = type;
}
private void appendKeyPropertyListString(final StringBuilder sb, final String metricName) {
// Order matters here - keys have to be sorted
sb.append("name=").append(metricName);
sb.append(",type=" + type);
}
@Override
public ObjectName createMetricName(String metricName) throws MalformedObjectNameException {
public String createMetricStringName(String metricName) {
String groupName = TableMetrics.class.getPackage().getName();
StringBuilder mbeanName = new StringBuilder();
mbeanName.append(groupName).append(":");
mbeanName.append("type=" + type);
mbeanName.append(",name=").append(metricName);
return new ObjectName(mbeanName.toString());
appendKeyPropertyListString(mbeanName, metricName);
return mbeanName.toString();
}
@Override
public String getDomain() {
return TableMetrics.class.getPackage().getName();
}
@Override
public String getKeyProperty(String property) {
switch (property) {
case "type": return type;
default: return null;
}
}
@Override
public Hashtable<String,String> getKeyPropertyList() {
Hashtable<String, String> res = new Hashtable<>();
res.put("type", type);
return res;
}
@Override
public String getKeyPropertyListString(String metricName) {
final StringBuilder sb = new StringBuilder();
appendKeyPropertyListString(sb, metricName);
return sb.toString();
}
@Override
public ObjectName createMetricName(String metricName) throws MalformedObjectNameException {
return new TableMetricObjectName(this, metricName);
}
}

View File

@ -24,6 +24,8 @@ package org.apache.cassandra.net;
import static java.util.Collections.emptyMap;
import jakarta.json.JsonArray;
import jakarta.json.JsonObject;
import java.net.UnknownHostException;
import java.util.HashMap;
import java.util.Map;
@ -32,9 +34,6 @@ import java.util.logging.Logger;
import java.util.stream.Collectors;
import java.util.stream.Stream;
import javax.json.JsonArray;
import javax.json.JsonObject;
import org.apache.cassandra.metrics.DroppedMessageMetrics;
import com.scylladb.jmx.api.APIClient;

View File

@ -122,6 +122,20 @@ public interface MessagingServiceMBean {
*/
public Map<String, Long> getTimeoutsPerHost();
/**
* Back-pressure rate limiting per host
*/
default public Map<String, Double> getBackPressurePerHost() {
throw new UnsupportedOperationException();
}
/**
* Enable/Disable back-pressure
*/
default public void setBackPressureEnabled(boolean enabled) {
throw new UnsupportedOperationException();
}
/**
* Number of timeouts since last check.
*/

View File

@ -24,12 +24,11 @@
package org.apache.cassandra.service;
import jakarta.ws.rs.core.MultivaluedHashMap;
import jakarta.ws.rs.core.MultivaluedMap;
import java.util.concurrent.ExecutionException;
import java.util.logging.Logger;
import javax.ws.rs.core.MultivaluedHashMap;
import javax.ws.rs.core.MultivaluedMap;
import org.apache.cassandra.metrics.CacheMetrics;
import com.scylladb.jmx.api.APIClient;

View File

@ -0,0 +1,66 @@
package org.apache.cassandra.service;
import static com.sun.jmx.mbeanserver.MXBeanMappingFactory.DEFAULT;
import jakarta.xml.bind.annotation.XmlRootElement;
import java.io.InvalidObjectException;
import java.util.List;
import javax.management.openmbean.CompositeData;
import javax.management.openmbean.OpenDataException;
import com.sun.jmx.mbeanserver.MXBeanMapping;
@SuppressWarnings("restriction")
@XmlRootElement
public class PerTableSSTableInfo {
private static final MXBeanMapping mxBeanMapping;
static {
try {
mxBeanMapping = DEFAULT.mappingForType(PerTableSSTableInfo.class, DEFAULT);
} catch (OpenDataException e) {
throw new RuntimeException(e);
}
}
private String keyspace;
private List<SSTableInfo> sstables;
private String table;
public String getTable() {
return table;
}
public void setTable(String table) {
this.table = table;
}
public String getKeyspace() {
return keyspace;
}
public void setKeyspace(String keyspace) {
this.keyspace = keyspace;
}
public List<SSTableInfo> getSSTables() {
return sstables;
}
public void setSSTableInfos(List<SSTableInfo> sstableInfos) {
this.sstables = sstableInfos;
}
public CompositeData toCompositeData() {
try {
return (CompositeData) mxBeanMapping.toOpenValue(this);
} catch (OpenDataException e) {
throw new Error(e); // should not reach.
}
}
public static PerTableSSTableInfo from(CompositeData data) throws InvalidObjectException {
return (PerTableSSTableInfo) mxBeanMapping.fromOpenValue(data);
}
}

View File

@ -0,0 +1,153 @@
package org.apache.cassandra.service;
import jakarta.xml.bind.annotation.adapters.XmlJavaTypeAdapter;
import java.util.Collections;
import java.util.Date;
import java.util.HashMap;
import java.util.List;
import java.util.Map;
import com.fasterxml.jackson.annotation.JsonIgnore;
import com.fasterxml.jackson.annotation.JsonProperty;
import com.scylladb.jmx.utils.DateXmlAdapter;
public class SSTableInfo {
private long size;
@JsonProperty("data_size")
private long dataSize;
@JsonProperty("index_size")
private long indexSize;
@JsonProperty("filter_size")
private long filterSize;
@XmlJavaTypeAdapter(type = Date.class, value = DateXmlAdapter.class)
private Date timestamp;
private long generation;
private long level;
private String version;
private Map<String, String> properties;
public void setProperties(Map<String, String> properties) {
this.properties = properties;
}
@JsonIgnore
private Map<String, Map<String, String>> extendedProperties;
public String getVersion() {
return version;
}
public void setVersion(String version) {
this.version = version;
}
public long getSize() {
return size;
}
public void setSize(long size) {
this.size = size;
}
public long getDataSize() {
return dataSize;
}
public void setDataSize(long dataSize) {
this.dataSize = dataSize;
}
public long getIndexSize() {
return indexSize;
}
public void setIndexSize(long indexSize) {
this.indexSize = indexSize;
}
public long getFilterSize() {
return filterSize;
}
public void setFilterSize(long filterSize) {
this.filterSize = filterSize;
}
public Date getTimestamp() {
return timestamp;
}
public void setTimestamp(Date timestamp) {
this.timestamp = timestamp;
}
public long getGeneration() {
return generation;
}
public void setGeneration(long generation) {
this.generation = generation;
}
public long getLevel() {
return level;
}
public void setLevel(long level) {
this.level = level;
}
public Map<String, String> getProperties() {
return properties;
}
public Map<String, Map<String, String>> getExtendedProperties() {
return extendedProperties;
}
public void setExtendedProperties(Map<String, Map<String, String>> extendedProperties) {
this.extendedProperties = extendedProperties;
}
@JsonProperty("properties")
private void unpackProperties(List<Map<String, String>> maps) {
Map<String, String> result = new HashMap<>();
for (Map<String, String> map : maps) {
String key = map.get("key");
String value = map.get("value");
result.put(key, value);
}
properties = result;
}
@JsonProperty("extended_properties")
private void unpackNested(List<Map<String, Object>> properties) {
Map<String, Map<String, String>> result = new HashMap<String, Map<String, String>>();
for (Map<String, Object> map : properties) {
Object name = map.get("group");
if (name != null) {
Map<String, String> dst = new HashMap<>();
List<?> value = (List<?>) map.get("attributes");
for (Object v : value) {
Map<?, ?> subMap = (Map<?, ?>) v;
dst.put(String.valueOf(subMap.get("key")), String.valueOf(subMap.get("value")));
}
result.put(String.valueOf(name), dst);
} else {
for (Map.Entry<String, Object> e : map.entrySet()) {
result.put(e.getKey(), Collections.singletonMap(String.valueOf(e.getValue()), ""));
}
}
}
extendedProperties = result;
}
}

View File

@ -25,14 +25,16 @@ package org.apache.cassandra.service;
import static java.util.Collections.emptySet;
import jakarta.json.JsonArray;
import jakarta.ws.rs.core.MultivaluedHashMap;
import jakarta.ws.rs.core.MultivaluedMap;
import java.util.HashSet;
import java.util.List;
import java.util.Map;
import java.util.Set;
import java.util.logging.Logger;
import javax.ws.rs.core.MultivaluedHashMap;
import javax.ws.rs.core.MultivaluedMap;
import org.apache.cassandra.db.ColumnFamilyStore;
import org.apache.cassandra.metrics.CASClientRequestMetrics;
import org.apache.cassandra.metrics.ClientRequestMetrics;
@ -54,7 +56,8 @@ public class StorageProxy extends MetricsMBean implements StorageProxyMBean {
new ClientRequestMetrics("RangeSlice", "/storage_proxy/metrics/range"),
new ClientRequestMetrics("Write", "storage_proxy/metrics/write"),
new CASClientRequestMetrics("CASWrite", "storage_proxy/metrics/cas_write"),
new CASClientRequestMetrics("CASRead", "storage_proxy/metrics/cas_read"));
new CASClientRequestMetrics("CASRead", "storage_proxy/metrics/cas_read"),
new ClientRequestMetrics("ViewWrite", "storage_proxy/metrics/view_write"));
}
@Override
@ -286,4 +289,11 @@ public class StorageProxy extends MetricsMBean implements StorageProxyMBean {
log(" getHintedHandoffDisabledDCs()");
return emptySet();
}
@Override
public int getNumberOfTables() {
// TODO: could be like 1000% more efficient
JsonArray mbeans = client.getJsonArray("/column_family/");
return mbeans.size();
}
}

View File

@ -95,6 +95,15 @@ public interface StorageProxyMBean {
public long getReadRepairRepairedBackground();
default public int getOtcBacklogExpirationInterval() {
throw new UnsupportedOperationException();
}
default void setOtcBacklogExpirationInterval(int intervalInMillis) {
throw new UnsupportedOperationException();
}
/** Returns each live node's schema version */
public Map<String, List<String>> getSchemaVersions();
public int getNumberOfTables();
}

View File

@ -24,16 +24,20 @@ package org.apache.cassandra.service;
import static java.util.Arrays.asList;
import jakarta.json.JsonArray;
import jakarta.json.JsonObject;
import jakarta.ws.rs.core.GenericType;
import jakarta.ws.rs.core.MultivaluedHashMap;
import jakarta.ws.rs.core.MultivaluedMap;
import java.io.IOException;
import java.net.InetAddress;
import java.net.UnknownHostException;
import java.nio.ByteBuffer;
import java.util.Arrays;
import java.util.ArrayList;
import java.util.Collection;
import java.util.Collections;
import java.util.HashMap;
import java.util.HashSet;
import java.util.LinkedHashMap;
import java.util.List;
import java.util.Map;
import java.util.Map.Entry;
@ -44,9 +48,8 @@ import java.util.concurrent.ExecutionException;
import java.util.concurrent.TimeoutException;
import java.util.concurrent.atomic.AtomicLong;
import java.util.logging.Logger;
import java.util.stream.Collectors;
import javax.json.JsonArray;
import javax.json.JsonObject;
import javax.management.ListenerNotFoundException;
import javax.management.MBeanNotificationInfo;
import javax.management.Notification;
@ -54,9 +57,15 @@ import javax.management.NotificationBroadcaster;
import javax.management.NotificationBroadcasterSupport;
import javax.management.NotificationFilter;
import javax.management.NotificationListener;
import javax.management.openmbean.CompositeData;
import javax.management.openmbean.CompositeDataSupport;
import javax.management.openmbean.CompositeType;
import javax.management.openmbean.OpenDataException;
import javax.management.openmbean.OpenType;
import javax.management.openmbean.SimpleType;
import javax.management.openmbean.TabularData;
import javax.ws.rs.core.MultivaluedHashMap;
import javax.ws.rs.core.MultivaluedMap;
import javax.management.openmbean.TabularDataSupport;
import javax.management.openmbean.TabularType;
import org.apache.cassandra.metrics.StorageMetrics;
import org.apache.cassandra.repair.RepairParallelism;
@ -64,7 +73,8 @@ import org.apache.cassandra.repair.RepairParallelism;
import com.google.common.base.Joiner;
import com.scylladb.jmx.api.APIClient;
import com.scylladb.jmx.metrics.MetricsMBean;
import com.scylladb.jmx.utils.FileUtils;
import com.scylladb.jmx.api.utils.FileUtils;
import com.google.common.base.Throwables;
/**
* This abstraction contains the token/identifier of this node on the identifier
@ -75,6 +85,41 @@ public class StorageService extends MetricsMBean implements StorageServiceMBean,
private static final Logger logger = Logger.getLogger(StorageService.class.getName());
private static final Timer timer = new Timer("Storage Service Repair", true);
private static final String[] COUNTER_NAMES = new String[]{"raw", "count", "error", "string"};
private static final String[] COUNTER_DESCS = new String[]
{ "partition key in raw hex bytes",
"value of this partition for given sampler",
"value is within the error bounds plus or minus of this",
"the partition key turned into a human readable format" };
private static final CompositeType COUNTER_COMPOSITE_TYPE;
private static final TabularType COUNTER_TYPE;
private static final String[] OPERATION_NAMES = new String[]{"read", "write"};
private static final String[] SAMPLER_NAMES = new String[]{"cardinality", "partitions"};
private static final String[] SAMPLER_DESCS = new String[]
{ "cardinality of partitions",
"list of counter results" };
private static final String SAMPLING_RESULTS_NAME = "SAMPLING_RESULTS";
private static final CompositeType SAMPLING_RESULT;
static
{
try
{
OpenType<?>[] counterTypes = new OpenType[] { SimpleType.STRING, SimpleType.LONG, SimpleType.LONG, SimpleType.STRING };
COUNTER_COMPOSITE_TYPE = new CompositeType(SAMPLING_RESULTS_NAME, SAMPLING_RESULTS_NAME, COUNTER_NAMES, COUNTER_DESCS, counterTypes);
COUNTER_TYPE = new TabularType(SAMPLING_RESULTS_NAME, SAMPLING_RESULTS_NAME, COUNTER_COMPOSITE_TYPE, COUNTER_NAMES);
OpenType<?>[] samplerTypes = new OpenType[] { SimpleType.LONG, COUNTER_TYPE };
SAMPLING_RESULT = new CompositeType(SAMPLING_RESULTS_NAME, SAMPLING_RESULTS_NAME, SAMPLER_NAMES, SAMPLER_DESCS, samplerTypes);
} catch (OpenDataException e)
{
throw Throwables.propagate(e);
}
}
private final NotificationBroadcasterSupport notificationBroadcasterSupport = new NotificationBroadcasterSupport();
@Override
@ -269,7 +314,7 @@ public class StorageService extends MetricsMBean implements StorageServiceMBean,
@Override
public Map<List<String>, List<String>> getRangeToEndpointMap(String keyspace) {
log(" getRangeToEndpointMap(String keyspace)");
return client.getMapListStrValue("/storage_service/range/" + keyspace);
return client.getMapListStrValue("/storage_service/range_to_endpoint_map/" + keyspace);
}
/**
@ -475,6 +520,12 @@ public class StorageService extends MetricsMBean implements StorageServiceMBean,
return client.getListInetAddressValue("");
}
@Override
public void checkAndRepairCdcStreams() throws IOException {
log(" checkAndRepairCdcStreams() throws IOException");
client.post("/storage_service/cdc_streams_check_and_repair");
}
/**
* Takes the snapshot for the given keyspaces. A snapshot name must be
* specified.
@ -486,10 +537,24 @@ public class StorageService extends MetricsMBean implements StorageServiceMBean,
*/
@Override
public void takeSnapshot(String tag, String... keyspaceNames) throws IOException {
takeSnapshot(tag, null, keyspaceNames);
}
@Override
public void takeSnapshot(String tag, Map<String, String> options, String... keyspaceNames) throws IOException {
log(" takeSnapshot(String tag, String... keyspaceNames) throws IOException");
MultivaluedMap<String, String> queryParams = new MultivaluedHashMap<String, String>();
APIClient.set_query_param(queryParams, "tag", tag);
if (keyspaceNames.length == 1 && keyspaceNames[0].indexOf('.') != -1) {
String[] parts = keyspaceNames[0].split("\\.");
keyspaceNames = new String[] { parts[0] };
APIClient.set_query_param(queryParams, "cf", parts[1]);
}
APIClient.set_query_param(queryParams, "kn", APIClient.join(keyspaceNames));
if (options.containsKey("skipFlush")) {
APIClient.set_query_param(queryParams, "sf", options.get("skipFlush"));
}
client.post("/storage_service/snapshots", queryParams);
}
@ -591,6 +656,13 @@ public class StorageService extends MetricsMBean implements StorageServiceMBean,
client.post("/storage_service/keyspace_compaction/" + keyspaceName, queryParams);
}
@Override
public void forceKeyspaceCompactionForTokenRange(String keyspaceName, String startToken, String endToken,
String... tableNames) throws IOException, ExecutionException, InterruptedException {
// TODO: actually handle token ranges.
forceKeyspaceCompaction(keyspaceName, tableNames);
}
/**
* Trigger a cleanup of keys on a single keyspace
*/
@ -625,7 +697,7 @@ public class StorageService extends MetricsMBean implements StorageServiceMBean,
APIClient.set_bool_query_param(queryParams, "disable_snapshot", disableSnapshot);
APIClient.set_bool_query_param(queryParams, "skip_corrupted", skipCorrupted);
APIClient.set_query_param(queryParams, "cf", APIClient.join(columnFamilies));
return client.getIntValue("/storage_service/keyspace_scrub/" + keyspaceName);
return client.getIntValue("/storage_service/keyspace_scrub/" + keyspaceName, queryParams);
}
/**
@ -639,7 +711,7 @@ public class StorageService extends MetricsMBean implements StorageServiceMBean,
MultivaluedMap<String, String> queryParams = new MultivaluedHashMap<String, String>();
APIClient.set_bool_query_param(queryParams, "exclude_current_version", excludeCurrentVersion);
APIClient.set_query_param(queryParams, "cf", APIClient.join(columnFamilies));
return client.getIntValue("/storage_service/keyspace_upgrade_sstables/" + keyspaceName);
return client.getIntValue("/storage_service/keyspace_upgrade_sstables/" + keyspaceName, queryParams);
}
/**
@ -851,7 +923,7 @@ public class StorageService extends MetricsMBean implements StorageServiceMBean,
@Deprecated
public int forceRepairAsync(String keyspace, boolean isSequential, Collection<String> dataCenters,
Collection<String> hosts, boolean primaryRange, boolean repairedAt, String... columnFamilies)
throws IOException {
throws IOException {
log(" forceRepairAsync(String keyspace, boolean isSequential, Collection<String> dataCenters, Collection<String> hosts, boolean primaryRange, boolean repairedAt, String... columnFamilies) throws IOException");
return repairRangeAsync(null, null, keyspace, isSequential, dataCenters, hosts, primaryRange, repairedAt,
columnFamilies);
@ -919,13 +991,21 @@ public class StorageService extends MetricsMBean implements StorageServiceMBean,
* the host id to remove
*/
@Override
public void removeNode(String hostIdString) {
log(" removeNode(String token)");
public void removeNode(String hostIdString, String ignoreNodes) {
log(" removeNode(String token, String ignoreNodes)");
MultivaluedMap<String, String> queryParams = new MultivaluedHashMap<String, String>();
APIClient.set_query_param(queryParams, "host_id", hostIdString);
if (ignoreNodes != null) {
APIClient.set_query_param(queryParams, "ignore_nodes", ignoreNodes);
}
client.post("/storage_service/remove_node", queryParams);
}
public void removeNode(String hostIdString) {
String ignoreNodes = null;
removeNode(hostIdString, ignoreNodes);
}
/**
* Get the status of a token removal.
*/
@ -1095,6 +1175,12 @@ public class StorageService extends MetricsMBean implements StorageServiceMBean,
return client.getListStrValue("/storage_service/keyspaces", queryParams);
}
@Override
public Map<String, String> getViewBuildStatuses(String keyspace, String view) {
log(" getViewBuildStatuses()");
return client.getMapStrValue("storage_service/view_build_statuses/" + keyspace + "/" + view);
}
/**
* Change endpointsnitch class and dynamic-ness (and dynamic attributes) at
* runtime
@ -1270,7 +1356,36 @@ public class StorageService extends MetricsMBean implements StorageServiceMBean,
*/
@Override
public void rebuild(String sourceDc) {
log(" rebuild(String sourceDc)");
rebuild(sourceDc, null, null, null);
}
/**
* Same as {@link #rebuild(String)}, but only for specified keyspace and
* ranges.
*
* @param sourceDc
* Name of DC from which to select sources for streaming or null
* to pick any node
* @param keyspace
* Name of the keyspace which to rebuild or null to rebuild all
* keyspaces.
* @param tokens
* Range of tokens to rebuild or null to rebuild all token
* ranges. In the format of:
* "(start_token_1,end_token_1],(start_token_2,end_token_2],...(start_token_n,end_token_n]"
*/
@Override
public void rebuild(String sourceDc, String keyspace, String tokens, String specificSources) {
log(" rebuild(String sourceDc, String keyspace, String tokens, String specificSources)");
if (keyspace != null) {
throw new UnsupportedOperationException("Rebuild: 'keyspace' not yet supported");
}
if (tokens != null) {
throw new UnsupportedOperationException("Rebuild: 'token range' not yet supported");
}
if (specificSources != null) {
throw new UnsupportedOperationException("Rebuild: 'specific sources' not yet supported");
}
if (sourceDc != null) {
MultivaluedMap<String, String> queryParams = new MultivaluedHashMap<String, String>();
APIClient.set_query_param(queryParams, "source_dc", sourceDc);
@ -1310,15 +1425,25 @@ public class StorageService extends MetricsMBean implements StorageServiceMBean,
* The parent keyspace name
* @param cfName
* The ColumnFamily name where SSTables belong
* @param isLoadAndStream
* Whether or not arbitrary SSTables should be loaded (and streamed to the owning nodes)
*/
@Override
public void loadNewSSTables(String ksName, String cfName) {
log(" loadNewSSTables(String ksName, String cfName)");
public void loadNewSSTables(String ksName, String cfName, boolean isLoadAndStream) {
log(" loadNewSSTables(String ksName, String cfName, boolean isLoadAndStream)");
MultivaluedMap<String, String> queryParams = new MultivaluedHashMap<String, String>();
queryParams.add("cf", cfName);
if (isLoadAndStream) {
queryParams.add("load_and_stream", "true");
}
client.post("/storage_service/sstables/" + ksName, queryParams);
}
@Override
public void loadNewSSTables(String ksName, String cfName) {
loadNewSSTables(ksName, cfName, false);
}
/**
* Return a List of Tokens representing a sample of keys across all
* ColumnFamilyStores.
@ -1389,13 +1514,7 @@ public class StorageService extends MetricsMBean implements StorageServiceMBean,
log("enableAutoCompaction(String ks, String... columnFamilies)");
MultivaluedMap<String, String> queryParams = new MultivaluedHashMap<String, String>();
APIClient.set_query_param(queryParams, "cf", APIClient.join(columnFamilies));
try {
client.post("/storage_service/auto_compaction/" + ks, queryParams);
} catch (RuntimeException e) {
// FIXME should throw the right exception
throw new IOException(e.getMessage());
}
client.post("/storage_service/auto_compaction/" + ks, queryParams);
}
@Override
@ -1614,7 +1733,7 @@ public class StorageService extends MetricsMBean implements StorageServiceMBean,
public int scrub(boolean disableSnapshot, boolean skipCorrupted, boolean checkData, int jobs, String keyspaceName,
String... columnFamilies) throws IOException, ExecutionException, InterruptedException {
// "jobs" not (yet) relevant for scylla. (though possibly useful...)
return scrub(disableSnapshot, skipCorrupted, checkData, 0, keyspaceName, columnFamilies);
return scrub(disableSnapshot, skipCorrupted, checkData, keyspaceName, columnFamilies);
}
@Override
@ -1658,4 +1777,99 @@ public class StorageService extends MetricsMBean implements StorageServiceMBean,
log(" resumeBootstrap");
return false;
}
@Override
public List<CompositeData> getSSTableInfo(String keyspace, String table) {
if (keyspace == null && table != null) {
throw new IllegalArgumentException("Missing keyspace name");
}
MultivaluedMap<String, String> queryParams = null;
if (keyspace != null) {
queryParams = new MultivaluedHashMap<String, String>();
queryParams.add("keyspace", keyspace);
}
if (table != null) {
queryParams.add("cf", table);
}
return client.get("/storage_service/sstable_info", queryParams)
.get(new GenericType<List<PerTableSSTableInfo>>() {
}).stream().map((i) -> i.toCompositeData()).collect(Collectors.toList());
}
@Override
public List<CompositeData> getSSTableInfo() {
return getSSTableInfo(null, null);
}
@Override
public int scrub(boolean disableSnapshot, boolean skipCorrupted, boolean checkData, boolean reinsertOverflowedTTL,
int jobs, String keyspaceName, String... columnFamilies)
throws IOException, ExecutionException, InterruptedException {
MultivaluedMap<String, String> queryParams = new MultivaluedHashMap<String, String>();
APIClient.set_bool_query_param(queryParams, "disable_snapshot", disableSnapshot);
APIClient.set_bool_query_param(queryParams, "skip_corrupted", skipCorrupted);
APIClient.set_query_param(queryParams, "cf", APIClient.join(columnFamilies));
return client.getIntValue("/storage_service/keyspace_scrub/" + keyspaceName, queryParams);
}
@Override
public int scrub(boolean disableSnapshot, String scrubMode, boolean checkData, boolean reinsertOverflowedTTL,
int jobs, String keyspaceName, String... columnFamilies)
throws IOException, ExecutionException, InterruptedException {
MultivaluedMap<String, String> queryParams = new MultivaluedHashMap<String, String>();
APIClient.set_bool_query_param(queryParams, "disable_snapshot", disableSnapshot);
if (!"".equals(scrubMode)) {
APIClient.set_query_param(queryParams, "scrub_mode", scrubMode);
}
APIClient.set_query_param(queryParams, "cf", APIClient.join(columnFamilies));
return client.getIntValue("/storage_service/keyspace_scrub/" + keyspaceName, queryParams);
}
@Override
public long getUptime() {
log("getUptime()");
return client.getLongValue("/system/uptime_ms");
}
@Override
public CompositeData getToppartitions(String sampler, List<String> keyspaceFilters, List<String> tableFilters, int duration, int capacity, int count) throws OpenDataException {
return getToppartitions(Arrays.asList(sampler), keyspaceFilters, tableFilters, duration, capacity, count).get(sampler.toLowerCase());
}
@Override
public Map<String, CompositeData> getToppartitions(List<String> samplers, List<String> keyspaceFilters, List<String> tableFilters, int duration, int capacity, int count) throws OpenDataException {
MultivaluedMap<String, String> queryParams = new MultivaluedHashMap<String, String>();
APIClient.set_query_param(queryParams, "duration", Integer.toString(duration));
APIClient.set_query_param(queryParams, "capacity", Integer.toString(capacity));
APIClient.set_query_param(queryParams, "keyspace_filters", keyspaceFilters != null ? APIClient.join(keyspaceFilters.toArray(new String[0])) : null);
APIClient.set_query_param(queryParams, "table_filters", tableFilters != null ? APIClient.join(tableFilters.toArray(new String[0])) : null);
JsonObject result = client.getJsonObj("/storage_service/toppartitions", queryParams);
Map<String, CompositeData> resultsMap = new HashMap<>();
for (String operation : OPERATION_NAMES) {
JsonArray counters = result.getJsonArray(operation);
long cardinality = result.getJsonNumber(operation + "_cardinality").longValue();
long size = 0;
TabularDataSupport tabularResult = new TabularDataSupport(COUNTER_TYPE);
if (counters != null) {
size = (count > counters.size()) ? counters.size() : count;
for (int i = 0; i < size; i++) {
JsonObject counter = counters.getJsonObject(i);
tabularResult.put(new CompositeDataSupport(COUNTER_COMPOSITE_TYPE, COUNTER_NAMES,
new Object[] { counter.getString("partition"), // raw
counter.getJsonNumber("count").longValue(), // count
counter.getJsonNumber("error").longValue(), // error
counter.getString("partition") })); // string
}
}
resultsMap.put(operation + "s", new CompositeDataSupport(SAMPLING_RESULT, SAMPLER_NAMES, new Object[] { cardinality, tabularResult }));
}
return resultsMap;
}
}

View File

@ -35,7 +35,9 @@ import java.util.concurrent.ExecutionException;
import java.util.concurrent.TimeoutException;
import javax.management.NotificationEmitter;
import javax.management.openmbean.CompositeData;
import javax.management.openmbean.TabularData;
import javax.management.openmbean.OpenDataException;
public interface StorageServiceMBean extends NotificationEmitter {
/**
@ -212,6 +214,7 @@ public interface StorageServiceMBean extends NotificationEmitter {
public List<InetAddress> getNaturalEndpoints(String keyspaceName, ByteBuffer key);
public void checkAndRepairCdcStreams() throws IOException;
/**
* Takes the snapshot for the given keyspaces. A snapshot name must be
* specified.
@ -247,6 +250,14 @@ public interface StorageServiceMBean extends NotificationEmitter {
*/
public void takeColumnFamilySnapshot(String keyspaceName, String columnFamilyName, String tag) throws IOException;
/**
* @deprecated use {@link #takeSnapshot(String tag, Map options, String... entities)} instead.
*/
@Deprecated
default public void takeMultipleTableSnapshot(String tag, String... tableList) throws IOException {
takeMultipleColumnFamilySnapshot(tag, tableList);
}
/**
* Takes the snapshot of a multiple column family from different keyspaces.
* A snapshot name must be specified.
@ -259,6 +270,18 @@ public interface StorageServiceMBean extends NotificationEmitter {
*/
public void takeMultipleColumnFamilySnapshot(String tag, String... columnFamilyList) throws IOException;
/**
* Takes the snapshot of a multiple column family from different keyspaces. A snapshot name must be specified.
*
* @param tag
* the tag given to the snapshot; may not be null or empty
* @param options
* Map of options (skipFlush is the only supported option for now)
* @param entities
* list of keyspaces / tables in the form of empty | ks1 ks2 ... | ks1.cf1,ks2.cf2,...
*/
public void takeSnapshot(String tag, Map<String, String> options, String... entities) throws IOException;
/**
* Remove the snapshot with the given name from the given keyspaces. If no
* tag is specified we will remove all snapshots.
@ -297,6 +320,20 @@ public interface StorageServiceMBean extends NotificationEmitter {
public void forceKeyspaceCompaction(String keyspaceName, String... tableNames)
throws IOException, ExecutionException, InterruptedException;
@Deprecated
default public int relocateSSTables(String keyspace, String ... cfnames) throws IOException, ExecutionException, InterruptedException {
return relocateSSTables(0, keyspace, cfnames);
}
default public int relocateSSTables(int jobs, String keyspace, String ... cfnames) throws IOException, ExecutionException, InterruptedException {
// Node tool op disabled in scylla
throw new UnsupportedOperationException("relocateSSTables");
}
/**
* Forces major compaction of specified token range in a single keyspace
*/
public void forceKeyspaceCompactionForTokenRange(String keyspaceName, String startToken, String endToken, String... tableNames) throws IOException, ExecutionException, InterruptedException;
/**
* Trigger a cleanup of keys on a single keyspace
*/
@ -313,6 +350,10 @@ public interface StorageServiceMBean extends NotificationEmitter {
* scrubbed.
*
* Scrubbed CFs will be snapshotted first, if disableSnapshot is false
*
* scrubMode controls what scrub does when encountering corruption.
* It replaces skipCorrupted where skipCorrupted is equivalent to scrubMode="SKIP".
* Can be one of: "ABORT", "SKIP", "SEGREGATE", or "VALIDATE".
*/
@Deprecated
public int scrub(boolean disableSnapshot, boolean skipCorrupted, String keyspaceName, String... tableNames)
@ -322,9 +363,19 @@ public interface StorageServiceMBean extends NotificationEmitter {
public int scrub(boolean disableSnapshot, boolean skipCorrupted, boolean checkData, String keyspaceName,
String... tableNames) throws IOException, ExecutionException, InterruptedException;
@Deprecated
public int scrub(boolean disableSnapshot, boolean skipCorrupted, boolean checkData, int jobs, String keyspaceName,
String... columnFamilies) throws IOException, ExecutionException, InterruptedException;
@Deprecated
public int scrub(boolean disableSnapshot, boolean skipCorrupted, boolean checkData, boolean reinsertOverflowedTTL,
int jobs, String keyspaceName, String... columnFamilies)
throws IOException, ExecutionException, InterruptedException;
public int scrub(boolean disableSnapshot, String scrubMode, boolean checkData, boolean reinsertOverflowedTTL,
int jobs, String keyspaceName, String... columnFamilies)
throws IOException, ExecutionException, InterruptedException;
/**
* Verify (checksums of) the given keyspace. If tableNames array is empty,
* all CFs are verified.
@ -346,6 +397,15 @@ public interface StorageServiceMBean extends NotificationEmitter {
public int upgradeSSTables(String keyspaceName, boolean excludeCurrentVersion, int jobs, String... tableNames)
throws IOException, ExecutionException, InterruptedException;
/**
* Rewrites all sstables from the given tables to remove deleted data.
* The tombstone option defines the granularity of the procedure: ROW removes deleted partitions and rows, CELL also removes overwritten or deleted cells.
*/
default public int garbageCollect(String tombstoneOption, int jobs, String keyspaceName, String... tableNames) throws IOException, ExecutionException, InterruptedException {
// Node tool op disabled in scylla
throw new UnsupportedOperationException("garbageCollect");
}
/**
* Flush all memtables for the given column families, or all columnfamilies
* for the given keyspace if none are explicitly listed.
@ -453,8 +513,11 @@ public interface StorageServiceMBean extends NotificationEmitter {
* removeToken removes token (and all data associated with enpoint that had
* it) from the ring
*/
@Deprecated
public void removeNode(String token);
public void removeNode(String token, String ignoreNodes);
/**
* Get the status of a token removal.
*/
@ -540,6 +603,8 @@ public interface StorageServiceMBean extends NotificationEmitter {
public List<String> getNonSystemKeyspaces();
public Map<String, String> getViewBuildStatuses(String keyspace, String view);
public List<String> getNonLocalStrategyKeyspaces();
/**
@ -561,6 +626,21 @@ public interface StorageServiceMBean extends NotificationEmitter {
public void updateSnitch(String epSnitchClassName, Boolean dynamic, Integer dynamicUpdateInterval,
Integer dynamicResetInterval, Double dynamicBadnessThreshold) throws ClassNotFoundException;
/*
* Update dynamic_snitch_update_interval_in_ms
*/
default public void setDynamicUpdateInterval(int dynamicUpdateInterval) {
// afaict not used by nodetool.
throw new UnsupportedOperationException("setDynamicUpdateInterval");
}
/*
* Get dynamic_snitch_update_interval_in_ms
*/
default public int getDynamicUpdateInterval() {
throw new UnsupportedOperationException("getDynamicUpdateInterval");
}
// allows a user to forcibly 'kill' a sick node
public void stopGossiping();
@ -596,6 +676,78 @@ public interface StorageServiceMBean extends NotificationEmitter {
public boolean isJoined();
default public boolean isDrained() {
throw new UnsupportedOperationException();
}
default public boolean isDraining() {
throw new UnsupportedOperationException();
}
default public void setRpcTimeout(long value) {
throw new UnsupportedOperationException();
}
default public long getRpcTimeout() {
throw new UnsupportedOperationException();
}
default public void setReadRpcTimeout(long value) {
throw new UnsupportedOperationException();
}
default public long getReadRpcTimeout() {
throw new UnsupportedOperationException();
}
default public void setRangeRpcTimeout(long value) {
throw new UnsupportedOperationException();
}
default public long getRangeRpcTimeout() {
throw new UnsupportedOperationException();
}
default public void setWriteRpcTimeout(long value) {
throw new UnsupportedOperationException();
}
default public long getWriteRpcTimeout() {
throw new UnsupportedOperationException();
}
default public void setCounterWriteRpcTimeout(long value) {
throw new UnsupportedOperationException();
}
default public long getCounterWriteRpcTimeout() {
throw new UnsupportedOperationException();
}
default public void setCasContentionTimeout(long value) {
throw new UnsupportedOperationException();
}
default public long getCasContentionTimeout() {
throw new UnsupportedOperationException();
}
default public void setTruncateRpcTimeout(long value) {
throw new UnsupportedOperationException();
}
default public long getTruncateRpcTimeout() {
throw new UnsupportedOperationException();
}
default public void setStreamingSocketTimeout(int value) {
throw new UnsupportedOperationException();
}
default public int getStreamingSocketTimeout() {
throw new UnsupportedOperationException();
}
public void setStreamThroughputMbPerSec(int value);
public int getStreamThroughputMbPerSec();
@ -607,6 +759,13 @@ public interface StorageServiceMBean extends NotificationEmitter {
public int getCompactionThroughputMbPerSec();
public void setCompactionThroughputMbPerSec(int value);
default public int getConcurrentCompactors() {
throw new UnsupportedOperationException();
}
default public void setConcurrentCompactors(int value) {
throw new UnsupportedOperationException();
}
public boolean isIncrementalBackupsEnabled();
public void setIncrementalBackupsEnabled(boolean value);
@ -623,6 +782,16 @@ public interface StorageServiceMBean extends NotificationEmitter {
*/
public void rebuild(String sourceDc);
/**
* Same as {@link #rebuild(String)}, but only for specified keyspace and ranges.
*
* @param sourceDc Name of DC from which to select sources for streaming or null to pick any node
* @param keyspace Name of the keyspace which to rebuild or null to rebuild all keyspaces.
* @param tokens Range of tokens to rebuild or null to rebuild all token ranges. In the format of:
* "(start_token_1,end_token_1],(start_token_2,end_token_2],...(start_token_n,end_token_n]"
*/
public void rebuild(String sourceDc, String keyspace, String tokens, String specificSources);
/** Starts a bulk load and blocks until it completes. */
public void bulkLoad(String directory);
@ -641,7 +810,16 @@ public interface StorageServiceMBean extends NotificationEmitter {
* The parent keyspace name
* @param cfName
* The ColumnFamily name where SSTables belong
* @param isLoadAndStream
* Whether or not arbitrary SSTables should be loaded (and streamed to the owning nodes)
*/
public void loadNewSSTables(String ksName, String cfName, boolean isLoadAndStream);
/**
* @deprecated use {@link #loadNewSSTables(String ksName, String cfName, boolean isLoadAndStream)} instead.
* This is kept for backward compatibility.
*/
@Deprecated
public void loadNewSSTables(String ksName, String cfName);
/**
@ -663,6 +841,10 @@ public interface StorageServiceMBean extends NotificationEmitter {
public void resetLocalSchema() throws IOException;
default public void reloadLocalSchema() {
throw new UnsupportedOperationException();
}
/**
* Enables/Disables tracing for the whole system. Only thrift requests can
* start tracing currently.
@ -719,4 +901,15 @@ public interface StorageServiceMBean extends NotificationEmitter {
* Sets the hinted handoff throttle in kb per second, per delivery thread.
*/
public boolean resumeBootstrap();
public List<CompositeData> getSSTableInfo(String keyspace, String table);
public List<CompositeData> getSSTableInfo();
/** retun the system uptime */
public long getUptime();
public CompositeData getToppartitions(String sampler, List<String> keyspaceFilters, List<String> tableFilters, int duration, int capacity, int count) throws OpenDataException;
public Map<String, CompositeData> getToppartitions(List<String> samplers, List<String> keyspaceFilters, List<String> tableFilters, int duration, int capacity, int count) throws OpenDataException;
}

View File

@ -24,15 +24,14 @@
package org.apache.cassandra.streaming;
import jakarta.json.JsonArray;
import jakarta.json.JsonObject;
import java.io.Serializable;
import java.net.InetAddress;
import java.net.UnknownHostException;
import java.util.HashMap;
import java.util.Map;
import javax.json.JsonArray;
import javax.json.JsonObject;
import com.google.common.base.Objects;
/**

View File

@ -24,6 +24,8 @@
package org.apache.cassandra.streaming;
import jakarta.json.JsonArray;
import jakarta.json.JsonObject;
import java.io.Serializable;
import java.net.InetAddress;
import java.net.UnknownHostException;
@ -32,9 +34,6 @@ import java.util.HashSet;
import java.util.Map;
import java.util.Set;
import javax.json.JsonArray;
import javax.json.JsonObject;
import com.google.common.collect.Iterables;
/**

View File

@ -24,12 +24,12 @@
package org.apache.cassandra.streaming;
import jakarta.json.JsonArray;
import jakarta.json.JsonObject;
import java.util.HashSet;
import java.util.Set;
import java.util.logging.Logger;
import javax.json.JsonArray;
import javax.json.JsonObject;
import javax.management.ListenerNotFoundException;
import javax.management.MBeanNotificationInfo;
import javax.management.NotificationBroadcasterSupport;

View File

@ -24,13 +24,12 @@
package org.apache.cassandra.streaming;
import jakarta.json.JsonArray;
import jakarta.json.JsonObject;
import java.util.ArrayList;
import java.util.Collection;
import java.util.UUID;
import javax.json.JsonArray;
import javax.json.JsonObject;
import com.google.common.base.Objects;
/**