Commit Graph

490 Commits

Author SHA1 Message Date
Yaron Kaikov
1e836211ce release: prepare for 4.1.9 2020-10-26 18:13:45 +02:00
Yaron Kaikov
398e4bd1db release: prepare for 4.1.8 2020-09-28 20:25:30 +03:00
Yaron Kaikov
5a327a4106 release: prepare for 4.1.7 2020-09-17 21:30:51 +03:00
Yaron Kaikov
ecc32f1fc5 release: prepare for 4.1.6 2020-08-30 21:35:04 +03:00
Yaron Kaikov
1ec049de11 release: prepare for 4.1.5 2020-08-17 21:55:23 +03:00
Avi Kivity
553e816430 dist: debian: do not require root during package build
Debian package builds provide a root environment for the installation
scripts, since that's what typical installation scripts expect. To
avoid providing actual root, a "fakeroot" system is used where syscalls
are intercepted and any effect that requires root (like chown) is emulated.

However, fakeroot sporadically fails for us, aborting the package build.
Since our install scripts don't really require root (when operating in
the --packaging mode), we can just tell dpkg-buildpackage that we don't
need fakeroot. This ought to fix the sporadic failures.

As a side effect, package builds are faster.

Follows scylla.git's b608af870b0a1ad88b91a72bddeff0c321877f9e.

Refs scylladb/scylla#6655.

(cherry picked from commit 626fd75173)
2020-08-17 10:00:00 +03:00
Yaron Kaikov
d2dd8d027c release: prepare for 4.1.4 2020-08-09 08:49:50 +03:00
Yaron Kaikov
7db22144ac release: prepare for 4.1.3 2020-07-29 08:26:23 +03:00
Takuya ASADA
faea829ba0 dist/debian: don't install systemd unit by install.sh, use debian/*.service
Installing *.service by install.sh script causes the error on installing .deb
package, use debian/*.service instead.

Fixes scylladb/scylla#6010
Related scylladb/scylla#5640
Related 29285b28e2

(cherry picked from commit 2883a8dc63)
2020-07-27 14:19:55 +03:00
Hagit Segev
75618699ec release: prepare for 4.1.2 2020-07-14 23:56:23 +03:00
Hagit Segev
5de17ec482 release: prepare for 4.1.1 2020-06-25 08:06:51 +03:00
Yaron Kaikov
9f366b16fb release: prepare for 4.1.0 2020-06-18 14:43:15 +03:00
Hagit Segev
843abf8376 release: prepare for 4.1.rc2 2020-06-08 16:37:53 +03:00
Hagit Segev
67a43e31e6 release: prepare for 4.1.rc1 2020-05-26 18:57:49 +03:00
Yaron Kaikov
ee72ec2288 release: prepare for 4.1.rc0 2020-05-11 11:32:22 +03:00
Takuya ASADA
773a82d539 dist: allow specify JVM options from sysconfig (#93)
Add SCYLLA_JMX_JVM_OPTS on sysconfig to specify JVM options.

Reviewed-by: Ľuboš Koščo <lubos@scylladb.com>

Fixes #58
2020-01-28 12:43:03 +02:00
Takuya ASADA
46681753cd dist: add /usr/lib/scylla/jmx for compatibility (#91)
On the commit 4c8660d, we dropped /usr/lib/scylla/jmx since it likely no user
script invoke scripts under the directory.
However, we found there are possibility scylla-jmx.service tries to load .jar
file from /usr/lib/scylla/jmx, when user upgraded from older version of scylla.
Because /etc/sysconfig/scylla-jmx is marked as 'noreplace' on our rpm,
yum upgrade may keep old sysconfig file when it modified by user, that may
causes to load .jar from /usr/lib/scylla/jmx since we specify the path in the
sysconfig file.

To avoid the issue it's better to have symlinks on /usr/lib/scylla/jmx for
safety.

See #90
2020-01-16 15:51:39 +02:00
Takuya ASADA
29601254fc dist/redhat: call systemctl --daemon-reload when upgraded (#92)
Since %systemd_post does not call systemctl --daemon-reload, we need to call it
manually to apply changes.

Fixes #90
2020-01-08 13:35:38 +02:00
Takuya ASADA
4c8660d41a dist: drop symlink to scripts (#89)
This is scylla-jmx part of https://github.com/scylladb/scylla/pull/5530

After we stopped replacing /usr/lib/scylla with symlink,
creating symlink on /opt/scylladb/scripts/jmx become meaningless,
so we can drop it now.

We able to introduce new symlink on /usr/lib/scylla, but it likely no user
directly invoke scripts under /usr/lib/scylla/jmx, so we don't have to
do that.
2019-12-30 13:51:33 +02:00
Takuya ASADA
2f34a97c6e dist/debian: add AdoptOpenJDK for Debian 10
Since Debian 10 dropped OpenJDK 8, so we need to switch to other
JVM distribution which still maintain OpenJDK 8 for Debian.
We can use AdoptOpenJDK for such environment.

See scylladb/scylla#5463

Signed-off-by: Takuya ASADA <syuu@scylladb.com>
Message-Id: <20191218124838.35017-1-syuu@scylladb.com>
2019-12-18 15:32:04 +02:00
Takuya ASADA
236ffa6c98 dist/debian: add Conflicts with openjdk-11
Since Debian variants can install multiple JRE, "Depends: openjdk-8-jre" may
not mean default JRE is openjdk-8.
We should block openjdk-11 for now, until we support it.

Related with scylladb/scylla#5463

Signed-off-by: Takuya ASADA <syuu@scylladb.com>
Message-Id: <20191218111835.25618-1-syuu@scylladb.com>
2019-12-18 14:08:28 +02:00
Takuya ASADA
771cf6ea50 dist/redhat: force xz compression on rpm binary payload
Same as 301c835cbf,
Fedora 31 switched the default compression to zstd, which isn't readable
by some older rpm distributions (CentOS 7 in particular). Tell it to use
the older xz compression instead, so packages produced on Fedora 31 can
be installed on older distributions.

See: https://github.com/scylladb/scylla/pull/5310

Signed-off-by: Takuya ASADA <syuu@scylladb.com>
Message-Id: <20191210191441.108774-1-syuu@scylladb.com>
2019-12-10 22:18:29 +02:00
Takuya ASADA
31e6bcf9be dist/redhat: fix rpmbuild error on Fedora 31
Same as scylladb/scylla-ami#53, it seems like rpm macro %systemd_postun requires
 one argument starting from Fedora 31, otherwise it causes the error.
The solution is passing systemd unit name just like
%systemd_post and %systemd_preun.

see: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/thread/TU3T2ZYY67SMAJFR2TD4HY6SCPPDVS5V/

Fixes #87

Signed-off-by: Takuya ASADA <syuu@scylladb.com>
Message-Id: <20191205120514.9382-1-syuu@scylladb.com>
2019-12-05 17:03:10 +02:00
Alexandros Bantis
d8c47603d9 Create a HTTP client per instance (#86)
Create javax HTTP client once per instance instead of per request.

Fixes #82
2019-11-19 17:28:09 +02:00
Pekka Enberg
f45ae1833e Add Pekka as a "code owner" on GitHub (#85)
Add myself as a "code owner" so that I am assigned a review
automatically:

https://help.github.com/en/github/creating-cloning-and-archiving-repositories/about-code-owners

I also wanted to add Amnon and Calle, but apparently you need to have
write permissions in order to be a code owner.

The purpose of this automation is to ensure Scylla JMX pull requests
show up in my github.com/pulls page. Thanks Maciej Zimnoch for the tip!
2019-11-14 02:18:33 -08:00
dependabot[bot]
8e1beb11f4 Upgrade jackson-databind from 2.9.9 to 2.9.10.1 (#84)
This upgrades jackson-databind dependency from version 2.9.9 to 2.9.10.1, which fixes various security vulnerabilities:

https://www.cvedetails.com/vulnerability-list/vendor_id-15866/product_id-42991/Fasterxml-Jackson-databind.html
2019-11-13 19:57:55 +02:00
Glauber Costa
27fed6136a Run scylla-jmx in a systemd slice (#79)
Scylla now supports server-defined systemd slices that are used to provide
isolation between components.

This patch adds scylla-jmx to the helper slice. This will guarantee that
scylla-jmx does not use too much resources, influencing the server performance.

Signed-off-by: Glauber Costa <glauber@scylladb.com>
2019-11-13 18:28:14 +02:00
Amos Kong
fa00e84794 build_deb.sh: don't generate scylla-jmx.service from mustache template (#81)
Takuya had untemplataize scylla-jmx.service in commit e8355087ea
But the build_deb.sh still tries to generate service file from a deleted
mustache template file -- scylla-jmx.spec.mustache. It wrongly redirects
a path to service file, then scylla-jmx would fail to start.

Fixes #80
2019-10-01 18:35:50 +03:00
Calle Wilund
f915f8fc7a sstableinfo: Fix deserizalization of "properties"
Refs #76

Since the incoming json uses swagger "key", "value" syntax
we need to do explicit deserialization of this property
as well (not just extended props).

Message-Id: <20190930115432.27801-1-calle@scylladb.com>
2019-09-30 15:29:52 +03:00
Avi Kivity
dc7f37b901 Merge "nonroot installer" from Takuya
"This is nonroot installer patchset v4, for scylla-jmx."

* 'nonroot_v4' of https://github.com/syuu1228/scylla-jmx:
  install.sh: add --nonroot mode
  dist/common/systemd: untemplataize *.service, use drop-in units instead
  dist: move package build script to install.sh
2019-09-10 14:46:04 +03:00
Takuya ASADA
9ef12f4651 install.sh: add --nonroot mode
This implements the way to install Scylla without requires root privilege,
not distribution dependent, does not uses package manager.
2019-09-04 09:54:42 +09:00
Takuya ASADA
e8355087ea dist/common/systemd: untemplataize *.service, use drop-in units instead
Since systemd unit can override parameters using drop-in unit, we don't need
mustache template for them.

Also, drop --disttype option on install.sh since it does not required anymore,
introduce --sysconfdir instead for non-redhat distributions.
2019-09-04 08:54:05 +09:00
Takuya ASADA
a1044e3bd1 dist: move package build script to install.sh
Move package build script from .rpm/.deb to single script, install.sh.
We need this to support nonroot mode, and also to keep package build script
consistent between .rpm/.deb.
2019-09-04 08:30:37 +09:00
Pekka Enberg
04ea3ab7e0 Merge 'Implement sstable_info command' from Calle
"Fixes #76

Implements JMX level call for "sstable_info" REST api command.

Requires seastar patch:
json: Make date formatter use RFC8601/RFC3339 format

Requires scylla patch set "Implement sstable_info API command (info on sstables)"

Forwards call to REST sstable_info and packs the data
into CompositeData for JMX consumption."
* 'sstabledesc' of git://github.com/elcallio/scylla-jmx:
  storage_service: Add "getSSTableInfo" command/attribute
  service: Add objects for deserializing sstable_info json
  scylla-apiclient: Add Date json serializer helper
  APIClient: Add jackson JSON serializer support to client object
  apiclient/pom.xml: Add jackson JSON support libs for REST client
2019-08-13 14:40:25 +03:00
Calle Wilund
133b2e4728 storage_service: Add "getSSTableInfo" command/attribute
Fixes #76

Requires seastar patch:
 json: Make date formatter use RFC8601/RFC3339 format

Requires scylla patch set "Sstabledesc"

Forwards call to REST sstable_info and packs the data
into CompositeData for JMX consumption.
2019-08-06 08:12:14 +00:00
Amnon Heiman
71170f5713 CompactionMetrics: use the pending compaction API (#75)
The PendingTasksByTableName metric should use the pending_tasks_by_table
API to get the real value of the pending compaction.

Fixes #74

Signed-off-by: Amnon Heiman <amnon@scylladb.com>
2019-08-05 14:12:48 +03:00
Amnon Heiman
ff0723abc6 ColumnFamilyStore: Mbean API support the hex format param (#69)
Cassandra 3.0 version of the JMX added a parameter that allows accepting
the parameter as hex.

This breaks the current implementation with a NoSuchMethodException.

This patch adds the missing implementation.

For a full support, a follow up patch in Scylla is needed, but for the
current functionality it would work.

After this patch usage example:

nodetool getsstables keyspace1 standard1 39303138374b4d343830

Signed-off-by: Amnon Heiman <amnon@scylladb.com>
2019-07-29 10:09:04 +03:00
Calle Wilund
cb42205061 service: Add objects for deserializing sstable_info json
Objects + serial logic to automate the transform of
scylla REST json object for sstable_info into
compositedata that can be consumed by nodetool
2019-07-24 14:31:10 +00:00
Calle Wilund
b2f3eeee05 scylla-apiclient: Add Date json serializer helper
To handle RFC8601 formattedd dates in JAXB
2019-07-24 14:30:02 +00:00
Calle Wilund
d8efa60ab7 APIClient: Add jackson JSON serializer support to client object
Allows java ws to deserialize json objects directly.
2019-07-24 14:28:38 +00:00
Calle Wilund
bbc817013e apiclient/pom.xml: Add jackson JSON support libs for REST client 2019-07-24 14:27:56 +00:00
Calle Wilund
263735379e scylla-jmx: Remove depdendency-reduced-pom.xml from tracking
This file is (re-)generated by maven on occasions. It should
not be version controlled.

Add to .gitignore as well.
2019-07-23 11:05:34 +03:00
Amnon Heiman
f0d2df3d15 StorageProxy.java: Add view write metrics
nodetool proxyhistograms command look for the view write metric.

While we do not report that metric yet, we still want the command to
succeed.

After this patch:
$ nodetool proxyhistograms
proxy histograms
Percentile       Read Latency      Write Latency      Range Latency   CAS Read Latency  CAS Write Latency View Write Latency
                     (micros)           (micros)           (micros)           (micros)           (micros)           (micros)
50%                    326.00             110.00             424.50               0.00               0.00               0.00
75%                   1253.00             193.25             877.75               0.00               0.00               0.00
95%                   2935.90            1007.25            5182.55               0.00               0.00               0.00
98%                   3100.00            1040.60            5492.00               0.00               0.00               0.00
99%                   3100.00            1058.00            5492.00               0.00               0.00               0.00
Min                     34.00               9.00              36.00               0.00               0.00               0.00
Max                   3100.00            1058.00            5492.00               0.00               0.00               0.00

See scylladb/scylla#4470

Signed-off-by: Amnon Heiman <amnon@scylladb.com>
Message-Id: <20190514063316.28040-1-amnon@scylladb.com>
2019-07-21 19:20:37 +03:00
Amnon Heiman
c7bce65919 APIMBeanServer: Support both Table and Tables as metric name
Fixes #71

Signed-off-by: Amnon Heiman <amnon@scylladb.com>
2019-07-17 10:56:44 +03:00
Pekka Enberg
4303f06426 Merge "Make API client a separate module for reuse" from Ľuboš
There's an effort to implement a version of "nodetool" that uses
Scylla's REST API directly, Let's make the API client a separate module,
so nodetool can use it.

* 'scylla-apiclient' of https://github.com/tarzanek/scylla-jmx:
  fix README for building instructions
  trigger build from parent maven to have the local repo properly set up
  cleanup commented implicit steps in mvn
  make scylla-apiclient a separate module so the jar can be reused
2019-07-10 22:31:12 +03:00
Lubos Kosco
183eb6158a fix README for building instructions 2019-07-08 11:02:45 +02:00
Lubos Kosco
4296c7d3ae trigger build from parent maven to have the local repo properly set up 2019-07-08 10:54:04 +02:00
Lubos Kosco
222990d821 cleanup commented implicit steps in mvn 2019-07-08 10:04:24 +02:00
Lubos Kosco
91ae4ec8ee make scylla-apiclient a separate module so the jar can be reused 2019-07-01 17:33:08 +02:00
Amnon Heiman
9dae28e2f0 ColumnFamilyStore: finishLocalSampling should respect count limit
When calling nodetool toppartitions with size limit, finishLocalSampling
should respect that and limit the number of the results.

Example:
$ nodetool toppartitions -k 2 keyspace1 standard1 20
WRITES Sampler:
  Cardinality: ~2 (256 capacity)
  Top 2 partitions:
	Partition                Count       +/-
	38333032394d4f4d5030         4         3
	4e353937383137503330         4         3

READS Sampler:
  Cardinality: ~2 (256 capacity)
  Top 2 partitions:
	Nothing recorded during sampling period...

Fixes #66

Signed-off-by: Amnon Heiman <amnon@scylladb.com>
2019-06-23 15:31:18 +03:00