Commit Graph

133 Commits

Author SHA1 Message Date
Amnon Heiman
a568e52cb5 Adding the RecentEstimatedHistogram
Some of the jmx methods uses the notion of recent estimated histogram.
In origin the implementation uses an estimated histogram and clean the
histogram values on each call.

The RecentEstimatedHistogram mimic this behaviour, it store the latest
values of the last call. In each call new values are stored in the
histogram and the results is the delta between the last two calls.

Signed-off-by: Amnon Heiman <amnon@cloudius-systems.com>
2015-10-21 14:19:57 +03:00
Pekka Enberg
be8eea6bd4 Merge "Adding cache support to the jmx proxy" from Amnon
"Some of the nodetool commands calls multiple time to the same methods,
 specifically nodetool status and ring calls the endpoint snitch get rack
 and datacenter many times during the same command. It result in a long
 execution time.

 This series adds the ability to cache results when needed. Typically
 cached result will be for a few seconds, there is no need to a clean up
 mechanism as we don't expect there will be too much information in the
 cache.

 Currently the only cached results are for the endpoint snitch but it is
 easy to add more if we see that it is needed."
2015-10-07 12:32:59 +03:00
Amnon Heiman
6ed255f2e3 EndpointSnitchInfo: Cache results for data center and rack
This cache the get data center and get rack results for 10s, it has a
direct impact on nodetool status and nodetool ring

Signed-off-by: Amnon Heiman <amnon@cloudius-systems.com>
2015-10-07 10:24:50 +03:00
Amnon Heiman
929d0aedd4 EndpointSnitchInfo formatting 2015-10-07 10:14:49 +03:00
Amnon Heiman
c5c0fb41fe APIClient: Adding cache support to the APIClient
Some operations are not changing frequently and are called multiple time
during a nodetool execution.

This patch adds the ability to cache results for a define period of time
(typically it will be for a few seconds) so that during the same
nodetool command call, the results will be retrieved from the cache.

It is currently only implemented for string values, other commands will
be added when needed.

Signed-off-by: Amnon Heiman <amnon@cloudius-systems.com>
2015-10-07 10:13:05 +03:00
Amnon Heiman
38af24d49d Adding CacheEntry to be used by the APIClient cache 2015-10-07 09:43:37 +03:00
Pekka Enberg
a90e080ee6 Merge "Complete the nodetool cfstats support" from Amnon
"This series complete the scylla series to support the nodetool cfstatus support.

After this series it will be possible to call nodetoold cfstatus and get a meaningfull output.

An output example:
./bin/nodetool cfstats keyspace1
Keyspace: keyspace1
        Read Count: 87657
        Read Latency: 1.1418900715287998 ms.
        Write Count: 87177
        Write Latency: 0.022303761313190406 ms.
        Pending Flushes: 0
                Table: standard1
                SSTable count: 8
                SSTables in each level: [               Space used (live): 92356832
                Space used (total): 92356832
                Space used by snapshots (total): 0
                Off heap memory used (total): 106430512
                SSTable Compression Ratio: 0.0
                Number of keys (estimate): 328672
                Memtable cell count: 100000
                Memtable data size: 84800254
                Memtable off heap memory used: 105906176
                Memtable switch count: 4
                Local read count: 92854
                Local read latency: 1.039 ms
                Local write count: 93880
                Local write latency: 1.045 ms
                Pending flushes: 0
                Bloom filter false positives: 0
                Bloom filter false ratio: 0.00000
                Bloom filter space used: 208416
                Bloom filter off heap memory used: 524336
                Index summary off heap memory used: 0
                Compression metadata off heap memory used: 0
                Compacted partition minimum bytes: 259
                Compacted partition maximum bytes: 310"
2015-10-06 12:23:09 +03:00
Pekka Enberg
e7637a643a Merge branch 'branch-0.9'
Amnon Heiman (1):
      EndPointSnitchInfo: Support null as host name

Asias He (1):
      rpm: Fix build in centos container

Takuya ASADA (2):
      dist: check hostname is resolvable
      dist: does not need maven for running time
2015-10-06 12:21:40 +03:00
Amnon Heiman
a8292e3e09 ColumnFamilyMetrics: add estimated row count histogram
This adds the estimated row count histogram support to
ColumnFamilyMetrics

Signed-off-by: Amnon Heiman <amnon@cloudius-systems.com>
2015-10-04 12:54:31 +03:00
Amnon Heiman
54fe3d7ccf ColumnFamilyMetrics: Use schedule for the column family timer
The logic of that timer, should be that after some defined time from the
previous request a new one will be sent, the actuall rate is
meaningless and only cause delyed request to be sent in a higher
frequency after the delayed reponse was returned.

Signed-off-by: Amnon Heiman <amnon@cloudius-systems.com>
2015-10-04 12:49:51 +03:00
Amnon Heiman
f16c49a860 LatencyMetrics: combine the different constructors
This combine the different constructor into one constructor with the
logic and another one that calls it with a default value.

Signed-off-by: Amnon Heiman <amnon@cloudius-systems.com>
2015-10-04 12:48:26 +03:00
Amnon Heiman
9c7ab3fcf6 APIHistogram: Use the new histogram parameter name
The use of mean and variance as histogram parameter names makes more
sense.

This also make it safe to use an empty histogram that holds no samples.

Signed-off-by: Amnon Heiman <amnon@cloudius-systems.com>
2015-10-04 12:46:44 +03:00
Amnon Heiman
d67e6e60fc EndPointSnitchInfo: Support null as host name
In origin an undocumented feature (bug?) is that passing null as a host
name for getRack and getDatacenter returns the rack or datacenter
according to the loopbck address.

This follow the same behaviour, so when the host is null, the function
will not fail but will call the API with the local loopback address

Signed-off-by: Amnon Heiman <amnon@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
2015-09-29 09:30:08 +03:00
Takuya ASADA
efc8d5c1e6 dist: does not need maven for running time
Signed-off-by: Takuya ASADA <syuu@cloudius-systems.com>
2015-09-25 17:11:11 -07:00
Takuya ASADA
f035be4435 dist: does not need maven for running time
Signed-off-by: Takuya ASADA <syuu@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
2015-09-25 09:45:06 +03:00
Takuya ASADA
86ae0a38a9 dist: check hostname is resolvable
Warning for #6

Signed-off-by: Takuya ASADA <syuu@cloudius-systems.com>
[ penberg: edit warning text ]
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
2015-09-24 15:36:24 +03:00
Asias He
4dd0b95083 rpm: Fix build in centos container
mock will call /usr/sbin/mock, but we need /usr/bin/mock.

Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
2015-09-24 08:24:24 +03:00
Pekka Enberg
61d085a1f0 Merge branch 'branch-0.9' 2015-09-22 13:33:50 +03:00
Pekka Enberg
d14ddd5a0c Change license to AGPLv3
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
2015-09-22 13:33:24 +03:00
Pekka Enberg
2076107fbc Merge branch 'branch-0.9'
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

Conflicts:
	SCYLLA-VERSION-GEN
2015-09-22 10:35:31 +03:00
Amnon Heiman
7255b7847e JMX run scripts: Set the rmi port to listen on the jmx port
This set the rmi port to listen on the jmx port, so the java will not
choose a random port for it.

Signed-off-by: Amnon Heiman <amnon@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
2015-09-22 10:19:53 +03:00
Avi Kivity
8b05482550 release: prepare for next version cycle 2015-09-21 16:17:32 -07:00
Shlomi Livne
3163a60101 release: bump up version to 0.9
Signed-off-by: Shlomi Livne <shlomi@cloudius-systems.com>
2015-09-21 16:16:40 -07:00
Takuya ASADA
709bc37fcd dist: use jre-1.8.0/bin/java instead of /usr/bin/java, since 1.8.0 is not default on CentOS
Signed-off-by: Takuya ASADA <syuu@cloudius-systems.com>
2015-09-21 21:10:26 +02:00
Takuya ASADA
ab549e0dc7 dist: we need JDK, instead JRE on building time
Signed-off-by: Takuya ASADA <syuu@cloudius-systems.com>
2015-09-21 21:10:26 +02:00
Shlomi Livne
cdb8aa82bf dist: fix jvm option mistake in run script
-Dcom.sun.management.jmemote.ssl=false need to be
-Dcom.sun.management.jmxremote.ssl=false

Signed-off-by: Shlomi Livne <shlomi@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
2015-09-21 10:09:02 +03:00
Amnon Heiman
41a09f3b67 scripts: A startup script for the JMX proxy
The startup scripts helps in running the JMX proxy, wrap the java
command with help and default values.

Signed-off-by: Amnon Heiman <amnon@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
2015-09-21 10:06:10 +03:00
Amnon Heiman
197c8c7e05 README: Use of uber-jar and open jmx
This update the README to reflect the changes in the maven and with the
updated command line that open the jmx for external connection so the
nodetool will be able to connect to the jmx proxy.

Signed-off-by: Amnon Heiman <amnon@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
2015-09-18 07:50:09 +03:00
Shlomi Livne
f162d740e0 Fix args for booting java process
args must be specified before jar file, updated args

Signed-off-by: Shlomi Livne <shlomi@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
2015-09-17 14:52:56 +03:00
Pekka Enberg
72751b75d8 Merge "Auto shutdown and depricated methods in column family" from Amnon
"This series addresses two issues:

 It uses the column family timer to verify that the API is available. After a
 number of retry (currently set to 30s) the JMX proxy will shutdown.

 Second, it reduce the number of unimplemented depricated method, following
 origin way of calling the counters implementation from the depricated methods."
2015-09-17 14:19:57 +03:00
Amnon Heiman
5feb4d3841 Adding depricated implementation to column_family
Some of the depricated method in column family are replaced by counters
in Origin. Still, some tools, like nodetool uses the depricated API.

This adds an implementation to some of the depricated method that is
based on the column family counters similiar to origin.

Signed-off-by: Amnon Heiman <amnon@cloudius-systems.com>
2015-09-17 13:43:19 +03:00
Amnon Heiman
713f8a5c75 Shuting down the the proxy after MAX_RETRY
This uses the column family timer to check that the API is still
available.

After MAX_RETRY (currently set to 30s) the proxy will shutdown.
This address issue #2

Signed-off-by: Amnon Heiman <amnon@cloudius-systems.com>
2015-09-17 13:38:01 +03:00
Shlomi Livne
7ce1b07962 dist/redhat: Fix a small bug in script
The prefix should include version without release.

Without this fix I get:

Complete!
Finish: build setup for scylla-jmx-0.8-20150917.b177d24.fc21.src.rpm
Start: rpmbuild scylla-jmx-0.8-20150917.b177d24.fc21.src.rpm
Building target platforms: x86_64
Building for target x86_64
Executing(%prep): /bin/sh -e /var/tmp/rpm-tmp.gNKOxr
+ umask 022
+ cd /builddir/build/BUILD
+ cd /builddir/build/BUILD
+ rm -rf scylla-jmx-0.8
+ /usr/bin/tar -xf
/builddir/build/SOURCES/scylla-jmx-0.8-20150917.b177d24.tar
+ cd scylla-jmx-0.8
/var/tmp/rpm-tmp.gNKOxr: line 33: cd: scylla-jmx-0.8: No such file or
directory
error: Bad exit status from /var/tmp/rpm-tmp.gNKOxr (%prep)
    Bad exit status from /var/tmp/rpm-tmp.gNKOxr (%prep)

RPM build errors:
ERROR:
Exception(build/rpmbuild/SRPMS/scylla-jmx-0.8-20150917.b177d24.fc21.src.rpm)
Config(fedora-21-x86_64) 0 minutes 51 seconds
INFO: Results and/or logs in: /home/shlomi/urchin-jmx/build/rpms
INFO: Cleaning up build root ('cleanup_on_failure=True')

Signed-off-by: Shlomi Livne <shlomi@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
2015-09-17 13:24:21 +03:00
Pekka Enberg
ab10c93a1d dist/redhat: Fix RPM package home page URL
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
2015-09-17 13:07:08 +03:00
Pekka Enberg
b177d24d46 dist/redhat: Add version numbering
This adds version proper version numbering for the scylla-jmx RPM. Please note
that the version number in the server is still hard-coded in the Maven POM
file as "1.0". We need to address that later.

Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
2015-09-17 10:23:25 +03:00
Takuya ASADA
3259ef9604 dist: move service command line to scripts/jmx_run
Signed-off-by: Takuya ASADA <syuu@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
2015-09-17 10:05:29 +03:00
Pekka Enberg
452b0e9856 Make urchin-mbean Jar include its dependencies
Use the maven-shade-plugin to package a single executable Jar file.
Fixes #3.

Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
2015-09-15 09:56:12 +03:00
Amnon Heiman
3e326bf2d0 APIClient: Fixing a problem with getString
The javax does not handle parsing JsonValue directly. So it is replaced
for the getString with a direct removal of the quotations.

The alternative is to replace the javax parser with a different one.

Signed-off-by: Amnon Heiman <amnon@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
2015-09-14 13:32:15 +03:00
Takuya ASADA
2058cdde7c dist: use mock to build rpm
Signed-off-by: Takuya ASADA <syuu@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
2015-09-14 11:44:17 +03:00
Takuya ASADA
b0dbe82dad dist: add dependency to scylla-server
Since systemd unit file already has dependency to scylla-server, we need to add
dependency to rpm as well.

Signed-off-by: Takuya ASADA <syuu@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
2015-09-14 11:03:14 +03:00
Amnon Heiman
cce767e47e APIClient: getString to return parsed json value
getString should return the string value without the sarounding quotes.

For getRaw was introduce that return the API values in its raw value,
and getString was changed to use the json parser to get the string
values without the quotes.

Signed-off-by: Amnon Heiman <amnon@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudus-systems.com>
2015-09-11 09:25:37 +03:00
Amnon Heiman
fe3b786053 Urchin-JMX: Adding the EndpointSnitchInfo implementation
This adds the call to the API for the EndpointSnitch, the url are based
on the endpoint_snitch_info.json swagger definition file.

Signed-off-by: Amnon Heiman <amnon@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudus-systems.com>
2015-09-11 09:25:00 +03:00
Pekka Enberg
aa546f3ccb dist/redhat: Fix typo in spec file
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
2015-09-07 13:47:38 +03:00
Takuya ASADA
795c7a5df6 .spec file and scripts to build RPM for Fedora
Signed-off-by: Takuya ASADA <syuu@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
2015-09-07 13:47:15 +03:00
Pekka Enberg
db6d48a695 Merge "Adding notification for the repair command" from Amnon
"The API is currently not supporting notification, but the repair
 command that perform via the nodetool relays on the notification to know
 when the command terminate.

 This series adds support for the repair notification, based on a timer
 and periodically check if a current repair command was terminated."
2015-08-31 09:41:01 +03:00
Amnon Heiman
67dca4da9d StorageService: Add notification suport for the repair command
This patch adds a notification support for the repair in StorageService.

When a repair command starts a timer is set to check the status of the
repair, when the repair complets it sends notification for the
successful or fail of the repair.

Because jconsole doesn't run method with variable number of parameters
an additional MBean method was added for async repair with only a
keyspace as its parameter.

All the forceAsycRepair methods are mapped to the asyncRepair metod that
replaces them in 2.2, when options will be supported in the repair, it
will be added to the jmx.

After this patch it is possible to register in the jconsole for
notification, perform a forceAsyncRepair and get the notification that
the repair complets
2015-08-27 21:02:28 +03:00
Amnon Heiman
e96e62b6bd Add the RepairParallelism enum
This takes the RepairParallelism enum from origin

Signed-off-by: Amnon Heiman <amnon@cloudius-systems.com>
2015-08-27 20:46:02 +03:00
Amnon Heiman
6fc2c5c720 APIClient add postInt
Sometimes a post command need to return, this adds a method to perform a
post command that returns an int value.

The general postGetVal, can be used for other types if needed.

Signed-off-by: Amnon Heiman <amnon@cloudius-systems.com>
2015-08-27 20:46:02 +03:00
Pekka Enberg
4598323f02 Merge "Adding the row estimated histogram to column family" from Amnon
"This series adds the JMX API to the row estimated histogram. It adds a
 get function in the APIClient to fetch the data. The implementation is
 based on the column_family.json for the API call and on utils.json for
 the estimated histogram definition."
2015-08-26 14:25:06 +03:00
Amnon Heiman
2cfe02acb8 LatencyMetrics: Support path parameter
This change adds path parameter support, for the cases that the path
parameter is the last on the path, for example in the ColumnFamily case

Signed-off-by: Amnon Heiman <amnon@cloudius-systems.com>
2015-08-24 17:13:34 +03:00