Commit Graph

193 Commits

Author SHA1 Message Date
Takuya ASADA
793f53e403 dist: fix 'missing-license-paragraph-in-dep5-copyright' message on building ubuntu package 2015-11-25 19:43:38 +09:00
Takuya ASADA
c079c5922c dist: make ubuntu package as 'debian non-native package' 2015-11-25 19:43:38 +09:00
Takuya ASADA
b007b3ae1b dist: specify maven repo directory 2015-11-24 19:22:54 +09:00
Takuya ASADA
dd2192ef97 dist: support ./SCYLLA-VERSION-GEN on ubuntu package
Signed-off-by: Takuya ASADA <syuu@scylladb.com>
2015-11-22 17:55:27 +02:00
Pekka Enberg
c7df07ac70 Merge "Support empty histogram from the API" from Amnon
"This series support API calls that returns an empty histogram. This is a
 typical scenario with counters that are not implemented yet, for example range
 latency.  The trigger for this series is the nodetoold proxyhistograms command

 After this series:
 ./bin/nodetool proxyhistograms
 proxy histograms
 Percentile      Read Latency     Write Latency     Range Latency
                     (micros)          (micros)          (micros)
 50%                654949.00         315852.00               NaN
 75%               8409007.00        4055269.00               NaN
 95%              20924300.00       17436917.00               NaN
 98%              25109160.00       20924300.00               NaN
 99%              25109160.00       25109160.00               NaN
 Min                 11865.00          11865.00               NaN
 Max              25109160.00       25109160.00               NaN"
2015-11-19 12:36:31 +02:00
Amnon Heiman
695e23bd4e RecentEstimatedHistogram: Support empty histogram
The RecentEstimatedHistogram updates its value from the API with an
array of recent values.

This array can be empty, in that case the getBuckets method should just
return a zero size array.

Signed-off-by: Amnon Heiman <amnon@scylladb.com>
2015-11-19 11:55:34 +02:00
Amnon Heiman
e0dea0c27e EstimatedHistogram: Support empty histogram
When creating an estimated histogram from buckets it is a valid option
to get a zero size array as the buckets array.

In that case the newOffsets method would get a negative value for its
size, which should result in a zero length array of offsets.

Signed-off-by: Amnon Heiman <amnon@scylladb.com>
2015-11-19 11:52:17 +02:00
Amnon Heiman
5362b2045d APIClient: Support Empty histogram
An empty histogram can return a valid response from the API but without
any buckets.
This is a valid scenario and common for counters of features that are
not supported yet.
In those cases, the APIClient should return a zero length array.

Signed-off-by: Amnon Heiman <amnon@scylladb.com>
2015-11-19 11:49:31 +02:00
Pekka Enberg
85ab591d27 Merge "FailureDetector getAllEndpointStates to use the updated API" from Amnon
"This series complete the change in the API that replaces the formatted string
functionality with an API that return an array of objects.

The series import from origin the helper classes: ApplicationState,
EndpointState and HeartBeatState and modify them to accept their values from
the API.

After this seris will be applied nodetool gossipinfo will have the same output
as origin has."
2015-11-18 16:29:34 +02:00
Amnon Heiman
9f9dc88643 FailureDetector: Change getAllEndpointStates implementation
This patch change getAllEndpointStates implementation. The proxy now
gets from the API a list of objects, it creates the endpoint map from it
and create the result string.

After this patch the nodetool gossipinfo should be formatted like
origin.

After this patch the nodetool gossipinfo return:

./bin/nodetool gossipinfo
127.0.0.2
  generation:1447850743
  heartbeat:78
  RACK:rack1
  DC:datacenter1
  HOST_ID:459137d7-2c7c-4b65-9ef8-f1c93b29dd6b
  RPC_ADDRESS:127.0.0.2
  RELEASE_VERSION:2.1.8
  LOAD:86677
  STATUS:NORMAL,9219539092146142451
  SCHEMA:59adb24e-f3cd-3e02-97f0-5b395827453f
  NET_VERSION:0
127.0.0.1
  generation:1447850742
  heartbeat:75
  RACK:rack1
  DC:datacenter1
  HOST_ID:5216770b-6fc5-4d5b-8c87-33304fd87bc8
  RPC_ADDRESS:127.0.0.1
  RELEASE_VERSION:2.1.8
  LOAD:12655
  STATUS:NORMAL,927478638459366287
  SCHEMA:59adb24e-f3cd-3e02-97f0-5b395827453f
  NET_VERSION:0

Fix #508

Signed-off-by: Amnon Heiman <amnon@scylladb.com>
2015-11-18 14:48:15 +02:00
Amnon Heiman
e01ece2fcd Import ApplicationState, EndpointState and HeartBeatState from origin
This patch import ApplicationState, EndpointState and HeartBeatState
from origin that are used to report the endpoint state map.

The classes where modified to be created by the API objects.

Signed-off-by: Amnon Heiman <amnon@scylladb.com>
2015-11-18 13:58:07 +02:00
Pekka Enberg
d06e6fdde1 Merge "Adding the messaging service counters" from Amnon
"This series adds the dropped, timeout and their recently version counter to the
 MessagingService."
2015-11-18 08:59:32 +02:00
Amnon Heiman
1292bd9ba4 MessagingService: Add the depricated getRecentTimeoutsPerHost and
getRecentTotalTimeouts

This patch adds the impelementation for the depricated method
getRecentTimeoutsPerHost and getRecentTotalTimeouts.

The implementatin is based on origin, the recent version of the method,
return the delta from the last call to the method.

Signed-off-by: Amnon Heiman <amnon@scylladb.com>
2015-11-17 11:57:07 +02:00
Amnon Heiman
db7aad26f5 MessagingService add dropped and recently dropped messages impl
This patch adds the implementation of the dropped messages and the
recent dropped messages.

The MessagingService holds a timer that periodically load the dropped
messages from the API and distribute the results between the
DroppedMessagesMetrics instances.

This mimic the timer behaviour in origin, only it does one API call for
all Verb.

Signed-off-by: Amnon Heiman <amnon@scylladb.com>
2015-11-17 11:56:03 +02:00
Amnon Heiman
896fd64de9 Import the DroppedMessageMetrics from origin
This patch import the DroppedMessageMetrics from origin, as oppose to
origin, it does not run timers but relay on the Messaging sevice.

This save the timer and API call for each of the Verb.

Signed-off-by: Amnon Heiman <amnon@scylladb.com>
2015-11-16 11:52:29 +02:00
Amnon Heiman
1cb048effe Create and register the APISettableMetrics
This patch adds the ability to create and register the
APISettableMettics.

Signed-off-by: Amnon Heiman <amnon@scylladb.com>
2015-11-16 11:29:21 +02:00
Amnon Heiman
ac049129de Adding the APISettableMeter
Sometimes it is required that a meter will not handle its own data, like
the APIMeter does.

This patch break the added functionality of APIMeter into two classes,
APISettableMeter is a Meter with a set value method and APIMeter adds
the functionality that reads from the API.

Signed-off-by: Amnon Heiman <amnon@scylladb.com>
2015-11-16 11:27:31 +02:00
Amnon Heiman
b783a0d09a MessagingService: Add dropped and timeout support
This adds the implementation for dropped messages and timeout messages
counters in MessagingService.

Signed-off-by: Amnon Heiman <amnon@scylladb.com>
2015-11-11 09:49:50 +02:00
Pekka Enberg
e530c13f87 Merge "Adding deprecated implementation to cache" from Amnon
"This series adds some deprecated methods implemenetation to the CacheService
depnding on its metrics.

It also stub the getDrainProgress in StorageService."
2015-11-11 09:40:04 +02:00
Amnon Heiman
fadfb9443c StorageService: Add describering functionality
This patch adds the describering method to StorageService, the
implementation is based on the storage_service API that is define in
storage_service.json

The implementation reflect the changes in the API, that returns an
object vs. the jmx_describe ring.

Signed-off-by: Amnon Heiman <amnon@scylladb.com>
Signed-off-by: Pekka Enberg <penberg@scylladb.com>
2015-11-11 09:14:05 +02:00
Pekka Enberg
81aa2a8279 Merge "JMX proxy to support configuration file" from Amnon
"Scylla's configuration can change the listening address and port of the API,
the jmx proxy need to use this same configuration.

This series adds the ability to add a path to a yaml configuration file and the
jmx proxy would read its configuration from there.  Configuration from system
properties/command line is still supported and the configuration hirarchy is as
follow from highest to lowest:
* command line
* configuration file according to the hirarchy:
  - command line
  - SCYLLA_CONF
  - SCYLLA_HOME
  - relative conf directory
* default values

The configuration definition was moved to a configuraion class that responsible
for getting the information from the command line and the configuration file."
2015-11-11 09:08:08 +02:00
Amnon Heiman
27c0eb8c99 StorageService: move should pass the parameter to the API
When calling the API move method, the proxy should pass the new_token
parameter.

Signed-off-by: Amnon Heiman <amnon@scylladb.com>
Signed-off-by: Pekka Enberg <penberg@scylladb.com>
2015-11-11 08:53:27 +02:00
Amnon Heiman
07b319d827 StorageService: Stub the getDrainProgress
Drain progress is not implemented yet, it is needed by the nodetool
command so it will not fail.

This patches the functionality until the API will be ready, which, in
that time it would be revert.

Signed-off-by: Amnon Heiman <amnon@scylladb.com>
2015-11-10 17:03:50 +02:00
Amnon Heiman
01477809ac CacheService: Add depricated unimplemented methods
This patch follow origin in the implementation of the depricated methods
in CacheService. It propogate the request to the relevant metrics.

Signed-off-by: Amnon Heiman <amnon@scylladb.com>
2015-11-10 16:54:48 +02:00
Amnon Heiman
54d451de88 CacheMetrics: Add recent hit rate
The depricated recent hit rate implementation was add from Origin as it
is still been used by external system.

Signed-off-by: Amnon Heiman <amnon@scylladb.com>
2015-11-10 15:58:51 +02:00
Amnon Heiman
43e07fda0e Use the configuration object
This patch changes the APIClient to read the connection string from the
configuration object.

Main uses the same configuraion API to print it's connecting message and
call the configuration setup.

Signed-off-by: Amnon Heiman <amnon@scylladb.com>
2015-11-08 12:12:27 +02:00
Amnon Heiman
39b50f63c7 Adding a configuration object for the jmx proxy
This patch adds a configuration object to the jmx proxy that support
both the system/command line based properties and it accept a yaml
configuration file. The later options allows the jmx to read scylla
configuration file and connect to it based on this configuration.

The configuration file reader uses a yaml parser that was added to the
pom.xml

If no configuration file is found in the command line, it would look for
SCYLLA_CONF then SCYLLA_HOME then for relative 'conf' directory

Signed-off-by: Amnon Heiman <amnon@scylladb.com>

need merge apiconfig
2015-11-08 12:12:27 +02:00
Avi Kivity
72f6f5dab4 Merge "Enabling nodetool netstats" from Amnon
"This series adds the jmx implementation to enable netstats.
After this series netstats should complete successfuly.
A run example:

$ ./bin/nodetool netstats
Mode: NORMAL
repair 397c91a0-8205-11e5-83e4-000000000001
repair 3977d5ba-8205-11e5-83e4-000000000001
repair 3977d624-8205-11e5-83e4-000000000001
repair 397c8fc8-8205-11e5-83e4-000000000001
.......
......
repair 3977d502-8205-11e5-83e4-000000000001
Read Repair Statistics:
Attempted: 1
Mismatch (Blocking): 0
Mismatch (Background): 0
Pool Name                    Active   Pending      Completed
Commands                        n/a         0          21182
Responses                       n/a         0            597"
2015-11-08 11:48:40 +02:00
Amnon Heiman
9b14550d0a StorageService: register the StreamManagerMBean
This patch adds the registration of StreamManagerMBean to
StorageService, similiar to the way it is done in origin.

After this patch the StreamManager will be available via Jconsole.

Signed-off-by: Amnon Heiman <amnon@scylladb.com>
2015-11-03 11:33:40 +02:00
Amnon Heiman
3cb168bed3 MessagingService: Add tasks statistics
This patch adds the implementation of:
getResponsePendingTasks()
getResponseCompletedTasks()
getDroppedMessages()

The implementation is based on the messaging_service API that defined in
messaging_service.json.

Signed-off-by: Amnon Heiman <amnon@scylladb.com>
2015-11-03 11:33:40 +02:00
Amnon Heiman
20778b2df6 Add StreamManagerMBean with its subclasses
The StreamManager getStreams returns an hirarchy of classes. This patch
import StreamManagerMBean with the class hirarchy and add an
implementation to StreamManager.

The implementation is based on the stream_manager API.

Signed-off-by: Amnon Heiman <amnon@scylladb.com>
2015-11-03 11:28:44 +02:00
Amnon Heiman
1db077618d APIClient: Add getMapStringIntegerValue and getMapStringLongValue
This patch adds two map reader function to the APIClient, one that parse
map<String,Integer> and one for map<String,Long>

Signed-off-by: Amnon Heiman <amnon@scylladb.com>
2015-11-03 11:18:33 +02:00
Takuya ASADA
2e11cb5471 dist: stop scylla-jmx when scylla-server stopped, don't respawn
Signed-off-by: Takuya ASADA <syuu@scylladb.com>
Signed-off-by: Pekka Enberg <penberg@scylladb.com>
2015-10-30 15:40:02 +02:00
Takuya ASADA
c596a9e462 dist: fix warning when building scylla-jmx ubuntu package
Signed-off-by: Takuya ASADA <syuu@scylladb.com>
2015-10-30 09:15:13 +02:00
Amnon Heiman
00694ce40a StorageService: getLoadString should return units
This address issue #512

Signed-off-by: Amnon Heiman <amnon@scylladb.com>
2015-10-29 15:54:29 +02:00
Takuya ASADA
b5d29a0796 dist: add devscripts to install debuild before start building package
Signed-off-by: Takuya ASADA <syuu@scylladb.com>
Signed-off-by: Pekka Enberg <penberg@scylladb.com>
2015-10-29 07:41:58 +02:00
Pekka Enberg
d88d017385 Merge "Populate the API exception" from Amnon
"The trigger for this patch is the nodetool remove command, the expected
behaviour is that the error that return from the API would be populate to the
JMX client.

In this first stage the API errors are send as a generic RuntimeException, in a
later phase specific methods would catch this exception and replace it with a
specific one.  Note that the thrown exceptions should be one that is known to the client,
or it would fail to handle it and in general should be equivalent to origin.

After this patch a call to the nodetool remove with a bad host name would result in:

./bin/nodetool removenode dcc0477a-b4a7-448a-bc79-27853f61b92d
error: Host ID not found.
-- StackTrace --
java.lang.RuntimeException: Host ID not found.
        at com.cloudius.urchin.api.APIClient.getException(APIClient.java:114)
        at com.cloudius.urchin.api.APIClient.post(APIClient.java:101)
....."
2015-10-28 11:36:25 +02:00
Amnon Heiman
649922fed8 Updating the http client and support API exception
The API returns errors with an HTTP code 400, 404 or 500 depending on
the cause with a json object that contains the failure reason.  The
error message should be populate to the JMX calling client, translated
to the appropriate exception.

This patch adds the ability to detect API failure and throw a runtime
exception with the returned message.

It is up to the calling method what to do with the exception, if it
would do nothing, the calling client would get a RuntimeException,
depends on origin MBean definition, the caller can catch the exception
and throw a specific kind.

Note that any exception that is thrown must be known to the JMX client
or it will not be able to process it.

As a first step, this patch replaces the jersey client to the newer
version under glassfish, which has an easy way of getting a Reply
object, and check its status before returning the results.

The only difference in the method that uses the APIClient is the use of
MultivaluedHashMap.

The following MBean implementation where changed

ColumnFamily
CompactionManager
Gossiper
EndpointSnitchInfo
CacheService
StorageProxy
StorageService

Signed-off-by: Amnon Heiman <amnon@scylladb.com>
2015-10-28 11:25:37 +02:00
Pekka Enberg
1c180c7164 Merge "Support Ubuntu 14.04LTS" from Takuya 2015-10-27 08:52:15 +02:00
Takuya ASADA
afdacb46ce dist: add build script for ubuntu 2015-10-27 04:17:13 +09:00
Takuya ASADA
2a7e668a44 dist: add debian/ directory to build .dep package for Ubuntu 2015-10-27 04:17:13 +09:00
Takuya ASADA
28efc831b1 dist: share scripts both on redhat and ubuntu 2015-10-27 04:17:13 +09:00
Takuya ASADA
a2ad9bb221 dist: use OpenJDK-7 on CentOS 2015-10-27 04:17:13 +09:00
Amnon Heiman
a707b487f0 StorageService: remove node should use hostid and not token
There is a confusion in origin, the MBean declare token as the parameter
to remove, but the implementation actually uses host id.

This patch modify scylla implementation to pass a host id as the
parameter to remove.

Signed-off-by: Amnon Heiman <amnon@cloudius-systems.com>
2015-10-26 10:31:43 +02:00
Pekka Enberg
dc76d2550e Merge "Support deprecated method in StorageProxy" from Amnon
"This series adds latency related depricated methods to the storage
 proxy.

 The implmenetation mimic origin, in which the depricated methods calls
 the counters that replaces them."
2015-10-26 09:15:20 +02:00
Amnon Heiman
ae26226339 ClientRequestMetrics: pass the url similiar to LatencyMetrics
In LatencyMetrics the URL is passed without the ending slash, this
patch use the same notation in ClientRequestMetrics.

Signed-off-by: Amnon Heiman <amnon@cloudius-systems.com>
2015-10-25 13:36:10 +02:00
Avi Kivity
77dc745b61 Downgrade Java compatibility level to 1.7.
Allows running on older Linux distributions.
2015-10-25 13:22:41 +02:00
Avi Kivity
ee7f5354b1 Avoid non-final variables in anonymous classes
Required by Java 7.
2015-10-25 13:22:05 +02:00
Amnon Heiman
392765fa2e StorageProxy: Add depricated method implementation based on metrics
This follow origin by adding the implementation for the depricated
metrics methods.

Similiar to origin, the implementation calls the implementation in the
metrics.

Signed-off-by: Amnon Heiman <amnon@cloudius-systems.com>
2015-10-22 10:43:29 +03:00
Pekka Enberg
b11c108fcf Remove obsolete comments from ColumnFamilyStore 2015-10-21 14:28:17 +03:00