"This series support API calls that returns an empty histogram. This is a
typical scenario with counters that are not implemented yet, for example range
latency. The trigger for this series is the nodetoold proxyhistograms command
After this series:
./bin/nodetool proxyhistograms
proxy histograms
Percentile Read Latency Write Latency Range Latency
(micros) (micros) (micros)
50% 654949.00 315852.00 NaN
75% 8409007.00 4055269.00 NaN
95% 20924300.00 17436917.00 NaN
98% 25109160.00 20924300.00 NaN
99% 25109160.00 25109160.00 NaN
Min 11865.00 11865.00 NaN
Max 25109160.00 25109160.00 NaN"
The RecentEstimatedHistogram updates its value from the API with an
array of recent values.
This array can be empty, in that case the getBuckets method should just
return a zero size array.
Signed-off-by: Amnon Heiman <amnon@scylladb.com>
When creating an estimated histogram from buckets it is a valid option
to get a zero size array as the buckets array.
In that case the newOffsets method would get a negative value for its
size, which should result in a zero length array of offsets.
Signed-off-by: Amnon Heiman <amnon@scylladb.com>
An empty histogram can return a valid response from the API but without
any buckets.
This is a valid scenario and common for counters of features that are
not supported yet.
In those cases, the APIClient should return a zero length array.
Signed-off-by: Amnon Heiman <amnon@scylladb.com>
"This series complete the change in the API that replaces the formatted string
functionality with an API that return an array of objects.
The series import from origin the helper classes: ApplicationState,
EndpointState and HeartBeatState and modify them to accept their values from
the API.
After this seris will be applied nodetool gossipinfo will have the same output
as origin has."
This patch change getAllEndpointStates implementation. The proxy now
gets from the API a list of objects, it creates the endpoint map from it
and create the result string.
After this patch the nodetool gossipinfo should be formatted like
origin.
After this patch the nodetool gossipinfo return:
./bin/nodetool gossipinfo
127.0.0.2
generation:1447850743
heartbeat:78
RACK:rack1
DC:datacenter1
HOST_ID:459137d7-2c7c-4b65-9ef8-f1c93b29dd6b
RPC_ADDRESS:127.0.0.2
RELEASE_VERSION:2.1.8
LOAD:86677
STATUS:NORMAL,9219539092146142451
SCHEMA:59adb24e-f3cd-3e02-97f0-5b395827453f
NET_VERSION:0
127.0.0.1
generation:1447850742
heartbeat:75
RACK:rack1
DC:datacenter1
HOST_ID:5216770b-6fc5-4d5b-8c87-33304fd87bc8
RPC_ADDRESS:127.0.0.1
RELEASE_VERSION:2.1.8
LOAD:12655
STATUS:NORMAL,927478638459366287
SCHEMA:59adb24e-f3cd-3e02-97f0-5b395827453f
NET_VERSION:0
Fix#508
Signed-off-by: Amnon Heiman <amnon@scylladb.com>
This patch import ApplicationState, EndpointState and HeartBeatState
from origin that are used to report the endpoint state map.
The classes where modified to be created by the API objects.
Signed-off-by: Amnon Heiman <amnon@scylladb.com>
getRecentTotalTimeouts
This patch adds the impelementation for the depricated method
getRecentTimeoutsPerHost and getRecentTotalTimeouts.
The implementatin is based on origin, the recent version of the method,
return the delta from the last call to the method.
Signed-off-by: Amnon Heiman <amnon@scylladb.com>
This patch adds the implementation of the dropped messages and the
recent dropped messages.
The MessagingService holds a timer that periodically load the dropped
messages from the API and distribute the results between the
DroppedMessagesMetrics instances.
This mimic the timer behaviour in origin, only it does one API call for
all Verb.
Signed-off-by: Amnon Heiman <amnon@scylladb.com>
This patch import the DroppedMessageMetrics from origin, as oppose to
origin, it does not run timers but relay on the Messaging sevice.
This save the timer and API call for each of the Verb.
Signed-off-by: Amnon Heiman <amnon@scylladb.com>
Sometimes it is required that a meter will not handle its own data, like
the APIMeter does.
This patch break the added functionality of APIMeter into two classes,
APISettableMeter is a Meter with a set value method and APIMeter adds
the functionality that reads from the API.
Signed-off-by: Amnon Heiman <amnon@scylladb.com>
"This series adds some deprecated methods implemenetation to the CacheService
depnding on its metrics.
It also stub the getDrainProgress in StorageService."
This patch adds the describering method to StorageService, the
implementation is based on the storage_service API that is define in
storage_service.json
The implementation reflect the changes in the API, that returns an
object vs. the jmx_describe ring.
Signed-off-by: Amnon Heiman <amnon@scylladb.com>
Signed-off-by: Pekka Enberg <penberg@scylladb.com>
"Scylla's configuration can change the listening address and port of the API,
the jmx proxy need to use this same configuration.
This series adds the ability to add a path to a yaml configuration file and the
jmx proxy would read its configuration from there. Configuration from system
properties/command line is still supported and the configuration hirarchy is as
follow from highest to lowest:
* command line
* configuration file according to the hirarchy:
- command line
- SCYLLA_CONF
- SCYLLA_HOME
- relative conf directory
* default values
The configuration definition was moved to a configuraion class that responsible
for getting the information from the command line and the configuration file."
When calling the API move method, the proxy should pass the new_token
parameter.
Signed-off-by: Amnon Heiman <amnon@scylladb.com>
Signed-off-by: Pekka Enberg <penberg@scylladb.com>
Drain progress is not implemented yet, it is needed by the nodetool
command so it will not fail.
This patches the functionality until the API will be ready, which, in
that time it would be revert.
Signed-off-by: Amnon Heiman <amnon@scylladb.com>
This patch follow origin in the implementation of the depricated methods
in CacheService. It propogate the request to the relevant metrics.
Signed-off-by: Amnon Heiman <amnon@scylladb.com>
The depricated recent hit rate implementation was add from Origin as it
is still been used by external system.
Signed-off-by: Amnon Heiman <amnon@scylladb.com>
This patch changes the APIClient to read the connection string from the
configuration object.
Main uses the same configuraion API to print it's connecting message and
call the configuration setup.
Signed-off-by: Amnon Heiman <amnon@scylladb.com>
This patch adds a configuration object to the jmx proxy that support
both the system/command line based properties and it accept a yaml
configuration file. The later options allows the jmx to read scylla
configuration file and connect to it based on this configuration.
The configuration file reader uses a yaml parser that was added to the
pom.xml
If no configuration file is found in the command line, it would look for
SCYLLA_CONF then SCYLLA_HOME then for relative 'conf' directory
Signed-off-by: Amnon Heiman <amnon@scylladb.com>
need merge apiconfig
"This series adds the jmx implementation to enable netstats.
After this series netstats should complete successfuly.
A run example:
$ ./bin/nodetool netstats
Mode: NORMAL
repair 397c91a0-8205-11e5-83e4-000000000001
repair 3977d5ba-8205-11e5-83e4-000000000001
repair 3977d624-8205-11e5-83e4-000000000001
repair 397c8fc8-8205-11e5-83e4-000000000001
.......
......
repair 3977d502-8205-11e5-83e4-000000000001
Read Repair Statistics:
Attempted: 1
Mismatch (Blocking): 0
Mismatch (Background): 0
Pool Name Active Pending Completed
Commands n/a 0 21182
Responses n/a 0 597"
This patch adds the registration of StreamManagerMBean to
StorageService, similiar to the way it is done in origin.
After this patch the StreamManager will be available via Jconsole.
Signed-off-by: Amnon Heiman <amnon@scylladb.com>
This patch adds the implementation of:
getResponsePendingTasks()
getResponseCompletedTasks()
getDroppedMessages()
The implementation is based on the messaging_service API that defined in
messaging_service.json.
Signed-off-by: Amnon Heiman <amnon@scylladb.com>
The StreamManager getStreams returns an hirarchy of classes. This patch
import StreamManagerMBean with the class hirarchy and add an
implementation to StreamManager.
The implementation is based on the stream_manager API.
Signed-off-by: Amnon Heiman <amnon@scylladb.com>
This patch adds two map reader function to the APIClient, one that parse
map<String,Integer> and one for map<String,Long>
Signed-off-by: Amnon Heiman <amnon@scylladb.com>
"The trigger for this patch is the nodetool remove command, the expected
behaviour is that the error that return from the API would be populate to the
JMX client.
In this first stage the API errors are send as a generic RuntimeException, in a
later phase specific methods would catch this exception and replace it with a
specific one. Note that the thrown exceptions should be one that is known to the client,
or it would fail to handle it and in general should be equivalent to origin.
After this patch a call to the nodetool remove with a bad host name would result in:
./bin/nodetool removenode dcc0477a-b4a7-448a-bc79-27853f61b92d
error: Host ID not found.
-- StackTrace --
java.lang.RuntimeException: Host ID not found.
at com.cloudius.urchin.api.APIClient.getException(APIClient.java:114)
at com.cloudius.urchin.api.APIClient.post(APIClient.java:101)
....."
The API returns errors with an HTTP code 400, 404 or 500 depending on
the cause with a json object that contains the failure reason. The
error message should be populate to the JMX calling client, translated
to the appropriate exception.
This patch adds the ability to detect API failure and throw a runtime
exception with the returned message.
It is up to the calling method what to do with the exception, if it
would do nothing, the calling client would get a RuntimeException,
depends on origin MBean definition, the caller can catch the exception
and throw a specific kind.
Note that any exception that is thrown must be known to the JMX client
or it will not be able to process it.
As a first step, this patch replaces the jersey client to the newer
version under glassfish, which has an easy way of getting a Reply
object, and check its status before returning the results.
The only difference in the method that uses the APIClient is the use of
MultivaluedHashMap.
The following MBean implementation where changed
ColumnFamily
CompactionManager
Gossiper
EndpointSnitchInfo
CacheService
StorageProxy
StorageService
Signed-off-by: Amnon Heiman <amnon@scylladb.com>
There is a confusion in origin, the MBean declare token as the parameter
to remove, but the implementation actually uses host id.
This patch modify scylla implementation to pass a host id as the
parameter to remove.
Signed-off-by: Amnon Heiman <amnon@cloudius-systems.com>
"This series adds latency related depricated methods to the storage
proxy.
The implmenetation mimic origin, in which the depricated methods calls
the counters that replaces them."
In LatencyMetrics the URL is passed without the ending slash, this
patch use the same notation in ClientRequestMetrics.
Signed-off-by: Amnon Heiman <amnon@cloudius-systems.com>
This follow origin by adding the implementation for the depricated
metrics methods.
Similiar to origin, the implementation calls the implementation in the
metrics.
Signed-off-by: Amnon Heiman <amnon@cloudius-systems.com>
"This series together with the cfhistogram series in scylla adds the
missing functionality so that nodetoold cfhistogram would work.
After both series will be apply an execution example is:
./bin/nodetool cfhistograms keyspace1 standard1
keyspace1/standard1 histograms
Percentile SSTables Write Latency Read Latency Partition Size Cell Count
(micros) (micros) (bytes)
50% 0.00 6866.00 4866323.00 310 5
75% 0.00 8239.00 10090808.00 310 5
95% 0.00 20501.00 17436917.00 310 5
98% 0.00 35425.00 25109160.00 310 5
99% 0.00 51012.00 25109160.00 310 5
Min 0.00 2300.00 654950.00 259 5
Max 0.00 20924300.00 25109160.00 310 5"
This patch uses the estimated latency that was added to the column
family metrics to get the recent and estimated latency.
It follows the same logic as origin does to call the logic in metrics.
The following method implementation will be added:
getMemtableColumnsCount
getRecentSSTablesPerReadHistogram
getSSTablesPerReadHistogram
getLifetimeReadLatencyHistogramMicros
getRecentReadLatencyHistogramMicros
getRecentReadLatencyMicros
getLifetimeWriteLatencyHistogramMicros
getRecentWriteLatencyHistogramMicros
getRecentWriteLatencyMicros
getRangeCount
getTotalRangeLatencyMicros
getLifetimeRangeLatencyHistogramMicros
getRecentRangeLatencyHistogramMicros
getRecentRangeLatencyMicros
Signed-off-by: Amnon Heiman <amnon@cloudius-systems.com>
This uses the recent estimated histogram and the API based estimated
histogram to support the sstable per read recent and total estimated
histogram.
Signed-off-by: Amnon Heiman <amnon@cloudius-systems.com>
