An empty histogram can return a valid response from the API but without
any buckets.
This is a valid scenario and common for counters of features that are
not supported yet.
In those cases, the APIClient should return a zero length array.
Signed-off-by: Amnon Heiman <amnon@scylladb.com>
This patch change getAllEndpointStates implementation. The proxy now
gets from the API a list of objects, it creates the endpoint map from it
and create the result string.
After this patch the nodetool gossipinfo should be formatted like
origin.
After this patch the nodetool gossipinfo return:
./bin/nodetool gossipinfo
127.0.0.2
generation:1447850743
heartbeat:78
RACK:rack1
DC:datacenter1
HOST_ID:459137d7-2c7c-4b65-9ef8-f1c93b29dd6b
RPC_ADDRESS:127.0.0.2
RELEASE_VERSION:2.1.8
LOAD:86677
STATUS:NORMAL,9219539092146142451
SCHEMA:59adb24e-f3cd-3e02-97f0-5b395827453f
NET_VERSION:0
127.0.0.1
generation:1447850742
heartbeat:75
RACK:rack1
DC:datacenter1
HOST_ID:5216770b-6fc5-4d5b-8c87-33304fd87bc8
RPC_ADDRESS:127.0.0.1
RELEASE_VERSION:2.1.8
LOAD:12655
STATUS:NORMAL,927478638459366287
SCHEMA:59adb24e-f3cd-3e02-97f0-5b395827453f
NET_VERSION:0
Fix#508
Signed-off-by: Amnon Heiman <amnon@scylladb.com>
This patch import ApplicationState, EndpointState and HeartBeatState
from origin that are used to report the endpoint state map.
The classes where modified to be created by the API objects.
Signed-off-by: Amnon Heiman <amnon@scylladb.com>
getRecentTotalTimeouts
This patch adds the impelementation for the depricated method
getRecentTimeoutsPerHost and getRecentTotalTimeouts.
The implementatin is based on origin, the recent version of the method,
return the delta from the last call to the method.
Signed-off-by: Amnon Heiman <amnon@scylladb.com>
This patch adds the implementation of the dropped messages and the
recent dropped messages.
The MessagingService holds a timer that periodically load the dropped
messages from the API and distribute the results between the
DroppedMessagesMetrics instances.
This mimic the timer behaviour in origin, only it does one API call for
all Verb.
Signed-off-by: Amnon Heiman <amnon@scylladb.com>
This patch import the DroppedMessageMetrics from origin, as oppose to
origin, it does not run timers but relay on the Messaging sevice.
This save the timer and API call for each of the Verb.
Signed-off-by: Amnon Heiman <amnon@scylladb.com>
Sometimes it is required that a meter will not handle its own data, like
the APIMeter does.
This patch break the added functionality of APIMeter into two classes,
APISettableMeter is a Meter with a set value method and APIMeter adds
the functionality that reads from the API.
Signed-off-by: Amnon Heiman <amnon@scylladb.com>
"This series adds some deprecated methods implemenetation to the CacheService
depnding on its metrics.
It also stub the getDrainProgress in StorageService."
This patch adds the describering method to StorageService, the
implementation is based on the storage_service API that is define in
storage_service.json
The implementation reflect the changes in the API, that returns an
object vs. the jmx_describe ring.
Signed-off-by: Amnon Heiman <amnon@scylladb.com>
Signed-off-by: Pekka Enberg <penberg@scylladb.com>
"Scylla's configuration can change the listening address and port of the API,
the jmx proxy need to use this same configuration.
This series adds the ability to add a path to a yaml configuration file and the
jmx proxy would read its configuration from there. Configuration from system
properties/command line is still supported and the configuration hirarchy is as
follow from highest to lowest:
* command line
* configuration file according to the hirarchy:
- command line
- SCYLLA_CONF
- SCYLLA_HOME
- relative conf directory
* default values
The configuration definition was moved to a configuraion class that responsible
for getting the information from the command line and the configuration file."
When calling the API move method, the proxy should pass the new_token
parameter.
Signed-off-by: Amnon Heiman <amnon@scylladb.com>
Signed-off-by: Pekka Enberg <penberg@scylladb.com>
Drain progress is not implemented yet, it is needed by the nodetool
command so it will not fail.
This patches the functionality until the API will be ready, which, in
that time it would be revert.
Signed-off-by: Amnon Heiman <amnon@scylladb.com>
This patch follow origin in the implementation of the depricated methods
in CacheService. It propogate the request to the relevant metrics.
Signed-off-by: Amnon Heiman <amnon@scylladb.com>
The depricated recent hit rate implementation was add from Origin as it
is still been used by external system.
Signed-off-by: Amnon Heiman <amnon@scylladb.com>
This patch changes the APIClient to read the connection string from the
configuration object.
Main uses the same configuraion API to print it's connecting message and
call the configuration setup.
Signed-off-by: Amnon Heiman <amnon@scylladb.com>
This patch adds a configuration object to the jmx proxy that support
both the system/command line based properties and it accept a yaml
configuration file. The later options allows the jmx to read scylla
configuration file and connect to it based on this configuration.
The configuration file reader uses a yaml parser that was added to the
pom.xml
If no configuration file is found in the command line, it would look for
SCYLLA_CONF then SCYLLA_HOME then for relative 'conf' directory
Signed-off-by: Amnon Heiman <amnon@scylladb.com>
need merge apiconfig
"This series adds the jmx implementation to enable netstats.
After this series netstats should complete successfuly.
A run example:
$ ./bin/nodetool netstats
Mode: NORMAL
repair 397c91a0-8205-11e5-83e4-000000000001
repair 3977d5ba-8205-11e5-83e4-000000000001
repair 3977d624-8205-11e5-83e4-000000000001
repair 397c8fc8-8205-11e5-83e4-000000000001
.......
......
repair 3977d502-8205-11e5-83e4-000000000001
Read Repair Statistics:
Attempted: 1
Mismatch (Blocking): 0
Mismatch (Background): 0
Pool Name Active Pending Completed
Commands n/a 0 21182
Responses n/a 0 597"
This patch adds the registration of StreamManagerMBean to
StorageService, similiar to the way it is done in origin.
After this patch the StreamManager will be available via Jconsole.
Signed-off-by: Amnon Heiman <amnon@scylladb.com>
This patch adds the implementation of:
getResponsePendingTasks()
getResponseCompletedTasks()
getDroppedMessages()
The implementation is based on the messaging_service API that defined in
messaging_service.json.
Signed-off-by: Amnon Heiman <amnon@scylladb.com>
The StreamManager getStreams returns an hirarchy of classes. This patch
import StreamManagerMBean with the class hirarchy and add an
implementation to StreamManager.
The implementation is based on the stream_manager API.
Signed-off-by: Amnon Heiman <amnon@scylladb.com>
This patch adds two map reader function to the APIClient, one that parse
map<String,Integer> and one for map<String,Long>
Signed-off-by: Amnon Heiman <amnon@scylladb.com>
The API returns errors with an HTTP code 400, 404 or 500 depending on
the cause with a json object that contains the failure reason. The
error message should be populate to the JMX calling client, translated
to the appropriate exception.
This patch adds the ability to detect API failure and throw a runtime
exception with the returned message.
It is up to the calling method what to do with the exception, if it
would do nothing, the calling client would get a RuntimeException,
depends on origin MBean definition, the caller can catch the exception
and throw a specific kind.
Note that any exception that is thrown must be known to the JMX client
or it will not be able to process it.
As a first step, this patch replaces the jersey client to the newer
version under glassfish, which has an easy way of getting a Reply
object, and check its status before returning the results.
The only difference in the method that uses the APIClient is the use of
MultivaluedHashMap.
The following MBean implementation where changed
ColumnFamily
CompactionManager
Gossiper
EndpointSnitchInfo
CacheService
StorageProxy
StorageService
Signed-off-by: Amnon Heiman <amnon@scylladb.com>
There is a confusion in origin, the MBean declare token as the parameter
to remove, but the implementation actually uses host id.
This patch modify scylla implementation to pass a host id as the
parameter to remove.
Signed-off-by: Amnon Heiman <amnon@cloudius-systems.com>
"This series adds latency related depricated methods to the storage
proxy.
The implmenetation mimic origin, in which the depricated methods calls
the counters that replaces them."
In LatencyMetrics the URL is passed without the ending slash, this
patch use the same notation in ClientRequestMetrics.
Signed-off-by: Amnon Heiman <amnon@cloudius-systems.com>
This follow origin by adding the implementation for the depricated
metrics methods.
Similiar to origin, the implementation calls the implementation in the
metrics.
Signed-off-by: Amnon Heiman <amnon@cloudius-systems.com>
"This series together with the cfhistogram series in scylla adds the
missing functionality so that nodetoold cfhistogram would work.
After both series will be apply an execution example is:
./bin/nodetool cfhistograms keyspace1 standard1
keyspace1/standard1 histograms
Percentile SSTables Write Latency Read Latency Partition Size Cell Count
(micros) (micros) (bytes)
50% 0.00 6866.00 4866323.00 310 5
75% 0.00 8239.00 10090808.00 310 5
95% 0.00 20501.00 17436917.00 310 5
98% 0.00 35425.00 25109160.00 310 5
99% 0.00 51012.00 25109160.00 310 5
Min 0.00 2300.00 654950.00 259 5
Max 0.00 20924300.00 25109160.00 310 5"
This patch uses the estimated latency that was added to the column
family metrics to get the recent and estimated latency.
It follows the same logic as origin does to call the logic in metrics.
The following method implementation will be added:
getMemtableColumnsCount
getRecentSSTablesPerReadHistogram
getSSTablesPerReadHistogram
getLifetimeReadLatencyHistogramMicros
getRecentReadLatencyHistogramMicros
getRecentReadLatencyMicros
getLifetimeWriteLatencyHistogramMicros
getRecentWriteLatencyHistogramMicros
getRecentWriteLatencyMicros
getRangeCount
getTotalRangeLatencyMicros
getLifetimeRangeLatencyHistogramMicros
getRecentRangeLatencyHistogramMicros
getRecentRangeLatencyMicros
Signed-off-by: Amnon Heiman <amnon@cloudius-systems.com>
This uses the recent estimated histogram and the API based estimated
histogram to support the sstable per read recent and total estimated
histogram.
Signed-off-by: Amnon Heiman <amnon@cloudius-systems.com>
This adds the depricated total and recent estimated histogram.
It uses the new RecentEstimatedHistogram for the recent value and the
API based estimated histogram for the total latency.
Signed-off-by: Amnon Heiman <amnon@cloudius-systems.com>
The EstimatedHistogramWrapper is a helper class that holds the API
related data, so that a class that uses an EstimatedHistogram can
replace it with the wrapper and keep most of its code as is.
Signed-off-by: Amnon Heiman <amnon@cloudius-systems.com>
This patch modify the CacheEntry to support both String and
EstimatedHistogram.
It is possible to add more supported types in the future when needed.
In the APIClient, the cache will now support both String and
EstimatedHistogram in a similiar way.
Signed-off-by: Amnon Heiman <amnon@cloudius-systems.com>
apiclient need to merge to cache
This patch allows to create an EstimatedHistogram from an array of data
value.
It will be used by the APIClient to return EstimatedHistogram
Signed-off-by: Amnon Heiman <amnon@cloudius-systems.com>
Some of the jmx methods uses the notion of recent estimated histogram.
In origin the implementation uses an estimated histogram and clean the
histogram values on each call.
The RecentEstimatedHistogram mimic this behaviour, it store the latest
values of the last call. In each call new values are stored in the
histogram and the results is the delta between the last two calls.
Signed-off-by: Amnon Heiman <amnon@cloudius-systems.com>
The existing scylla-jmx code had reversed logic for the success of the repair:
it reported to "nodetool repair" a failure when the repair was successful :-)
Note that "nodetool repair" waits until a FINISHED notification, and then reports
a failure if it previously got any SESSION_FAILED notification; So if repair was
successful, all we need to do is to avoid sending a "SESSION_FAILED" message.
But we don't need to send any additional "SESSION_SUCCESS" message to signal
success. That message type is only used to report progress to the user (a
"session" is part of the repair work, so seeing sessions completing shows
progress), but because Scylla doesn't support this progress report yet, we
can't send these notifications yet, and there's no point in sending one such
message at the end - it's only confusing (especially when the text is the same
as that of the FINISHED message).
Signed-off-by: Nadav Har'El <nyh@cloudius-systems.com>
This cache the get data center and get rack results for 10s, it has a
direct impact on nodetool status and nodetool ring
Signed-off-by: Amnon Heiman <amnon@cloudius-systems.com>
Some operations are not changing frequently and are called multiple time
during a nodetool execution.
This patch adds the ability to cache results for a define period of time
(typically it will be for a few seconds) so that during the same
nodetool command call, the results will be retrieved from the cache.
It is currently only implemented for string values, other commands will
be added when needed.
Signed-off-by: Amnon Heiman <amnon@cloudius-systems.com>
"This series complete the scylla series to support the nodetool cfstatus support.
After this series it will be possible to call nodetoold cfstatus and get a meaningfull output.
An output example:
./bin/nodetool cfstats keyspace1
Keyspace: keyspace1
Read Count: 87657
Read Latency: 1.1418900715287998 ms.
Write Count: 87177
Write Latency: 0.022303761313190406 ms.
Pending Flushes: 0
Table: standard1
SSTable count: 8
SSTables in each level: [ Space used (live): 92356832
Space used (total): 92356832
Space used by snapshots (total): 0
Off heap memory used (total): 106430512
SSTable Compression Ratio: 0.0
Number of keys (estimate): 328672
Memtable cell count: 100000
Memtable data size: 84800254
Memtable off heap memory used: 105906176
Memtable switch count: 4
Local read count: 92854
Local read latency: 1.039 ms
Local write count: 93880
Local write latency: 1.045 ms
Pending flushes: 0
Bloom filter false positives: 0
Bloom filter false ratio: 0.00000
Bloom filter space used: 208416
Bloom filter off heap memory used: 524336
Index summary off heap memory used: 0
Compression metadata off heap memory used: 0
Compacted partition minimum bytes: 259
Compacted partition maximum bytes: 310"
The logic of that timer, should be that after some defined time from the
previous request a new one will be sent, the actuall rate is
meaningless and only cause delyed request to be sent in a higher
frequency after the delayed reponse was returned.
Signed-off-by: Amnon Heiman <amnon@cloudius-systems.com>
This combine the different constructor into one constructor with the
logic and another one that calls it with a default value.
Signed-off-by: Amnon Heiman <amnon@cloudius-systems.com>
The use of mean and variance as histogram parameter names makes more
sense.
This also make it safe to use an empty histogram that holds no samples.
Signed-off-by: Amnon Heiman <amnon@cloudius-systems.com>
