The API returns errors with an HTTP code 400, 404 or 500 depending on
the cause with a json object that contains the failure reason. The
error message should be populate to the JMX calling client, translated
to the appropriate exception.
This patch adds the ability to detect API failure and throw a runtime
exception with the returned message.
It is up to the calling method what to do with the exception, if it
would do nothing, the calling client would get a RuntimeException,
depends on origin MBean definition, the caller can catch the exception
and throw a specific kind.
Note that any exception that is thrown must be known to the JMX client
or it will not be able to process it.
As a first step, this patch replaces the jersey client to the newer
version under glassfish, which has an easy way of getting a Reply
object, and check its status before returning the results.
The only difference in the method that uses the APIClient is the use of
MultivaluedHashMap.
The following MBean implementation where changed
ColumnFamily
CompactionManager
Gossiper
EndpointSnitchInfo
CacheService
StorageProxy
StorageService
Signed-off-by: Amnon Heiman <amnon@scylladb.com>
There is a confusion in origin, the MBean declare token as the parameter
to remove, but the implementation actually uses host id.
This patch modify scylla implementation to pass a host id as the
parameter to remove.
Signed-off-by: Amnon Heiman <amnon@cloudius-systems.com>
"This series adds latency related depricated methods to the storage
proxy.
The implmenetation mimic origin, in which the depricated methods calls
the counters that replaces them."
In LatencyMetrics the URL is passed without the ending slash, this
patch use the same notation in ClientRequestMetrics.
Signed-off-by: Amnon Heiman <amnon@cloudius-systems.com>
This follow origin by adding the implementation for the depricated
metrics methods.
Similiar to origin, the implementation calls the implementation in the
metrics.
Signed-off-by: Amnon Heiman <amnon@cloudius-systems.com>
"This series together with the cfhistogram series in scylla adds the
missing functionality so that nodetoold cfhistogram would work.
After both series will be apply an execution example is:
./bin/nodetool cfhistograms keyspace1 standard1
keyspace1/standard1 histograms
Percentile SSTables Write Latency Read Latency Partition Size Cell Count
(micros) (micros) (bytes)
50% 0.00 6866.00 4866323.00 310 5
75% 0.00 8239.00 10090808.00 310 5
95% 0.00 20501.00 17436917.00 310 5
98% 0.00 35425.00 25109160.00 310 5
99% 0.00 51012.00 25109160.00 310 5
Min 0.00 2300.00 654950.00 259 5
Max 0.00 20924300.00 25109160.00 310 5"
This patch uses the estimated latency that was added to the column
family metrics to get the recent and estimated latency.
It follows the same logic as origin does to call the logic in metrics.
The following method implementation will be added:
getMemtableColumnsCount
getRecentSSTablesPerReadHistogram
getSSTablesPerReadHistogram
getLifetimeReadLatencyHistogramMicros
getRecentReadLatencyHistogramMicros
getRecentReadLatencyMicros
getLifetimeWriteLatencyHistogramMicros
getRecentWriteLatencyHistogramMicros
getRecentWriteLatencyMicros
getRangeCount
getTotalRangeLatencyMicros
getLifetimeRangeLatencyHistogramMicros
getRecentRangeLatencyHistogramMicros
getRecentRangeLatencyMicros
Signed-off-by: Amnon Heiman <amnon@cloudius-systems.com>
This uses the recent estimated histogram and the API based estimated
histogram to support the sstable per read recent and total estimated
histogram.
Signed-off-by: Amnon Heiman <amnon@cloudius-systems.com>
This adds the depricated total and recent estimated histogram.
It uses the new RecentEstimatedHistogram for the recent value and the
API based estimated histogram for the total latency.
Signed-off-by: Amnon Heiman <amnon@cloudius-systems.com>
The EstimatedHistogramWrapper is a helper class that holds the API
related data, so that a class that uses an EstimatedHistogram can
replace it with the wrapper and keep most of its code as is.
Signed-off-by: Amnon Heiman <amnon@cloudius-systems.com>
This patch modify the CacheEntry to support both String and
EstimatedHistogram.
It is possible to add more supported types in the future when needed.
In the APIClient, the cache will now support both String and
EstimatedHistogram in a similiar way.
Signed-off-by: Amnon Heiman <amnon@cloudius-systems.com>
apiclient need to merge to cache
This patch allows to create an EstimatedHistogram from an array of data
value.
It will be used by the APIClient to return EstimatedHistogram
Signed-off-by: Amnon Heiman <amnon@cloudius-systems.com>
Some of the jmx methods uses the notion of recent estimated histogram.
In origin the implementation uses an estimated histogram and clean the
histogram values on each call.
The RecentEstimatedHistogram mimic this behaviour, it store the latest
values of the last call. In each call new values are stored in the
histogram and the results is the delta between the last two calls.
Signed-off-by: Amnon Heiman <amnon@cloudius-systems.com>
The existing scylla-jmx code had reversed logic for the success of the repair:
it reported to "nodetool repair" a failure when the repair was successful :-)
Note that "nodetool repair" waits until a FINISHED notification, and then reports
a failure if it previously got any SESSION_FAILED notification; So if repair was
successful, all we need to do is to avoid sending a "SESSION_FAILED" message.
But we don't need to send any additional "SESSION_SUCCESS" message to signal
success. That message type is only used to report progress to the user (a
"session" is part of the repair work, so seeing sessions completing shows
progress), but because Scylla doesn't support this progress report yet, we
can't send these notifications yet, and there's no point in sending one such
message at the end - it's only confusing (especially when the text is the same
as that of the FINISHED message).
Signed-off-by: Nadav Har'El <nyh@cloudius-systems.com>
This cache the get data center and get rack results for 10s, it has a
direct impact on nodetool status and nodetool ring
Signed-off-by: Amnon Heiman <amnon@cloudius-systems.com>
Some operations are not changing frequently and are called multiple time
during a nodetool execution.
This patch adds the ability to cache results for a define period of time
(typically it will be for a few seconds) so that during the same
nodetool command call, the results will be retrieved from the cache.
It is currently only implemented for string values, other commands will
be added when needed.
Signed-off-by: Amnon Heiman <amnon@cloudius-systems.com>
"This series complete the scylla series to support the nodetool cfstatus support.
After this series it will be possible to call nodetoold cfstatus and get a meaningfull output.
An output example:
./bin/nodetool cfstats keyspace1
Keyspace: keyspace1
Read Count: 87657
Read Latency: 1.1418900715287998 ms.
Write Count: 87177
Write Latency: 0.022303761313190406 ms.
Pending Flushes: 0
Table: standard1
SSTable count: 8
SSTables in each level: [ Space used (live): 92356832
Space used (total): 92356832
Space used by snapshots (total): 0
Off heap memory used (total): 106430512
SSTable Compression Ratio: 0.0
Number of keys (estimate): 328672
Memtable cell count: 100000
Memtable data size: 84800254
Memtable off heap memory used: 105906176
Memtable switch count: 4
Local read count: 92854
Local read latency: 1.039 ms
Local write count: 93880
Local write latency: 1.045 ms
Pending flushes: 0
Bloom filter false positives: 0
Bloom filter false ratio: 0.00000
Bloom filter space used: 208416
Bloom filter off heap memory used: 524336
Index summary off heap memory used: 0
Compression metadata off heap memory used: 0
Compacted partition minimum bytes: 259
Compacted partition maximum bytes: 310"
The logic of that timer, should be that after some defined time from the
previous request a new one will be sent, the actuall rate is
meaningless and only cause delyed request to be sent in a higher
frequency after the delayed reponse was returned.
Signed-off-by: Amnon Heiman <amnon@cloudius-systems.com>
This combine the different constructor into one constructor with the
logic and another one that calls it with a default value.
Signed-off-by: Amnon Heiman <amnon@cloudius-systems.com>
The use of mean and variance as histogram parameter names makes more
sense.
This also make it safe to use an empty histogram that holds no samples.
Signed-off-by: Amnon Heiman <amnon@cloudius-systems.com>
In origin an undocumented feature (bug?) is that passing null as a host
name for getRack and getDatacenter returns the rack or datacenter
according to the loopbck address.
This follow the same behaviour, so when the host is null, the function
will not fail but will call the API with the local loopback address
Signed-off-by: Amnon Heiman <amnon@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
Some of the depricated method in column family are replaced by counters
in Origin. Still, some tools, like nodetool uses the depricated API.
This adds an implementation to some of the depricated method that is
based on the column family counters similiar to origin.
Signed-off-by: Amnon Heiman <amnon@cloudius-systems.com>
This uses the column family timer to check that the API is still
available.
After MAX_RETRY (currently set to 30s) the proxy will shutdown.
This address issue #2
Signed-off-by: Amnon Heiman <amnon@cloudius-systems.com>
The javax does not handle parsing JsonValue directly. So it is replaced
for the getString with a direct removal of the quotations.
The alternative is to replace the javax parser with a different one.
Signed-off-by: Amnon Heiman <amnon@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
getString should return the string value without the sarounding quotes.
For getRaw was introduce that return the API values in its raw value,
and getString was changed to use the json parser to get the string
values without the quotes.
Signed-off-by: Amnon Heiman <amnon@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudus-systems.com>
This adds the call to the API for the EndpointSnitch, the url are based
on the endpoint_snitch_info.json swagger definition file.
Signed-off-by: Amnon Heiman <amnon@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudus-systems.com>
"The API is currently not supporting notification, but the repair
command that perform via the nodetool relays on the notification to know
when the command terminate.
This series adds support for the repair notification, based on a timer
and periodically check if a current repair command was terminated."
This patch adds a notification support for the repair in StorageService.
When a repair command starts a timer is set to check the status of the
repair, when the repair complets it sends notification for the
successful or fail of the repair.
Because jconsole doesn't run method with variable number of parameters
an additional MBean method was added for async repair with only a
keyspace as its parameter.
All the forceAsycRepair methods are mapped to the asyncRepair metod that
replaces them in 2.2, when options will be supported in the repair, it
will be added to the jmx.
After this patch it is possible to register in the jconsole for
notification, perform a forceAsyncRepair and get the notification that
the repair complets
Sometimes a post command need to return, this adds a method to perform a
post command that returns an int value.
The general postGetVal, can be used for other types if needed.
Signed-off-by: Amnon Heiman <amnon@cloudius-systems.com>
This change adds path parameter support, for the cases that the path
parameter is the last on the path, for example in the ColumnFamily case
Signed-off-by: Amnon Heiman <amnon@cloudius-systems.com>
The getEstimatedRowSizeHistogram is based on the column_family.json that
defined in the API.
The implementation in the MBean and in the metrics is now works with the
get histogram as long array method in the client.
Signed-off-by: Amnon Heiman <amnon@cloudius-systems.com>
This method returns the estimated histogram buckets. It is based on the
module definied in utils.json in the API.
Signed-off-by: Amnon Heiman <amnon@cloudius-systems.com>
This adds the implementation of the effectiveOwnership and getOwnership
in StorageService. It uses the API that defined in storage_service.json.
After this patch the effectiveOwnership and getOwnership will be
available via jconsole.
Signed-off-by: Amnon Heiman <amnon@cloudius-systems.com>
This adds the implementation to the stubed getMapInetAddressFloatValue,
it gets an array of 'key', 'value' and translate it into a map of key to
float.
Signed-off-by: Amnon Heiman <amnon@cloudius-systems.com>
Some of the depricated function in StorgeProxy uses the Metrics object
to get the information.
This adds an implementation for the following:
getReadOperations
getTotalReadLatencyMicros
getRecentReadLatencyMicros
getTotalReadLatencyHistogramMicros
getRangeOperations
getTotalRangeLatencyMicros
getRecentRangeLatencyMicros
getWriteOperations
getTotalWriteLatencyMicros
getRecentWriteLatencyMicros
It uses the implementation in ClientRequestMetrics without adding new
functionality.
Signed-off-by: Amnon Heiman <amnon@cloudius-systems.com>
This patch changes the MBeans to 2.1 version, with the changes an empty
stub implementation was added to the implementation so the code can
compile.
Signed-off-by: Amnon Heiman <amnon@cloudius-systems.com>
This patch cleans the file formatting of ColumnFamilyStore, CommitLog,
StorageProxy and StorageService.
Those files will be modified when moving to 2.1
Signed-off-by: Amnon Heiman <amnon@cloudius-systems.com>
This changes the Timer in ColumnFamilyMetrics, CommitLogMetrics and
LatencyMetrics to be an APITimer.
The APITimer is passed the url to get the data from.
Signed-off-by: Amnon Heiman <amnon@cloudius-systems.com>
The newTimer methods in the APIMetrics where modified to pass the url so
an APITimer would be returned from the registry.
Signed-off-by: Amnon Heiman <amnon@cloudius-systems.com>
This adds a Timer creation methods to the APIMetricsRegistry, the
newTimer method would create an APITimer and would pass it the url for
quering.
Signed-off-by: Amnon Heiman <amnon@cloudius-systems.com>
This is a clean up in the APIMetricsRegistry. It replaces snake case to
camel case, and perform a better formatting of the code.
Signed-off-by: Amnon Heiman <amnon@cloudius-systems.com>
The Timer object in the yammer library is used to regularly check a
histogram.
The APITimer is a Timer that uses the APIHistogram instead that in it
self calls the API to get its values.
Signed-off-by: Amnon Heiman <amnon@cloudius-systems.com>
This patch replace the Histogram data and statistic calculation from the
JMX proxy to the API.
This way the count, sum, min, max variance and square sum are calculated
always on the server.
When an update is perform, the API would return the statistic with a
sample of the last n elements as a sample.
This implementation insure that the counters are correct. The
implementation also allows to set the minimal update interval.
The implementation adds a mutable getter for each of the parameters so
they could be set to their new value.
Signed-off-by: Amnon Heiman <amnon@cloudius-systems.com>
The HistogramValue object is used by the APIClient to returns a
histogram object from the API.
It contains the values that are defined in utils.json
Signed-off-by: Amnon Heiman <amnon@cloudius-systems.com>
This adds the StorageMetrics, the implementation is based on the API.
The API definition are in storage_service under the metrics part.
After this patch, it will be possible to connect with jconsole and see
the StorageMetrics mbean.
Signed-off-by: Amnon Heiman <amnon@cloudius-systems.com>
This Adds the CommitLogMetrics object.
The implementation is based on the API, that define in commitlog.json
After this patch, it will be possible to connect with jconsole and see
the CommitLogMetrics.
Signed-off-by: Amnon Heiman <amnon@cloudius-systems.com>
"The column family matrics is a set of data related to the column family.
This series adds an API based on the ColumnFamilyMetrics mbean.
It has a stub implementation, just so the JMX proxy would get a response."
"The CacheMetric is used to return statistics on the cache. This series adds
the definition of the CacheMetric and initilized it in the CacheSerivce.
After this series it will be possible to connect with jconsole and see that
there are 3 mbean available for key, row and counter cache statistics."
The CompactionMetrics is used to report statistics about compaction. It
is based on calling the API that defined in compaction_manager.json
Signed-off-by: Amnon Heiman <amnon@cloudius-systems.com>
This adds the CompactionManagerMBean and its implementation
CompactionManager.
The implementation is based on the API, that is define in
compaction_manager.json
Signed-off-by: Amnon Heiman <amnon@cloudius-systems.com>
This patch adds the getListMapStrValue that returns a list of maps and a
helper method to create a map from json list, it
also adds a stub method for CQL queries.
Signed-off-by: Amnon Heiman <amnon@cloudius-systems.com>
CacheService holds an instance of cache metrics per: key, row and counter.
This instantiate the different metrics. After this petch it will be
possible to connect with jsconsole and see those metrics
Signed-off-by: Amnon Heiman <amnon@cloudius-systems.com>
The CacheMetrics holds information per a specific Cache type (key, row
or counter)
It the clas was modified to use the API. The API definition can be found
in cache_service.json under metrics.
Signed-off-by: Amnon Heiman <amnon@cloudius-systems.com>
The ColumnFamilyMetrics is used to report statistic of a column family.
This adds an adaptation for the ColumnFamilyMetrics that works with the
API that define in column_family.json
It adds the ColumnFamilyMetrics to the ColumnFamilyStore and expose the
keyspace name, so the ColumnFamilyMetrics would have access to it.
Signed-off-by: Amnon Heiman <amnon@cloudius-systems.com>
The latency metrics holds a single counter, so there is no need to add
the total_latency to the url it is pointed to.
Removing it, can shorten the url in the API definitions.
This patch adds the implementation for the ColumnFamilyStore by adding
calls to the relevent API that define in column_family.json
This patch also do some clean up, by removing leading underscore from
parameter names and trailing white spaces.
Signed-off-by: Amnon Heiman <amnon@cloudius-systems.com>
This adds a helper function to make a string out of a map, by default,
key/names are joing by '=' and entries are joined by ',' but it can be
modified.
It also adds the implementation for getIntArr, which return an array of
ints.
Signed-off-by: Amnon Heiman <amnon@cloudius-systems.com>
This adds the CacheServiceMBean and its implementation the CacheService
class.
The implementation is based on calls to the API that define in
cache_service.json
Signed-off-by: Amnon Heiman <amnon@cloudius-systems.com>
This adds the StorageProxyMBean and implementation.
To keep the same API the relevant matrics where added.
Signed-off-by: Amnon Heiman <amnon@cloudius-systems.com>
LatencyMetrics
This import the implementation of CASClientRequestMetrics
ClientRequestMetrics and LatencyMetrics with modification to use the
APIMetrics with a given URL.
The Metrics where added to keep the same naming of of the MBean as they
are in Origin.
Signed-off-by: Amnon Heiman <amnon@cloudius-systems.com>
This adds the getSetStringValue getMapStringListStrValue, the first
return a set of string and the second a map from a string to a list of
string.
Signed-off-by: Amnon Heiman <amnon@cloudius-systems.com>
This change update the internal value on each time click, after this
change the rate calculation will be done on the proxy at the price of
continuesly query the server. It is yet to be determine if this is a
problem, if so, it is possible to do and store those calculation on the
server and just pull them on demend.
Signed-off-by: Amnon Heiman <amnon@cloudius-systems.com>
This change accept a URL for the histogram creation methods and passing
it to the default registry so the created histogram will be the
APIHistogram instead of Histogram.
Signed-off-by: Amnon Heiman <amnon@cloudius-systems.com>
From Amnon:
"This series adopt the yammer Metric library to use with out API. The
library it self is taken as is by adding it to the Maven dependency. The
required adaptation is that information will not be added in push to the
library Metric, instead, when required, the specific Metric would use
the APIClient to fetch information from the system using the API.
This approach would mean that there will be little adaptation when code
would be passed from origin."
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
Conflicts:
pom.xml
This adds the ColumnFamilyStoreMBean with most of its method stub. The
only current functionality is returnning the column family name and the
registration of the StoreMBean.
Signed-off-by: Amnon Heiman <amnon@cloudius-systems.com>
This adds the following method implementation that are supported:
getLiveNodes()
getUnreachableNodes()
getTokens()
getTokens(String endpoint)
getCommitLogLocation()
getTokenToEndpointMap()
After applying this series, those functionality will be available via
jconsole
The series also complete the API by adding an implementation based on
the swagger storage_service.json definition. An implementation that
will be added to the API should be reflected in the MBean.
Signed-off-by: Amnon Heiman <amnon@cloudius-systems.com>
This patch enhence the APICLient with additional support to query
parameters in getStringValue, getListStrValue. New method that will be
aded, will be added with both variation, where the base comunication
method: get can accept a null query parameter object.
An implementation was added for getMapStrValue that retreive a list of
key, value and creates a map out of it.
post and delete with and without query parameters.
getIntValue with query parameters.
getMapListStrValue, getListInetAddressValue, getMapStringSnapshotTabularDataValue
And the helper functions for the query parameter:
join, set_query_param, set_bool_query_param
Signed-off-by: Amnon Heiman <amnon@cloudius-systems.com>
This adds the functionality that is used by the JMX from Origin.
The following files where import: FileUtils.java Pair.java
SnapshotDetailsTabularData.java, un needed functionality was removed to
minimize the dependency and they were placed in a util directory.
Signed-off-by: Amnon Heiman <amnon@cloudius-systems.com>
This is a copy of the Metrics class from yammer, with two differences:
Add a url Metrics that needs it to retrieve data and uses the
APIMetricsRegistry as the Metric factory.
Signed-off-by: Amnon Heiman <amnon@cloudius-systems.com>
The MetricsRegistry in yammer used to create metrics and register them
in the JMX.
The APIMetricsRegistry extends the functionality by accepting a URL
that the API will perform to retrieve the relevant data, it also creates
the API version of each of the requested Metric (i.e. APIMeter,
APICounter or APIHistogram)
Signed-off-by: Amnon Heiman <amnon@cloudius-systems.com>
Three of the Metric in the yammer library are based on data that is
pushed to them: Counter, Histogram and Meter.
This patch modify the specific functionality by inherit the original
Meter, keeping its functionality and API, and modify the way the data is
collected.
For Counter: A call to count will be implementing by calling the API to
retreive a value.
For Meter: A call to count will be implementing by calling the API to
retreive a value, similiar to count, but the timer aspect of the Meter
remains the same.
For histogram: Histogram uses an internal sample container. To mimic the
behaviour, a call to any of the historgram get functionality will update
the sample container first. A timestamp of the last update limits the
number of updates that will be done by the histogram.
Signed-off-by: Amnon Heiman <amnon@cloudius-systems.com>
The DefaultNameFactory which implement the MetricNameFactory responsible
for the metrics names.
They are imported from Origin and placed under urchin package.
Signed-off-by: Amnon Heiman <amnon@cloudius-systems.com>
This adds the FailureDetectorMBean and its implementation.
Setting the phi is not supported yet and will be added when the code
will be completed inside the failure_detector implementation.
Signed-off-by: Amnon Heiman <amnon@cloudius-systems.com>
This adds the implementation of the Gossiper MBean.
To test, run an urchin server, run the API and use jconsole to connect
to the API.
Signed-off-by: Amnon Heiman <amnon@cloudius-systems.com>
This adds a post method to the APIClient to perform POST command on the
API and uses the queryParams API to add query parameters to a query.
Signed-off-by: Amnon Heiman <amnon@cloudius-systems.com>
This adds two of the supported CommitLog method, the
getActiveSegmentNames and the getArchivingSegmentNames.
The API for both returns full path, so the JMX API takes the file name
from the path and remove duplicates.
Signed-off-by: Amnon Heiman <amnon@cloudius-systems.com>
This was a code that was modified from Origin, in the JMX API
initilization is done in the custructor, which mean that it would enter
an endless loop.
When initiliation in the constructor this should be used.
Signed-off-by: Amnon Heiman <amnon@cloudius-systems.com>
The urchin files (Client and Main) where moved to:
com.cloudius.urchin.api and com.cloudius.urchin.main respectively.
Signed-off-by: Amnon Heiman <amnon@cloudius-systems.com>
Signed-off-by: Avi Kivity <avi@cloudius-systems.com>
This adds a stub of the StorageService.
After applying it, it will be possible to use JConsole to connect to the
process and run any of the command and inquire all the properties.
Some deprecated methods uses RepairParallelism class, an empty
implementation for it is added. It would be removed with future version
of the MBean, when those methods will be removed from the MBean
As this is a stub, the only effect is a printout on the screen.
Signed-off-by: Amnon Heiman <amnon@cloudius-systems.com>
