Commit Graph

59 Commits

Author SHA1 Message Date
Amnon Heiman
695e23bd4e RecentEstimatedHistogram: Support empty histogram
The RecentEstimatedHistogram updates its value from the API with an
array of recent values.

This array can be empty, in that case the getBuckets method should just
return a zero size array.

Signed-off-by: Amnon Heiman <amnon@scylladb.com>
2015-11-19 11:55:34 +02:00
Amnon Heiman
e0dea0c27e EstimatedHistogram: Support empty histogram
When creating an estimated histogram from buckets it is a valid option
to get a zero size array as the buckets array.

In that case the newOffsets method would get a negative value for its
size, which should result in a zero length array of offsets.

Signed-off-by: Amnon Heiman <amnon@scylladb.com>
2015-11-19 11:52:17 +02:00
Amnon Heiman
5362b2045d APIClient: Support Empty histogram
An empty histogram can return a valid response from the API but without
any buckets.
This is a valid scenario and common for counters of features that are
not supported yet.
In those cases, the APIClient should return a zero length array.

Signed-off-by: Amnon Heiman <amnon@scylladb.com>
2015-11-19 11:49:31 +02:00
Amnon Heiman
1cb048effe Create and register the APISettableMetrics
This patch adds the ability to create and register the
APISettableMettics.

Signed-off-by: Amnon Heiman <amnon@scylladb.com>
2015-11-16 11:29:21 +02:00
Amnon Heiman
ac049129de Adding the APISettableMeter
Sometimes it is required that a meter will not handle its own data, like
the APIMeter does.

This patch break the added functionality of APIMeter into two classes,
APISettableMeter is a Meter with a set value method and APIMeter adds
the functionality that reads from the API.

Signed-off-by: Amnon Heiman <amnon@scylladb.com>
2015-11-16 11:27:31 +02:00
Amnon Heiman
43e07fda0e Use the configuration object
This patch changes the APIClient to read the connection string from the
configuration object.

Main uses the same configuraion API to print it's connecting message and
call the configuration setup.

Signed-off-by: Amnon Heiman <amnon@scylladb.com>
2015-11-08 12:12:27 +02:00
Amnon Heiman
39b50f63c7 Adding a configuration object for the jmx proxy
This patch adds a configuration object to the jmx proxy that support
both the system/command line based properties and it accept a yaml
configuration file. The later options allows the jmx to read scylla
configuration file and connect to it based on this configuration.

The configuration file reader uses a yaml parser that was added to the
pom.xml

If no configuration file is found in the command line, it would look for
SCYLLA_CONF then SCYLLA_HOME then for relative 'conf' directory

Signed-off-by: Amnon Heiman <amnon@scylladb.com>

need merge apiconfig
2015-11-08 12:12:27 +02:00
Amnon Heiman
1db077618d APIClient: Add getMapStringIntegerValue and getMapStringLongValue
This patch adds two map reader function to the APIClient, one that parse
map<String,Integer> and one for map<String,Long>

Signed-off-by: Amnon Heiman <amnon@scylladb.com>
2015-11-03 11:18:33 +02:00
Amnon Heiman
649922fed8 Updating the http client and support API exception
The API returns errors with an HTTP code 400, 404 or 500 depending on
the cause with a json object that contains the failure reason.  The
error message should be populate to the JMX calling client, translated
to the appropriate exception.

This patch adds the ability to detect API failure and throw a runtime
exception with the returned message.

It is up to the calling method what to do with the exception, if it
would do nothing, the calling client would get a RuntimeException,
depends on origin MBean definition, the caller can catch the exception
and throw a specific kind.

Note that any exception that is thrown must be known to the JMX client
or it will not be able to process it.

As a first step, this patch replaces the jersey client to the newer
version under glassfish, which has an easy way of getting a Reply
object, and check its status before returning the results.

The only difference in the method that uses the APIClient is the use of
MultivaluedHashMap.

The following MBean implementation where changed

ColumnFamily
CompactionManager
Gossiper
EndpointSnitchInfo
CacheService
StorageProxy
StorageService

Signed-off-by: Amnon Heiman <amnon@scylladb.com>
2015-10-28 11:25:37 +02:00
Amnon Heiman
a7c30493cd APIClient to support EstimatedHistogram
This patch modify the CacheEntry to support both String and
EstimatedHistogram.

It is possible to add more supported types in the future when needed.

In the APIClient, the cache will now support both String and
EstimatedHistogram in a similiar way.

Signed-off-by: Amnon Heiman <amnon@cloudius-systems.com>

apiclient need to merge to cache
2015-10-21 14:20:48 +03:00
Amnon Heiman
931571c3ed EstimatedHistogram: Add constructor from data
This patch allows to create an EstimatedHistogram from an array of data
value.

It will be used by the APIClient to return EstimatedHistogram

Signed-off-by: Amnon Heiman <amnon@cloudius-systems.com>
2015-10-21 14:20:04 +03:00
Amnon Heiman
a568e52cb5 Adding the RecentEstimatedHistogram
Some of the jmx methods uses the notion of recent estimated histogram.
In origin the implementation uses an estimated histogram and clean the
histogram values on each call.

The RecentEstimatedHistogram mimic this behaviour, it store the latest
values of the last call. In each call new values are stored in the
histogram and the results is the delta between the last two calls.

Signed-off-by: Amnon Heiman <amnon@cloudius-systems.com>
2015-10-21 14:19:57 +03:00
Amnon Heiman
c5c0fb41fe APIClient: Adding cache support to the APIClient
Some operations are not changing frequently and are called multiple time
during a nodetool execution.

This patch adds the ability to cache results for a define period of time
(typically it will be for a few seconds) so that during the same
nodetool command call, the results will be retrieved from the cache.

It is currently only implemented for string values, other commands will
be added when needed.

Signed-off-by: Amnon Heiman <amnon@cloudius-systems.com>
2015-10-07 10:13:05 +03:00
Amnon Heiman
38af24d49d Adding CacheEntry to be used by the APIClient cache 2015-10-07 09:43:37 +03:00
Amnon Heiman
9c7ab3fcf6 APIHistogram: Use the new histogram parameter name
The use of mean and variance as histogram parameter names makes more
sense.

This also make it safe to use an empty histogram that holds no samples.

Signed-off-by: Amnon Heiman <amnon@cloudius-systems.com>
2015-10-04 12:46:44 +03:00
Amnon Heiman
3e326bf2d0 APIClient: Fixing a problem with getString
The javax does not handle parsing JsonValue directly. So it is replaced
for the getString with a direct removal of the quotations.

The alternative is to replace the javax parser with a different one.

Signed-off-by: Amnon Heiman <amnon@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
2015-09-14 13:32:15 +03:00
Amnon Heiman
cce767e47e APIClient: getString to return parsed json value
getString should return the string value without the sarounding quotes.

For getRaw was introduce that return the API values in its raw value,
and getString was changed to use the json parser to get the string
values without the quotes.

Signed-off-by: Amnon Heiman <amnon@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudus-systems.com>
2015-09-11 09:25:37 +03:00
Pekka Enberg
db6d48a695 Merge "Adding notification for the repair command" from Amnon
"The API is currently not supporting notification, but the repair
 command that perform via the nodetool relays on the notification to know
 when the command terminate.

 This series adds support for the repair notification, based on a timer
 and periodically check if a current repair command was terminated."
2015-08-31 09:41:01 +03:00
Amnon Heiman
6fc2c5c720 APIClient add postInt
Sometimes a post command need to return, this adds a method to perform a
post command that returns an int value.

The general postGetVal, can be used for other types if needed.

Signed-off-by: Amnon Heiman <amnon@cloudius-systems.com>
2015-08-27 20:46:02 +03:00
Amnon Heiman
bde40aed53 APIClient: Add a getter for estimated histogram buckets
This method returns the estimated histogram buckets. It is based on the
module definied in utils.json in the API.

Signed-off-by: Amnon Heiman <amnon@cloudius-systems.com>
2015-08-24 17:13:34 +03:00
Amnon Heiman
7e7e0ca367 APIClient: Add the getMapInetAddressFloatValue implementation
This adds the implementation to the stubed getMapInetAddressFloatValue,
it gets an array of 'key', 'value' and translate it into a map of key to
float.

Signed-off-by: Amnon Heiman <amnon@cloudius-systems.com>
2015-08-24 13:12:16 +03:00
Amnon Heiman
62eec8aaf2 Changing the newTimer in APIMetrics to return APITimer
The newTimer methods in the APIMetrics where modified to pass the url so
an APITimer would be returned from the registry.

Signed-off-by: Amnon Heiman <amnon@cloudius-systems.com>
2015-07-14 18:10:53 +03:00
Amnon Heiman
9d143efb07 Adding Timer creation to APIMetricsRegistry
This adds a Timer creation methods to the APIMetricsRegistry, the
newTimer method would create an APITimer and would pass it the url for
quering.

Signed-off-by: Amnon Heiman <amnon@cloudius-systems.com>
2015-07-14 18:10:53 +03:00
Amnon Heiman
479a0769d1 Clean up the APIMetricsRegistry
This is a clean up in the APIMetricsRegistry. It replaces snake case to
camel case, and perform a better formatting of the code.

Signed-off-by: Amnon Heiman <amnon@cloudius-systems.com>
2015-07-14 18:10:53 +03:00
Amnon Heiman
2d97995229 Adding the APITimer
The Timer object in the yammer library is used to regularly check a
histogram.

The APITimer is a Timer that uses the APIHistogram instead that in it
self calls the API to get its values.

Signed-off-by: Amnon Heiman <amnon@cloudius-systems.com>
2015-07-14 18:10:53 +03:00
Amnon Heiman
e2cdc95b81 APIHistogram: Moving the histogram data collection to the API
This patch replace the Histogram data and statistic calculation from the
JMX proxy to the API.

This way the count, sum, min, max variance and square sum are calculated
always on the server.

When an update is perform, the API would return the statistic with a
sample of the last n elements as a sample.

This implementation insure that the counters are correct. The
implementation also allows to set the minimal update interval.

The implementation adds a mutable getter for each of the parameters so
they could be set to their new value.

Signed-off-by: Amnon Heiman <amnon@cloudius-systems.com>
2015-07-14 18:10:53 +03:00
Amnon Heiman
e02313923b APIClient add a getJsonObj and getHistogram methods
The getJsonObj returns a json object. The getHistogram returns a
HistogramValue object.

Signed-off-by: Amnon Heiman <amnon@cloudius-systems.com>
2015-07-14 18:10:53 +03:00
Amnon Heiman
5e19158c0f Adding the HistogramValue object
The HistogramValue object is used by the APIClient to returns a
histogram object from the API.

It contains the values that are defined in utils.json

Signed-off-by: Amnon Heiman <amnon@cloudius-systems.com>
2015-07-14 18:10:53 +03:00
Avi Kivity
3cef1a6f7a Merge "Adding the column family metrics API" from Amnon
"The column family matrics is a set of data related to the column family.

This series adds an API based on the ColumnFamilyMetrics mbean.
It has a stub implementation, just so the JMX proxy would get a response."
2015-07-05 17:34:48 +03:00
Amnon Heiman
1b36b044fb Instantiating the CompactionManger in Main 2015-07-02 09:23:29 +03:00
Amnon Heiman
ad23e435a1 Enhancing the APIClient
This patch adds the getListMapStrValue that returns a list of maps and a
helper method to create a map from json list, it
also adds a stub method for CQL queries.

Signed-off-by: Amnon Heiman <amnon@cloudius-systems.com>
2015-07-02 09:18:34 +03:00
Amnon Heiman
919785bc04 Importing the EstimateHistogram from origin 2015-06-30 18:38:18 +03:00
Amnon Heiman
c3e9bd6ef7 APIClinet: Add mapToString helper function and intArr impl
This adds a helper function to make a string out of a map, by default,
key/names are joing by '=' and entries are joined by ',' but it can be
modified.

It also adds the implementation for getIntArr, which return an array of
ints.

Signed-off-by: Amnon Heiman <amnon@cloudius-systems.com>
2015-06-25 09:47:37 +03:00
Amnon Heiman
a7be16ed99 Initilizing the CacheService from Main
This initilize the CacheService MBean in main

Signed-off-by: Amnon Heiman <amnon@cloudius-systems.com>
2015-06-24 16:50:26 +03:00
Amnon Heiman
4c63735586 Main starting the StorageProxy API
This start the StorageProxy API

Signed-off-by: Amnon Heiman <amnon@cloudius-systems.com>
2015-06-18 12:08:16 +03:00
Amnon Heiman
37c09679ff APIClient: add the getLongArrValue implementation
This adds the implementation for getLongArrValue with and without query
parameters support.

Signed-off-by: Amnon Heiman <amnon@cloudius-systems.com>
2015-06-18 12:08:16 +03:00
Amnon Heiman
331c6c13b3 APIClient adding getSetStringValue getMapStringListStrValue
This adds the getSetStringValue getMapStringListStrValue, the first
return a set of string and the second a map from a string to a list of
string.

Signed-off-by: Amnon Heiman <amnon@cloudius-systems.com>
2015-06-18 12:08:16 +03:00
Amnon Heiman
1cf0279644 APIMeter update the included value on each click
This change update the internal value on each time click, after this
change the rate calculation will be done on the proxy at the price of
continuesly query the server. It is yet to be determine if this is a
problem, if so, it is possible to do and store those calculation on the
server and just pull them on demend.

Signed-off-by: Amnon Heiman <amnon@cloudius-systems.com>
2015-06-18 12:08:16 +03:00
Amnon Heiman
d1f85080b4 APIMetrics: Creating an APIHistogram instead of Histogram
This change accept a URL for the histogram creation methods and passing
it to the default registry so the created histogram will be the
APIHistogram instead of Histogram.

Signed-off-by: Amnon Heiman <amnon@cloudius-systems.com>
2015-06-18 12:08:15 +03:00
Pekka Enberg
a600152de7 Merge 'Adding JMX Metric support'
From Amnon:

 "This series adopt the yammer Metric library to use with out API. The
  library it self is taken as is by adding it to the Maven dependency. The
  required adaptation is that information will not be added in push to the
  library Metric, instead, when required, the specific Metric would use
  the APIClient to fetch information from the system using the API.

  This approach would mean that there will be little adaptation when code
  would be passed from origin."

Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

Conflicts:
	pom.xml
2015-06-17 15:10:30 +03:00
Amnon Heiman
815e5b429c Urching-JMX start the column family registration
This adds a call to the ColumnFamilyStore registration, which runs a
timer and check if there are any changes in the registered Mbean.
2015-06-16 17:16:09 +03:00
Amnon Heiman
37a0e6926b Urchin-JMX Cleanup and add a getJsonArray method to the APIClient
This patch adds a method that returns a JsonArray by the APIClient.

It also changes an errounouse string == compare to equals.
2015-06-16 17:15:28 +03:00
Amnon Heiman
67abbeb060 Enhencing the APIClient
This patch enhence the APICLient with additional support to query
parameters in getStringValue, getListStrValue. New method that will be
aded, will be added with both variation, where the base comunication
method: get can accept a null query parameter object.

An implementation was added for getMapStrValue that retreive a list of
key, value and creates a map out of it.

post and delete with and without query parameters.

getIntValue with query parameters.

getMapListStrValue, getListInetAddressValue, getMapStringSnapshotTabularDataValue

And the helper functions for the query parameter:
join, set_query_param, set_bool_query_param

Signed-off-by: Amnon Heiman <amnon@cloudius-systems.com>
2015-06-15 16:35:50 +03:00
Amnon Heiman
565d62d4db Importing FileUtils.java Pair.java SnapshotDetailsTabularData.java
This adds the functionality that is used by the JMX from Origin.

The following files where import: FileUtils.java  Pair.java
SnapshotDetailsTabularData.java, un needed functionality was removed to
minimize the dependency and they were placed in a util directory.

Signed-off-by: Amnon Heiman <amnon@cloudius-systems.com>
2015-06-15 16:35:44 +03:00
Amnon Heiman
7aba32d2fa Adding the APIMetrics
This is a copy of the Metrics class from yammer, with two differences:
Add a url Metrics that needs it to retrieve data and uses the
APIMetricsRegistry as the Metric factory.

Signed-off-by: Amnon Heiman <amnon@cloudius-systems.com>
2015-06-11 14:11:25 +03:00
Amnon Heiman
1212e09ffd Adding the APIMetricsRegistry the metric factory
The MetricsRegistry in yammer used to create metrics and register them
in the JMX.

The APIMetricsRegistry extends the functionality by accepting a URL
that the API will perform to retrieve the relevant data, it also creates
the API version of each of the requested Metric (i.e. APIMeter,
APICounter or APIHistogram)

Signed-off-by: Amnon Heiman <amnon@cloudius-systems.com>
2015-06-11 14:08:10 +03:00
Amnon Heiman
dd2d50a45c Overriden the Metric data collection
Three of the Metric in the yammer library are based on data that is
pushed to them: Counter, Histogram and Meter.

This patch modify the specific functionality by inherit the original
Meter, keeping its functionality and API, and modify the way the data is
collected.

For Counter: A call to count will be implementing by calling the API to
retreive a value.

For Meter: A call to count will be implementing by calling the API to
retreive a value, similiar to count, but the timer aspect of the Meter
remains the same.

For histogram: Histogram uses an internal sample container. To mimic the
behaviour, a call to any of the historgram get functionality will update
the sample container first. A timestamp of the last update limits the
number of updates that will be done by the histogram.

Signed-off-by: Amnon Heiman <amnon@cloudius-systems.com>
2015-06-11 11:53:51 +03:00
Amnon Heiman
5eba35e439 Passing the metrics nameFactory from Origin
The DefaultNameFactory which implement the MetricNameFactory responsible
for the metrics names.

They are imported from Origin and placed under urchin package.

Signed-off-by: Amnon Heiman <amnon@cloudius-systems.com>
2015-06-11 11:48:36 +03:00
Amnon Heiman
9889727810 Add Failure detector to Main 2015-06-03 19:45:39 +03:00
Amnon Heiman
c7aff5bb5f Adding post and parameters support to the API Client
This adds a post method to the APIClient to perform POST command on the
API and uses the queryParams API to add query parameters to a query.

Signed-off-by: Amnon Heiman <amnon@cloudius-systems.com>
2015-05-25 18:36:05 +03:00