Commit Graph

54 Commits

Author SHA1 Message Date
Pekka Enberg
b11c108fcf Remove obsolete comments from ColumnFamilyStore 2015-10-21 14:28:17 +03:00
Pekka Enberg
8e0fb98fc4 Merge "Adding estimated histogram support for nodetool cfhistogram" from Amnon
"This series together with the cfhistogram series in scylla adds the
 missing functionality so that nodetoold cfhistogram would work.

 After both series will be apply an execution example is:

 ./bin/nodetool cfhistograms keyspace1 standard1
 keyspace1/standard1 histograms
 Percentile  SSTables     Write Latency      Read Latency    Partition Size        Cell Count
                               (micros)          (micros)           (bytes)
 50%             0.00           6866.00        4866323.00               310                 5
 75%             0.00           8239.00       10090808.00               310                 5
 95%             0.00          20501.00       17436917.00               310                 5
 98%             0.00          35425.00       25109160.00               310                 5
 99%             0.00          51012.00       25109160.00               310                 5
 Min             0.00           2300.00         654950.00               259                 5
 Max             0.00       20924300.00       25109160.00               310                 5"
2015-10-21 14:26:40 +03:00
Amnon Heiman
3667682075 ColumnFamilyStore: add support for estimated latency
This patch uses the estimated latency that was added to the column
family metrics to get the recent and estimated latency.

It follows the same logic as origin does to call the logic in metrics.

The following method implementation will be added:

getMemtableColumnsCount
getRecentSSTablesPerReadHistogram
getSSTablesPerReadHistogram
getLifetimeReadLatencyHistogramMicros
getRecentReadLatencyHistogramMicros
getRecentReadLatencyMicros
getLifetimeWriteLatencyHistogramMicros
getRecentWriteLatencyHistogramMicros
getRecentWriteLatencyMicros
getRangeCount
getTotalRangeLatencyMicros
getLifetimeRangeLatencyHistogramMicros
getRecentRangeLatencyHistogramMicros
getRecentRangeLatencyMicros

Signed-off-by: Amnon Heiman <amnon@cloudius-systems.com>
2015-10-21 14:22:06 +03:00
Amnon Heiman
db3b3cdeee ColumnFamilyMetrics: add estimted per read histogram support
This uses the recent estimated histogram and the API based estimated
histogram to support the sstable per read recent and total estimated
histogram.

Signed-off-by: Amnon Heiman <amnon@cloudius-systems.com>
2015-10-21 14:22:06 +03:00
Amnon Heiman
960aa6f509 LatencyMetrics: Support totalLatencyHistogram and recentLatencyHistogram
This adds the depricated total and recent estimated histogram.
It uses the new RecentEstimatedHistogram for the recent value and the
API based estimated histogram for the total latency.

Signed-off-by: Amnon Heiman <amnon@cloudius-systems.com>
2015-10-21 14:22:06 +03:00
Amnon Heiman
6df716e5f7 Add an EstimateHistogramWrapper
The EstimatedHistogramWrapper is a helper class that holds the API
related data, so that a class that uses an EstimatedHistogram can
replace it with the wrapper and keep most of its code as is.

Signed-off-by: Amnon Heiman <amnon@cloudius-systems.com>
2015-10-21 14:21:53 +03:00
Amnon Heiman
3bf092c83b StorageService: handle loadNewSStable and forceKeyspaceCleanup
keyspace cleanup was changed from GET to POST and load sstable was
missing the keyspace name.

Signed-off-by: Amnon Heiman <amnon@cloudius-systems.com>
2015-10-21 12:27:59 +03:00
Nadav Har'El
f05c6709dd Fix "nodetool repair" error
The existing scylla-jmx code had reversed logic for the success of the repair:
it reported to "nodetool repair" a failure when the repair was successful :-)

Note that "nodetool repair" waits until a FINISHED notification, and then reports
a failure if it previously got any SESSION_FAILED notification; So if repair was
successful, all we need to do is to avoid sending a "SESSION_FAILED" message.
But we don't need to send any additional "SESSION_SUCCESS" message to signal
success. That message type is only used to report progress to the user (a
"session" is part of the repair work, so seeing sessions completing shows
progress), but because Scylla doesn't support this progress report yet, we
can't send these notifications yet, and there's no point in sending one such
message at the end - it's only confusing (especially when the text is the same
as that of the FINISHED message).

Signed-off-by: Nadav Har'El <nyh@cloudius-systems.com>
2015-10-15 09:50:35 +03:00
Amnon Heiman
6ed255f2e3 EndpointSnitchInfo: Cache results for data center and rack
This cache the get data center and get rack results for 10s, it has a
direct impact on nodetool status and nodetool ring

Signed-off-by: Amnon Heiman <amnon@cloudius-systems.com>
2015-10-07 10:24:50 +03:00
Amnon Heiman
929d0aedd4 EndpointSnitchInfo formatting 2015-10-07 10:14:49 +03:00
Pekka Enberg
a90e080ee6 Merge "Complete the nodetool cfstats support" from Amnon
"This series complete the scylla series to support the nodetool cfstatus support.

After this series it will be possible to call nodetoold cfstatus and get a meaningfull output.

An output example:
./bin/nodetool cfstats keyspace1
Keyspace: keyspace1
        Read Count: 87657
        Read Latency: 1.1418900715287998 ms.
        Write Count: 87177
        Write Latency: 0.022303761313190406 ms.
        Pending Flushes: 0
                Table: standard1
                SSTable count: 8
                SSTables in each level: [               Space used (live): 92356832
                Space used (total): 92356832
                Space used by snapshots (total): 0
                Off heap memory used (total): 106430512
                SSTable Compression Ratio: 0.0
                Number of keys (estimate): 328672
                Memtable cell count: 100000
                Memtable data size: 84800254
                Memtable off heap memory used: 105906176
                Memtable switch count: 4
                Local read count: 92854
                Local read latency: 1.039 ms
                Local write count: 93880
                Local write latency: 1.045 ms
                Pending flushes: 0
                Bloom filter false positives: 0
                Bloom filter false ratio: 0.00000
                Bloom filter space used: 208416
                Bloom filter off heap memory used: 524336
                Index summary off heap memory used: 0
                Compression metadata off heap memory used: 0
                Compacted partition minimum bytes: 259
                Compacted partition maximum bytes: 310"
2015-10-06 12:23:09 +03:00
Amnon Heiman
a8292e3e09 ColumnFamilyMetrics: add estimated row count histogram
This adds the estimated row count histogram support to
ColumnFamilyMetrics

Signed-off-by: Amnon Heiman <amnon@cloudius-systems.com>
2015-10-04 12:54:31 +03:00
Amnon Heiman
54fe3d7ccf ColumnFamilyMetrics: Use schedule for the column family timer
The logic of that timer, should be that after some defined time from the
previous request a new one will be sent, the actuall rate is
meaningless and only cause delyed request to be sent in a higher
frequency after the delayed reponse was returned.

Signed-off-by: Amnon Heiman <amnon@cloudius-systems.com>
2015-10-04 12:49:51 +03:00
Amnon Heiman
f16c49a860 LatencyMetrics: combine the different constructors
This combine the different constructor into one constructor with the
logic and another one that calls it with a default value.

Signed-off-by: Amnon Heiman <amnon@cloudius-systems.com>
2015-10-04 12:48:26 +03:00
Amnon Heiman
d67e6e60fc EndPointSnitchInfo: Support null as host name
In origin an undocumented feature (bug?) is that passing null as a host
name for getRack and getDatacenter returns the rack or datacenter
according to the loopbck address.

This follow the same behaviour, so when the host is null, the function
will not fail but will call the API with the local loopback address

Signed-off-by: Amnon Heiman <amnon@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
2015-09-29 09:30:08 +03:00
Amnon Heiman
5feb4d3841 Adding depricated implementation to column_family
Some of the depricated method in column family are replaced by counters
in Origin. Still, some tools, like nodetool uses the depricated API.

This adds an implementation to some of the depricated method that is
based on the column family counters similiar to origin.

Signed-off-by: Amnon Heiman <amnon@cloudius-systems.com>
2015-09-17 13:43:19 +03:00
Amnon Heiman
713f8a5c75 Shuting down the the proxy after MAX_RETRY
This uses the column family timer to check that the API is still
available.

After MAX_RETRY (currently set to 30s) the proxy will shutdown.
This address issue #2

Signed-off-by: Amnon Heiman <amnon@cloudius-systems.com>
2015-09-17 13:38:01 +03:00
Amnon Heiman
fe3b786053 Urchin-JMX: Adding the EndpointSnitchInfo implementation
This adds the call to the API for the EndpointSnitch, the url are based
on the endpoint_snitch_info.json swagger definition file.

Signed-off-by: Amnon Heiman <amnon@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudus-systems.com>
2015-09-11 09:25:00 +03:00
Pekka Enberg
db6d48a695 Merge "Adding notification for the repair command" from Amnon
"The API is currently not supporting notification, but the repair
 command that perform via the nodetool relays on the notification to know
 when the command terminate.

 This series adds support for the repair notification, based on a timer
 and periodically check if a current repair command was terminated."
2015-08-31 09:41:01 +03:00
Amnon Heiman
67dca4da9d StorageService: Add notification suport for the repair command
This patch adds a notification support for the repair in StorageService.

When a repair command starts a timer is set to check the status of the
repair, when the repair complets it sends notification for the
successful or fail of the repair.

Because jconsole doesn't run method with variable number of parameters
an additional MBean method was added for async repair with only a
keyspace as its parameter.

All the forceAsycRepair methods are mapped to the asyncRepair metod that
replaces them in 2.2, when options will be supported in the repair, it
will be added to the jmx.

After this patch it is possible to register in the jconsole for
notification, perform a forceAsyncRepair and get the notification that
the repair complets
2015-08-27 21:02:28 +03:00
Amnon Heiman
e96e62b6bd Add the RepairParallelism enum
This takes the RepairParallelism enum from origin

Signed-off-by: Amnon Heiman <amnon@cloudius-systems.com>
2015-08-27 20:46:02 +03:00
Amnon Heiman
2cfe02acb8 LatencyMetrics: Support path parameter
This change adds path parameter support, for the cases that the path
parameter is the last on the path, for example in the ColumnFamily case

Signed-off-by: Amnon Heiman <amnon@cloudius-systems.com>
2015-08-24 17:13:34 +03:00
Amnon Heiman
0ae282b548 Add the getEstimatedRowSizeHistogram implementation
The getEstimatedRowSizeHistogram is based on the column_family.json that
defined in the API.

The implementation in the MBean and in the metrics is now works with the
get histogram as long array method in the client.

Signed-off-by: Amnon Heiman <amnon@cloudius-systems.com>
2015-08-24 17:13:34 +03:00
Amnon Heiman
b53be3a4ec StorageService: Add the effectiveOwnership and getOwnership implementation
This adds the implementation of the effectiveOwnership and getOwnership
in StorageService. It uses the API that defined in storage_service.json.

After this patch the effectiveOwnership and getOwnership will be
available via jconsole.

Signed-off-by: Amnon Heiman <amnon@cloudius-systems.com>
2015-08-24 13:12:33 +03:00
Amnon Heiman
631a6537d9 Add depricated implemention to the StorageProxy
Some of the depricated function in StorgeProxy uses the Metrics object
to get the information.

This adds an implementation for the following:
getReadOperations
getTotalReadLatencyMicros
getRecentReadLatencyMicros
getTotalReadLatencyHistogramMicros
getRangeOperations
getTotalRangeLatencyMicros
getRecentRangeLatencyMicros
getWriteOperations
getTotalWriteLatencyMicros
getRecentWriteLatencyMicros

It uses the implementation in ClientRequestMetrics without adding new
functionality.

Signed-off-by: Amnon Heiman <amnon@cloudius-systems.com>
2015-08-10 09:00:47 +03:00
Amnon Heiman
d1e16178df Moving the MBean to 2.1 (8)
This patch changes the MBeans to 2.1 version, with the changes an empty
stub implementation was added to the implementation so the code can
compile.

Signed-off-by: Amnon Heiman <amnon@cloudius-systems.com>
2015-07-30 12:01:05 +03:00
Amnon Heiman
057a3e27fd Cleaning file formatting
This patch cleans the file formatting of ColumnFamilyStore, CommitLog,
StorageProxy and StorageService.

Those files will be modified when moving to 2.1

Signed-off-by: Amnon Heiman <amnon@cloudius-systems.com>
2015-07-30 11:37:03 +03:00
Amnon Heiman
6a89f19988 Use APITimer in ColumnFamilyMetrics, CommitLogMetrics and LatencyMetrics
This changes the Timer in ColumnFamilyMetrics, CommitLogMetrics and
LatencyMetrics to be an APITimer.
The APITimer is passed the url to get the data from.

Signed-off-by: Amnon Heiman <amnon@cloudius-systems.com>
2015-07-14 18:10:53 +03:00
Amnon Heiman
7c8b1a86ca Adding the StorageMetrics
This adds the StorageMetrics, the implementation is based on the API.
The API definition are in storage_service under the metrics part.

After this patch, it will be possible to connect with jconsole and see
the StorageMetrics mbean.

Signed-off-by: Amnon Heiman <amnon@cloudius-systems.com>
2015-07-07 10:28:22 +03:00
Amnon Heiman
68cdf4acf4 Adding the CommitLogMetrics
This Adds the CommitLogMetrics object.
The implementation is based on the API, that define in commitlog.json

After this patch, it will be possible to connect with jconsole and see
the CommitLogMetrics.

Signed-off-by: Amnon Heiman <amnon@cloudius-systems.com>
2015-07-06 14:24:59 +03:00
Avi Kivity
3cef1a6f7a Merge "Adding the column family metrics API" from Amnon
"The column family matrics is a set of data related to the column family.

This series adds an API based on the ColumnFamilyMetrics mbean.
It has a stub implementation, just so the JMX proxy would get a response."
2015-07-05 17:34:48 +03:00
Avi Kivity
16368ca915 Merge "Adding the CacheMetric counters"
"The CacheMetric is used to return statistics on the cache.  This series adds
the definition of the CacheMetric and initilized it in the CacheSerivce.

After this series it will be possible to connect with jconsole and see that
there are 3 mbean available for key, row and counter cache statistics."
2015-07-05 16:47:53 +03:00
Amnon Heiman
81b26d2b53 Adding the CompactionMetrics
The CompactionMetrics is used to report statistics about compaction. It
is based on calling the API that defined in compaction_manager.json

Signed-off-by: Amnon Heiman <amnon@cloudius-systems.com>
2015-07-02 10:08:00 +03:00
Amnon Heiman
84168d40fb Adding the CompactionManagerMBean
This adds the CompactionManagerMBean and its implementation
CompactionManager.

The implementation is based on the API, that is define in
compaction_manager.json

Signed-off-by: Amnon Heiman <amnon@cloudius-systems.com>
2015-07-02 09:21:30 +03:00
Amnon Heiman
366b744d7b Instantiate the CacheMetrics
CacheService holds an instance of cache metrics per: key, row and counter.

This instantiate the different metrics. After this petch it will be
possible to connect with jsconsole and see those metrics

Signed-off-by: Amnon Heiman <amnon@cloudius-systems.com>
2015-07-01 10:11:21 +03:00
Amnon Heiman
19a7daf368 Adding the CacheMetrics class
The CacheMetrics holds information per a specific Cache type (key, row
or counter)

It the clas was modified to use the API. The API definition can be found
in cache_service.json under metrics.

Signed-off-by: Amnon Heiman <amnon@cloudius-systems.com>
2015-07-01 10:08:32 +03:00
Amnon Heiman
648c17a93a Adding the ColumnFamilyMetrics
The ColumnFamilyMetrics is used to report statistic of a column family.

This adds an adaptation for the ColumnFamilyMetrics that works with the
API that define in column_family.json

It adds the ColumnFamilyMetrics to the ColumnFamilyStore and expose the
keyspace name, so the ColumnFamilyMetrics would have access to it.

Signed-off-by: Amnon Heiman <amnon@cloudius-systems.com>
2015-06-30 18:38:18 +03:00
Amnon Heiman
baf7d101d1 LatencyMetrics: simplify the URL
The latency metrics holds a single counter, so there is no need to add
the total_latency to the url it is pointed to.

Removing it, can shorten the url in the API definitions.
2015-06-30 18:36:20 +03:00
Amnon Heiman
ae6021233f ColumnFamilyStore: Cleanup and adding missing implementation
This patch adds the implementation for the ColumnFamilyStore by adding
calls to the relevent API that define in column_family.json

This patch also do some clean up, by removing leading underscore from
parameter names and trailing white spaces.

Signed-off-by: Amnon Heiman <amnon@cloudius-systems.com>
2015-06-25 09:47:37 +03:00
Amnon Heiman
769b222173 Adding the CacheServiceMBean and CacheService
This adds the CacheServiceMBean and its implementation the CacheService
class.

The implementation is based on calls to the API that define in
cache_service.json

Signed-off-by: Amnon Heiman <amnon@cloudius-systems.com>
2015-06-24 16:48:56 +03:00
Amnon Heiman
a34df20a74 Adding the StorageProxyMBean
This adds the StorageProxyMBean and implementation.
To keep the same API the relevant matrics where added.

Signed-off-by: Amnon Heiman <amnon@cloudius-systems.com>
2015-06-18 12:08:16 +03:00
Amnon Heiman
8082ac854b Adding the CASClientRequestMetrics ClientRequestMetrics and
LatencyMetrics

This import the implementation of CASClientRequestMetrics
ClientRequestMetrics and LatencyMetrics with modification to use the
APIMetrics with a given URL.

The Metrics where added to keep the same naming of of the MBean as they
are in Origin.

Signed-off-by: Amnon Heiman <amnon@cloudius-systems.com>
2015-06-18 12:08:16 +03:00
Amnon Heiman
3e4cefb6bd Urchin-JMX adding the ColumnFamilyStoreMBean
This adds the ColumnFamilyStoreMBean with most of its method stub. The
only current functionality is returnning the column family name and the
registration of the StoreMBean.

Signed-off-by: Amnon Heiman <amnon@cloudius-systems.com>
2015-06-16 17:16:09 +03:00
Amnon Heiman
afac410a0d Adding StorageService MBean implementation
This adds the following method implementation that are supported:
getLiveNodes()
getUnreachableNodes()
getTokens()
getTokens(String endpoint)
getCommitLogLocation()
getTokenToEndpointMap()

After applying this series, those functionality will be available via
jconsole

The series also complete the API by adding an implementation based on
the swagger storage_service.json definition.  An implementation that
will be added to the API should be reflected in the MBean.

Signed-off-by: Amnon Heiman <amnon@cloudius-systems.com>
2015-06-15 16:36:22 +03:00
Amnon Heiman
c671310d0d Adding the FailureDetector MBean support
This adds the FailureDetectorMBean and its implementation.
Setting the phi is not supported yet and will be added when the code
will be completed inside the failure_detector implementation.

Signed-off-by: Amnon Heiman <amnon@cloudius-systems.com>
2015-06-03 19:45:39 +03:00
Amnon Heiman
8847343ebd Adding the Gossiper MBean implementation
This adds the implementation of the Gossiper MBean.

To test, run an urchin server, run the API and use jconsole to connect
to the API.

Signed-off-by: Amnon Heiman <amnon@cloudius-systems.com>
2015-05-25 18:36:12 +03:00
Amnon Heiman
b7e1f5a979 JMX API: Adding the CommitLog implementation
This adds two of the supported CommitLog method, the
getActiveSegmentNames and the getArchivingSegmentNames.

The API for both returns full path, so the JMX API takes the file name
from the path and remove duplicates.

Signed-off-by: Amnon Heiman <amnon@cloudius-systems.com>
2015-05-19 18:01:54 +03:00
Amnon Heiman
0ef065ebc7 JMX-API: Fix an endless loop in EndpointSnitchInfo
This was a code that was modified from Origin, in the JMX API
initilization is done in the custructor, which mean that it would enter
an endless loop.

When initiliation in the constructor this should be used.

Signed-off-by: Amnon Heiman <amnon@cloudius-systems.com>
2015-05-19 18:01:53 +03:00
Amnon Heiman
b0d8fd0de0 JMX API: Adding the EndpointSnitchInfoMBean stab
This adds a stab to the EndpointSnitchInfoMBean

Signed-off-by: Amnon Heiman <amnon@cloudius-systems.com>
2015-05-18 15:52:53 +03:00
Amnon Heiman
cd2b174cd3 JMX API: Add the GossiperMBean stab
This adds a stab implementation of the GossiperMBean

Signed-off-by: Amnon Heiman <amnon@cloudius-systems.com>
2015-05-18 15:33:55 +03:00