Currently, the nodetool uses the jmx server for the uptime, this is
confusing is what we expect is Scylla uptime.
This patch exposes the API uptime using MBean.
Relates to #154
Signed-off-by: Amnon Heiman <amnon@scylladb.com>
Closes#155
The scrub command was not supported from node_tool, but now when we want
to enable it the current API is not compatible with the 3.11 MBean
definition.
This patch adds the definition to the MBean and the implementation to
StorageService.
It also address two problems with the old scrub implementation, just
in case someone will use them.
1. Implementation didn't pass the parameters to the API.
2. A stub implementation called itself instead of calling an actual
implementation.
This patch will enable to test the command from nodetool additional
changes may come on top of it if more command line options will be
supported.
Signed-off-by: Amnon Heiman <amnon@scylladb.com>
Fixes#76
Requires seastar patch:
json: Make date formatter use RFC8601/RFC3339 format
Requires scylla patch set "Sstabledesc"
Forwards call to REST sstable_info and packs the data
into CompositeData for JMX consumption.
Almost 100% null implementations, which is ok for most purposes
currently used by scylla. Some of these new calls (like dropped
mutations etc) should perhaps however be implemented.
Tested with the nodetool dtests. So sparsely.
Needed when/if scylla-tools-java is upgraded to origin 3.11,
otherwise noodtool breaks.
Message-Id: <20180730113741.14952-1-calle@scylladb.com>
nodetool from scylla-tools-java still uses it. Currently `nodetool compact` fails like this:
error: forceKeyspaceCompaction(java.lang.String, [Ljava.lang.String;)
-- StackTrace --
java.lang.NoSuchMethodException: forceKeyspaceCompaction(java.lang.String, [Ljava.lang.String;)
Fixesscylladb/scylla#2261
Probably broken by 3e146845b4
Message-Id: <1491470483-6147-1-git-send-email-tgrabiec@scylladb.com>
This patch change the sorting of tokensEndpointMap so it will use the
order returned by the API.
See Scylladb/scylla#1945
Signed-off-by: Amnon Heiman <amnon@scylladb.com>
This patch adds the functionality of takeMultipleColumnFamilySnapshot to
StorageService.
It follow origin logic of first check that all keyspaces and column
families exists and has no snapshot with that name and then run snapshot
on each of the combinations.
Two methods where added to simplify the implementation, but that can be
reused. One to get a map from keyspace to column family and one with the
current snapshots.
Fixesscylladb/scylla#1133
Signed-off-by: Amnon Heiman <amnon@scylladb.com>
Message-Id: <1461659678-22030-1-git-send-email-amnon@scylladb.com>
When getting the tokens of the current node, we use the get_token api
call with the local broadcast address.
The current implementation that tries to figure it out from the
configuration is prone to error.
Currently in a configuration where the broadcast address is set to the
local API and the listening API is set to 127.0.0.1 we get a call to
nodetool info will return an exception:
ID : 54185d5d-6f62-4884-814c-5d17c2776de9
Gossip active : true
Thrift active : true
Native Transport active: true
Load : 178.09 KB
Generation No : 1458349593
Uptime (seconds) : 11
Heap Memory (MB) : 47.23 / 247.50
Off Heap Memory (MB) : 2.75
error: Index: 0, Size: 0
-- StackTrace --
java.lang.IndexOutOfBoundsException: Index: 0, Size: 0
at java.util.ArrayList.rangeCheck(ArrayList.java:653)
at java.util.ArrayList.get(ArrayList.java:429)
at org.apache.cassandra.tools.NodeProbe.getEndpoint(NodeProbe.java:812)
at org.apache.cassandra.tools.NodeProbe.getDataCenter(NodeProbe.java:830)
at org.apache.cassandra.tools.NodeTool$Info.execute(NodeTool.java:425)
at org.apache.cassandra.tools.NodeTool$NodeToolCmd.run(NodeTool.java:288)
at org.apache.cassandra.tools.NodeTool.main(NodeTool.java:202)
Becasue getTokens will return an empty list.
This patch changed how broadcast address is deduct. It Adds a reverse
mapping from hostid to ip address and use it with the get local id to
find the ip address in use.
This implementation would probably be replaced by a single API call in
the future.
After the change a call to nodetool info works.
Fixesscylladb/scylla#1027
Signed-off-by: Amnon Heiman <amnon@scylladb.com>
Message-Id: <1458405434-8491-3-git-send-email-amnon@scylladb.com>
The 'nodetool scrub' command ends up calling the variant that is not
wired up to the Scylla API which causes the following error to be
printed out to the user:
[penberg@nero scylla-tools-java]$ ./bin/nodetool scrub
error: For input string: ""
-- StackTrace --
java.lang.NumberFormatException: For input string: ""
at java.lang.NumberFormatException.forInputString(NumberFormatException.java:65)
at java.lang.Integer.parseInt(Integer.java:592)
at java.lang.Integer.parseInt(Integer.java:615)
at com.scylladb.jmx.api.APIClient.getIntValue(APIClient.java:216)
at com.scylladb.jmx.api.APIClient.getIntValue(APIClient.java:220)
at org.apache.cassandra.service.StorageService.scrub(StorageService.java:1291)
Fix the problem by implementing the said scrub() variant.
Message-Id: <1458035736-26349-1-git-send-email-penberg@scylladb.com>
This patch takes the implementation of getTokenToEndpointMap from Origin
which sorts the map result.
Fixesscylladb/scylla#722
Signed-off-by: Amnon Heiman <amnon@scylladb.com>
Message-Id: <1456142885-20838-1-git-send-email-amnon@scylladb.com>
Fix the stubbed implementation of forceRepairRangeAsync() which is
used, for example, when the "--start-token"/"--end-token" options are
passed to "nodetool repair".
forceRepairRangeAsync() works similarly to the existing forceRepairAsync()
just sending the additional start/end tokens as two new options to the
REST API. Unlike the parallel Cassandra code, we don't do any fancy
processing on these tokens to intersect them with the node's token ranges -
we'll do this intersection in the C++ code, where the repair is actually
done.
Signed-off-by: Nadav Har'El <nyh@scylladb.com>
Message-Id: <1455808238-25692-1-git-send-email-nyh@scylladb.com>
StorageService.getTokens should return only the tokens of the current
node, not all the tokens.
Signed-off-by: Amnon Heiman <amnon@scylladb.com>
Message-Id: <1454240935-21903-1-git-send-email-amnon@scylladb.com>
This patch uses the system api to set log level.
After this patch the nodetool setloglevel would support modifying a log
level of a log object.
Signed-off-by: Amnon Heiman <amnon@scylladb.com>
Message-Id: <1453367412-29722-1-git-send-email-amnon@scylladb.com>
The API was modify to return the load map as a map of string to double
instead of formatted string.
This patch change the code to support the udpated API.
Signed-off-by: Amnon Heiman <amnon@scylladb.com>
This patch clean the redundant output the jmx proxy creates.
It set the trace level of the called method to finest and remove some
println leftovers.
Fixes#22
Signed-off-by: Amnon Heiman <amnon@scylladb.com>
The API now uses explicit parameters to pass the parameters to repair.
This patch changes how the parameters are passed to the API to be
compatible with the changed API.
Signed-off-by: Amnon Heiman <amnon@scylladb.com>
Reviewed-by: Nadav Har'El <nyh@scylladb.com>
"nodetool repair" ends up calling one of the dozen forceAsyncRepair()
functions. This function ignored its option rather than passing it on,
so this patch fixes that.
Note that there are still many more forceAsyncRepair() overloads which
similarly ignore their options, and it is possible that certain invocation
of "nodetool repair" will need them, so we will need to fix all of them
in the future.
After this patch, "nodetool repair" no longer works because now Scylla
needs to be fixed to understand the "parallelism" and "incremental" options
passed to it.
Signed-off-by: Nadav Har'El <nyh@scylladb.com>
Scylla's repair REST API (see scylla/api/storage_service.cc) takes all
repair options as one "options" string. The options are separated by ",",
and for each option, the name and value are separated by ":". The existing
code wrongly used "=" instead of ":", so this patch fixes it.
Signed-off-by: Nadav Har'El <nyh@scylladb.com>
The describeRingJMX method, returns a formated output. The output should
be similiar to origin as oppose to the current implementation that
returns a json representation.
After the change an example of nodetool describering:
$ nodetool describering keyspace1
Schema Version:1074c31b-1f39-3df2-90ff-7f0b64bb3ea4
TokenRange:
TokenRange(start_token:7485973865401664349,
end_token:-338297331236877217, endpoints:[127.0.0.1],
rpc_endpoints:[127.0.0.1],
endpoint_details:[EndpointDetails(host:127.0.0.1,
datacenter:datacenter1, rack:rack1)])
TokenRange(start_token:-338297331236877217,
end_token:7485973865401664349, endpoints:[127.0.0.2],
rpc_endpoints:[127.0.0.2],
endpoint_details:[EndpointDetails(host:127.0.0.2,
datacenter:datacenter1, rack:rack1)])
On sycall-jmx:
Fixes#21
Signed-off-by: Amnon Heiman <amnon@scylladb.com>
repairAsync() builds an "options" argument from the options map it gets,
but then forgot to pass this argument to the request :-)
This is part of issue scylladb/#714.
Signed-off-by: Nadav Har'El <nyh@scylladb.com>
The API uses the source_dc as a query parameter, the jmx should use the
same.
In addition, the rebuild method can get null as a datacenter value and
in that case it should not pass a parameter.
Fixesscylladb/scylla#668.
Signed-off-by: Amnon Heiman <amnon@scylladb.com>
Similiar to origin, the load map should return a formated load value.
After this patch the nodetool status command:
$nodetool status
Datacenter: datacenter1
=======================
Status=Up/Down
|/ State=Normal/Leaving/Joining/Moving
-- Address Load Tokens Owns Host ID
Rack
UN 127.0.0.1 394.97 MB 256 ?
292a6c7f-2063-484c-b54d-9015216f1750 rack1
UN 127.0.0.2 151.07 MB 256 ?
102b6ecd-2081-4073-8172-bf818c35e27b rack1
Under scylla-jmx
Fixes#18
Signed-off-by: Amnon Heiman <amnon@scylladb.com>
"This series adds some deprecated methods implemenetation to the CacheService
depnding on its metrics.
It also stub the getDrainProgress in StorageService."
This patch adds the describering method to StorageService, the
implementation is based on the storage_service API that is define in
storage_service.json
The implementation reflect the changes in the API, that returns an
object vs. the jmx_describe ring.
Signed-off-by: Amnon Heiman <amnon@scylladb.com>
Signed-off-by: Pekka Enberg <penberg@scylladb.com>
When calling the API move method, the proxy should pass the new_token
parameter.
Signed-off-by: Amnon Heiman <amnon@scylladb.com>
Signed-off-by: Pekka Enberg <penberg@scylladb.com>
Drain progress is not implemented yet, it is needed by the nodetool
command so it will not fail.
This patches the functionality until the API will be ready, which, in
that time it would be revert.
Signed-off-by: Amnon Heiman <amnon@scylladb.com>
"This series adds the jmx implementation to enable netstats.
After this series netstats should complete successfuly.
A run example:
$ ./bin/nodetool netstats
Mode: NORMAL
repair 397c91a0-8205-11e5-83e4-000000000001
repair 3977d5ba-8205-11e5-83e4-000000000001
repair 3977d624-8205-11e5-83e4-000000000001
repair 397c8fc8-8205-11e5-83e4-000000000001
.......
......
repair 3977d502-8205-11e5-83e4-000000000001
Read Repair Statistics:
Attempted: 1
Mismatch (Blocking): 0
Mismatch (Background): 0
Pool Name Active Pending Completed
Commands n/a 0 21182
Responses n/a 0 597"
This patch adds the registration of StreamManagerMBean to
StorageService, similiar to the way it is done in origin.
After this patch the StreamManager will be available via Jconsole.
Signed-off-by: Amnon Heiman <amnon@scylladb.com>
The API returns errors with an HTTP code 400, 404 or 500 depending on
the cause with a json object that contains the failure reason. The
error message should be populate to the JMX calling client, translated
to the appropriate exception.
This patch adds the ability to detect API failure and throw a runtime
exception with the returned message.
It is up to the calling method what to do with the exception, if it
would do nothing, the calling client would get a RuntimeException,
depends on origin MBean definition, the caller can catch the exception
and throw a specific kind.
Note that any exception that is thrown must be known to the JMX client
or it will not be able to process it.
As a first step, this patch replaces the jersey client to the newer
version under glassfish, which has an easy way of getting a Reply
object, and check its status before returning the results.
The only difference in the method that uses the APIClient is the use of
MultivaluedHashMap.
The following MBean implementation where changed
ColumnFamily
CompactionManager
Gossiper
EndpointSnitchInfo
CacheService
StorageProxy
StorageService
Signed-off-by: Amnon Heiman <amnon@scylladb.com>
There is a confusion in origin, the MBean declare token as the parameter
to remove, but the implementation actually uses host id.
This patch modify scylla implementation to pass a host id as the
parameter to remove.
Signed-off-by: Amnon Heiman <amnon@cloudius-systems.com>
The existing scylla-jmx code had reversed logic for the success of the repair:
it reported to "nodetool repair" a failure when the repair was successful :-)
Note that "nodetool repair" waits until a FINISHED notification, and then reports
a failure if it previously got any SESSION_FAILED notification; So if repair was
successful, all we need to do is to avoid sending a "SESSION_FAILED" message.
But we don't need to send any additional "SESSION_SUCCESS" message to signal
success. That message type is only used to report progress to the user (a
"session" is part of the repair work, so seeing sessions completing shows
progress), but because Scylla doesn't support this progress report yet, we
can't send these notifications yet, and there's no point in sending one such
message at the end - it's only confusing (especially when the text is the same
as that of the FINISHED message).
Signed-off-by: Nadav Har'El <nyh@cloudius-systems.com>
"The API is currently not supporting notification, but the repair
command that perform via the nodetool relays on the notification to know
when the command terminate.
This series adds support for the repair notification, based on a timer
and periodically check if a current repair command was terminated."
This patch adds a notification support for the repair in StorageService.
When a repair command starts a timer is set to check the status of the
repair, when the repair complets it sends notification for the
successful or fail of the repair.
Because jconsole doesn't run method with variable number of parameters
an additional MBean method was added for async repair with only a
keyspace as its parameter.
All the forceAsycRepair methods are mapped to the asyncRepair metod that
replaces them in 2.2, when options will be supported in the repair, it
will be added to the jmx.
After this patch it is possible to register in the jconsole for
notification, perform a forceAsyncRepair and get the notification that
the repair complets
This adds the implementation of the effectiveOwnership and getOwnership
in StorageService. It uses the API that defined in storage_service.json.
After this patch the effectiveOwnership and getOwnership will be
available via jconsole.
Signed-off-by: Amnon Heiman <amnon@cloudius-systems.com>
This patch changes the MBeans to 2.1 version, with the changes an empty
stub implementation was added to the implementation so the code can
compile.
Signed-off-by: Amnon Heiman <amnon@cloudius-systems.com>
This patch cleans the file formatting of ColumnFamilyStore, CommitLog,
StorageProxy and StorageService.
Those files will be modified when moving to 2.1
Signed-off-by: Amnon Heiman <amnon@cloudius-systems.com>
This adds the StorageMetrics, the implementation is based on the API.
The API definition are in storage_service under the metrics part.
After this patch, it will be possible to connect with jconsole and see
the StorageMetrics mbean.
Signed-off-by: Amnon Heiman <amnon@cloudius-systems.com>
