Running 'nodetool status' now reports the following if the JMX proxy is
not able to connect to an API server:
nodetool: Unable to connect to Scylla API server: java.net.ConnectException: Connection refused
See 'nodetool help' or 'nodetool help <command>'.
instead of the scary-looking:
error: javax.ws.rs.ProcessingException (no security manager: RMI class loader disabled)
-- StackTrace --
java.lang.ClassNotFoundException: javax.ws.rs.ProcessingException (no security manager: RMI class loader disabled)
at sun.rmi.server.LoaderHandler.loadClass(LoaderHandler.java:393)
at sun.rmi.server.LoaderHandler.loadClass(LoaderHandler.java:185)
at java.rmi.server.RMIClassLoader$2.loadClass(RMIClassLoader.java:637)
at java.rmi.server.RMIClassLoader.loadClass(RMIClassLoader.java:264)
at sun.rmi.server.MarshalInputStream.resolveClass(MarshalInputStream.java:214)
That happens because the MBean propagates a
'javax.ws.rs.ProcessingException' to nodetool which does not have it in
it's classpath and loading via RMI fails.
Fixes#25.
Message-Id: <1457697628-31792-1-git-send-email-penberg@scylladb.com>
"By default Origin accept local JMX connection. This series import the
code from origin to set the jmx to listen to local traffic only and
change the run script so that the default behaviuor would be local only
traffic."
This patch set the jmx proxy to listen on local traffic by default and
adds a command line switch to allow remote conectivity.
Signed-off-by: Amnon Heiman <amnon@scylladb.com>
This patch init the jmx proxy from the RMIServerSocketFactoryImp init
function. This way the jmx can be set to listen on local port only.
Signed-off-by: Amnon Heiman <amnon@scylladb.com>
The RMIServerSocketFactoryImp is the way origin handle local port
configuration.
When used, the jmx can be set to listen on local traffic only.
Signed-off-by: Amnon Heiman <amnon@scylladb.com>
"This series depends on scylla patch fixing the stream information.
Now that the API report on file information in the stream they need to be
populated to the jmx.
After this patch the nodetool netstats report about file information:
$ nodetool netstats
Mode: NORMAL
Bootstrap ee150e80-dcef-11e5-bee0-000000000000
/127.0.0.2
Sending 1 files, 0 bytes total. Already sent 1 files, 8391192 bytes total
txnofile 8391192/8391192 bytes(100%) sent to idx:0/127.0.0.2
Read Repair Statistics:
Attempted: 6
Mismatch (Blocking): 0
Mismatch (Background): 0
Pool Name Active Pending Completed
Commands n/a 0 16268
Responses n/a 0 2
Fixes scylladb/scylla#948"
This patch adds the streaming session files receiving and sending
information. It is needed for the streaming information.
The constructor now expect the file information, so the
sessionInfoCompositeData was changed to add an empty value for them.
Signed-off-by: Amnon Heiman <amnon@scylladb.com>
This will allow to creat ProgressInfo object from json object and json
Array it needed to report stream file information.
Signed-off-by: Amnon Heiman <amnon@scylladb.com>
This patch takes the implementation of getTokenToEndpointMap from Origin
which sorts the map result.
Fixesscylladb/scylla#722
Signed-off-by: Amnon Heiman <amnon@scylladb.com>
Message-Id: <1456142885-20838-1-git-send-email-amnon@scylladb.com>
Fix the stubbed implementation of forceRepairRangeAsync() which is
used, for example, when the "--start-token"/"--end-token" options are
passed to "nodetool repair".
forceRepairRangeAsync() works similarly to the existing forceRepairAsync()
just sending the additional start/end tokens as two new options to the
REST API. Unlike the parallel Cassandra code, we don't do any fancy
processing on these tokens to intersect them with the node's token ranges -
we'll do this intersection in the C++ code, where the repair is actually
done.
Signed-off-by: Nadav Har'El <nyh@scylladb.com>
Message-Id: <1455808238-25692-1-git-send-email-nyh@scylladb.com>
The dropped messages thread pull information from the API, in various
scenario it can face a connection problem (specifically on startup and
shutdown) or other related exception, when scylla shutds down. It shold
ignore the connection problem, as it is been taken care of by another
thread that check the status and will shutdown when needed.
For other exception, it logs them while continue to connect.
Fixesscylladb/scylla#902
Signed-off-by: Amnon Heiman <amnon@scylladb.com>
Message-Id: <1455799819-17957-1-git-send-email-amnon@scylladb.com>
StorageService.getTokens should return only the tokens of the current
node, not all the tokens.
Signed-off-by: Amnon Heiman <amnon@scylladb.com>
Message-Id: <1454240935-21903-1-git-send-email-amnon@scylladb.com>
After observing logs of scylla-jmx, I started to notice
the following message:
Running '/bin/journalctl --unit scylla-jmx.service'
[stdout] -- Logs begin at Sat 2016-01-23 10:02:51 UTC, end at Sat 2016-01-23 10:07:26 UTC. --
[stdout] Jan 23 10:05:15 ip-172-30-0-9 systemd[1]: Started Scylla JMX.
[stdout] Jan 23 10:05:15 ip-172-30-0-9 systemd[1]: Starting Scylla JMX...
[stdout] Jan 23 10:05:16 ip-172-30-0-9 scylla-jmx[2685]: Using config file: /etc/scylla/scylla.yaml
[stdout] Jan 23 10:05:22 ip-172-30-0-9 scylla-jmx[2685]: Connecting to http://127.0.0.1:10000
[stdout] Jan 23 10:05:22 ip-172-30-0-9 scylla-jmx[2685]: Starting the JMX server
[stdout] Jan 23 10:05:29 ip-172-30-0-9 scylla-jmx[2685]: SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
[stdout] Jan 23 10:05:29 ip-172-30-0-9 scylla-jmx[2685]: SLF4J: Defaulting to no-operation (NOP) logger implementation
[stdout] Jan 23 10:05:29 ip-172-30-0-9 scylla-jmx[2685]: SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.
So we're potentially losing a lot of information on our jmx service logs.
Let's update the log4j dependencies, and add the other ones that are
necessary for the logging to work.
Signed-off-by: Lucas Meneghel Rodrigues <lmr@scylladb.com>
Message-Id: <1453746313-15054-1-git-send-email-lmr@scylladb.com>
This patch uses the system api to set log level.
After this patch the nodetool setloglevel would support modifying a log
level of a log object.
Signed-off-by: Amnon Heiman <amnon@scylladb.com>
Message-Id: <1453367412-29722-1-git-send-email-amnon@scylladb.com>
This removes a debug print that was left in the code by accident.
Signed-off-by: Amnon Heiman <amnon@scylladb.com>
Message-Id: <1452673361-8242-1-git-send-email-amnon@scylladb.com>
The serial garbage collector has the smallest memory footprint and the
smallest impact on the rest of the system, esp. in large multicores.
Message-Id: <1452433737-4413-1-git-send-email-avi@scylladb.com>
Restart=always leads to the following loop:
1. scylla terminates abnormally
2. scylla-jmx sees that, and terminates
3. systemd sees that scylla-jmx terminated, and restarts it.
4. scylla-jmx requires scylla, so systemd starts it.
5. goto 1.
To prevent the loop, set Restart=on-abnormal; systemd will restart scylla-jmx
if some JVM bug got it killed, but not otherwise.
The downside to this patch is that if scylla-server goes down, so does
scylla-jmx, but if scylla-server is then restarted, scylla-jmx stays down.
To get scylla and scylla-jmx to start together, we need to create
scylla.service that requires both of them.
"This series will enable straming support and the nodetool netstats command.
After this series:
$ nodetool netstats
Mode: NORMAL
Bootstrap 331955a0-aeff-11e5-895c-000000000000
/127.0.0.2
Sending 1 files, 140724545317112 bytes total. Already sent 0 files, 0 bytes total
Read Repair Statistics:
Attempted: 6
Mismatch (Blocking): 0
Mismatch (Background): 0
Pool Name Active Pending Completed
Commands n/a 0 121
Responses n/a 0 64
Fixes scylladb/scylla #731"
This patch import and modify the StreamingMetrics from orgin. It will
pull periodically the API to check for the current stream and when it
will find any, it will register their MBean.
After this patch during streaming (ie. node is adding to the cluster) it
will be possible to check with jconsole and see the stream.
A nodetool netstats example:
$ nodetool netstats
Mode: NORMAL
Bootstrap 331955a0-aeff-11e5-895c-000000000000
/127.0.0.2
Sending 1 files, 140724545317112 bytes total. Already sent 0
files, 0 bytes total
Read Repair Statistics:
Attempted: 6
Mismatch (Blocking): 0
Mismatch (Background): 0
Pool Name Active Pending Completed
Commands n/a 0 85
Responses n/a 0 46
Signed-off-by: Amnon Heiman <amnon@scylladb.com>
The API of the session info returns parameters in snake case instead of
camel case.
This patch chagne the expected field to match the API. It was also
modified to accept empty fields and store them as null.
Signed-off-by: Amnon Heiman <amnon@scylladb.com>
"The storage_service api was changed to return a map of string, double instead
of formatted numbers. This change update the JMX proxy to support this API.
While going over the code a potential bug was found and was fix.
The series adds method to the APIClient to return a map of string, double and
uses that function to call the API."
The API was modify to return the load map as a map of string to double
instead of formatted string.
This patch change the code to support the udpated API.
Signed-off-by: Amnon Heiman <amnon@scylladb.com>
This patch adds a method to the APIClient that return a map of String
and Double.
It support both simple and with query parameters.
Signed-off-by: Amnon Heiman <amnon@scylladb.com>
This patch clean the redundant output the jmx proxy creates.
It set the trace level of the called method to finest and remove some
println leftovers.
Fixes#22
Signed-off-by: Amnon Heiman <amnon@scylladb.com>
The API now uses explicit parameters to pass the parameters to repair.
This patch changes how the parameters are passed to the API to be
compatible with the changed API.
Signed-off-by: Amnon Heiman <amnon@scylladb.com>
Reviewed-by: Nadav Har'El <nyh@scylladb.com>
"nodetool repair" ends up calling one of the dozen forceAsyncRepair()
functions. This function ignored its option rather than passing it on,
so this patch fixes that.
Note that there are still many more forceAsyncRepair() overloads which
similarly ignore their options, and it is possible that certain invocation
of "nodetool repair" will need them, so we will need to fix all of them
in the future.
After this patch, "nodetool repair" no longer works because now Scylla
needs to be fixed to understand the "parallelism" and "incremental" options
passed to it.
Signed-off-by: Nadav Har'El <nyh@scylladb.com>
Scylla's repair REST API (see scylla/api/storage_service.cc) takes all
repair options as one "options" string. The options are separated by ",",
and for each option, the name and value are separated by ":". The existing
code wrongly used "=" instead of ":", so this patch fixes it.
Signed-off-by: Nadav Har'El <nyh@scylladb.com>
The describeRingJMX method, returns a formated output. The output should
be similiar to origin as oppose to the current implementation that
returns a json representation.
After the change an example of nodetool describering:
$ nodetool describering keyspace1
Schema Version:1074c31b-1f39-3df2-90ff-7f0b64bb3ea4
TokenRange:
TokenRange(start_token:7485973865401664349,
end_token:-338297331236877217, endpoints:[127.0.0.1],
rpc_endpoints:[127.0.0.1],
endpoint_details:[EndpointDetails(host:127.0.0.1,
datacenter:datacenter1, rack:rack1)])
TokenRange(start_token:-338297331236877217,
end_token:7485973865401664349, endpoints:[127.0.0.2],
rpc_endpoints:[127.0.0.2],
endpoint_details:[EndpointDetails(host:127.0.0.2,
datacenter:datacenter1, rack:rack1)])
On sycall-jmx:
Fixes#21
Signed-off-by: Amnon Heiman <amnon@scylladb.com>
repairAsync() builds an "options" argument from the options map it gets,
but then forgot to pass this argument to the request :-)
This is part of issue scylladb/#714.
Signed-off-by: Nadav Har'El <nyh@scylladb.com>
The API uses the source_dc as a query parameter, the jmx should use the
same.
In addition, the rebuild method can get null as a datacenter value and
in that case it should not pass a parameter.
Fixesscylladb/scylla#668.
Signed-off-by: Amnon Heiman <amnon@scylladb.com>
"After this series the nodetool compacthistory displays:
$ nodetool compactionhistory
Compaction History:
id keyspace_name columnfamily_name compacted_at bytes_in bytes_out rows_merged
09d71860-a3f3-11e5-b1cf-000000000000 system peers 1450269994214 365 365
09d73f70-a3f3-11e5-88b4-000000000001 system local 1450269994215 816 690"
This changes the CompactionManager getCompactionHistory to use the new
get_compaction_history API.
It uses the CompactionHistoryTabularData to parse and report the
results.
After this patch nodetool compactionhistory would work.
Signed-off-by: Amnon Heiman <amnon@scylladb.com>
This patch import and modify CompactionHistoryTabularData from origin.
It will be used by the getCompactionHistory method in CompactionManager.
Signed-off-by: Amnon Heiman <amnon@scylladb.com>
Since scylla-jmx supports to read configuration from scylla.yaml, we
don't need to pass these parameters from program arguments.
Fixes#17.
Signed-off-by: Takuya ASADA <syuu@scylladb.com>
There was a confusion in the API between key and keyspace.
It was changed in the API so the JMX should be modified accordingly.
After this change
nodetool listsnapshots
Will show the current snapshots.
On scylla-jmx:
Fixes#15
Signed-off-by: Amnon Heiman <amnon@scylladb.com>
Similiar to origin, the load map should return a formated load value.
After this patch the nodetool status command:
$nodetool status
Datacenter: datacenter1
=======================
Status=Up/Down
|/ State=Normal/Leaving/Joining/Moving
-- Address Load Tokens Owns Host ID
Rack
UN 127.0.0.1 394.97 MB 256 ?
292a6c7f-2063-484c-b54d-9015216f1750 rack1
UN 127.0.0.2 151.07 MB 256 ?
102b6ecd-2081-4073-8172-bf818c35e27b rack1
Under scylla-jmx
Fixes#18
Signed-off-by: Amnon Heiman <amnon@scylladb.com>
