Fixes#57
The usage of TableMetricsObjectName-yada-yada relies on translating the
"fake" objectname to a canonical one on remote
publication/serialization. However, the implementation of
ObjectName.getInstance has changed in JDK (micro) updates so it no
longer applies overridable methods -> wrong name published.
Fix by doing explicit ObjectName instansiation.
Message-Id: <20181023132005.23099-1-calle@scylladb.com>
Before we were discarding the initial repository while
overriding it with TableRepository. This was a mistake that
caused dtests to fail. Proper solution is to keep the initial
repository inside TableRepository. That way whatever was registered
at the time of JmxMBeanServer creation is still handled properly.
Signed-off-by: Piotr Jastrzebski <piotr@scylladb.com>
Message-Id: <22181859012fd20ddf37e049a145bc94a3a91a33.1527844328.git.piotr@scylladb.com>
Default implementation stores MBeans in the following map:
<domain name> -> (<properties as a single string> -> NamedObject)
This is problematic because NamedObject contains ObjectName that
has both domain and properties inside itself.
This means we're storing the same data twice.
For domain "" we want to store MBeans in a more compact way using map:
ObjectName -> DynamicMBean
which is equivalent to NamedObject.
Signed-off-by: Piotr Jastrzebski <piotr@scylladb.com>
Before this change it was taking JMX Server 270 seconds to start
when Scylla had 2000 tables. After the change it takes only 2 seconds.
Signed-off-by: Piotr Jastrzebski <piotr@scylladb.com>
Next patch will introduce new ObjectName implementation that
will use less memory. This new object won't be serializable.
This means it won't be possible to transport it to a remote
caller. We want to keep this new object local to JMX server as well.
This patch makes sure that every ObjectName returned
from APIBeanServer is transformed into a regular ObjectName.
It also makes sure that every ObjectInstance returned from
APIBeanServer has its ObjectName swapped with a regular ObjectName.
Signed-off-by: Piotr Jastrzebski <piotr@scylladb.com>
JmxMBeanServer is a concrete implementation of a MBeanServer.
We want to use it directly because we need to bypass calls to
JmxMBeanServer.registerMBean and JmxMBeanServer.unregisterMBean.
They take ObjectName as parameter, copy it using
ObjectName.getInstance(ObjectName) and pass it to registerMBean
and unregisterMBean of JmxMBeanServer.getMBeanServerInterceptor().
We want to avoid this copy and pass the ObjectName argument directly
to JmxMBeanServer.getMBeanServerInterceptor().
To do that this patch:
1. changes all MBeanServer variables to JmxMBeanServer
2. creates JmxMBeanServer in APIBuilder making sure accessing
interceptors is allowed
3. makes sure that JmxMBeanServer.getMBeanServerInterceptor().registerMBean
is called wherever JmxMBeanServer.registerMBean was called
4. makes sure that JmxMBeanServer.getMBeanServerInterceptor().unregisterMBean
is called whenever JmxMBeanServer.unregisterMBean was called
Next patch will use different ObjectName implementation that will
use less memory and this patch is crucial because without it every ObjectName
is transformed with ObjectName.getInstance which turns the object into
a regular ObjectName.
Signed-off-by: Piotr Jastrzebski <piotr@scylladb.com>
When parsing the snapshot disk sizes, it should be long and not int.
See scylladb/scylla#2104
Signed-off-by: Amnon Heiman <amnon@scylladb.com>
Message-Id: <1487760019-1354-1-git-send-email-amnon@scylladb.com>
Actually removes a bunch of code to manage the JMX connector,
since as of jdk8u102, the standard jmx connector answers to
property setting bind address -> can restrict access.
Note that the RMI connector will now (as is jdk normal)
_bind_ to 0.0.0.0, but it will not answer non-local requests
if "remote" is not enabled. This is the default jdk behaviour.
In any case, we rely on setting the appropriate properties
instead, and also allow pass-through of -D flags to java,
which in turn means those who wish can turn on both auth
and ssl, set key/trust stores etc etc.
Message-Id: <1485357178-20714-1-git-send-email-calle@scylladb.com>
The getMapStrValue return a map from the API. This change the
implementation to use linkedHashMap so the map will be sorted according
to the API order.
Signed-off-by: Amnon Heiman <amnon@scylladb.com>
Currently, we have a scary looking dtest failure when attempting to force flush a
Nodetool command '/data/jenkins/workspace/scylla-1.3-dtest/label/monster/mode/release/smp/1/scylla/resources/cassandra/bin/nodetool -h localhost -p 7100 flush' failed; exit status: 2; stderr: Picked up JAVA_TOOL_OPTIONS: -Dfile.encoding=UTF8
error: javax.ws.rs.ProcessingException (no security manager: RMI class loader disabled)
-- StackTrace --
java.lang.ClassNotFoundException: javax.ws.rs.ProcessingException (no security manager: RMI class loader disabled)
at sun.rmi.server.LoaderHandler.loadClass(LoaderHandler.java:396)
at sun.rmi.server.LoaderHandler.loadClass(LoaderHandler.java:186)
at java.rmi.server.RMIClassLoader$2.loadClass(RMIClassLoader.java:637)
at java.rmi.server.RMIClassLoader.loadClass(RMIClassLoader.java:264)
at sun.rmi.server.MarshalInputStream.resolveClass(MarshalInputStream.java:219)
at java.io.ObjectInputStream.readNonProxyDesc(ObjectInputStream.java:1620)
at java.io.ObjectInputStream.readClassDesc(ObjectInputStream.java:1521)
at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1781)
at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1353)
at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2018)
at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1942)
at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1808)
at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1353)
at java.io.ObjectInputStream.readObject(ObjectInputStream.java:373)
at sun.rmi.transport.StreamRemoteCall.executeCall(StreamRemoteCall.java:245)
at sun.rmi.server.UnicastRef.invoke(UnicastRef.java:162)
at com.sun.jmx.remote.internal.PRef.invoke(Unknown Source)
at javax.management.remote.rmi.RMIConnectionImpl_Stub.invoke(Unknown Source)
at javax.management.remote.rmi.RMIConnector$RemoteMBeanServerConnection.invoke(RMIConnector.java:1020)
at javax.management.MBeanServerInvocationHandler.invoke(MBeanServerInvocationHandler.java:298)
at com.sun.proxy.$Proxy7.forceKeyspaceFlush(Unknown Source)
at org.apache.cassandra.tools.NodeProbe.forceKeyspaceFlush(NodeProbe.java:290)
at org.apache.cassandra.tools.NodeTool$Flush.execute(NodeTool.java:1227)
at org.apache.cassandra.tools.NodeTool$NodeToolCmd.run(NodeTool.java:288)
at org.apache.cassandra.tools.NodeTool.main(NodeTool.java:202)
The problem is rather innocent: the API call fails and we leak
javax.ws.rs.ProcessingException, which is not available in nodetool's
classpath. In fact, we already fixed the problem for GETs in commit
02e0598 ("APIClient: Fix error handling if connection to API server
fails") so do the same thing for POSTs.
Message-Id: <1471589525-26435-1-git-send-email-penberg@scylladb.com>
When removing the pull based timers in the API the sum method in the
APITimer was left stubed by mistake.
This patch take the sum from the histogram as it should be.
Another missed changes are the units, in the yammer library the Timer
does unit conversion before returning the values.
This patch takes the unit conversion from the yammer library to be
compatible.
Signed-off-by: Amnon Heiman <amnon@scylladb.com>
Message-Id: <1464173726-7482-1-git-send-email-amnon@scylladb.com>
Some of the specific functionality is needed from the APIMeter and
APITimer.
The MetricsRegistry now return the specific objects so they can be used
in their adapted form.
Signed-off-by: Amnon Heiman <amnon@scylladb.com>
The yammer Timer object calculate rate based on a timer, which causes
periodic calls to the API.
This replaces the implementation so that a timer would get all its
values from the API.
For object registration the APITimer still inherit from Timer but
override all its functionality.
Signed-off-by: Amnon Heiman <amnon@scylladb.com>
With the move to APITimer, in many occasion a histogram will not update
itself, instead it will be updated by the APITimer.
This breaks the update values functionality so that a histogram that is
included in an APITimer will not try to update it self.
Signed-off-by: Amnon Heiman <amnon@scylladb.com>
This replaces the APIMeter implementation so it will not pull the API
regularly.
To by complient with the yammer object registration mechanism, it still
inherit from Meter, but all the functionalities are overriden.
Now all the data is taken from the API including the rate statistics.
Signed-off-by: Amnon Heiman <amnon@scylladb.com>
APIMeter will be modified not to use pulling and to retrieve the derived
information from the API.
To support that the APIClient was changed so it would be able to cache
json objects and histogram.
Signed-off-by: Amnon Heiman <amnon@scylladb.com>
The mx4j implementation of queryName does not handle correctly pattern
matching.
This patch identify that a name contains a patern and do the patern
matching as it should have been done by the mx4j MBeanServer.
Fixes#28
Message-Id: <1462447227-8367-1-git-send-email-amnon@scylladb.com>
With the addition of the APIMBeanServer there is no longer a need for
the pulling functionality to be perform for MBean registration.
Signed-off-by: Amnon Heiman <amnon@scylladb.com>
This register the APIBuilder as the MBeanServerBuilder which will cause
the APIMBeanServer to be used as the MBeanServer.
Signed-off-by: Amnon Heiman <amnon@scylladb.com>
The APIBuilder is an implementation for the MBeanServerBuilder that is
used to instantiate the APIMBeanServer as the platform MBeanServer.
Signed-off-by: Amnon Heiman <amnon@scylladb.com>
The APIMBeanServer is serve as a proxy for the MBeanServer.
It intercept calls to the MBeanServer and check for the column family
and stream registeration before they are perform.
Current implementation override queryNames as it's the one that is being
used by nodetool.
Additional methods can be override in the future if needed.
Signed-off-by: Amnon Heiman <amnon@scylladb.com>
The MX4J introspector does not support *MXBean interfaces name, This
causes a problem with the garbage collector and java related MBeans.
To bypass that limitation the APIMBeanIntrospector inherit from
MBeanIntrospector and override the relevant functionality so MXBean
will be treated like MBean.
Signed-off-by: Amnon Heiman <amnon@scylladb.com>
Sometimes it is usefull to get a reverse of the map return by the API.
For example instread of ip address to hostid to get the hostid to ip
address.
Though it is possible to reverse a map, there is no need, it's easier to
generate the reverse mapping.
Signed-off-by: Amnon Heiman <amnon@scylladb.com>
Message-Id: <1458405434-8491-2-git-send-email-amnon@scylladb.com>
Make the error messages returned by Scylla API server human readable
from 'nodetool'.
For example, if an API URL is missing, print out the following error:
[penberg@nero scylla-tools-java]$ ./bin/nodetool getcompactionthreshold ks test4
nodetool: Scylla API server HTTP GET to URL 'column_family/minimum_compaction/ks:test4' failed: Not found
See 'nodetool help' or 'nodetool help <command>'.
instead of the scary-looking error that we now print:
[penberg@nero scylla-tools-java]$ ./bin/nodetool getcompactionthreshold ks test4
error: Not found
-- StackTrace --
java.lang.RuntimeException: Not found
at com.scylladb.jmx.api.APIClient.getException(APIClient.java:116)
at com.scylladb.jmx.api.APIClient.getRawValue(APIClient.java:160)
at com.scylladb.jmx.api.APIClient.getRawValue(APIClient.java:174)
at com.scylladb.jmx.api.APIClient.getIntValue(APIClient.java:216)
at com.scylladb.jmx.api.APIClient.getIntValue(APIClient.java:220)
at org.apache.cassandra.db.ColumnFamilyStore.getMinimumCompactionThreshold(ColumnFamilyStore.java:475)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at sun.reflect.misc.Trampoline.invoke(MethodUtil.java:71)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
[snip]
Message-Id: <1458032300-17704-1-git-send-email-penberg@scylladb.com>
Running 'nodetool status' now reports the following if the JMX proxy is
not able to connect to an API server:
nodetool: Unable to connect to Scylla API server: java.net.ConnectException: Connection refused
See 'nodetool help' or 'nodetool help <command>'.
instead of the scary-looking:
error: javax.ws.rs.ProcessingException (no security manager: RMI class loader disabled)
-- StackTrace --
java.lang.ClassNotFoundException: javax.ws.rs.ProcessingException (no security manager: RMI class loader disabled)
at sun.rmi.server.LoaderHandler.loadClass(LoaderHandler.java:393)
at sun.rmi.server.LoaderHandler.loadClass(LoaderHandler.java:185)
at java.rmi.server.RMIClassLoader$2.loadClass(RMIClassLoader.java:637)
at java.rmi.server.RMIClassLoader.loadClass(RMIClassLoader.java:264)
at sun.rmi.server.MarshalInputStream.resolveClass(MarshalInputStream.java:214)
That happens because the MBean propagates a
'javax.ws.rs.ProcessingException' to nodetool which does not have it in
it's classpath and loading via RMI fails.
Fixes#25.
Message-Id: <1457697628-31792-1-git-send-email-penberg@scylladb.com>
This patch init the jmx proxy from the RMIServerSocketFactoryImp init
function. This way the jmx can be set to listen on local port only.
Signed-off-by: Amnon Heiman <amnon@scylladb.com>
The RMIServerSocketFactoryImp is the way origin handle local port
configuration.
When used, the jmx can be set to listen on local traffic only.
Signed-off-by: Amnon Heiman <amnon@scylladb.com>
This patch adds a method to the APIClient that return a map of String
and Double.
It support both simple and with query parameters.
Signed-off-by: Amnon Heiman <amnon@scylladb.com>
There was a confusion in the API between key and keyspace.
It was changed in the API so the JMX should be modified accordingly.
After this change
nodetool listsnapshots
Will show the current snapshots.
On scylla-jmx:
Fixes#15
Signed-off-by: Amnon Heiman <amnon@scylladb.com>
The RecentEstimatedHistogram updates its value from the API with an
array of recent values.
This array can be empty, in that case the getBuckets method should just
return a zero size array.
Signed-off-by: Amnon Heiman <amnon@scylladb.com>
When creating an estimated histogram from buckets it is a valid option
to get a zero size array as the buckets array.
In that case the newOffsets method would get a negative value for its
size, which should result in a zero length array of offsets.
Signed-off-by: Amnon Heiman <amnon@scylladb.com>
An empty histogram can return a valid response from the API but without
any buckets.
This is a valid scenario and common for counters of features that are
not supported yet.
In those cases, the APIClient should return a zero length array.
Signed-off-by: Amnon Heiman <amnon@scylladb.com>
Sometimes it is required that a meter will not handle its own data, like
the APIMeter does.
This patch break the added functionality of APIMeter into two classes,
APISettableMeter is a Meter with a set value method and APIMeter adds
the functionality that reads from the API.
Signed-off-by: Amnon Heiman <amnon@scylladb.com>
