Commit Graph

428 Commits

Author SHA1 Message Date
elcallio
a44c18c621 Add metric/mbean base types + metrics JMX object factory 2016-10-24 11:43:51 +00:00
Calle Wilund
3e146845b4 StorageService: update to c3 compat
Note: some calls that are not (yet) applicable to scylla are 
unimplemented.
2016-10-24 11:43:51 +00:00
Calle Wilund
b4e483b179 StorageProxy: update to c3 compat 2016-10-24 11:43:51 +00:00
Calle Wilund
de28e68532 GCInspector: Add SuppressWarnings("restriction") 2016-10-24 11:43:51 +00:00
Calle Wilund
3a4adcb676 CacheService: update to c3 compat 2016-10-24 11:43:51 +00:00
Calle Wilund
85e1b07544 MessagingService: update to c3 compat
Note: c3 adds configurable size threshold counting of messages sent, 
dividing info "large"/"small" partitions (+gossiper). Message bulk 
queries in v3 mbean reflects this. 

Scylla does not (yet?) have such a threshold divider, so this is 
highly incomplete and just delegates to old apis that "sort-of" fit.
2016-10-24 11:43:51 +00:00
Calle Wilund
f4759f05e7 EndpointSnitchInfo: update to c3 compat 2016-10-24 11:43:51 +00:00
Calle Wilund
68ce437b03 Gossiper: update to c3 compat 2016-10-24 11:43:51 +00:00
Calle Wilund
b7a6554ee9 FailureDetector: update to c3 compat 2016-10-24 11:43:51 +00:00
Calle Wilund
3efcd5103b CompactionManager: update to c3 compat 2016-10-24 11:43:51 +00:00
Calle Wilund
39e4cd8f3f CommitLog: update to c3 compat 2016-10-24 11:43:51 +00:00
Calle Wilund
85b39d7fbe ColumnFamilyStore: update to c3 compat
Note: some calls still unimplemented
2016-10-24 11:43:51 +00:00
Calle Wilund
9a44228c71 APIClient: Add some "post" overloads 2016-10-24 11:43:51 +00:00
Pekka Enberg
45e2c982f7 Merge "Connect with remote cluster" from Yan
"Tools (like jconsole) cannot connect with remote cluster without this fix."
2016-10-18 12:05:44 +03:00
yan cui
82ae18605a connect with remote cluster 2016-10-17 18:38:10 -07:00
Pekka Enberg
c07f5c034f APIClient: Fix error handling for POST if API call fails
Currently, we have a scary looking dtest failure when attempting to force flush a

  Nodetool command '/data/jenkins/workspace/scylla-1.3-dtest/label/monster/mode/release/smp/1/scylla/resources/cassandra/bin/nodetool -h localhost -p 7100 flush' failed; exit status: 2; stderr: Picked up JAVA_TOOL_OPTIONS: -Dfile.encoding=UTF8
  error: javax.ws.rs.ProcessingException (no security manager: RMI class loader disabled)
  -- StackTrace --
  java.lang.ClassNotFoundException: javax.ws.rs.ProcessingException (no security manager: RMI class loader disabled)
          at sun.rmi.server.LoaderHandler.loadClass(LoaderHandler.java:396)
          at sun.rmi.server.LoaderHandler.loadClass(LoaderHandler.java:186)
          at java.rmi.server.RMIClassLoader$2.loadClass(RMIClassLoader.java:637)
          at java.rmi.server.RMIClassLoader.loadClass(RMIClassLoader.java:264)
          at sun.rmi.server.MarshalInputStream.resolveClass(MarshalInputStream.java:219)
          at java.io.ObjectInputStream.readNonProxyDesc(ObjectInputStream.java:1620)
          at java.io.ObjectInputStream.readClassDesc(ObjectInputStream.java:1521)
          at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1781)
          at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1353)
          at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2018)
          at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1942)
          at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1808)
          at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1353)
          at java.io.ObjectInputStream.readObject(ObjectInputStream.java:373)
          at sun.rmi.transport.StreamRemoteCall.executeCall(StreamRemoteCall.java:245)
          at sun.rmi.server.UnicastRef.invoke(UnicastRef.java:162)
          at com.sun.jmx.remote.internal.PRef.invoke(Unknown Source)
          at javax.management.remote.rmi.RMIConnectionImpl_Stub.invoke(Unknown Source)
          at javax.management.remote.rmi.RMIConnector$RemoteMBeanServerConnection.invoke(RMIConnector.java:1020)
          at javax.management.MBeanServerInvocationHandler.invoke(MBeanServerInvocationHandler.java:298)
          at com.sun.proxy.$Proxy7.forceKeyspaceFlush(Unknown Source)
          at org.apache.cassandra.tools.NodeProbe.forceKeyspaceFlush(NodeProbe.java:290)
          at org.apache.cassandra.tools.NodeTool$Flush.execute(NodeTool.java:1227)
          at org.apache.cassandra.tools.NodeTool$NodeToolCmd.run(NodeTool.java:288)
          at org.apache.cassandra.tools.NodeTool.main(NodeTool.java:202)

The problem is rather innocent: the API call fails and we leak
javax.ws.rs.ProcessingException, which is not available in nodetool's
classpath. In fact, we already fixed the problem for GETs in commit
02e0598 ("APIClient: Fix error handling if connection to API server
fails") so do the same thing for POSTs.
Message-Id: <1471589525-26435-1-git-send-email-penberg@scylladb.com>
2016-08-19 14:44:22 +03:00
Tomasz Grabiec
27e3d1745a scylla-jmx: Exit on unknown parameter rather than infinite-loop
Ran into this while trying to use ccm with not up-to-date scylla-jmx.

Symptoms:

  $ ccm start
  Error starting node node1

and empty ~/.ccm/scylla-3/node1/logs/system.log.jmx
Message-Id: <1468399926-3565-1-git-send-email-tgrabiec@scylladb.com>
2016-07-13 12:05:42 +03:00
Amnon Heiman
4672cd360f scylla-jmx.service.in: Depend on scylla-server
The correct dependency between the jmx and the scylla-server is:
The scylla-jmx should not run if the scylla-server is not running, it
should shutdown when the scylla-server shuts down.

Starting the scylla-jmx should not start the scylla-server, instead, if
the scylla-server is not running it should fail to start.

This patch changes the setup to do so.

Signed-off-by: Amnon Heiman <amnon@scylladb.com>
Message-Id: <1467184319-3395-1-git-send-email-amnon@scylladb.com>
2016-06-29 11:36:43 +03:00
Pekka Enberg
36ae2fcfd7 dist: Execute Maven in batch mode
The batch mode produces much more readable logs because it's designed
for non-interactive builds and doesn't have the fancy download progress
meters.
Message-Id: <1464770158-32482-1-git-send-email-penberg@scylladb.com>
2016-06-06 16:19:22 +03:00
Pekka Enberg
c6edab5990 dist/redhat: Fix RPM package build
Commit 12daaf5 ("dist/redhat: fix rpm build error") did not fix the
error, at least not on our Jenkins build machines.

Looking at the RPM build logs, we create the build directory:

+ cd /builddir/build/BUILD
+ mkdir build

but then change directory to "scylla-jmx-1.2.rc1":

+ cd /builddir/build/BUILD
+ cd scylla-jmx-1.2.rc1
+ mvn install

and therefore fail the copy:

+ cp dist/common/systemd/scylla-jmx.service.in build/scylla-jmx.service
cp: cannot create regular file 'build/scylla-jmx.service': No such
file or directory

I don't know why Takuya put the "mkdir" in the "prep" section but
something like this should unblock the build.
2016-06-01 10:59:52 +03:00
Amnon Heiman
9e97cb530a APITimer: sum should return a value and values are in ns
When removing the pull based timers in the API the sum method in the
APITimer was left stubed by mistake.

This patch take the sum from the histogram as it should be.

Another missed changes are the units, in the yammer library the Timer
does unit conversion before returning the values.

This patch takes the unit conversion from the yammer library to be
compatible.

Signed-off-by: Amnon Heiman <amnon@scylladb.com>
Message-Id: <1464173726-7482-1-git-send-email-amnon@scylladb.com>
2016-05-25 14:16:40 +03:00
Takuya ASADA
12daaf546d dist/redhat: fix rpm build error
Since build/ is not exist, 'cp dist/common/systemd/scylla-jmx.service.in build/scylla-jmx.service' will fail.
So create build/ before starting build stage.

Signed-off-by: Takuya ASADA <syuu@scylladb.com>
Message-Id: <1464164524-20867-1-git-send-email-syuu@scylladb.com>
2016-05-25 13:24:04 +03:00
Pekka Enberg
4eb02743cd Merge "Removing counter pulling from the JMX" from Amnon
"This series uses the API that was added to scylla to remove the counter
 pulling and rely on the statistics collected by the API.

 The series extends the APIMeter, APIHistogram and APITimer to remove
 their pulling part and to fetch the information when needed from the
 API.

 For performence reason those objects will be cached, so that in the
 typical case of of multiple requests of different fields will cause a
 single API call."
2016-05-24 17:20:52 +03:00
Takuya ASADA
225d8ace17 dist/ubuntu: chmod a+rx on build_deb.sh
Add permission to execute build_deb.sh

Signed-off-by: Takuya ASADA <syuu@scylladb.com>
Message-Id: <1464083473-1701-2-git-send-email-syuu@scylladb.com>
2016-05-24 12:53:15 +03:00
Takuya ASADA
e3c5acfcad dist: Support systemd for Ubuntu 15.10/16.04
Since Ubuntu 15.10/16.04 has moved to systemd, share CentOS/Fedora's systemd unit file with Ubuntu.

Fixes scylladb/scylla#1283

Signed-off-by: Takuya ASADA <syuu@scylladb.com>
Message-Id: <1464083473-1701-1-git-send-email-syuu@scylladb.com>
2016-05-24 12:53:15 +03:00
Avi Kivity
f6710465ef dist: change scylla-jmx process name from 'java' to 'scylla-jmx'
Helps in top, pgrep and friends.  Unfortunately the only reasonable way
to do it is to create a symlink to /usr/bin/java and run that.
Message-Id: <1463580254-8369-1-git-send-email-avi@scylladb.com>
2016-05-20 13:38:28 +03:00
Amnon Heiman
1e4edeb858 MessagingService: Move to APITimer and drop the pulling
With the change to APITimer there is no longer a need to periodically
pull the API.

The verb will be register on the object initialization and will be
updated whenever they are been used.

Signed-off-by: Amnon Heiman <amnon@scylladb.com>
2016-05-17 13:04:55 +03:00
Amnon Heiman
7756f4751a DroppedMessageMetrics: Change APISettableMeter to APIMeter
The APIMeter replaces the APISettableMeter as the Meter implementation.

Signed-off-by: Amnon Heiman <amnon@scylladb.com>
2016-05-17 12:27:36 +03:00
Amnon Heiman
74d062851c CompactionMetrics: Use APITimer
This change the Timer from Timer to APITimer.

Signed-off-by: Amnon Heiman <amnon@scylladb.com>
2016-05-17 12:25:51 +03:00
Amnon Heiman
78bb2ef25a CacheService: Do not get non existing caches metrics
With the change in the meter implementation, retrieving a non existing
metrics would take time.

For this, the CacheService would mark caches that are not supported with
null url, so the metrics will be register but will return 0 for all
request (instead of going to the API that will return 0).

Signed-off-by: Amnon Heiman <amnon@scylladb.com>
2016-05-17 12:20:35 +03:00
Amnon Heiman
8a73d6a840 CacheMetrics: Switch to APITimer
CacheMetrics is a general counter that is used for all possible caches.
For caches that we do not support, there is no need to go and fetch
their values.

When moving to the APITimer each such request will take longer (the
value will not be available as it use it) now it will be possible to
supply a null as a url which would cause to the metrics to return 0 for
all counters.

Signed-off-by: Amnon Heiman <amnon@scylladb.com>
2016-05-17 11:32:59 +03:00
Amnon Heiman
ad49d05780 ClientRequestMetrics: Using the APITimer
The APITimer uses a different endpoint not to break existing API.

Signed-off-by: Amnon Heiman <amnon@scylladb.com>
2016-05-17 11:30:55 +03:00
Amnon Heiman
2c07ca2e09 LatencyMetrics: Move to APITimer
The APITimer uses a different endpoint not to break existing API.

The addNano functionality was removed as all of the values are updated
from the APi.

Signed-off-by: Amnon Heiman <amnon@scylladb.com>
2016-05-17 11:26:18 +03:00
Amnon Heiman
801af00ddb APIMetrics, APIMetricsRegistry: Return APIMeter and APITimer
Some of the specific functionality is needed from the APIMeter and
APITimer.

The MetricsRegistry now return the specific objects so they can be used
in their adapted form.

Signed-off-by: Amnon Heiman <amnon@scylladb.com>
2016-05-17 11:22:43 +03:00
Amnon Heiman
d3b8ef1ae5 APITimer: Non pull based Timer
The yammer Timer object calculate rate based on a timer, which causes
periodic calls to the API.

This replaces the implementation so that a timer would get all its
values from the API.

For object registration the APITimer still inherit from Timer but
override all its functionality.

Signed-off-by: Amnon Heiman <amnon@scylladb.com>
2016-05-17 11:12:01 +03:00
Amnon Heiman
eca6451832 APIHistogram: Support APITimer
With the move to APITimer, in many occasion a histogram will not update
itself, instead it will be updated by the APITimer.

This breaks the update values functionality so that a histogram that is
included in an APITimer will not try to update it self.

Signed-off-by: Amnon Heiman <amnon@scylladb.com>
2016-05-17 11:08:05 +03:00
Amnon Heiman
4d1f8ed7c9 APIMeter: Move out of pull mode
This replaces the APIMeter implementation so it will not pull the API
regularly.

To by complient with the yammer object registration mechanism, it still
inherit from Meter, but all the functionalities are overriden.

Now all the data is taken from the API including the rate statistics.

Signed-off-by: Amnon Heiman <amnon@scylladb.com>
2016-05-17 10:03:12 +03:00
Amnon Heiman
adf466519f APIClient: Support for non pull APIMeter
APIMeter will be modified not to use pulling and to retrieve the derived
information from the API.

To support that the APIClient was changed so it would be able to cache
json objects and histogram.

Signed-off-by: Amnon Heiman <amnon@scylladb.com>
2016-05-17 09:35:42 +03:00
Amnon Heiman
a028e59699 CacheEntry: Support caching of jsonobject
This will allow to store json objects in the cache.

Signed-off-by: Amnon Heiman <amnon@scylladb.com>
2016-05-17 09:32:50 +03:00
Amnon Heiman
2ff16fa2a5 scylla-jmx: set the APIBuilder in the command line
When setting the jmx through the command line, the jmx server creates
even before the main is called.
For the APIServer to take effect the builder should be set via system
properties.

This patch also add an option to run the java process with debug ports
open so an extern debug will be able to connect to the app.

Signed-off-by: Amnon Heiman <amnon@scylladb.com>
Message-Id: <1462447227-8367-2-git-send-email-amnon@scylladb.com>
2016-05-05 16:40:23 +03:00
Amnon Heiman
853963f833 APIMBeanServer: overload the queryName implementation
The mx4j implementation of queryName does not handle correctly pattern
matching.

This patch identify that a name contains a patern and do the patern
matching as it should have been done by the mx4j MBeanServer.

Fixes #28

Message-Id: <1462447227-8367-1-git-send-email-amnon@scylladb.com>
2016-05-05 16:40:20 +03:00
Avi Kivity
f8112b5d57 Merge "Remove the pulling mode for MBean registration" from Amnon
"This series replaces that mechanism with an implementation of the MBeanServer
that intercept the relevant MBean call and call the relevant registration
function.

The pulling mechanism was removed from Main."
2016-05-02 15:40:37 +03:00
Amnon Heiman
50c8ff548f Main: remove the pulling registration
With the addition of the APIMBeanServer there is no longer a need for
the pulling functionality to be perform for MBean registration.

Signed-off-by: Amnon Heiman <amnon@scylladb.com>
2016-05-02 15:03:23 +03:00
Amnon Heiman
3e95c89310 RMIServerSocketFactoryImpl: regsiter the APIBuilder
This register the APIBuilder as the MBeanServerBuilder which will cause
the APIMBeanServer to be used as the MBeanServer.

Signed-off-by: Amnon Heiman <amnon@scylladb.com>
2016-05-02 15:03:23 +03:00
Amnon Heiman
1daa5eb030 Adding the APIBuilder
The APIBuilder is an implementation for the MBeanServerBuilder that is
used to instantiate the APIMBeanServer as the platform MBeanServer.

Signed-off-by: Amnon Heiman <amnon@scylladb.com>
2016-05-02 15:03:16 +03:00
Amnon Heiman
4e02c52aee Adding the APIMBeanServer
The APIMBeanServer is serve as a proxy for the MBeanServer.
It intercept calls to the MBeanServer and check for the column family
and stream registeration before they are perform.

Current implementation override queryNames as it's the one that is being
used by nodetool.

Additional methods can be override in the future if needed.

Signed-off-by: Amnon Heiman <amnon@scylladb.com>
2016-05-02 15:02:08 +03:00
Amnon Heiman
645d04083c APIMBeanIntrospector: Creating an introspector for the MBeanserver
The MX4J introspector does not support *MXBean interfaces name, This
causes a problem with the garbage collector and java related MBeans.

To bypass that limitation the APIMBeanIntrospector inherit from
MBeanIntrospector and override the relevant functionality so MXBean
will be treated like MBean.

Signed-off-by: Amnon Heiman <amnon@scylladb.com>
2016-05-02 15:00:57 +03:00
Amnon Heiman
5f17e6e0db pom: Add mx4j dependency
The mx4j is used to implement the API MBean Server.

Signed-off-by: Amnon Heiman <amnon@scylladb.com>
2016-05-02 14:55:42 +03:00
Amnon Heiman
0bfaba0a82 StreamingMetrics: Preparation for removing pull mode
This patch expose the check stream registration as an external static
method.

Signed-off-by: Amnon Heiman <amnon@scylladb.com>
2016-05-02 14:55:42 +03:00
Amnon Heiman
5c33a8afa7 ColumnFamilyStore: Preparation for removing the pull mode
This expose the ColumnFamilyStore registration via static method.

It would allow an external object (ie. MBeanServer) to update the
registration on demand.

Signed-off-by: Amnon Heiman <amnon@scylladb.com>
2016-05-02 14:55:32 +03:00