Commit Graph

403 Commits

Author SHA1 Message Date
Pekka Enberg
12ba9915a2 Merge "Cassandra 3.x compatibility" from Calle
"This is a pretty hefty change set. Base goal is to update JMX impl to
 Cassandra 3.x compatibility. While the base MBeans are easy enough, the
 metrics parts of the stew are trickier, since Cassandra 3 changed a
 number of things around.  To that end, and to clean up this code
 somewhat, this basically changes all about metrics management in
 scylla-jmx. Some background:

 Cassandra uses codehaul metrics for actual measurement. Now,
 obviously, in this proxy project we need not measure anything, since
 actual happenings are in the Scylla process. Previous version of the
 code however still utilized a (not-so-pretty reflection-hacked-into)
 version of codehaul metrics because they also provided the system for
 exposing the data through JMX. I.e. we added a bunch of stuff we
 really did not need, to avoid dealing with some of that we did.

 In Cassandra 3, v3 of codehaul is used, which the Cassandra devs
 apparently did not like the JMX integration of. Thus they decided to
 deal with JMX exposure themselves. which makes sense, because they
 want to control the syntax/structure. But given this, we no longer
 have any reason to utilize codehaul, since it does nothing for us.

 These change sets instead adapts the cassandra JMX bindings somewhat,
 and adds a wholly own structure of metric point binding, using
 java.util.function interfaces to provide flexible and late-ish binding
 to actual data query objects. End result is a much slimmer set of
 objects/functions to bind metrics (which of course are just queries to
 Scylla API).

 Also, MX4J has been dropped, since it is at best broken. Instead, we
 use simple wrapping of the system management server object to deal with
 dynamically populating transients objects like column families.

 Removed most statefulness (beyond binding) in MBean impls, all
 "bookeeping" of sub-objects and bind status now uses the actual mbean
 server. I.e. remove race conditions + lighter bookkeep.

 Since this is Java, and everything is tied together in a ball of yarn,
 most of the changes here are not self-contained. I.e. some of these
 will, applied individually, break the build. They are still kept as
 individual patches though, mainly for readability."
2016-11-09 11:03:53 +02:00
Pekka Enberg
b61a5b8439 Revert "Implement deprecated metrics in CommitLog and CompactionManager"
This reverts commit 9b63a35da6 in
preparation for Cassandra 3.x compatibility changes.
2016-11-09 11:03:31 +02:00
Pekka Enberg
c08442b158 Revert "Adding missing method implementation"
This reverts commit 8e5d649048 in
preparation for Cassandra 3.x compatibility changes.
2016-11-09 11:03:07 +02:00
Amnon Heiman
8e5d649048 Adding missing method implementation
This series fill some missing functionality by using already existing
API.

The idea is to use existing code in places that it was not used.

Also, in places were a stub value is in place, the methods returns a
stab value.
Message-Id: <1478619479-10023-1-git-send-email-amnon@scylladb.com>
2016-11-09 10:29:46 +02:00
Pekka Enberg
9b63a35da6 Implement deprecated metrics in CommitLog and CompactionManager
Implement some deprecated metrics in CommitLog and CompactionManager,
that can easily just be a wrapper to the non-deprecated metrics API.

Message-Id: <1478591291-30344-1-git-send-email-penberg@scylladb.com>
2016-11-08 10:59:41 +02:00
Calle Wilund
ae6a000807 ColumnFamilyStore: Remove compaction parameter API usage
Do manual mangling of in/out data in JMX instead. Saves on
controversy over more or less pointless API additions.
2016-11-01 09:44:17 +00:00
Pekka Enberg
954f40e550 Revert "scylla-jmx.service.in: Depend on scylla-server"
This reverts commit 4672cd360f.

Fixes #34
2016-10-31 14:02:22 +02:00
elcallio
434ce947b0 Code formatting + source cleanup (eclipse) 2016-10-24 11:43:52 +00:00
elcallio
9c2d6cec51 Remove yammer/codehale dependencies and augumentations 2016-10-24 11:43:52 +00:00
elcallio
824638594b Clean up and simplify Main startup 2016-10-24 11:43:52 +00:00
elcallio
1709ff2d02 API accessor
* Make config an instance object
* Add functional interfaces
* http options
* Remove dead code
* Clean up/format
2016-10-24 11:43:52 +00:00
elcallio
f4f3c44dc1 Rework StreamManager 2016-10-24 11:43:52 +00:00
Calle Wilund
4ed049739a Storage service: Fix 3.x style notifications (repair) 2016-10-24 11:43:51 +00:00
elcallio
4ec7d58249 Rework service.* beans 2016-10-24 11:43:51 +00:00
elcallio
fec8b44942 Rework MessagingService 2016-10-24 11:43:51 +00:00
Calle Wilund
3fe9cfc232 EndpointSnitchInfo: Fix getRack/DC host handling
I.e. our localhost might be (and probably is) different from scyllas
"fb::broadcast", and if not, try to get numerical asap.
2016-10-24 11:43:51 +00:00
elcallio
21a343d003 Rework EnpointSnitchInfo 2016-10-24 11:43:51 +00:00
elcallio
80762eb60a Rework gms beans 2016-10-24 11:43:51 +00:00
elcallio
e49b4ef322 Rework CompactionManager 2016-10-24 11:43:51 +00:00
elcallio
1470b37193 Rework CommitLog 2016-10-24 11:43:51 +00:00
elcallio
e55863e375 Rework ColumnFamilyStore 2016-10-24 11:43:51 +00:00
elcallio
4b83a9388e Make APIMBeanServer simply wrap actual mbeanserver 2016-10-24 11:43:51 +00:00
elcallio
781821ac9e Make APIMBean name derivation check interface fields as well. 2016-10-24 11:43:51 +00:00
elcallio
cd9deafc51 Rework all org.apache.cassandra.metrics types to new style
I.e. bind only JMX object via registry.
2016-10-24 11:43:51 +00:00
elcallio
319dadb79c Add TableMetrics - c3 version of ColumnFamilyMetrics
Using new, slimmer, metrics binding
2016-10-24 11:43:51 +00:00
elcallio
a44c18c621 Add metric/mbean base types + metrics JMX object factory 2016-10-24 11:43:51 +00:00
Calle Wilund
3e146845b4 StorageService: update to c3 compat
Note: some calls that are not (yet) applicable to scylla are 
unimplemented.
2016-10-24 11:43:51 +00:00
Calle Wilund
b4e483b179 StorageProxy: update to c3 compat 2016-10-24 11:43:51 +00:00
Calle Wilund
de28e68532 GCInspector: Add SuppressWarnings("restriction") 2016-10-24 11:43:51 +00:00
Calle Wilund
3a4adcb676 CacheService: update to c3 compat 2016-10-24 11:43:51 +00:00
Calle Wilund
85e1b07544 MessagingService: update to c3 compat
Note: c3 adds configurable size threshold counting of messages sent, 
dividing info "large"/"small" partitions (+gossiper). Message bulk 
queries in v3 mbean reflects this. 

Scylla does not (yet?) have such a threshold divider, so this is 
highly incomplete and just delegates to old apis that "sort-of" fit.
2016-10-24 11:43:51 +00:00
Calle Wilund
f4759f05e7 EndpointSnitchInfo: update to c3 compat 2016-10-24 11:43:51 +00:00
Calle Wilund
68ce437b03 Gossiper: update to c3 compat 2016-10-24 11:43:51 +00:00
Calle Wilund
b7a6554ee9 FailureDetector: update to c3 compat 2016-10-24 11:43:51 +00:00
Calle Wilund
3efcd5103b CompactionManager: update to c3 compat 2016-10-24 11:43:51 +00:00
Calle Wilund
39e4cd8f3f CommitLog: update to c3 compat 2016-10-24 11:43:51 +00:00
Calle Wilund
85b39d7fbe ColumnFamilyStore: update to c3 compat
Note: some calls still unimplemented
2016-10-24 11:43:51 +00:00
Calle Wilund
9a44228c71 APIClient: Add some "post" overloads 2016-10-24 11:43:51 +00:00
Pekka Enberg
45e2c982f7 Merge "Connect with remote cluster" from Yan
"Tools (like jconsole) cannot connect with remote cluster without this fix."
2016-10-18 12:05:44 +03:00
yan cui
82ae18605a connect with remote cluster 2016-10-17 18:38:10 -07:00
Pekka Enberg
c07f5c034f APIClient: Fix error handling for POST if API call fails
Currently, we have a scary looking dtest failure when attempting to force flush a

  Nodetool command '/data/jenkins/workspace/scylla-1.3-dtest/label/monster/mode/release/smp/1/scylla/resources/cassandra/bin/nodetool -h localhost -p 7100 flush' failed; exit status: 2; stderr: Picked up JAVA_TOOL_OPTIONS: -Dfile.encoding=UTF8
  error: javax.ws.rs.ProcessingException (no security manager: RMI class loader disabled)
  -- StackTrace --
  java.lang.ClassNotFoundException: javax.ws.rs.ProcessingException (no security manager: RMI class loader disabled)
          at sun.rmi.server.LoaderHandler.loadClass(LoaderHandler.java:396)
          at sun.rmi.server.LoaderHandler.loadClass(LoaderHandler.java:186)
          at java.rmi.server.RMIClassLoader$2.loadClass(RMIClassLoader.java:637)
          at java.rmi.server.RMIClassLoader.loadClass(RMIClassLoader.java:264)
          at sun.rmi.server.MarshalInputStream.resolveClass(MarshalInputStream.java:219)
          at java.io.ObjectInputStream.readNonProxyDesc(ObjectInputStream.java:1620)
          at java.io.ObjectInputStream.readClassDesc(ObjectInputStream.java:1521)
          at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1781)
          at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1353)
          at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2018)
          at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1942)
          at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1808)
          at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1353)
          at java.io.ObjectInputStream.readObject(ObjectInputStream.java:373)
          at sun.rmi.transport.StreamRemoteCall.executeCall(StreamRemoteCall.java:245)
          at sun.rmi.server.UnicastRef.invoke(UnicastRef.java:162)
          at com.sun.jmx.remote.internal.PRef.invoke(Unknown Source)
          at javax.management.remote.rmi.RMIConnectionImpl_Stub.invoke(Unknown Source)
          at javax.management.remote.rmi.RMIConnector$RemoteMBeanServerConnection.invoke(RMIConnector.java:1020)
          at javax.management.MBeanServerInvocationHandler.invoke(MBeanServerInvocationHandler.java:298)
          at com.sun.proxy.$Proxy7.forceKeyspaceFlush(Unknown Source)
          at org.apache.cassandra.tools.NodeProbe.forceKeyspaceFlush(NodeProbe.java:290)
          at org.apache.cassandra.tools.NodeTool$Flush.execute(NodeTool.java:1227)
          at org.apache.cassandra.tools.NodeTool$NodeToolCmd.run(NodeTool.java:288)
          at org.apache.cassandra.tools.NodeTool.main(NodeTool.java:202)

The problem is rather innocent: the API call fails and we leak
javax.ws.rs.ProcessingException, which is not available in nodetool's
classpath. In fact, we already fixed the problem for GETs in commit
02e0598 ("APIClient: Fix error handling if connection to API server
fails") so do the same thing for POSTs.
Message-Id: <1471589525-26435-1-git-send-email-penberg@scylladb.com>
2016-08-19 14:44:22 +03:00
Tomasz Grabiec
27e3d1745a scylla-jmx: Exit on unknown parameter rather than infinite-loop
Ran into this while trying to use ccm with not up-to-date scylla-jmx.

Symptoms:

  $ ccm start
  Error starting node node1

and empty ~/.ccm/scylla-3/node1/logs/system.log.jmx
Message-Id: <1468399926-3565-1-git-send-email-tgrabiec@scylladb.com>
2016-07-13 12:05:42 +03:00
Amnon Heiman
4672cd360f scylla-jmx.service.in: Depend on scylla-server
The correct dependency between the jmx and the scylla-server is:
The scylla-jmx should not run if the scylla-server is not running, it
should shutdown when the scylla-server shuts down.

Starting the scylla-jmx should not start the scylla-server, instead, if
the scylla-server is not running it should fail to start.

This patch changes the setup to do so.

Signed-off-by: Amnon Heiman <amnon@scylladb.com>
Message-Id: <1467184319-3395-1-git-send-email-amnon@scylladb.com>
2016-06-29 11:36:43 +03:00
Pekka Enberg
36ae2fcfd7 dist: Execute Maven in batch mode
The batch mode produces much more readable logs because it's designed
for non-interactive builds and doesn't have the fancy download progress
meters.
Message-Id: <1464770158-32482-1-git-send-email-penberg@scylladb.com>
2016-06-06 16:19:22 +03:00
Pekka Enberg
c6edab5990 dist/redhat: Fix RPM package build
Commit 12daaf5 ("dist/redhat: fix rpm build error") did not fix the
error, at least not on our Jenkins build machines.

Looking at the RPM build logs, we create the build directory:

+ cd /builddir/build/BUILD
+ mkdir build

but then change directory to "scylla-jmx-1.2.rc1":

+ cd /builddir/build/BUILD
+ cd scylla-jmx-1.2.rc1
+ mvn install

and therefore fail the copy:

+ cp dist/common/systemd/scylla-jmx.service.in build/scylla-jmx.service
cp: cannot create regular file 'build/scylla-jmx.service': No such
file or directory

I don't know why Takuya put the "mkdir" in the "prep" section but
something like this should unblock the build.
2016-06-01 10:59:52 +03:00
Amnon Heiman
9e97cb530a APITimer: sum should return a value and values are in ns
When removing the pull based timers in the API the sum method in the
APITimer was left stubed by mistake.

This patch take the sum from the histogram as it should be.

Another missed changes are the units, in the yammer library the Timer
does unit conversion before returning the values.

This patch takes the unit conversion from the yammer library to be
compatible.

Signed-off-by: Amnon Heiman <amnon@scylladb.com>
Message-Id: <1464173726-7482-1-git-send-email-amnon@scylladb.com>
2016-05-25 14:16:40 +03:00
Takuya ASADA
12daaf546d dist/redhat: fix rpm build error
Since build/ is not exist, 'cp dist/common/systemd/scylla-jmx.service.in build/scylla-jmx.service' will fail.
So create build/ before starting build stage.

Signed-off-by: Takuya ASADA <syuu@scylladb.com>
Message-Id: <1464164524-20867-1-git-send-email-syuu@scylladb.com>
2016-05-25 13:24:04 +03:00
Pekka Enberg
4eb02743cd Merge "Removing counter pulling from the JMX" from Amnon
"This series uses the API that was added to scylla to remove the counter
 pulling and rely on the statistics collected by the API.

 The series extends the APIMeter, APIHistogram and APITimer to remove
 their pulling part and to fetch the information when needed from the
 API.

 For performence reason those objects will be cached, so that in the
 typical case of of multiple requests of different fields will cause a
 single API call."
2016-05-24 17:20:52 +03:00
Takuya ASADA
225d8ace17 dist/ubuntu: chmod a+rx on build_deb.sh
Add permission to execute build_deb.sh

Signed-off-by: Takuya ASADA <syuu@scylladb.com>
Message-Id: <1464083473-1701-2-git-send-email-syuu@scylladb.com>
2016-05-24 12:53:15 +03:00
Takuya ASADA
e3c5acfcad dist: Support systemd for Ubuntu 15.10/16.04
Since Ubuntu 15.10/16.04 has moved to systemd, share CentOS/Fedora's systemd unit file with Ubuntu.

Fixes scylladb/scylla#1283

Signed-off-by: Takuya ASADA <syuu@scylladb.com>
Message-Id: <1464083473-1701-1-git-send-email-syuu@scylladb.com>
2016-05-24 12:53:15 +03:00