Commit Graph

310 Commits

Author SHA1 Message Date
Pekka Enberg
36ae2fcfd7 dist: Execute Maven in batch mode
The batch mode produces much more readable logs because it's designed
for non-interactive builds and doesn't have the fancy download progress
meters.
Message-Id: <1464770158-32482-1-git-send-email-penberg@scylladb.com>
2016-06-06 16:19:22 +03:00
Pekka Enberg
c6edab5990 dist/redhat: Fix RPM package build
Commit 12daaf5 ("dist/redhat: fix rpm build error") did not fix the
error, at least not on our Jenkins build machines.

Looking at the RPM build logs, we create the build directory:

+ cd /builddir/build/BUILD
+ mkdir build

but then change directory to "scylla-jmx-1.2.rc1":

+ cd /builddir/build/BUILD
+ cd scylla-jmx-1.2.rc1
+ mvn install

and therefore fail the copy:

+ cp dist/common/systemd/scylla-jmx.service.in build/scylla-jmx.service
cp: cannot create regular file 'build/scylla-jmx.service': No such
file or directory

I don't know why Takuya put the "mkdir" in the "prep" section but
something like this should unblock the build.
2016-06-01 10:59:52 +03:00
Amnon Heiman
9e97cb530a APITimer: sum should return a value and values are in ns
When removing the pull based timers in the API the sum method in the
APITimer was left stubed by mistake.

This patch take the sum from the histogram as it should be.

Another missed changes are the units, in the yammer library the Timer
does unit conversion before returning the values.

This patch takes the unit conversion from the yammer library to be
compatible.

Signed-off-by: Amnon Heiman <amnon@scylladb.com>
Message-Id: <1464173726-7482-1-git-send-email-amnon@scylladb.com>
2016-05-25 14:16:40 +03:00
Takuya ASADA
12daaf546d dist/redhat: fix rpm build error
Since build/ is not exist, 'cp dist/common/systemd/scylla-jmx.service.in build/scylla-jmx.service' will fail.
So create build/ before starting build stage.

Signed-off-by: Takuya ASADA <syuu@scylladb.com>
Message-Id: <1464164524-20867-1-git-send-email-syuu@scylladb.com>
2016-05-25 13:24:04 +03:00
Pekka Enberg
4eb02743cd Merge "Removing counter pulling from the JMX" from Amnon
"This series uses the API that was added to scylla to remove the counter
 pulling and rely on the statistics collected by the API.

 The series extends the APIMeter, APIHistogram and APITimer to remove
 their pulling part and to fetch the information when needed from the
 API.

 For performence reason those objects will be cached, so that in the
 typical case of of multiple requests of different fields will cause a
 single API call."
2016-05-24 17:20:52 +03:00
Takuya ASADA
225d8ace17 dist/ubuntu: chmod a+rx on build_deb.sh
Add permission to execute build_deb.sh

Signed-off-by: Takuya ASADA <syuu@scylladb.com>
Message-Id: <1464083473-1701-2-git-send-email-syuu@scylladb.com>
2016-05-24 12:53:15 +03:00
Takuya ASADA
e3c5acfcad dist: Support systemd for Ubuntu 15.10/16.04
Since Ubuntu 15.10/16.04 has moved to systemd, share CentOS/Fedora's systemd unit file with Ubuntu.

Fixes scylladb/scylla#1283

Signed-off-by: Takuya ASADA <syuu@scylladb.com>
Message-Id: <1464083473-1701-1-git-send-email-syuu@scylladb.com>
2016-05-24 12:53:15 +03:00
Avi Kivity
f6710465ef dist: change scylla-jmx process name from 'java' to 'scylla-jmx'
Helps in top, pgrep and friends.  Unfortunately the only reasonable way
to do it is to create a symlink to /usr/bin/java and run that.
Message-Id: <1463580254-8369-1-git-send-email-avi@scylladb.com>
2016-05-20 13:38:28 +03:00
Amnon Heiman
1e4edeb858 MessagingService: Move to APITimer and drop the pulling
With the change to APITimer there is no longer a need to periodically
pull the API.

The verb will be register on the object initialization and will be
updated whenever they are been used.

Signed-off-by: Amnon Heiman <amnon@scylladb.com>
2016-05-17 13:04:55 +03:00
Amnon Heiman
7756f4751a DroppedMessageMetrics: Change APISettableMeter to APIMeter
The APIMeter replaces the APISettableMeter as the Meter implementation.

Signed-off-by: Amnon Heiman <amnon@scylladb.com>
2016-05-17 12:27:36 +03:00
Amnon Heiman
74d062851c CompactionMetrics: Use APITimer
This change the Timer from Timer to APITimer.

Signed-off-by: Amnon Heiman <amnon@scylladb.com>
2016-05-17 12:25:51 +03:00
Amnon Heiman
78bb2ef25a CacheService: Do not get non existing caches metrics
With the change in the meter implementation, retrieving a non existing
metrics would take time.

For this, the CacheService would mark caches that are not supported with
null url, so the metrics will be register but will return 0 for all
request (instead of going to the API that will return 0).

Signed-off-by: Amnon Heiman <amnon@scylladb.com>
2016-05-17 12:20:35 +03:00
Amnon Heiman
8a73d6a840 CacheMetrics: Switch to APITimer
CacheMetrics is a general counter that is used for all possible caches.
For caches that we do not support, there is no need to go and fetch
their values.

When moving to the APITimer each such request will take longer (the
value will not be available as it use it) now it will be possible to
supply a null as a url which would cause to the metrics to return 0 for
all counters.

Signed-off-by: Amnon Heiman <amnon@scylladb.com>
2016-05-17 11:32:59 +03:00
Amnon Heiman
ad49d05780 ClientRequestMetrics: Using the APITimer
The APITimer uses a different endpoint not to break existing API.

Signed-off-by: Amnon Heiman <amnon@scylladb.com>
2016-05-17 11:30:55 +03:00
Amnon Heiman
2c07ca2e09 LatencyMetrics: Move to APITimer
The APITimer uses a different endpoint not to break existing API.

The addNano functionality was removed as all of the values are updated
from the APi.

Signed-off-by: Amnon Heiman <amnon@scylladb.com>
2016-05-17 11:26:18 +03:00
Amnon Heiman
801af00ddb APIMetrics, APIMetricsRegistry: Return APIMeter and APITimer
Some of the specific functionality is needed from the APIMeter and
APITimer.

The MetricsRegistry now return the specific objects so they can be used
in their adapted form.

Signed-off-by: Amnon Heiman <amnon@scylladb.com>
2016-05-17 11:22:43 +03:00
Amnon Heiman
d3b8ef1ae5 APITimer: Non pull based Timer
The yammer Timer object calculate rate based on a timer, which causes
periodic calls to the API.

This replaces the implementation so that a timer would get all its
values from the API.

For object registration the APITimer still inherit from Timer but
override all its functionality.

Signed-off-by: Amnon Heiman <amnon@scylladb.com>
2016-05-17 11:12:01 +03:00
Amnon Heiman
eca6451832 APIHistogram: Support APITimer
With the move to APITimer, in many occasion a histogram will not update
itself, instead it will be updated by the APITimer.

This breaks the update values functionality so that a histogram that is
included in an APITimer will not try to update it self.

Signed-off-by: Amnon Heiman <amnon@scylladb.com>
2016-05-17 11:08:05 +03:00
Amnon Heiman
4d1f8ed7c9 APIMeter: Move out of pull mode
This replaces the APIMeter implementation so it will not pull the API
regularly.

To by complient with the yammer object registration mechanism, it still
inherit from Meter, but all the functionalities are overriden.

Now all the data is taken from the API including the rate statistics.

Signed-off-by: Amnon Heiman <amnon@scylladb.com>
2016-05-17 10:03:12 +03:00
Amnon Heiman
adf466519f APIClient: Support for non pull APIMeter
APIMeter will be modified not to use pulling and to retrieve the derived
information from the API.

To support that the APIClient was changed so it would be able to cache
json objects and histogram.

Signed-off-by: Amnon Heiman <amnon@scylladb.com>
2016-05-17 09:35:42 +03:00
Amnon Heiman
a028e59699 CacheEntry: Support caching of jsonobject
This will allow to store json objects in the cache.

Signed-off-by: Amnon Heiman <amnon@scylladb.com>
2016-05-17 09:32:50 +03:00
Amnon Heiman
2ff16fa2a5 scylla-jmx: set the APIBuilder in the command line
When setting the jmx through the command line, the jmx server creates
even before the main is called.
For the APIServer to take effect the builder should be set via system
properties.

This patch also add an option to run the java process with debug ports
open so an extern debug will be able to connect to the app.

Signed-off-by: Amnon Heiman <amnon@scylladb.com>
Message-Id: <1462447227-8367-2-git-send-email-amnon@scylladb.com>
2016-05-05 16:40:23 +03:00
Amnon Heiman
853963f833 APIMBeanServer: overload the queryName implementation
The mx4j implementation of queryName does not handle correctly pattern
matching.

This patch identify that a name contains a patern and do the patern
matching as it should have been done by the mx4j MBeanServer.

Fixes #28

Message-Id: <1462447227-8367-1-git-send-email-amnon@scylladb.com>
2016-05-05 16:40:20 +03:00
Avi Kivity
f8112b5d57 Merge "Remove the pulling mode for MBean registration" from Amnon
"This series replaces that mechanism with an implementation of the MBeanServer
that intercept the relevant MBean call and call the relevant registration
function.

The pulling mechanism was removed from Main."
2016-05-02 15:40:37 +03:00
Amnon Heiman
50c8ff548f Main: remove the pulling registration
With the addition of the APIMBeanServer there is no longer a need for
the pulling functionality to be perform for MBean registration.

Signed-off-by: Amnon Heiman <amnon@scylladb.com>
2016-05-02 15:03:23 +03:00
Amnon Heiman
3e95c89310 RMIServerSocketFactoryImpl: regsiter the APIBuilder
This register the APIBuilder as the MBeanServerBuilder which will cause
the APIMBeanServer to be used as the MBeanServer.

Signed-off-by: Amnon Heiman <amnon@scylladb.com>
2016-05-02 15:03:23 +03:00
Amnon Heiman
1daa5eb030 Adding the APIBuilder
The APIBuilder is an implementation for the MBeanServerBuilder that is
used to instantiate the APIMBeanServer as the platform MBeanServer.

Signed-off-by: Amnon Heiman <amnon@scylladb.com>
2016-05-02 15:03:16 +03:00
Amnon Heiman
4e02c52aee Adding the APIMBeanServer
The APIMBeanServer is serve as a proxy for the MBeanServer.
It intercept calls to the MBeanServer and check for the column family
and stream registeration before they are perform.

Current implementation override queryNames as it's the one that is being
used by nodetool.

Additional methods can be override in the future if needed.

Signed-off-by: Amnon Heiman <amnon@scylladb.com>
2016-05-02 15:02:08 +03:00
Amnon Heiman
645d04083c APIMBeanIntrospector: Creating an introspector for the MBeanserver
The MX4J introspector does not support *MXBean interfaces name, This
causes a problem with the garbage collector and java related MBeans.

To bypass that limitation the APIMBeanIntrospector inherit from
MBeanIntrospector and override the relevant functionality so MXBean
will be treated like MBean.

Signed-off-by: Amnon Heiman <amnon@scylladb.com>
2016-05-02 15:00:57 +03:00
Amnon Heiman
5f17e6e0db pom: Add mx4j dependency
The mx4j is used to implement the API MBean Server.

Signed-off-by: Amnon Heiman <amnon@scylladb.com>
2016-05-02 14:55:42 +03:00
Amnon Heiman
0bfaba0a82 StreamingMetrics: Preparation for removing pull mode
This patch expose the check stream registration as an external static
method.

Signed-off-by: Amnon Heiman <amnon@scylladb.com>
2016-05-02 14:55:42 +03:00
Amnon Heiman
5c33a8afa7 ColumnFamilyStore: Preparation for removing the pull mode
This expose the ColumnFamilyStore registration via static method.

It would allow an external object (ie. MBeanServer) to update the
registration on demand.

Signed-off-by: Amnon Heiman <amnon@scylladb.com>
2016-05-02 14:55:32 +03:00
Avi Kivity
9f2f92c379 Merge "Ubuntu 16.04 support" from Takuya 2016-05-01 11:14:50 +03:00
Takuya ASADA
19a0e72752 dist/ubuntu: fix build error on Ubuntu 16.04 2016-04-30 07:51:07 +00:00
Takuya ASADA
72aae6bf3b dist/ubuntu: resolve build time dependency by mk-build-deps command
Resolve dependency packages from debian/control, intead of doing apt-get manually.
2016-04-30 07:44:25 +00:00
Takuya ASADA
d8d3023334 dist/ubuntu: generate correct distribution codename on debian/changelog
Since we supported more than one version of Ubuntu, need to generate each codename on changelog.
2016-04-30 07:42:35 +00:00
Takuya ASADA
ef15d95416 dist: #!/bin/bash for all scripts
Like we did on scylla-server, switch to bash.
2016-04-30 07:40:42 +00:00
Amnon Heiman
c63ec3e96b StorageService: Add takeMultipleColumnFamilySnapshot support
This patch adds the functionality of takeMultipleColumnFamilySnapshot to
StorageService.

It follow origin logic of first check that all keyspaces and column
families exists and has no snapshot with that name and then run snapshot
on each of the combinations.

Two methods where added to simplify the implementation, but that can be
reused. One to get a map from keyspace to column family and one with the
current snapshots.

Fixes scylladb/scylla#1133

Signed-off-by: Amnon Heiman <amnon@scylladb.com>
Message-Id: <1461659678-22030-1-git-send-email-amnon@scylladb.com>
2016-04-26 14:28:33 +03:00
Amnon Heiman
5903271c4d EndpointState: log and ignore not supported states
During upgrade or version inconsistency. The API can return an un
supported state.

Instead of throwing an expcetion the state will be ignore and a warning
will be written to the log.

An example (state where modified in the API)
$ nodetool gossipinfo
/127.0.0.1
  generation:1460450456
  heartbeat:32

The log shows:

Apr 12, 2016 3:40:20 PM org.apache.cassandra.gms.EndpointState
addApplicationState
WARNING: Unknown application state with id:25

Fixes scylladb/scylla#1164.

Signed-off-by: Amnon Heiman <amnon@scylladb.com>
Message-Id: <1460465073-3567-1-git-send-email-amnon@scylladb.com>
2016-04-12 15:53:16 +03:00
Takuya ASADA
a2ec7f2f60 dist/ubuntu: drop classical sysv init script, only support Upstart for Ubuntu 14.04LTS
Drop sysv init script on scylla-jmx.
Same as a5bb6c4b1b

Signed-off-by: Takuya ASADA <syuu@scylladb.com>
Message-Id: <1459346746-3433-2-git-send-email-syuu@scylladb.com>
2016-03-30 18:10:18 +03:00
Takuya ASADA
0c5c1debde dist: do not auto-start scylla-server job on Ubuntu package install time
Same as f1d18e9980
Fixes scylladb/scylla#1134

Signed-off-by: Takuya ASADA <syuu@scylladb.com>
Message-Id: <1459346746-3433-1-git-send-email-syuu@scylladb.com>
2016-03-30 18:10:17 +03:00
Pekka Enberg
524763cfed dist/ubuntu: Use tilde for release candidate builds
The version number ordering rules are different for rpm and deb. Use
tilde ('~') for the latter to ensure a release candidate is ordered
_before_ a final version.
2016-03-22 12:23:48 +02:00
Amnon Heiman
94f144e9b3 StorageService: Get the broadcast address from the API
When getting the tokens of the current node, we use the get_token api
call with the local broadcast address.

The current implementation that tries to figure it out from the
configuration is prone to error.

Currently in a configuration where the broadcast address is set to the
local API and the listening API is set to 127.0.0.1 we get a call to
nodetool info will return an exception:
ID                     : 54185d5d-6f62-4884-814c-5d17c2776de9
Gossip active          : true
Thrift active          : true
Native Transport active: true
Load                   : 178.09 KB
Generation No          : 1458349593
Uptime (seconds)       : 11
Heap Memory (MB)       : 47.23 / 247.50
Off Heap Memory (MB)   : 2.75
error: Index: 0, Size: 0
-- StackTrace --
java.lang.IndexOutOfBoundsException: Index: 0, Size: 0
	at java.util.ArrayList.rangeCheck(ArrayList.java:653)
	at java.util.ArrayList.get(ArrayList.java:429)
	at org.apache.cassandra.tools.NodeProbe.getEndpoint(NodeProbe.java:812)
	at org.apache.cassandra.tools.NodeProbe.getDataCenter(NodeProbe.java:830)
	at org.apache.cassandra.tools.NodeTool$Info.execute(NodeTool.java:425)
	at org.apache.cassandra.tools.NodeTool$NodeToolCmd.run(NodeTool.java:288)
	at org.apache.cassandra.tools.NodeTool.main(NodeTool.java:202)

Becasue getTokens will return an empty list.

This patch changed how broadcast address is deduct. It Adds a reverse
mapping from hostid to ip address and use it with the get local id to
find the ip address in use.

This implementation would probably be replaced by a single API call in
the future.

After the change a call to nodetool info works.

Fixes scylladb/scylla#1027

Signed-off-by: Amnon Heiman <amnon@scylladb.com>
Message-Id: <1458405434-8491-3-git-send-email-amnon@scylladb.com>
2016-03-22 09:43:13 +02:00
Amnon Heiman
a60c3156c6 ApiClient: Add getReverseMapStrValue method
Sometimes it is usefull to get a reverse of the map return by the API.
For example instread of ip address to hostid to get the hostid to ip
address.

Though it is possible to reverse a map, there is no need, it's easier to
generate the reverse mapping.

Signed-off-by: Amnon Heiman <amnon@scylladb.com>
Message-Id: <1458405434-8491-2-git-send-email-amnon@scylladb.com>
2016-03-22 09:42:57 +02:00
Amnon Heiman
8f90d413a1 ProcessingException was changed to IllegalStateException
This patch fix the exception handling for connection problem, instead of
ProcessingException it now expect IllegalStateException.

The rest of the functionality remains the same.

Fixes #26

Signed-off-by: Amnon Heiman <amnon@scylladb.com>
Message-Id: <1458602355-23601-1-git-send-email-amnon@scylladb.com>
2016-03-22 08:55:35 +02:00
Pekka Enberg
d7e3dae323 dist/ubuntu: Relax Java dependencies
The requirement for Java 7 is too strict, especially as it's end-of-life.

Fixes #1029.
Message-Id: <1458132593-25935-1-git-send-email-penberg@scylladb.com>
2016-03-21 14:43:08 +02:00
Pekka Enberg
2cd5a5f048 StorageService: Fix scrub() variant API wiring
The 'nodetool scrub' command ends up calling the variant that is not
wired up to the Scylla API which causes the following error to be
printed out to the user:

  [penberg@nero scylla-tools-java]$ ./bin/nodetool scrub
  error: For input string: ""
  -- StackTrace --
  java.lang.NumberFormatException: For input string: ""
          at java.lang.NumberFormatException.forInputString(NumberFormatException.java:65)
          at java.lang.Integer.parseInt(Integer.java:592)
          at java.lang.Integer.parseInt(Integer.java:615)
          at com.scylladb.jmx.api.APIClient.getIntValue(APIClient.java:216)
          at com.scylladb.jmx.api.APIClient.getIntValue(APIClient.java:220)
          at org.apache.cassandra.service.StorageService.scrub(StorageService.java:1291)

Fix the problem by implementing the said scrub() variant.
Message-Id: <1458035736-26349-1-git-send-email-penberg@scylladb.com>
2016-03-16 08:35:30 +02:00
Pekka Enberg
c4d8d7087e APIClient: Make API server errors human readable
Make the error messages returned by Scylla API server human readable
from 'nodetool'.

For example, if an API URL is missing, print out the following error:

  [penberg@nero scylla-tools-java]$ ./bin/nodetool getcompactionthreshold ks test4
  nodetool: Scylla API server HTTP GET to URL 'column_family/minimum_compaction/ks:test4' failed: Not found
  See 'nodetool help' or 'nodetool help <command>'.

instead of the scary-looking error that we now print:

  [penberg@nero scylla-tools-java]$ ./bin/nodetool getcompactionthreshold ks test4
  error: Not found
  -- StackTrace --
  java.lang.RuntimeException: Not found
          at com.scylladb.jmx.api.APIClient.getException(APIClient.java:116)
          at com.scylladb.jmx.api.APIClient.getRawValue(APIClient.java:160)
          at com.scylladb.jmx.api.APIClient.getRawValue(APIClient.java:174)
          at com.scylladb.jmx.api.APIClient.getIntValue(APIClient.java:216)
          at com.scylladb.jmx.api.APIClient.getIntValue(APIClient.java:220)
          at org.apache.cassandra.db.ColumnFamilyStore.getMinimumCompactionThreshold(ColumnFamilyStore.java:475)
          at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
          at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
          at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
          at java.lang.reflect.Method.invoke(Method.java:498)
          at sun.reflect.misc.Trampoline.invoke(MethodUtil.java:71)
          at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
          at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)

          [snip]
Message-Id: <1458032300-17704-1-git-send-email-penberg@scylladb.com>
2016-03-16 08:34:47 +02:00
Pekka Enberg
02e0598506 APIClient: Fix error handling if connection to API server fails
Running 'nodetool status' now reports the following if the JMX proxy is
not able to connect to an API server:

  nodetool: Unable to connect to Scylla API server: java.net.ConnectException: Connection refused
  See 'nodetool help' or 'nodetool help <command>'.

instead of the scary-looking:

  error: javax.ws.rs.ProcessingException (no security manager: RMI class loader disabled)
  -- StackTrace --
  java.lang.ClassNotFoundException: javax.ws.rs.ProcessingException (no security manager: RMI class loader disabled)
          at sun.rmi.server.LoaderHandler.loadClass(LoaderHandler.java:393)
          at sun.rmi.server.LoaderHandler.loadClass(LoaderHandler.java:185)
          at java.rmi.server.RMIClassLoader$2.loadClass(RMIClassLoader.java:637)
          at java.rmi.server.RMIClassLoader.loadClass(RMIClassLoader.java:264)
          at sun.rmi.server.MarshalInputStream.resolveClass(MarshalInputStream.java:214)

That happens because the MBean propagates a
'javax.ws.rs.ProcessingException' to nodetool which does not have it in
it's classpath and loading via RMI fails.

Fixes #25.

Message-Id: <1457697628-31792-1-git-send-email-penberg@scylladb.com>
2016-03-14 11:50:29 +02:00
Pekka Enberg
a38bbfd603 Merge "JMX to listen on local traffic by default" from Amnon
"By default Origin accept local JMX connection. This series import the
 code from origin to set the jmx to listen to local traffic only and
 change the run script so that the default behaviuor would be local only
 traffic."
2016-03-11 14:33:54 +02:00