`!=` compares references not values.
Use !"".equals(scrubMode) instead, as it also covers
the null scrubMode case.
Signed-off-by: Benny Halevy <bhalevy@scylladb.com>
Closes#179
This information is translated to {"load_and_stream", "true"} entry in the
POST request to Scylla's HTTP API at `storage_service/sstables/{keyspace}`
endpoint.
More about this feature: scylladb/scylla#7846
This change is a consequence of scylladb/scylla-tools-java#253.
Support new scrubMode option and deprecate skipCorrupted
that's equivalent to scrubMode="SKIP".
Signed-off-by: Benny Halevy <bhalevy@scylladb.com>
Closes#175
In line with the previous API, the getToppartitions function returned
results for one specified sampler (reads OR writes). This forced
the user to call the function once for each sampler, which is
suboptimal.
This commit changes the signature so that results for both samplers
are returned and the user can then pick whichever they need.
As part of making the toppartitions API more generic
(i.e. being able to consider multiple tables
and keyspaces specified by the user) this commit adds
a JMX endpoint to call the generic Scylla REST API
introduced in #7864. It has been put inside
storage_service as being now able to query more than
one column family makes it no longer suitable for the
'column_family' group.
Fixes#4520
Currently, the nodetool uses the jmx server for the uptime, this is
confusing is what we expect is Scylla uptime.
This patch exposes the API uptime using MBean.
Relates to #154
Signed-off-by: Amnon Heiman <amnon@scylladb.com>
Closes#155
This is a seemingly pointless change. The RW-lock code is 100%
correct (afaict), yet we've seen repeated cases of test runs
hanging in JMX query because this lock is seemingly left held
by what seems to be the reaper task.
There is no explanation for this, no sign of exceptions/errors
that could explain the lock being broken. Nor any known JDK/JVM
bugs.
Yet, in tests, it seems that replacing the lock with a more
coarse, yet proven, synchronized, fixes the issue. So there.
I officially hate this patch, and it should not exist.
This patch adds the compaction id in getCompactions if it returns by the
API, if it's not the current behaviour will be used and it will return none.
After this patch a call to nodetool compactionstats -H
Will return:
id compaction type keyspace table completed total unit progress
c942bd30-7a62-11eb-84bc-576502584f9a COMPACTION keyspace1 standard1 1062 8576 keys 12.38%
c9429620-7a62-11eb-8afb-576402584f9a COMPACTION keyspace1 standard1 972 8448 keys 11.51%
Active compaction remaining time : 0h00m00s
Fixesscylladb/scylla#7927
Signed-off-by: Amnon Heiman <amnon@scylladb.com>
Right now, in the finishLocalSampling method of the ColumnFamilyStore
we return the size of the list of returned partitions. Instead, we should
be propagating the actual cardinality of the sampled set.
Let's just read the read_cardinality and write_cardinality properties
of the scylla's REST API response.
Fixes#148
Fixes#134
Refs #135
Replaces previous refresh calls with ones bound to registration
check objects, which provides some sync between threads doing
refresh, and reduced redundant calls.
Also adds repeated reaping of dead objects, i.e. every 5 minutes
we try to remove dead CF:s (not adding new ones), to reduce
idle footprint.
Fixes#133
TableMetricObjectName is not serializable as such, since
it depends on a lexicon object etc.
Use writeReplace to put a regular ObjectName in
the stream instead.
The scrub command was not supported from node_tool, but now when we want
to enable it the current API is not compatible with the 3.11 MBean
definition.
This patch adds the definition to the MBean and the implementation to
StorageService.
It also address two problems with the old scrub implementation, just
in case someone will use them.
1. Implementation didn't pass the parameters to the API.
2. A stub implementation called itself instead of calling an actual
implementation.
This patch will enable to test the command from nodetool additional
changes may come on top of it if more command line options will be
supported.
Signed-off-by: Amnon Heiman <amnon@scylladb.com>
Refs #76
Since the incoming json uses swagger "key", "value" syntax
we need to do explicit deserialization of this property
as well (not just extended props).
Message-Id: <20190930115432.27801-1-calle@scylladb.com>
"Fixes #76
Implements JMX level call for "sstable_info" REST api command.
Requires seastar patch:
json: Make date formatter use RFC8601/RFC3339 format
Requires scylla patch set "Implement sstable_info API command (info on sstables)"
Forwards call to REST sstable_info and packs the data
into CompositeData for JMX consumption."
* 'sstabledesc' of git://github.com/elcallio/scylla-jmx:
storage_service: Add "getSSTableInfo" command/attribute
service: Add objects for deserializing sstable_info json
scylla-apiclient: Add Date json serializer helper
APIClient: Add jackson JSON serializer support to client object
apiclient/pom.xml: Add jackson JSON support libs for REST client
Fixes#76
Requires seastar patch:
json: Make date formatter use RFC8601/RFC3339 format
Requires scylla patch set "Sstabledesc"
Forwards call to REST sstable_info and packs the data
into CompositeData for JMX consumption.
The PendingTasksByTableName metric should use the pending_tasks_by_table
API to get the real value of the pending compaction.
Fixes#74
Signed-off-by: Amnon Heiman <amnon@scylladb.com>
Cassandra 3.0 version of the JMX added a parameter that allows accepting
the parameter as hex.
This breaks the current implementation with a NoSuchMethodException.
This patch adds the missing implementation.
For a full support, a follow up patch in Scylla is needed, but for the
current functionality it would work.
After this patch usage example:
nodetool getsstables keyspace1 standard1 39303138374b4d343830
Signed-off-by: Amnon Heiman <amnon@scylladb.com>
When calling nodetool toppartitions with size limit, finishLocalSampling
should respect that and limit the number of the results.
Example:
$ nodetool toppartitions -k 2 keyspace1 standard1 20
WRITES Sampler:
Cardinality: ~2 (256 capacity)
Top 2 partitions:
Partition Count +/-
38333032394d4f4d5030 4 3
4e353937383137503330 4 3
READS Sampler:
Cardinality: ~2 (256 capacity)
Top 2 partitions:
Nothing recorded during sampling period...
Fixes#66
Signed-off-by: Amnon Heiman <amnon@scylladb.com>
delete commands do not return a value, still, it is possible that the
command will return a value different than OK.
In such a case, the error should be propagate to the caller via an
exception.
Fixes#65
Signed-off-by: Amnon Heiman <amnon@scylladb.com>
Message-Id: <20190618135312.2776-1-amnon@scylladb.com>
This patch adds the implementation for begin and finish local sampling
of a column family.
There is a difference in the implementation of Cassandra API and Scylla.
In Cassandra and the JMX an external source start and stop the sampling.
In Scylla, a single API call start the sampling and return with the
result. In Scylla the API call always return sampling of the read and of
the writes.
To bridge the difference, the begin sampling command will use a Future
when calling the API. The finish method will wait for the future to end.
Because of the different implementation, it is possible that two
consecutive calls will be made to start sampling one for the read and
one for the write, similarly, two calls will be made to finish for read
and write.
The implementation would ignore the second call to start and will
store the result, so the second call to finish will be served from the
stored result.
Note, that the use of future is only for safety, the way we expect it to
work, the caller to the begin sampling will sleep anyhow while waiting
for the result.
To avoid breaking the MBean compatibility we piggyback the duration on
top of the sampler string.
If no duration is given, a default duration will be taken, this is also
just as a precaution, we will modify the nodetool implementation to
pass that information.
There is a known issue with cardinality, that will need to be addressed.
Also we return a value in the raw column to match what Cassandra JMX
returns, but it's a duplication of the partition key.
See scylladb/scylla#2811
Signed-off-by: Amnon Heiman <amnon@scylladb.com>
Message-Id: <20190128143505.5241-1-amnon@scylladb.com>
Fixes#57
The usage of TableMetricsObjectName-yada-yada relies on translating the
"fake" objectname to a canonical one on remote
publication/serialization. However, the implementation of
ObjectName.getInstance has changed in JDK (micro) updates so it no
longer applies overridable methods -> wrong name published.
Fix by doing explicit ObjectName instansiation.
Message-Id: <20181023132005.23099-1-calle@scylladb.com>
Almost 100% null implementations, which is ok for most purposes
currently used by scylla. Some of these new calls (like dropped
mutations etc) should perhaps however be implemented.
Tested with the nodetool dtests. So sparsely.
Needed when/if scylla-tools-java is upgraded to origin 3.11,
otherwise noodtool breaks.
Message-Id: <20180730113741.14952-1-calle@scylladb.com>
Before we were discarding the initial repository while
overriding it with TableRepository. This was a mistake that
caused dtests to fail. Proper solution is to keep the initial
repository inside TableRepository. That way whatever was registered
at the time of JmxMBeanServer creation is still handled properly.
Signed-off-by: Piotr Jastrzebski <piotr@scylladb.com>
Message-Id: <22181859012fd20ddf37e049a145bc94a3a91a33.1527844328.git.piotr@scylladb.com>
Default implementation stores MBeans in the following map:
<domain name> -> (<properties as a single string> -> NamedObject)
This is problematic because NamedObject contains ObjectName that
has both domain and properties inside itself.
This means we're storing the same data twice.
For domain "" we want to store MBeans in a more compact way using map:
ObjectName -> DynamicMBean
which is equivalent to NamedObject.
Signed-off-by: Piotr Jastrzebski <piotr@scylladb.com>
Before this change it was taking JMX Server 270 seconds to start
when Scylla had 2000 tables. After the change it takes only 2 seconds.
Signed-off-by: Piotr Jastrzebski <piotr@scylladb.com>
This is a new extention of ObjectName that uses less memory.
TableMetricNameFactory and AllTableMetricNameFactory can
create it instead of regular ObjectName to save memory.
It is possible to save memory because each name created by
TableMetricNameFactory (or AllTableMetricNameFactory) shares
most of its data with other names created by the same factory
and there's no need to create multiple copies.
Signed-off-by: Piotr Jastrzebski <piotr@scylladb.com>
Next patch will introduce new ObjectName implementation that
will use less memory. This new object won't be serializable.
This means it won't be possible to transport it to a remote
caller. We want to keep this new object local to JMX server as well.
This patch makes sure that every ObjectName returned
from APIBeanServer is transformed into a regular ObjectName.
It also makes sure that every ObjectInstance returned from
APIBeanServer has its ObjectName swapped with a regular ObjectName.
Signed-off-by: Piotr Jastrzebski <piotr@scylladb.com>
JmxMBeanServer is a concrete implementation of a MBeanServer.
We want to use it directly because we need to bypass calls to
JmxMBeanServer.registerMBean and JmxMBeanServer.unregisterMBean.
They take ObjectName as parameter, copy it using
ObjectName.getInstance(ObjectName) and pass it to registerMBean
and unregisterMBean of JmxMBeanServer.getMBeanServerInterceptor().
We want to avoid this copy and pass the ObjectName argument directly
to JmxMBeanServer.getMBeanServerInterceptor().
To do that this patch:
1. changes all MBeanServer variables to JmxMBeanServer
2. creates JmxMBeanServer in APIBuilder making sure accessing
interceptors is allowed
3. makes sure that JmxMBeanServer.getMBeanServerInterceptor().registerMBean
is called wherever JmxMBeanServer.registerMBean was called
4. makes sure that JmxMBeanServer.getMBeanServerInterceptor().unregisterMBean
is called whenever JmxMBeanServer.unregisterMBean was called
Next patch will use different ObjectName implementation that will
use less memory and this patch is crucial because without it every ObjectName
is transformed with ObjectName.getInstance which turns the object into
a regular ObjectName.
Signed-off-by: Piotr Jastrzebski <piotr@scylladb.com>
When a node is part of a cluster but is down (like in the situation where
a cluster is taken down and up again but not all nodes are up). There is
no application_state information for that node.
This patch check that the information exists before using it to prevent
null pointer exception.
After this patch, a call to nodetool gossipinfo would return the
available information without failing.
See scylladb/scylla#3330
Signed-off-by: Amnon Heiman <amnon@scylladb.com>
Message-Id: <20180329115345.29357-1-amnon@scylladb.com>
