When we add product name customization, we mistakenly defined the
parameter on each package build script.
Number of script is increasing since we recently added relocatable
python3 package, we should merge it in single place.
Also we should save the parameter on relocatable package, just like
version-release parameters.
So move the definition to SCYLLA-VERSION-GEN, save it to
build/SCYLLA-PRODUCT-FILE then archive it to relocatable package.
Signed-off-by: Takuya ASADA <syuu@scylladb.com>
Message-Id: <20190422105304.23454-1-syuu@scylladb.com>
This patch adds the implementation for begin and finish local sampling
of a column family.
There is a difference in the implementation of Cassandra API and Scylla.
In Cassandra and the JMX an external source start and stop the sampling.
In Scylla, a single API call start the sampling and return with the
result. In Scylla the API call always return sampling of the read and of
the writes.
To bridge the difference, the begin sampling command will use a Future
when calling the API. The finish method will wait for the future to end.
Because of the different implementation, it is possible that two
consecutive calls will be made to start sampling one for the read and
one for the write, similarly, two calls will be made to finish for read
and write.
The implementation would ignore the second call to start and will
store the result, so the second call to finish will be served from the
stored result.
Note, that the use of future is only for safety, the way we expect it to
work, the caller to the begin sampling will sleep anyhow while waiting
for the result.
To avoid breaking the MBean compatibility we piggyback the duration on
top of the sampler string.
If no duration is given, a default duration will be taken, this is also
just as a precaution, we will modify the nodetool implementation to
pass that information.
There is a known issue with cardinality, that will need to be addressed.
Also we return a value in the raw column to match what Cassandra JMX
returns, but it's a duplication of the partition key.
See scylladb/scylla#2811
Signed-off-by: Amnon Heiman <amnon@scylladb.com>
Message-Id: <20190128143505.5241-1-amnon@scylladb.com>
On some Fedora environment dh build tries to run
dh_strip_nondeterminism, and fails sice Fedora does not provide such
command.
To prevent the build error we need to skip it.
Fixes#62
See 5bf9a03d65
Signed-off-by: Takuya ASADA <syuu@scylladb.com>
Message-Id: <20190125223351.20381-1-syuu@scylladb.com>
Current scylla.spec fails build on Fedora 27, since python2-pystache is
new package name that renamed on Fedora 28.
But Fedora 28's python2-pystache has tag "Provides: pystache",
so we can depends on old package name, this way we can build scylla.spec both
on Fedora 27/28.
Signed-off-by: Takuya ASADA <syuu@scylladb.com>
Message-Id: <20181028175757.32224-1-syuu@scylladb.com>
- we don't use mock anymore, so drop mock directory
- build_rpm.sh usage need to update
- build_rpm.sh should install rpmbuild
Signed-off-by: Takuya ASADA <syuu@scylladb.com>
Message-Id: <20181024222136.3332-2-syuu@scylladb.com>
- we don't use pbuilder anymore, so drop pbuilderrc
- on build_deb.sh is_debian / is_ubuntu functions are unused now, drop them
Signed-off-by: Takuya ASADA <syuu@scylladb.com>
Message-Id: <20181024222136.3332-1-syuu@scylladb.com>
"
This patchset adds relocatable package support for scylla-jmx, and also
support generating .rpm/.deb from relocatable package.
- Scripts are based on relocatable .rpm/.deb support patchset for main repo
(not merged https://github.com/syuu1228/scylla/tree/reloc_rpmdeb_v4)
- Single .rpm package provided for CentOS7/Fedora(unofficial)
- Single .deb package provided for Ubuntu 14/16/18, Debian 8/9
"
* 'reloc_v1' of https://github.com/syuu1228/scylla-jmx:
dist/debian: use relocatable package to produce .deb
dist/redhat: use relocatable package to produce .rpm
reloc: add support relocatable package
To align build system with scylla main repo, adding relocatable package
support.
On scylla-jmx, we don't provide libraries and linker since it's Java
program, just contains .jar file and dist/ directory.
Fixes#57
The usage of TableMetricsObjectName-yada-yada relies on translating the
"fake" objectname to a canonical one on remote
publication/serialization. However, the implementation of
ObjectName.getInstance has changed in JDK (micro) updates so it no
longer applies overridable methods -> wrong name published.
Fix by doing explicit ObjectName instansiation.
Message-Id: <20181023132005.23099-1-calle@scylladb.com>
We found on some Debian environment Ubuntu .deb build fails with
gpg error because lack of Ubuntu GPG key, so we need to install it before
start pbuilder.
Same as on Ubuntu, it needs to install Debian GPG key.
See scylladb/scylla#3823
Signed-off-by: Takuya ASADA <syuu@scylladb.com>
Message-Id: <20181008110724.18335-1-syuu@scylladb.com>
To automatically rename packages on enterprise release, added package name
prefix as a variable on build_deb.sh.
Signed-off-by: Takuya ASADA <syuu@scylladb.com>
Message-Id: <20180828010445.11920-2-syuu@scylladb.com>
To automatically rename packages on enterprise release, added package name
prefix as a variable on build_rpm.sh.
Signed-off-by: Takuya ASADA <syuu@scylladb.com>
Message-Id: <20180822072105.9420-2-syuu@scylladb.com>
Almost 100% null implementations, which is ok for most purposes
currently used by scylla. Some of these new calls (like dropped
mutations etc) should perhaps however be implemented.
Tested with the nodetool dtests. So sparsely.
Needed when/if scylla-tools-java is upgraded to origin 3.11,
otherwise noodtool breaks.
Message-Id: <20180730113741.14952-1-calle@scylladb.com>
mock outputs files owned by root. This causes attempts
by scripts that want to junk the working directory (typically
continuous integration) to fail on permission errors.
Fixup those permissions after the fact.
Message-Id: <20180719163258.4393-1-avi@scylladb.com>
Use is_debian()/is_ubuntu() to detect target distribution, also install
pystache by path since package name is different between Fedora and
CentOS.
Signed-off-by: Takuya ASADA <syuu@scylladb.com>
Message-Id: <20180703183211.3455-1-syuu@scylladb.com>
Currently we are using *.in files for templates, applying parameters by sed
command one-by-one.
This patch will replace them by Mustache, it's simple and easy syntax template
language.
Signed-off-by: Takuya ASADA <syuu@scylladb.com>
Message-Id: <20180606183113.25275-1-syuu@scylladb.com>
On Ubuntu 18.04 build fails by java.security.InvalidAlgorithmParameterException,
while downloading .pom file from HTTPS URL.
Looks like it's ca-certs problem, can fix by running update-ca-certificates.
Fixed#52
Signed-off-by: Takuya ASADA <syuu@scylladb.com>
Message-Id: <20180601092954.28319-1-syuu@scylladb.com>
Before we were discarding the initial repository while
overriding it with TableRepository. This was a mistake that
caused dtests to fail. Proper solution is to keep the initial
repository inside TableRepository. That way whatever was registered
at the time of JmxMBeanServer creation is still handled properly.
Signed-off-by: Piotr Jastrzebski <piotr@scylladb.com>
Message-Id: <22181859012fd20ddf37e049a145bc94a3a91a33.1527844328.git.piotr@scylladb.com>
"
Use more memory efficient MBeans repository and remove quadratic behaviour on startup.
This reduces memory usage for 2000 tables from 127M to 82M and reduces start time
from 270 seconds to 2 seconds.
Changes since last version:
1. Fix registered map to handle multiple JMX servers and to properly deregister mbeans
2. Clean up TableRepository code.
"
* 'speedup_2' of https://github.com/haaawk/scylla-jmx:
Use more efficient MBeans repository
Remove unnecessary quadratic algorithm from MetricsMBean.register
"
Introduce more memory efficient version of ObjectName and use it in JMX Server.
The original version stores the same data multiple times in different forms.
Big part of data is shared by multiple instances of ObjectName.
Original class keeps a separate copy for each instance.
The new version keeps only one copy that's shared by all instances.
"
* 'speedup_1' of https://github.com/haaawk/scylla-jmx:
Introduce and use TableMetricObjectName
Ensure regular ObjectName is returned to remote callers
Use JmxMBeanServer instead of MBeanServer
Default implementation stores MBeans in the following map:
<domain name> -> (<properties as a single string> -> NamedObject)
This is problematic because NamedObject contains ObjectName that
has both domain and properties inside itself.
This means we're storing the same data twice.
For domain "" we want to store MBeans in a more compact way using map:
ObjectName -> DynamicMBean
which is equivalent to NamedObject.
Signed-off-by: Piotr Jastrzebski <piotr@scylladb.com>
Before this change it was taking JMX Server 270 seconds to start
when Scylla had 2000 tables. After the change it takes only 2 seconds.
Signed-off-by: Piotr Jastrzebski <piotr@scylladb.com>
This is a new extention of ObjectName that uses less memory.
TableMetricNameFactory and AllTableMetricNameFactory can
create it instead of regular ObjectName to save memory.
It is possible to save memory because each name created by
TableMetricNameFactory (or AllTableMetricNameFactory) shares
most of its data with other names created by the same factory
and there's no need to create multiple copies.
Signed-off-by: Piotr Jastrzebski <piotr@scylladb.com>
Next patch will introduce new ObjectName implementation that
will use less memory. This new object won't be serializable.
This means it won't be possible to transport it to a remote
caller. We want to keep this new object local to JMX server as well.
This patch makes sure that every ObjectName returned
from APIBeanServer is transformed into a regular ObjectName.
It also makes sure that every ObjectInstance returned from
APIBeanServer has its ObjectName swapped with a regular ObjectName.
Signed-off-by: Piotr Jastrzebski <piotr@scylladb.com>
JmxMBeanServer is a concrete implementation of a MBeanServer.
We want to use it directly because we need to bypass calls to
JmxMBeanServer.registerMBean and JmxMBeanServer.unregisterMBean.
They take ObjectName as parameter, copy it using
ObjectName.getInstance(ObjectName) and pass it to registerMBean
and unregisterMBean of JmxMBeanServer.getMBeanServerInterceptor().
We want to avoid this copy and pass the ObjectName argument directly
to JmxMBeanServer.getMBeanServerInterceptor().
To do that this patch:
1. changes all MBeanServer variables to JmxMBeanServer
2. creates JmxMBeanServer in APIBuilder making sure accessing
interceptors is allowed
3. makes sure that JmxMBeanServer.getMBeanServerInterceptor().registerMBean
is called wherever JmxMBeanServer.registerMBean was called
4. makes sure that JmxMBeanServer.getMBeanServerInterceptor().unregisterMBean
is called whenever JmxMBeanServer.unregisterMBean was called
Next patch will use different ObjectName implementation that will
use less memory and this patch is crucial because without it every ObjectName
is transformed with ObjectName.getInstance which turns the object into
a regular ObjectName.
Signed-off-by: Piotr Jastrzebski <piotr@scylladb.com>
The default epel-7-x86_64 wisely disables networking, however our
maven build accesses the maven repository during the build process.
Recognize the target name and redirect it to our networking-enabled
configuration.
Message-Id: <20180408122138.16672-1-avi@scylladb.com>
When a node is part of a cluster but is down (like in the situation where
a cluster is taken down and up again but not all nodes are up). There is
no application_state information for that node.
This patch check that the information exists before using it to prevent
null pointer exception.
After this patch, a call to nodetool gossipinfo would return the
available information without failing.
See scylladb/scylla#3330
Signed-off-by: Amnon Heiman <amnon@scylladb.com>
Message-Id: <20180329115345.29357-1-amnon@scylladb.com>
This patch renames all the packages generated from this repository from
"scylla" prefix to "scylla-enterprise" prefix. The changes are ported
from the 2017.1 enterprise repository.
Message-Id: <20180326132049.32320-1-penberg@scylladb.com>
We supported Ubuntu 18.04 on scylla-server, need to support on jmx/tools too.
Signed-off-by: Takuya ASADA <syuu@scylladb.com>
Message-Id: <1521189263-17592-1-git-send-email-syuu@scylladb.com>
ubuntu 14 default shell does not respect the string substituation and return an
error when using -Dcom.sun.management.jmxremote.host flag with
scyll-jmx.
For example:
$ scyll-jmx -Dcom.sun.management.jmxremote.host=10.0.0.1
/usr/lib/scylla/jmx/scylla-jmx: 101: /usr/lib/scylla/jmx/scylla-jmx: Bad substitution
This patch change the shell interpreter to bash, instead of the default shell.
Fixes#49
Signed-off-by: Amnon Heiman <amnon@scylladb.com>
Message-Id: <20180129131602.7380-1-amnon@scylladb.com>
Since we want to support cross building, we shouldn't hardcode GPG file path,
even these files provided on recent version of mock.
This fixes build error on some older build environment such as CentOS-7.2.
See scylladb/scylla#3002
Signed-off-by: Takuya ASADA <syuu@scylladb.com>
Message-Id: <1512668704-6775-1-git-send-email-syuu@scylladb.com>
Since we run pbuilder as root, .deb packages owned by root user so we need to
run 'rm -rf build' as root as well.
Signed-off-by: Takuya ASADA <syuu@scylladb.com>
Message-Id: <1512373941-7018-1-git-send-email-syuu@scylladb.com>
Examples:
# scripts/scylla-jmx -l /usr/lib/scylla/jmx ""
Result: script stuck
# scripts/scylla-jmx "" -l /usr/lib/scylla/jmx
Unknown parameter: /usr/lib/scylla/jmx
Result: wrongly shift arguments
Above two problem caused by a redundant argument parse ("$PARAM_PORT"),
the variable isn't set, so it's a covert empty string. The others valid
9 options are all parsed with right case, and API_ADDR also be set by
other case path, so it's safe to remove $PARAM_PORT.
This patch removed the redundant argument, and skipped empty string in
arguments.
Signed-off-by: Amos Kong <amos@scylladb.com>
Message-Id: <74c3f75d58bab1a8348f2c87f58825eed4c5b705.1510501133.git.amos@scylladb.com>
This commit e80a5e3cb3 introduced an issue,
it wrongly passes an empty string to scylla-jmx cmdline, which causes
scylla-jmx script stuck. The cmdline real executes is:
# scylla-jmx -l /usr/lib/scylla/jmx ""
Wrapped quotation marks of env variable can't be parsed away if we use
${SCYLLA_PARAMS} in ExecStart cmdline, but $SCYLLA_PARAMS works.
Another problem is the variable can't be re-used insider env file,
reference: [1]. Let's split the parameters to multiple env variables,
and combine them in ExecStart cmdline.
[1] https://unix.stackexchange.com/questions/358998/systemd-environmentfile-re-using-variables-howFixesscylladb/scylla#2935
Signed-off-by: Amos Kong <amos@scylladb.com>
Message-Id: <c983103c08a3f901037fd282a14df5bb7f85dddd.1510494507.git.amos@scylladb.com>
Using systemd to run scylla-jmx won't load params from
/etc/defaults/scylla-jmx because it expects params to be sent using the
command line.
This patch enhances the sysconfig file by adding the available options
(commented) and passes the right options to systemd `ExecStart` when
defined. That way scylla-jmx is ran with the params defined into
/etc/defaults/scylla-jmx.
Maybe not the most elegant way to do it, but 1/ it works 2/ I didn't
find a better solution to fix that problem.
Message-Id: <20170927085236.6704-1-fdevillamil@synthesio.com>
"Column family is not being passed, so the requests fail to reach the
correct endpoint."
* 'compact' of git://github.com/glommer/scylla-jmx:
fix auto_compaction request strings
We are moving to aptly to release .deb packages, that requires debian repository
structure changes.
After the change, we will share 'pool' directory between distributions.
However, our .deb package name on specific release is exactly same between
distributions, so we have file name confliction.
To avoid the problem, we need to append distribution name on package version.
Signed-off-by: Takuya ASADA <syuu@scylladb.com>
Message-Id: <1502351777-11559-1-git-send-email-syuu@scylladb.com>
"This patchset makes sure that we provide a mock configuration file and won't rely
on defaults that might not work for us"
* 'ourmock' of http://github.com/glommer/scylla-jmx:
default to our in-tree mock config when building on CentOS
provide mock configuration file
scylla.git does a similar thing, albeit in a more complicated fashion,
testing for whether or not a rebuild is asked for and etc.
For us, the build process is a lot simpler, so it is better to just
point to the file when we detect that we are on CentOS and no explicit
target is given.
Signed-off-by: Glauber Costa <glauber@scylladb.com>
This adds the ability to build CentOS packages from Fedora - scylla
already has it. Aside from that, this guarantees that the build will
work in any system. In particular, by default rpmbuild is not given
network access in some of the systems I tested, causing the build to
fail as it tries to contact the maven repo.
This file is just a copy of the one provided by Centos' mock, with
the option to allow rpmbuild network added.
Signed-off-by: Glauber Costa <glauber@scylladb.com>
Since we switched to pbuilder and supported cross build, we no longer able to
use $DISTRIBUTION and $RELEASE in the script.
Use $TARGET instead.
Signed-off-by: Takuya ASADA <syuu@scylladb.com>
Message-Id: <1500540534-27951-1-git-send-email-syuu@scylladb.com>
