Commit 92761964 authored by Paras Garg's avatar Paras Garg

1. Added JNI

2. Added YSCB basic code
3. Added future for client
4. Added properties in client
parent b9e98c52
......@@ -45,9 +45,10 @@ count:
Jclient: $(OBJS)
$(CXX) -o libHelloImpl.so -shared .build/Hello.o $(LIBS)
$(CXX) -o libhpdosclient.so -L/usr/local/lib -shared $^ $(LIBS)
@echo "jclient "$<" successfully!"
sudo cp libhpdosclient.so /usr/lib
@echo "Copied libhpdosclient.so to /usr/lib"
#jcompile: javac $(JSRCS) -d JBUILD
JniHeader:
......
Steps to build jni client
> make JniHeader <br>
> make JClient <br>
> java -cp jsrc JClient
Running YSCB
> mvn compile <br>
./bin/ycsb load hpdos -P workloads/workloadb -threads 1
mvn -pl site.ycsb:hpdos-binding -am clean package -Dcheckstyle.skip
to do
delete client endpoint on close
threading in client and hashing in client
resolve double delete for get and put
add cache add support for invalidation
interface client API through endpointGroup
Endpointgroup to manage list of servers and caches, Invalidation
./bin/ycsb shell hpdos
./bin/ycsb run hpdos -P workloads/workloada
./bin/ycsb load hpdos -P workloads/workloada
Options:
-P file Specify workload file
-cp path Additional Java classpath entries
-jvm-args args Additional arguments to the JVM
-p key=value Override workload property
-s Print status to stderr
-target n Target ops/sec (default: unthrottled)
-threads n Number of client threads (default: 1)
\ No newline at end of file
# Copyright (c) 2015, 2017 YCSB contributors.
# All rights reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License"); you
# may not use this file except in compliance with the License. You
# may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
# implied. See the License for the specific language governing
# permissions and limitations under the License. See accompanying
# LICENSE file.
# For more info, see: http://EditorConfig.org
root = true
[*.java]
indent_style = space
indent_size = 2
continuation_indent_size = 4
[*.md]
indent_style = space
indent_size = 2
continuation_indent_size = 4
[*.xml]
indent_style = space
indent_size = 2
continuation_indent_size = 4
# ignore compiled byte code
target
# ignore output files from testing
output*
# ignore standard Eclipse files
.project
.classpath
.settings
.checkstyle
# ignore standard IntelliJ files
.idea/
*.iml
*.ipr
*.iws
# ignore standard Vim and Emacs temp files
*.swp
*~
# ignore standard Mac OS X files/dirs
.DS_Store
/differentbin/
# Copyright (c) 2010 Yahoo! Inc., 2012 - 2015 YCSB contributors.
# All rights reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License"); you
# may not use this file except in compliance with the License. You
# may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
# implied. See the License for the specific language governing
# permissions and limitations under the License. See accompanying
# LICENSE file.
# more info here about TravisCI and Java projects
# http://docs.travis-ci.com/user/languages/java/
language: java
jdk:
- openjdk8
- openjdk11
- oraclejdk11
addons:
hosts:
- myshorthost
hostname: myshorthost
postgresql: "9.5"
install:
- mvn -N io.takari:maven:0.7.7:wrapper -Dmaven=3.6.3
- ./mvnw install -q -DskipTests=true
script: ./mvnw test -q
before_script:
- psql -c 'CREATE database test;' -U postgres
- psql -c 'CREATE TABLE usertable (YCSB_KEY VARCHAR(255) PRIMARY KEY not NULL, YCSB_VALUE JSONB not NULL);' -U postgres -d test
- psql -c 'GRANT ALL PRIVILEGES ON DATABASE test to postgres;' -U postgres
# Services to start for tests.
services:
- ignite
- mongodb
- postgresql
# temporarily disable riak. failing, docs offline.
# - riak
# Can't use container based infra because of hosts/hostname
sudo: true
This diff is collapsed.
This diff is collapsed.
=========================================================================
NOTICE file for use with, and corresponding to Section 4 of,
the Apache License, Version 2.0,
in this case for the YCSB project.
=========================================================================
This product includes software developed by
Yahoo! Inc. (www.yahoo.com)
Copyright (c) 2010 Yahoo! Inc. All rights reserved.
This product includes software developed by
Google Inc. (www.google.com)
Copyright (c) 2015 Google Inc. All rights reserved.
<!--
Copyright (c) 2010 Yahoo! Inc., 2012 - 2016 YCSB contributors.
All rights reserved.
Licensed under the Apache License, Version 2.0 (the "License"); you
may not use this file except in compliance with the License. You
may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
implied. See the License for the specific language governing
permissions and limitations under the License. See accompanying
LICENSE file.
-->
YCSB
====================================
[![Build Status](https://travis-ci.org/brianfrankcooper/YCSB.png?branch=master)](https://travis-ci.org/brianfrankcooper/YCSB)
Links
-----
* To get here, use https://ycsb.site
* [Our project docs](https://github.com/brianfrankcooper/YCSB/wiki)
* [The original announcement from Yahoo!](https://labs.yahoo.com/news/yahoo-cloud-serving-benchmark/)
Getting Started
---------------
1. Download the [latest release of YCSB](https://github.com/brianfrankcooper/YCSB/releases/latest):
```sh
curl -O --location https://github.com/brianfrankcooper/YCSB/releases/download/0.17.0/ycsb-0.17.0.tar.gz
tar xfvz ycsb-0.17.0.tar.gz
cd ycsb-0.17.0
```
2. Set up a database to benchmark. There is a README file under each binding
directory.
3. Run YCSB command.
On Linux:
```sh
bin/ycsb.sh load basic -P workloads/workloada
bin/ycsb.sh run basic -P workloads/workloada
```
On Windows:
```bat
bin/ycsb.bat load basic -P workloads\workloada
bin/ycsb.bat run basic -P workloads\workloada
```
Running the `ycsb` command without any argument will print the usage.
See https://github.com/brianfrankcooper/YCSB/wiki/Running-a-Workload
for a detailed documentation on how to run a workload.
See https://github.com/brianfrankcooper/YCSB/wiki/Core-Properties for
the list of available workload properties.
Building from source
--------------------
YCSB requires the use of Maven 3; if you use Maven 2, you may see [errors
such as these](https://github.com/brianfrankcooper/YCSB/issues/406).
To build the full distribution, with all database bindings:
mvn clean package
To build a single database binding:
mvn -pl site.ycsb:mongodb-binding -am clean package
<!--
Copyright (c) 2015 YCSB contributors. All rights reserved.
Licensed under the Apache License, Version 2.0 (the "License"); you
may not use this file except in compliance with the License. You
may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
implied. See the License for the specific language governing
permissions and limitations under the License. See accompanying
LICENSE file.
-->
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<parent>
<groupId>site.ycsb</groupId>
<artifactId>root</artifactId>
<version>0.18.0-SNAPSHOT</version>
<relativePath>../../</relativePath>
</parent>
<artifactId>datastore-specific-descriptor</artifactId>
<name>Per Datastore Binding descriptor</name>
<packaging>jar</packaging>
<description>
This module contains the assembly descriptor used by the individual components
to build binding-specific distributions.
</description>
<dependencies>
<dependency>
<groupId>site.ycsb</groupId>
<artifactId>core</artifactId>
<version>${project.version}</version>
</dependency>
</dependencies>
</project>
<!--
Copyright (c) 2015 YCSB contributors. All rights reserved.
Licensed under the Apache License, Version 2.0 (the "License"); you
may not use this file except in compliance with the License. You
may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
implied. See the License for the specific language governing
permissions and limitations under the License. See accompanying
LICENSE file.
-->
<assembly xmlns="http://maven.apache.org/plugins/maven-assembly-plugin/assembly/1.1.2" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/plugins/maven-assembly-plugin/assembly/1.1.2 http://maven.apache.org/xsd/assembly-1.1.2.xsd">
<id>dist</id>
<includeBaseDirectory>true</includeBaseDirectory>
<baseDirectory>ycsb-${artifactId}-${version}</baseDirectory>
<files>
<file>
<source>README.md</source>
<outputDirectory></outputDirectory>
</file>
</files>
<fileSets>
<fileSet>
<directory>..</directory>
<outputDirectory></outputDirectory>
<fileMode>0644</fileMode>
<includes>
<include>LICENSE.txt</include>
<include>NOTICE.txt</include>
</includes>
</fileSet>
<fileSet>
<directory>../bin</directory>
<outputDirectory>bin</outputDirectory>
<fileMode>0755</fileMode>
<includes>
<include>ycsb*</include>
</includes>
</fileSet>
<fileSet>
<directory>../bin</directory>
<outputDirectory>bin</outputDirectory>
<fileMode>0644</fileMode>
<includes>
<include>bindings.properties</include>
</includes>
</fileSet>
<fileSet>
<directory>../workloads</directory>
<outputDirectory>workloads</outputDirectory>
<fileMode>0644</fileMode>
</fileSet>
<fileSet>
<directory>src/main/conf</directory>
<outputDirectory>conf</outputDirectory>
<fileMode>0644</fileMode>
</fileSet>
</fileSets>
<dependencySets>
<dependencySet>
<outputDirectory>lib</outputDirectory>
<includes>
<include>site.ycsb:core</include>
</includes>
<scope>provided</scope>
<useTransitiveFiltering>true</useTransitiveFiltering>
</dependencySet>
<dependencySet>
<outputDirectory>lib</outputDirectory>
<includes>
<include>*:jar:*</include>
</includes>
<excludes>
<exclude>*:sources</exclude>
</excludes>
</dependencySet>
</dependencySets>
</assembly>
<!--
Copyright (c) 2015-2016 YCSB contributors. All rights reserved.
Licensed under the Apache License, Version 2.0 (the "License"); you
may not use this file except in compliance with the License. You
may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
implied. See the License for the specific language governing
permissions and limitations under the License. See accompanying
LICENSE file.
-->
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<parent>
<groupId>site.ycsb</groupId>
<artifactId>root</artifactId>
<version>0.18.0-SNAPSHOT</version>
</parent>
<artifactId>binding-parent</artifactId>
<name>YCSB Datastore Binding Parent</name>
<packaging>pom</packaging>
<description>
This module acts as the parent for new datastore bindings.
It creates a datastore specific binary artifact.
</description>
<modules>
<module>datastore-specific-descriptor</module>
</modules>
<properties>
<!-- See the test-on-jdk9 profile below. Default to 'jdk9 works' -->
<skipJDK9Tests>false</skipJDK9Tests>
<skipJDK10Tests>false</skipJDK10Tests>
<skipJDK11Tests>false</skipJDK11Tests>
</properties>
<build>
<pluginManagement>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-assembly-plugin</artifactId>
<version>${maven.assembly.version}</version>
<dependencies>
<dependency>
<groupId>site.ycsb</groupId>
<artifactId>datastore-specific-descriptor</artifactId>
<version>${project.version}</version>
</dependency>
</dependencies>
<configuration>
<descriptorRefs>
<descriptorRef>datastore-specific-assembly</descriptorRef>
</descriptorRefs>
<finalName>ycsb-${project.artifactId}-${project.version}</finalName>
<formats>
<format>tar.gz</format>
</formats>
<appendAssemblyId>false</appendAssemblyId>
<tarLongFileMode>posix</tarLongFileMode>
</configuration>
<executions>
<execution>
<phase>package</phase>
<goals>
<goal>single</goal>
</goals>
</execution>
</executions>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-checkstyle-plugin</artifactId>
<executions>
<execution>
<id>validate</id>
<configuration>
<configLocation>../checkstyle.xml</configLocation>
</configuration>
</execution>
</executions>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-dependency-plugin</artifactId>
<version>${maven.dependency.version}</version>
</plugin>
</plugins>
</pluginManagement>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-dependency-plugin</artifactId>
<executions>
<execution>
<id>stage-dependencies</id>
<phase>package</phase>
<goals>
<goal>copy-dependencies</goal>
</goals>
<configuration>
<includeScope>runtime</includeScope>
</configuration>
</execution>
</executions>
</plugin>
</plugins>
</build>
<profiles>
<!-- If the binding defines a README, presume we should make an assembly. -->
<profile>
<id>datastore-binding</id>
<activation>
<file>
<exists>README.md</exists>
</file>
</activation>
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-assembly-plugin</artifactId>
</plugin>
</plugins>
</build>
</profile>
<!-- If the binding doesn't work with jdk9, it should redefine the
skipJDK9 property
-->
<profile>
<id>tests-on-jdk9</id>
<activation>
<jdk>9</jdk>
</activation>
<properties>
<skipTests>${skipJDK9Tests}</skipTests>
</properties>
</profile>
<!-- If the binding doesn't work with jdk10, it should redefine the
skipJDK10 property
-->
<profile>
<id>tests-on-jdk10</id>
<activation>
<jdk>10</jdk>
</activation>
<properties>
<skipTests>${skipJDK10Tests}</skipTests>
</properties>
</profile>
<!-- If the binding doesn't work with jdk11, it should redefine the
skipJDK11 property
-->
<profile>
<id>tests-on-jdk11</id>
<activation>
<jdk>11</jdk>
</activation>
<properties>
<skipTests>${skipJDK11Tests}</skipTests>
</properties>
</profile>
<!-- When doing a YCSB release, we want to make sure specific bindings aren't included in the maven repo -->
<profile>
<id>ycsb-release</id>
<properties>
<!-- Set the deploy plugin to skip wherever this property is inherited -->
<maven.deploy.skip>true</maven.deploy.skip>
</properties>
<build>
<plugins>
<!-- We still want to make sure that *this* module gets deployed. -->
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-deploy-plugin</artifactId>
<executions>
<execution>
<id>but-still-deploy-the-binding-parent</id>
<goals>
<goal>deploy</goal>
</goals>
<phase>deploy</phase>
<inherited>false</inherited>
<configuration>
<skip>false</skip>
</configuration>
</execution>
</executions>
</plugin>
</plugins>
</build>
</profile>
</profiles>
</project>
<!--
Copyright (c) 2015 YCSB contributors. All rights reserved.
Licensed under the Apache License, Version 2.0 (the "License"); you
may not use this file except in compliance with the License. You
may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
implied. See the License for the specific language governing
permissions and limitations under the License. See accompanying
LICENSE file.
-->
# Apache Cassandra 2.x CQL binding
Binding for [Apache Cassandra](http://cassandra.apache.org), using the CQL API
via the [DataStax
driver](http://docs.datastax.com/en/developer/java-driver/2.1/java-driver/whatsNew2.html).
To run against the (deprecated) Cassandra Thrift API, use the `cassandra-10` binding.
## Creating a table for use with YCSB
For keyspace `ycsb`, table `usertable`:
cqlsh> create keyspace ycsb
WITH REPLICATION = {'class' : 'SimpleStrategy', 'replication_factor': 3 };
cqlsh> USE ycsb;
cqlsh> create table usertable (
y_id varchar primary key,
field0 varchar,
field1 varchar,
field2 varchar,
field3 varchar,
field4 varchar,
field5 varchar,
field6 varchar,
field7 varchar,
field8 varchar,
field9 varchar);
**Note that `replication_factor` and consistency levels (below) will affect performance.**
## Cassandra Configuration Parameters
- `hosts` (**required**)
- Cassandra nodes to connect to.
- No default.
* `port`
* CQL port for communicating with Cassandra cluster.
* Default is `9042`.
- `cassandra.keyspace`
Keyspace name - must match the keyspace for the table created (see above).
See http://docs.datastax.com/en/cql/3.1/cql/cql_reference/create_keyspace_r.html for details.
- Default value is `ycsb`
- `cassandra.username`
- `cassandra.password`
- Optional user name and password for authentication. See http://docs.datastax.com/en/cassandra/2.0/cassandra/security/security_config_native_authenticate_t.html for details.
* `cassandra.readconsistencylevel`
* `cassandra.writeconsistencylevel`
* Default value is `QUORUM`
- Consistency level for reads and writes, respectively. See the [DataStax documentation](http://docs.datastax.com/en/cassandra/2.0/cassandra/dml/dml_config_consistency_c.html) for details.
* `cassandra.maxconnections`
* `cassandra.coreconnections`
* Defaults for max and core connections can be found here: https://datastax.github.io/java-driver/2.1.8/features/pooling/#pool-size. Cassandra 2.0.X falls under protocol V2, Cassandra 2.1+ falls under protocol V3.
* `cassandra.connecttimeoutmillis`
* `cassandra.useSSL`
* Default value is false.
- To connect with SSL set this value to true.
* `cassandra.readtimeoutmillis`
* Defaults for connect and read timeouts can be found here: https://docs.datastax.com/en/drivers/java/2.0/com/datastax/driver/core/SocketOptions.html.
* `cassandra.tracing`
* Default is false
* https://docs.datastax.com/en/cql/3.3/cql/cql_reference/tracing_r.html
<?xml version="1.0" encoding="UTF-8"?>
<!--
Copyright (c) 2012-2016 YCSB contributors. All rights reserved.
Licensed under the Apache License, Version 2.0 (the "License"); you
may not use this file except in compliance with the License. You
may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
implied. See the License for the specific language governing
permissions and limitations under the License. See accompanying
LICENSE file.
-->
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">
<modelVersion>4.0.0</modelVersion>
<parent>
<groupId>site.ycsb</groupId>
<artifactId>binding-parent</artifactId>
<version>0.18.0-SNAPSHOT</version>
<relativePath>../binding-parent</relativePath>
</parent>
<artifactId>cassandra-binding</artifactId>
<name>Cassandra 2.1+ DB Binding</name>
<packaging>jar</packaging>
<properties>
<!-- Skip tests by default. will be activated by jdk8 profile -->
<skipTests>true</skipTests>
</properties>
<dependencies>
<!-- CQL driver -->
<dependency>
<groupId>com.datastax.cassandra</groupId>
<artifactId>cassandra-driver-core</artifactId>
<version>${cassandra.cql.version}</version>
</dependency>
<dependency>
<groupId>site.ycsb</groupId>
<artifactId>core</artifactId>
<version>${project.version}</version>
<scope>provided</scope>
</dependency>
<dependency>
<groupId>org.cassandraunit</groupId>
<artifactId>cassandra-unit</artifactId>
<version>3.0.0.1</version>
<classifier>shaded</classifier>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.slf4j</groupId>
<artifactId>slf4j-simple</artifactId>
<version>1.7.21</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
<version>4.12</version>
<scope>test</scope>
</dependency>
<!-- only for Cassandra test (Cassandra 2.2+ uses Sigar for collecting system information, and Sigar requires some native lib files) -->
<dependency>
<groupId>org.hyperic</groupId>
<artifactId>sigar-dist</artifactId>
<version>1.6.4.129</version>
<type>zip</type>
<scope>test</scope>
</dependency>
</dependencies>
<profiles>
<!-- Cassandra 2.2+ requires JDK8 to run, so none of our tests
will work unless we're using jdk8.
-->
<profile>
<id>jdk8-tests</id>
<activation>
<jdk>1.8</jdk>
</activation>
<properties>
<skipTests>false</skipTests>
</properties>
</profile>
</profiles>
<!-- sigar-dist can be downloaded from jboss repository -->
<repositories>
<repository>
<id>central2</id>
<name>sigar Repository</name>
<url>https://repository.jboss.org/nexus/content/groups/public-jboss/</url>
<layout>default</layout>
<snapshots>
<enabled>false</enabled>
</snapshots>
</repository>
</repositories>
<!-- unzip sigar-dist/lib files.
References:
http://stackoverflow.com/questions/5388661/unzip-dependency-in-maven
https://arviarya.wordpress.com/2013/09/22/sigar-access-operating-system-and-hardware-level-information/
-->
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-dependency-plugin</artifactId>
<executions>
<execution>
<id>unpack-sigar</id>
<phase>process-test-resources<!-- or any other valid maven phase --></phase>
<goals>
<goal>unpack-dependencies</goal>
</goals>
<configuration>
<includeGroupIds>org.hyperic</includeGroupIds>
<includeArtifactIds>sigar-dist</includeArtifactIds>
<includes>**/sigar-bin/lib/*</includes>
<excludes>**/sigar-bin/lib/*jar</excludes>
<outputDirectory>
${project.build.directory}/cassandra-dependency
</outputDirectory>
</configuration>
</execution>
</executions>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.8</version>
<configuration>
<argLine>-Djava.library.path=${project.build.directory}/cassandra-dependency/hyperic-sigar-1.6.4/sigar-bin/lib</argLine>
</configuration>
</plugin>
</plugins>
</build>
</project>
/*
* Copyright (c) 2014, Yahoo!, Inc. All rights reserved.
*
* Licensed under the Apache License, Version 2.0 (the "License"); you
* may not use this file except in compliance with the License. You
* may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
* implied. See the License for the specific language governing
* permissions and limitations under the License. See accompanying
* LICENSE file.
*/
/**
* The YCSB binding for <a href="http://cassandra.apache.org/">Cassandra</a>
* 2.1+ via CQL.
*/
package site.ycsb.db;
/**
* Copyright (c) 2015 YCSB contributors All rights reserved.
*
* Licensed under the Apache License, Version 2.0 (the "License"); you
* may not use this file except in compliance with the License. You
* may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
* implied. See the License for the specific language governing
* permissions and limitations under the License. See accompanying
* LICENSE file.
*/
package site.ycsb.db;
import static org.hamcrest.MatcherAssert.assertThat;
import static org.hamcrest.Matchers.hasEntry;
import static org.hamcrest.Matchers.hasSize;
import static org.hamcrest.Matchers.is;
import static org.hamcrest.Matchers.notNullValue;
import static org.hamcrest.Matchers.nullValue;
import com.google.common.collect.Sets;
import com.datastax.driver.core.ResultSet;
import com.datastax.driver.core.Row;
import com.datastax.driver.core.Session;
import com.datastax.driver.core.Statement;
import com.datastax.driver.core.querybuilder.Insert;
import com.datastax.driver.core.querybuilder.QueryBuilder;
import com.datastax.driver.core.querybuilder.Select;
import site.ycsb.ByteIterator;
import site.ycsb.Status;
import site.ycsb.StringByteIterator;
import site.ycsb.measurements.Measurements;
import site.ycsb.workloads.CoreWorkload;
import org.cassandraunit.CassandraCQLUnit;
import org.cassandraunit.dataset.cql.ClassPathCQLDataSet;
import org.junit.After;
import org.junit.Before;
import org.junit.ClassRule;
import org.junit.Test;
import java.util.HashMap;
import java.util.Map;
import java.util.Properties;
import java.util.Set;
/**
* Integration tests for the Cassandra client
*/
public class CassandraCQLClientTest {
// Change the default Cassandra timeout from 10s to 120s for slow CI machines
private final static long timeout = 120000L;
private final static String TABLE = "usertable";
private final static String HOST = "localhost";
private final static int PORT = 9142;
private final static String DEFAULT_ROW_KEY = "user1";
private CassandraCQLClient client;
private Session session;
@ClassRule
public static CassandraCQLUnit cassandraUnit = new CassandraCQLUnit(
new ClassPathCQLDataSet("ycsb.cql", "ycsb"), null, timeout);
@Before
public void setUp() throws Exception {
session = cassandraUnit.getSession();
Properties p = new Properties();
p.setProperty("hosts", HOST);
p.setProperty("port", Integer.toString(PORT));
p.setProperty("table", TABLE);
Measurements.setProperties(p);
final CoreWorkload workload = new CoreWorkload();
workload.init(p);
client = new CassandraCQLClient();
client.setProperties(p);
client.init();
}
@After
public void tearDownClient() throws Exception {
if (client != null) {
client.cleanup();
}
client = null;
}
@After
public void clearTable() throws Exception {
// Clear the table so that each test starts fresh.
final Statement truncate = QueryBuilder.truncate(TABLE);
if (cassandraUnit != null) {
cassandraUnit.getSession().execute(truncate);
}
}
@Test
public void testReadMissingRow() throws Exception {
final HashMap<String, ByteIterator> result = new HashMap<String, ByteIterator>();
final Status status = client.read(TABLE, "Missing row", null, result);
assertThat(result.size(), is(0));
assertThat(status, is(Status.NOT_FOUND));
}
private void insertRow() {
final String rowKey = DEFAULT_ROW_KEY;
Insert insertStmt = QueryBuilder.insertInto(TABLE);
insertStmt.value(CassandraCQLClient.YCSB_KEY, rowKey);
insertStmt.value("field0", "value1");
insertStmt.value("field1", "value2");
session.execute(insertStmt);
}
@Test
public void testRead() throws Exception {
insertRow();
final HashMap<String, ByteIterator> result = new HashMap<String, ByteIterator>();
final Status status = client.read(TABLE, DEFAULT_ROW_KEY, null, result);
assertThat(status, is(Status.OK));
assertThat(result.entrySet(), hasSize(11));
assertThat(result, hasEntry("field2", null));
final HashMap<String, String> strResult = new HashMap<String, String>();
for (final Map.Entry<String, ByteIterator> e : result.entrySet()) {
if (e.getValue() != null) {
strResult.put(e.getKey(), e.getValue().toString());
}
}
assertThat(strResult, hasEntry(CassandraCQLClient.YCSB_KEY, DEFAULT_ROW_KEY));
assertThat(strResult, hasEntry("field0", "value1"));
assertThat(strResult, hasEntry("field1", "value2"));
}
@Test
public void testReadSingleColumn() throws Exception {
insertRow();
final HashMap<String, ByteIterator> result = new HashMap<String, ByteIterator>();
final Set<String> fields = Sets.newHashSet("field1");
final Status status = client.read(TABLE, DEFAULT_ROW_KEY, fields, result);
assertThat(status, is(Status.OK));
assertThat(result.entrySet(), hasSize(1));
final Map<String, String> strResult = StringByteIterator.getStringMap(result);
assertThat(strResult, hasEntry("field1", "value2"));
}
@Test
public void testInsert() throws Exception {
final String key = "key";
final Map<String, String> input = new HashMap<String, String>();
input.put("field0", "value1");
input.put("field1", "value2");
final Status status = client.insert(TABLE, key, StringByteIterator.getByteIteratorMap(input));
assertThat(status, is(Status.OK));
// Verify result
final Select selectStmt =
QueryBuilder.select("field0", "field1")
.from(TABLE)
.where(QueryBuilder.eq(CassandraCQLClient.YCSB_KEY, key))
.limit(1);
final ResultSet rs = session.execute(selectStmt);
final Row row = rs.one();
assertThat(row, notNullValue());
assertThat(rs.isExhausted(), is(true));
assertThat(row.getString("field0"), is("value1"));
assertThat(row.getString("field1"), is("value2"));
}
@Test
public void testUpdate() throws Exception {
insertRow();
final Map<String, String> input = new HashMap<String, String>();
input.put("field0", "new-value1");
input.put("field1", "new-value2");
final Status status = client.update(TABLE,
DEFAULT_ROW_KEY,
StringByteIterator.getByteIteratorMap(input));
assertThat(status, is(Status.OK));
// Verify result
final Select selectStmt =
QueryBuilder.select("field0", "field1")
.from(TABLE)
.where(QueryBuilder.eq(CassandraCQLClient.YCSB_KEY, DEFAULT_ROW_KEY))
.limit(1);
final ResultSet rs = session.execute(selectStmt);
final Row row = rs.one();
assertThat(row, notNullValue());
assertThat(rs.isExhausted(), is(true));
assertThat(row.getString("field0"), is("new-value1"));
assertThat(row.getString("field1"), is("new-value2"));
}
@Test
public void testDelete() throws Exception {
insertRow();
final Status status = client.delete(TABLE, DEFAULT_ROW_KEY);
assertThat(status, is(Status.OK));
// Verify result
final Select selectStmt =
QueryBuilder.select("field0", "field1")
.from(TABLE)
.where(QueryBuilder.eq(CassandraCQLClient.YCSB_KEY, DEFAULT_ROW_KEY))
.limit(1);
final ResultSet rs = session.execute(selectStmt);
final Row row = rs.one();
assertThat(row, nullValue());
}
@Test
public void testPreparedStatements() throws Exception {
final int LOOP_COUNT = 3;
for (int i = 0; i < LOOP_COUNT; i++) {
testInsert();
testUpdate();
testRead();
testReadSingleColumn();
testReadMissingRow();
testDelete();
}
}
}
/**
* Copyright (c) 2015 YCSB Contributors. All rights reserved.
*
* Licensed under the Apache License, Version 2.0 (the "License"); you
* may not use this file except in compliance with the License. You
* may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
* implied. See the License for the specific language governing
* permissions and limitations under the License. See accompanying
* LICENSE file.
*/
CREATE TABLE usertable (
y_id varchar primary key,
field0 varchar,
field1 varchar,
field2 varchar,
field3 varchar,
field4 varchar,
field5 varchar,
field6 varchar,
field7 varchar,
field8 varchar,
field9 varchar);
<?xml version="1.0"?>
<!--
Copyright (c) 2012 - 2016 YCSB contributors. All rights reserved.
Licensed under the Apache License, Version 2.0 (the "License"); you
may not use this file except in compliance with the License. You
may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
implied. See the License for the specific language governing
permissions and limitations under the License. See accompanying
LICENSE file.
-->
<!DOCTYPE module PUBLIC
"-//Puppy Crawl//DTD Check Configuration 1.2//EN"
"http://www.puppycrawl.com/dtds/configuration_1_2.dtd">
<!--
Checkstyle configuration for Hadoop that is based on the sun_checks.xml file
that is bundled with Checkstyle and includes checks for:
- the Java Language Specification at
http://java.sun.com/docs/books/jls/second_edition/html/index.html
- the Sun Code Conventions at http://java.sun.com/docs/codeconv/
- the Javadoc guidelines at
http://java.sun.com/j2se/javadoc/writingdoccomments/index.html
- the JDK Api documentation http://java.sun.com/j2se/docs/api/index.html
- some best practices
Checkstyle is very configurable. Be sure to read the documentation at
http://checkstyle.sf.net (or in your downloaded distribution).
Most Checks are configurable, be sure to consult the documentation.
To completely disable a check, just comment it out or delete it from the file.
Finally, it is worth reading the documentation.
-->
<module name="Checker">
<!-- Checks that a package.html file exists for each package. -->
<!-- See http://checkstyle.sf.net/config_javadoc.html#PackageHtml -->
<module name="JavadocPackage"/>
<!-- Checks whether files end with a new line. -->
<!-- See http://checkstyle.sf.net/config_misc.html#NewlineAtEndOfFile -->
<!-- module name="NewlineAtEndOfFile"/-->
<!-- Checks that property files contain the same keys. -->
<!-- See http://checkstyle.sf.net/config_misc.html#Translation -->
<module name="Translation"/>
<module name="FileLength"/>
<module name="FileTabCharacter"/>
<module name="TreeWalker">
<!-- Checks for Javadoc comments. -->
<!-- See http://checkstyle.sf.net/config_javadoc.html -->
<module name="JavadocType">
<property name="scope" value="public"/>
<property name="allowMissingParamTags" value="true"/>
<!-- unfortunately we cannot add implNote, implSpec, apiNote and apiSpec to checkstyle -->
<property name="allowUnknownTags" value="true"/>
</module>
<module name="JavadocStyle"/>
<!-- Checks for Naming Conventions. -->
<!-- See http://checkstyle.sf.net/config_naming.html -->
<module name="ConstantName"/>
<module name="LocalFinalVariableName"/>
<module name="LocalVariableName"/>
<module name="MemberName"/>
<module name="MethodName"/>
<module name="PackageName"/>
<module name="ParameterName"/>
<module name="StaticVariableName"/>
<module name="TypeName"/>
<!-- Checks for Headers -->
<!-- See http://checkstyle.sf.net/config_header.html -->
<!-- <module name="Header"> -->
<!-- The follow property value demonstrates the ability -->
<!-- to have access to ANT properties. In this case it uses -->
<!-- the ${basedir} property to allow Checkstyle to be run -->
<!-- from any directory within a project. See property -->
<!-- expansion, -->
<!-- http://checkstyle.sf.net/config.html#properties -->
<!-- <property -->
<!-- name="headerFile" -->
<!-- value="${basedir}/java.header"/> -->
<!-- </module> -->
<!-- Following interprets the header file as regular expressions. -->
<!-- <module name="RegexpHeader"/> -->
<!-- Checks for imports -->
<!-- See http://checkstyle.sf.net/config_import.html -->
<module name="IllegalImport"/> <!-- defaults to sun.* packages -->
<module name="RedundantImport"/>
<module name="UnusedImports"/>
<!-- Checks for Size Violations. -->
<!-- See http://checkstyle.sf.net/config_sizes.html -->
<module name="LineLength">
<property name="max" value="120"/>
</module>
<module name="MethodLength"/>
<module name="ParameterNumber"/>
<!-- Checks for whitespace -->
<!-- See http://checkstyle.sf.net/config_whitespace.html -->
<module name="EmptyForIteratorPad"/>
<module name="MethodParamPad"/>
<module name="NoWhitespaceAfter"/>
<module name="NoWhitespaceBefore"/>
<module name="ParenPad"/>
<module name="TypecastParenPad"/>
<module name="WhitespaceAfter">
<property name="tokens" value="COMMA, SEMI"/>
</module>
<!-- Modifier Checks -->
<!-- See http://checkstyle.sf.net/config_modifiers.html -->
<module name="ModifierOrder"/>
<module name="RedundantModifier"/>
<!-- Checks for blocks. You know, those {}'s -->
<!-- See http://checkstyle.sf.net/config_blocks.html -->
<module name="AvoidNestedBlocks"/>
<module name="EmptyBlock">
<property name="option" value="text"/>
</module>
<module name="LeftCurly"/>
<module name="NeedBraces"/>
<module name="RightCurly"/>
<!-- Checks for common coding problems -->
<!-- See http://checkstyle.sf.net/config_coding.html -->
<!-- module name="AvoidInlineConditionals"/-->
<module name="EmptyStatement"/>
<module name="EqualsHashCode"/>
<module name="HiddenField">
<property name="ignoreConstructorParameter" value="true"/>
</module>
<module name="IllegalInstantiation"/>
<module name="InnerAssignment"/>
<module name="MissingSwitchDefault"/>
<module name="SimplifyBooleanExpression"/>
<module name="SimplifyBooleanReturn"/>
<!-- Checks for class design -->
<!-- See http://checkstyle.sf.net/config_design.html -->
<module name="FinalClass"/>
<module name="HideUtilityClassConstructor"/>
<module name="InterfaceIsType"/>
<module name="VisibilityModifier">
<property name="protectedAllowed" value="true"/>
</module>
<!-- Miscellaneous other checks. -->
<!-- See http://checkstyle.sf.net/config_misc.html -->
<module name="ArrayTypeStyle"/>
<module name="Indentation">
<property name="basicOffset" value="2" />
<property name="caseIndent" value="0" />
</module>
<!-- <module name="TodoComment"/> -->
<module name="UpperEll"/>
</module>
</module>
<!--
Copyright (c) 2015 YCSB contributors. All rights reserved.
Licensed under the Apache License, Version 2.0 (the "License"); you
may not use this file except in compliance with the License. You
may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
implied. See the License for the specific language governing
permissions and limitations under the License. See accompanying
LICENSE file.
-->
When used as a latency under load benchmark YCSB in it's original form suffers from
Coordinated Omission[1] and related measurement issue:
* Load is controlled by response time
* Measurement does not account for missing time
* Measurement starts at beginning of request rather than at intended beginning
* Measurement is limited in scope as the histogram does not provide data on overflow values
To provide a minimal correction patch the following were implemented:
1. Replace internal histogram implementation with HdrHistogram[2]:
HdrHistogram offers a dynamic range of measurement at a given precision and will
improve the fidelity of reporting. It allows capturing a much wider range of latencies.
HdrHistogram also supports compressed loss-less serialization which enable capturing
snapshot histograms from which lower resolution histograms can be constructed for plotting
latency over time. Snapshot interval histograms are serialized on status reporting which
must be enabled using the '-s' option.
2. Track intended operation start and report latencies from that point in time:
Assuming the benchmark sets a target schedule of execution in which every operation
is supposed to happen at a given time the benchmark should measure the latency between
intended start time and operation completion.
This required the introduction of a new measurement point and inevitably
includes measuring some of the internal preparation steps of the load generator.
These overhead should be negligible in the context of a network hop, but could
be corrected for by estimating the load-generator overheads (e.g. by measuring a
no-op DB or by measuring the setup time for an operation and deducting that from total).
This intended measurement point is only used when there is a target load (specified by
the -target paramaeter)
This branch supports the following new options:
* -p measurementtype=[histogram|hdrhistogram|hdrhistogram+histogram|timeseries] (default=histogram)
The new measurement types are hdrhistogram and hdrhistogram+histogram. Default is still
histogram, which is the old histogram. Ultimately we would remove the old measurement types
and use only HdrHistogram but the old measurement is left in there for comparison sake.
* -p measurement.interval=[op|intended|both] (default=op)
This new option deferentiates between measured intervals and adds the intended interval(as described)
above, and the option to record both the op and intended for comparison.
* -p hdrhistogram.fileoutput=[true|false] (default=false)
This new option will enable periodical writes of the interval histogram into an output file. The path can be set using '-p hdrhistogram.output.path=<PATH>'.
Example parameters:
-target 1000 -s -p workload=site.ycsb.workloads.CoreWorkload -p basicdb.verbose=false -p basicdb.simulatedelay=4 -p measurement.interval=both -p measurementtype=hdrhistogram -p hdrhistogram.fileoutput=true -p maxexecutiontime=60
Further changes made:
* -p status.interval=<number of seconds> (default=10)
Controls the number of seconds between status reports and therefore between HdrHistogram snapshots reported.
* -p basicdb.randomizedelay=[true|false] (default=true)
Controls weather the delay simulated by the mock DB is uniformly random or not.
Further suggestions:
1. Correction load control: currently after a pause the load generator will do
operations back to back to catchup, this leads to a flat out throughput mode
of testing as opposed to controlled load.
2. Move to async model: Scenarios where Ops have no dependency could delegate the
Op execution to a threadpool and thus separate the request rate control from the
synchronous execution of Ops. Measurement would start on queuing for execution.
1. https://groups.google.com/forum/#!msg/mechanical-sympathy/icNZJejUHfE/BfDekfBEs_sJ
2. https://github.com/HdrHistogram/HdrHistogram
\ No newline at end of file
<?xml version="1.0" encoding="UTF-8"?>
<!--
Copyright (c) 2012 - 2016 YCSB contributors. All rights reserved.
Licensed under the Apache License, Version 2.0 (the "License"); you
may not use this file except in compliance with the License. You
may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
implied. See the License for the specific language governing
permissions and limitations under the License. See accompanying
LICENSE file.
-->
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">
<modelVersion>4.0.0</modelVersion>
<parent>
<groupId>site.ycsb</groupId>
<artifactId>root</artifactId>
<version>0.18.0-SNAPSHOT</version>
</parent>
<artifactId>core</artifactId>
<name>Core YCSB</name>
<packaging>jar</packaging>
<properties>
<jackson.api.version>1.9.4</jackson.api.version>
</properties>
<dependencies>
<dependency>
<groupId>org.apache.htrace</groupId>
<artifactId>htrace-core4</artifactId>
<version>4.1.0-incubating</version>
</dependency>
<dependency>
<groupId>org.codehaus.jackson</groupId>
<artifactId>jackson-mapper-asl</artifactId>
<version>${jackson.api.version}</version>
</dependency>
<dependency>
<groupId>org.codehaus.jackson</groupId>
<artifactId>jackson-core-asl</artifactId>
<version>${jackson.api.version}</version>
</dependency>
<dependency>
<groupId>org.testng</groupId>
<artifactId>testng</artifactId>
<version>6.1.1</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.hdrhistogram</groupId>
<artifactId>HdrHistogram</artifactId>
<version>2.1.4</version>
</dependency>
</dependencies>
<build>
<resources>
<resource>
<directory>src/main/resources</directory>
<filtering>true</filtering>
</resource>
</resources>
</build>
<profiles>
<profile>
<!-- Build profile when running via yscb.sh or yscb.bat-->
<id>source-run</id>
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-dependency-plugin</artifactId>
<executions>
<execution>
<id>stage-dependencies</id>
<phase>package</phase>
<goals>
<goal>copy-dependencies</goal>
</goals>
<configuration>
<includeScope>runtime</includeScope>
</configuration>
</execution>
</executions>
</plugin>
</plugins>
</build>
</profile>
</profiles>
</project>
This diff is collapsed.
/**
* Copyright (c) 2017 YCSB contributors All rights reserved.
* <p>
* Licensed under the Apache License, Version 2.0 (the "License"); you
* may not use this file except in compliance with the License. You
* may obtain a copy of the License at
* <p>
* http://www.apache.org/licenses/LICENSE-2.0
* <p>
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
* implied. See the License for the specific language governing
* permissions and limitations under the License. See accompanying
* LICENSE file.
*/
package site.ycsb;
import java.util.HashMap;
import java.util.HashSet;
import java.util.Map;
import java.util.Map.Entry;
import java.util.Set;
import java.util.TreeMap;
import site.ycsb.workloads.TimeSeriesWorkload;
/**
* Basic DB for printing out time series workloads and/or tracking the distribution
* of keys and fields.
*/
public class BasicTSDB extends BasicDB {
/** Time series workload specific counters. */
protected static Map<Long, Integer> timestamps;
protected static Map<Integer, Integer> floats;
protected static Map<Integer, Integer> integers;
private String timestampKey;
private String valueKey;
private String tagPairDelimiter;
private String queryTimeSpanDelimiter;
private long lastTimestamp;
@Override
public void init() {
super.init();
synchronized (MUTEX) {
if (timestamps == null) {
timestamps = new HashMap<Long, Integer>();
floats = new HashMap<Integer, Integer>();
integers = new HashMap<Integer, Integer>();
}
}
timestampKey = getProperties().getProperty(
TimeSeriesWorkload.TIMESTAMP_KEY_PROPERTY,
TimeSeriesWorkload.TIMESTAMP_KEY_PROPERTY_DEFAULT);
valueKey = getProperties().getProperty(
TimeSeriesWorkload.VALUE_KEY_PROPERTY,
TimeSeriesWorkload.VALUE_KEY_PROPERTY_DEFAULT);
tagPairDelimiter = getProperties().getProperty(
TimeSeriesWorkload.PAIR_DELIMITER_PROPERTY,
TimeSeriesWorkload.PAIR_DELIMITER_PROPERTY_DEFAULT);
queryTimeSpanDelimiter = getProperties().getProperty(
TimeSeriesWorkload.QUERY_TIMESPAN_DELIMITER_PROPERTY,
TimeSeriesWorkload.QUERY_TIMESPAN_DELIMITER_PROPERTY_DEFAULT);
}
public Status read(String table, String key, Set<String> fields, Map<String, ByteIterator> result) {
delay();
if (verbose) {
StringBuilder sb = getStringBuilder();
sb.append("READ ").append(table).append(" ").append(key).append(" [ ");
if (fields != null) {
for (String f : fields) {
sb.append(f).append(" ");
}
} else {
sb.append("<all fields>");
}
sb.append("]");
System.out.println(sb);
}
if (count) {
Set<String> filtered = null;
if (fields != null) {
filtered = new HashSet<String>();
for (final String field : fields) {
if (field.startsWith(timestampKey)) {
String[] parts = field.split(tagPairDelimiter);
if (parts[1].contains(queryTimeSpanDelimiter)) {
parts = parts[1].split(queryTimeSpanDelimiter);
lastTimestamp = Long.parseLong(parts[0]);
} else {
lastTimestamp = Long.parseLong(parts[1]);
}
synchronized(timestamps) {
Integer ctr = timestamps.get(lastTimestamp);
if (ctr == null) {
timestamps.put(lastTimestamp, 1);
} else {
timestamps.put(lastTimestamp, ctr + 1);
}
}
} else {
filtered.add(field);
}
}
}
incCounter(reads, hash(table, key, filtered));
}
return Status.OK;
}
@Override
public Status update(String table, String key, Map<String, ByteIterator> values) {
delay();
boolean isFloat = false;
if (verbose) {
StringBuilder sb = getStringBuilder();
sb.append("UPDATE ").append(table).append(" ").append(key).append(" [ ");
if (values != null) {
final TreeMap<String, ByteIterator> tree = new TreeMap<String, ByteIterator>(values);
for (Map.Entry<String, ByteIterator> entry : tree.entrySet()) {
if (entry.getKey().equals(timestampKey)) {
sb.append(entry.getKey()).append("=")
.append(Utils.bytesToLong(entry.getValue().toArray())).append(" ");
} else if (entry.getKey().equals(valueKey)) {
final NumericByteIterator it = (NumericByteIterator) entry.getValue();
isFloat = it.isFloatingPoint();
sb.append(entry.getKey()).append("=")
.append(isFloat ? it.getDouble() : it.getLong()).append(" ");
} else {
sb.append(entry.getKey()).append("=").append(entry.getValue()).append(" ");
}
}
}
sb.append("]");
System.out.println(sb);
}
if (count) {
if (!verbose) {
isFloat = ((NumericByteIterator) values.get(valueKey)).isFloatingPoint();
}
int hash = hash(table, key, values);
incCounter(updates, hash);
synchronized(timestamps) {
Integer ctr = timestamps.get(lastTimestamp);
if (ctr == null) {
timestamps.put(lastTimestamp, 1);
} else {
timestamps.put(lastTimestamp, ctr + 1);
}
}
if (isFloat) {
incCounter(floats, hash);
} else {
incCounter(integers, hash);
}
}
return Status.OK;
}
@Override
public Status insert(String table, String key, Map<String, ByteIterator> values) {
delay();
boolean isFloat = false;
if (verbose) {
StringBuilder sb = getStringBuilder();
sb.append("INSERT ").append(table).append(" ").append(key).append(" [ ");
if (values != null) {
final TreeMap<String, ByteIterator> tree = new TreeMap<String, ByteIterator>(values);
for (Map.Entry<String, ByteIterator> entry : tree.entrySet()) {
if (entry.getKey().equals(timestampKey)) {
sb.append(entry.getKey()).append("=")
.append(Utils.bytesToLong(entry.getValue().toArray())).append(" ");
} else if (entry.getKey().equals(valueKey)) {
final NumericByteIterator it = (NumericByteIterator) entry.getValue();
isFloat = it.isFloatingPoint();
sb.append(entry.getKey()).append("=")
.append(isFloat ? it.getDouble() : it.getLong()).append(" ");
} else {
sb.append(entry.getKey()).append("=").append(entry.getValue()).append(" ");
}
}
}
sb.append("]");
System.out.println(sb);
}
if (count) {
if (!verbose) {
isFloat = ((NumericByteIterator) values.get(valueKey)).isFloatingPoint();
}
int hash = hash(table, key, values);
incCounter(inserts, hash);
synchronized(timestamps) {
Integer ctr = timestamps.get(lastTimestamp);
if (ctr == null) {
timestamps.put(lastTimestamp, 1);
} else {
timestamps.put(lastTimestamp, ctr + 1);
}
}
if (isFloat) {
incCounter(floats, hash);
} else {
incCounter(integers, hash);
}
}
return Status.OK;
}
@Override
public void cleanup() {
super.cleanup();
if (count && counter < 1) {
System.out.println("[TIMESTAMPS], Unique, " + timestamps.size());
System.out.println("[FLOATS], Unique series, " + floats.size());
System.out.println("[INTEGERS], Unique series, " + integers.size());
long minTs = Long.MAX_VALUE;
long maxTs = Long.MIN_VALUE;
for (final long ts : timestamps.keySet()) {
if (ts > maxTs) {
maxTs = ts;
}
if (ts < minTs) {
minTs = ts;
}
}
System.out.println("[TIMESTAMPS], Min, " + minTs);
System.out.println("[TIMESTAMPS], Max, " + maxTs);
}
}
@Override
protected int hash(final String table, final String key, final Map<String, ByteIterator> values) {
final TreeMap<String, ByteIterator> sorted = new TreeMap<String, ByteIterator>();
for (final Entry<String, ByteIterator> entry : values.entrySet()) {
if (entry.getKey().equals(valueKey)) {
continue;
} else if (entry.getKey().equals(timestampKey)) {
lastTimestamp = ((NumericByteIterator) entry.getValue()).getLong();
entry.getValue().reset();
continue;
}
sorted.put(entry.getKey(), entry.getValue());
}
// yeah it's ugly but gives us a unique hash without having to add hashers
// to all of the ByteIterators.
StringBuilder buf = new StringBuilder().append(table).append(key);
for (final Entry<String, ByteIterator> entry : sorted.entrySet()) {
entry.getValue().reset();
buf.append(entry.getKey())
.append(entry.getValue().toString());
}
return buf.toString().hashCode();
}
}
\ No newline at end of file
/**
* Copyright (c) 2010-2016 Yahoo! Inc., 2017 YCSB contributors All rights reserved.
* <p>
* Licensed under the Apache License, Version 2.0 (the "License"); you
* may not use this file except in compliance with the License. You
* may obtain a copy of the License at
* <p>
* http://www.apache.org/licenses/LICENSE-2.0
* <p>
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
* implied. See the License for the specific language governing
* permissions and limitations under the License. See accompanying
* LICENSE file.
*/
package site.ycsb;
/**
* A ByteIterator that iterates through a byte array.
*/
public class ByteArrayByteIterator extends ByteIterator {
private final int originalOffset;
private final byte[] str;
private int off;
private final int len;
public ByteArrayByteIterator(byte[] s) {
this.str = s;
this.off = 0;
this.len = s.length;
originalOffset = 0;
}
public ByteArrayByteIterator(byte[] s, int off, int len) {
this.str = s;
this.off = off;
this.len = off + len;
originalOffset = off;
}
@Override
public boolean hasNext() {
return off < len;
}
@Override
public byte nextByte() {
byte ret = str[off];
off++;
return ret;
}
@Override
public long bytesLeft() {
return len - off;
}
@Override
public void reset() {
off = originalOffset;
}
@Override
public byte[] toArray() {
int size = (int) bytesLeft();
byte[] bytes = new byte[size];
System.arraycopy(str, off, bytes, 0, size);
off = len;
return bytes;
}
}
/**
* Copyright (c) 2010-2016 Yahoo! Inc., 2017 YCSB contributors All rights reserved.
* <p>
* Licensed under the Apache License, Version 2.0 (the "License"); you
* may not use this file except in compliance with the License. You
* may obtain a copy of the License at
* <p>
* http://www.apache.org/licenses/LICENSE-2.0
* <p>
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
* implied. See the License for the specific language governing
* permissions and limitations under the License. See accompanying
* LICENSE file.
*/
package site.ycsb;
import java.nio.ByteBuffer;
import java.nio.CharBuffer;
import java.nio.charset.Charset;
import java.util.Iterator;
/**
* YCSB-specific buffer class. ByteIterators are designed to support
* efficient field generation, and to allow backend drivers that can stream
* fields (instead of materializing them in RAM) to do so.
* <p>
* YCSB originially used String objects to represent field values. This led to
* two performance issues.
* </p><p>
* First, it leads to unnecessary conversions between UTF-16 and UTF-8, both
* during field generation, and when passing data to byte-based backend
* drivers.
* </p><p>
* Second, Java strings are represented internally using UTF-16, and are
* built by appending to a growable array type (StringBuilder or
* StringBuffer), then calling a toString() method. This leads to a 4x memory
* overhead as field values are being built, which prevented YCSB from
* driving large object stores.
* </p>
* The StringByteIterator class contains a number of convenience methods for
* backend drivers that convert between Map&lt;String,String&gt; and
* Map&lt;String,ByteBuffer&gt;.
*
*/
public abstract class ByteIterator implements Iterator<Byte> {
@Override
public abstract boolean hasNext();
@Override
public Byte next() {
throw new UnsupportedOperationException();
}
public abstract byte nextByte();
/** @return byte offset immediately after the last valid byte */
public int nextBuf(byte[] buf, int bufOff) {
int sz = bufOff;
while (sz < buf.length && hasNext()) {
buf[sz] = nextByte();
sz++;
}
return sz;
}
public abstract long bytesLeft();
@Override
public void remove() {
throw new UnsupportedOperationException();
}
/** Resets the iterator so that it can be consumed again. Not all
* implementations support this call.
* @throws UnsupportedOperationException if the implementation hasn't implemented
* the method.
*/
public void reset() {
throw new UnsupportedOperationException();
}
/** Consumes remaining contents of this object, and returns them as a string. */
public String toString() {
Charset cset = Charset.forName("UTF-8");
CharBuffer cb = cset.decode(ByteBuffer.wrap(this.toArray()));
return cb.toString();
}
/** Consumes remaining contents of this object, and returns them as a byte array. */
public byte[] toArray() {
long left = bytesLeft();
if (left != (int) left) {
throw new ArrayIndexOutOfBoundsException("Too much data to fit in one array!");
}
byte[] ret = new byte[(int) left];
for (int i = 0; i < ret.length; i++) {
ret[i] = nextByte();
}
return ret;
}
}
This diff is collapsed.
/**
* Copyright (c) 2010-2016 Yahoo! Inc., 2017 YCSB contributors All rights reserved.
* <p>
* Licensed under the Apache License, Version 2.0 (the "License"); you
* may not use this file except in compliance with the License. You
* may obtain a copy of the License at
* <p>
* http://www.apache.org/licenses/LICENSE-2.0
* <p>
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
* implied. See the License for the specific language governing
* permissions and limitations under the License. See accompanying
* LICENSE file.
*/
package site.ycsb;
import site.ycsb.measurements.Measurements;
import java.util.Properties;
import java.util.concurrent.CountDownLatch;
import java.util.concurrent.ThreadLocalRandom;
import java.util.concurrent.locks.LockSupport;
/**
* A thread for executing transactions or data inserts to the database.
*/
public class ClientThread implements Runnable {
// Counts down each of the clients completing.
private final CountDownLatch completeLatch;
private static boolean spinSleep;
private DB db;
private boolean dotransactions;
private Workload workload;
private int opcount;
private double targetOpsPerMs;
private int opsdone;
private int threadid;
private int threadcount;
private Object workloadstate;
private Properties props;
private long targetOpsTickNs;
private final Measurements measurements;
/**
* Constructor.
*
* @param db the DB implementation to use
* @param dotransactions true to do transactions, false to insert data
* @param workload the workload to use
* @param props the properties defining the experiment
* @param opcount the number of operations (transactions or inserts) to do
* @param targetperthreadperms target number of operations per thread per ms
* @param completeLatch The latch tracking the completion of all clients.
*/
public ClientThread(DB db, boolean dotransactions, Workload workload, Properties props, int opcount,
double targetperthreadperms, CountDownLatch completeLatch) {
this.db = db;
this.dotransactions = dotransactions;
this.workload = workload;
this.opcount = opcount;
opsdone = 0;
if (targetperthreadperms > 0) {
targetOpsPerMs = targetperthreadperms;
targetOpsTickNs = (long) (1000000 / targetOpsPerMs);
}
this.props = props;
measurements = Measurements.getMeasurements();
spinSleep = Boolean.valueOf(this.props.getProperty("spin.sleep", "false"));
this.completeLatch = completeLatch;
}
public void setThreadId(final int threadId) {
threadid = threadId;
}
public void setThreadCount(final int threadCount) {
threadcount = threadCount;
}
public int getOpsDone() {
return opsdone;
}
@Override
public void run() {
try {
db.init();
} catch (DBException e) {
e.printStackTrace();
e.printStackTrace(System.out);
return;
}
try {
workloadstate = workload.initThread(props, threadid, threadcount);
} catch (WorkloadException e) {
e.printStackTrace();
e.printStackTrace(System.out);
return;
}
//NOTE: Switching to using nanoTime and parkNanos for time management here such that the measurements
// and the client thread have the same view on time.
//spread the thread operations out so they don't all hit the DB at the same time
// GH issue 4 - throws exception if _target>1 because random.nextInt argument must be >0
// and the sleep() doesn't make sense for granularities < 1 ms anyway
if ((targetOpsPerMs > 0) && (targetOpsPerMs <= 1.0)) {
long randomMinorDelay = ThreadLocalRandom.current().nextInt((int) targetOpsTickNs);
sleepUntil(System.nanoTime() + randomMinorDelay);
}
try {
if (dotransactions) {
long startTimeNanos = System.nanoTime();
while (((opcount == 0) || (opsdone < opcount)) && !workload.isStopRequested()) {
if (!workload.doTransaction(db, workloadstate)) {
break;
}
opsdone++;
throttleNanos(startTimeNanos);
}
} else {
long startTimeNanos = System.nanoTime();
while (((opcount == 0) || (opsdone < opcount)) && !workload.isStopRequested()) {
if (!workload.doInsert(db, workloadstate)) {
break;
}
opsdone++;
throttleNanos(startTimeNanos);
}
}
} catch (Exception e) {
e.printStackTrace();
e.printStackTrace(System.out);
System.exit(0);
}
try {
measurements.setIntendedStartTimeNs(0);
db.cleanup();
} catch (DBException e) {
e.printStackTrace();
e.printStackTrace(System.out);
} finally {
completeLatch.countDown();
}
}
private static void sleepUntil(long deadline) {
while (System.nanoTime() < deadline) {
if (!spinSleep) {
LockSupport.parkNanos(deadline - System.nanoTime());
}
}
}
private void throttleNanos(long startTimeNanos) {
//throttle the operations
if (targetOpsPerMs > 0) {
// delay until next tick
long deadline = startTimeNanos + opsdone * targetOpsTickNs;
sleepUntil(deadline);
measurements.setIntendedStartTimeNs(deadline);
}
}
/**
* The total amount of work this thread is still expected to do.
*/
int getOpsTodo() {
int todo = opcount - opsdone;
return todo < 0 ? 0 : todo;
}
}
/**
* Copyright (c) 2010-2016 Yahoo! Inc., 2017 YCSB contributors All rights reserved.
* <p>
* Licensed under the Apache License, Version 2.0 (the "License"); you
* may not use this file except in compliance with the License. You
* may obtain a copy of the License at
* <p>
* http://www.apache.org/licenses/LICENSE-2.0
* <p>
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
* implied. See the License for the specific language governing
* permissions and limitations under the License. See accompanying
* LICENSE file.
*/
package site.ycsb;
import java.util.HashMap;
import java.util.Map;
import java.util.Properties;
import java.util.Set;
import java.util.Vector;
/**
* A layer for accessing a database to be benchmarked. Each thread in the client
* will be given its own instance of whatever DB class is to be used in the test.
* This class should be constructed using a no-argument constructor, so we can
* load it dynamically. Any argument-based initialization should be
* done by init().
*
* Note that YCSB does not make any use of the return codes returned by this class.
* Instead, it keeps a count of the return values and presents them to the user.
*
* The semantics of methods such as insert, update and delete vary from database
* to database. In particular, operations may or may not be durable once these
* methods commit, and some systems may return 'success' regardless of whether
* or not a tuple with a matching key existed before the call. Rather than dictate
* the exact semantics of these methods, we recommend you either implement them
* to match the database's default semantics, or the semantics of your
* target application. For the sake of comparison between experiments we also
* recommend you explain the semantics you chose when presenting performance results.
*/
public abstract class DB {
/**
* Properties for configuring this DB.
*/
private Properties properties = new Properties();
/**
* Set the properties for this DB.
*/
public void setProperties(Properties p) {
properties = p;
}
/**
* Get the set of properties for this DB.
*/
public Properties getProperties() {
return properties;
}
/**
* Initialize any state for this DB.
* Called once per DB instance; there is one DB instance per client thread.
*/
public void init() throws DBException {
}
/**
* Cleanup any state for this DB.
* Called once per DB instance; there is one DB instance per client thread.
*/
public void cleanup() throws DBException {
}
/**
* Read a record from the database. Each field/value pair from the result will be stored in a HashMap.
*
* @param table The name of the table
* @param key The record key of the record to read.
* @param fields The list of fields to read, or null for all of them
* @param result A HashMap of field/value pairs for the result
* @return The result of the operation.
*/
public abstract Status read(String table, String key, Set<String> fields, Map<String, ByteIterator> result);
/**
* Perform a range scan for a set of records in the database. Each field/value pair from the result will be stored
* in a HashMap.
*
* @param table The name of the table
* @param startkey The record key of the first record to read.
* @param recordcount The number of records to read
* @param fields The list of fields to read, or null for all of them
* @param result A Vector of HashMaps, where each HashMap is a set field/value pairs for one record
* @return The result of the operation.
*/
public abstract Status scan(String table, String startkey, int recordcount, Set<String> fields,
Vector<HashMap<String, ByteIterator>> result);
/**
* Update a record in the database. Any field/value pairs in the specified values HashMap will be written into the
* record with the specified record key, overwriting any existing values with the same field name.
*
* @param table The name of the table
* @param key The record key of the record to write.
* @param values A HashMap of field/value pairs to update in the record
* @return The result of the operation.
*/
public abstract Status update(String table, String key, Map<String, ByteIterator> values);
/**
* Insert a record in the database. Any field/value pairs in the specified values HashMap will be written into the
* record with the specified record key.
*
* @param table The name of the table
* @param key The record key of the record to insert.
* @param values A HashMap of field/value pairs to insert in the record
* @return The result of the operation.
*/
public abstract Status insert(String table, String key, Map<String, ByteIterator> values);
/**
* Delete a record from the database.
*
* @param table The name of the table
* @param key The record key of the record to delete.
* @return The result of the operation.
*/
public abstract Status delete(String table, String key);
}
/**
* Copyright (c) 2010-2016 Yahoo! Inc., 2017 YCSB contributors All rights reserved.
* <p>
* Licensed under the Apache License, Version 2.0 (the "License"); you
* may not use this file except in compliance with the License. You
* may obtain a copy of the License at
* <p>
* http://www.apache.org/licenses/LICENSE-2.0
* <p>
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
* implied. See the License for the specific language governing
* permissions and limitations under the License. See accompanying
* LICENSE file.
*/
package site.ycsb;
/**
* Something bad happened while interacting with the database.
*/
public class DBException extends Exception {
/**
*
*/
private static final long serialVersionUID = 6646883591588721475L;
public DBException(String message) {
super(message);
}
public DBException() {
super();
}
public DBException(String message, Throwable cause) {
super(message, cause);
}
public DBException(Throwable cause) {
super(cause);
}
}
/**
* Copyright (c) 2010-2016 Yahoo! Inc., 2017 YCSB contributors All rights reserved.
* <p>
* Licensed under the Apache License, Version 2.0 (the "License"); you
* may not use this file except in compliance with the License. You
* may obtain a copy of the License at
* <p>
* http://www.apache.org/licenses/LICENSE-2.0
* <p>
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
* implied. See the License for the specific language governing
* permissions and limitations under the License. See accompanying
* LICENSE file.
*/
package site.ycsb;
import org.apache.htrace.core.Tracer;
import java.util.Properties;
/**
* Creates a DB layer by dynamically classloading the specified DB class.
*/
public final class DBFactory {
private DBFactory() {
// not used
}
public static DB newDB(String dbname, Properties properties, final Tracer tracer) throws UnknownDBException {
ClassLoader classLoader = DBFactory.class.getClassLoader();
DB ret;
try {
Class dbclass = classLoader.loadClass(dbname);
ret = (DB) dbclass.newInstance();
} catch (Exception e) {
e.printStackTrace();
return null;
}
ret.setProperties(properties);
return new DBWrapper(ret, tracer);
}
}
/**
* Copyright (c) 2010 Yahoo! Inc. All rights reserved.
* <p>
* Licensed under the Apache License, Version 2.0 (the "License"); you
* may not use this file except in compliance with the License. You
* may obtain a copy of the License at
* <p>
* http://www.apache.org/licenses/LICENSE-2.0
* <p>
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
* implied. See the License for the specific language governing
* permissions and limitations under the License. See accompanying
* LICENSE file.
*/
package site.ycsb;
import java.util.HashMap;
import java.util.Map;
import java.util.Random;
import java.util.Set;
import java.util.Vector;
import java.util.concurrent.ThreadLocalRandom;
import java.util.concurrent.locks.Lock;
import java.util.concurrent.locks.LockSupport;
import java.util.concurrent.locks.ReadWriteLock;
import java.util.concurrent.locks.ReentrantReadWriteLock;
import static java.util.concurrent.TimeUnit.MICROSECONDS;
/**
* Basic DB that just prints out the requested operations, instead of doing them against a database.
*/
public class GoodBadUglyDB extends DB {
public static final String SIMULATE_DELAY = "gbudb.delays";
public static final String SIMULATE_DELAY_DEFAULT = "200,1000,10000,50000,100000";
private static final ReadWriteLock DB_ACCESS = new ReentrantReadWriteLock();
private long[] delays;
public GoodBadUglyDB() {
delays = new long[]{200, 1000, 10000, 50000, 200000};
}
private void delay() {
final Random random = ThreadLocalRandom.current();
double p = random.nextDouble();
int mod;
if (p < 0.9) {
mod = 0;
} else if (p < 0.99) {
mod = 1;
} else if (p < 0.9999) {
mod = 2;
} else {
mod = 3;
}
// this will make mod 3 pauses global
Lock lock = mod == 3 ? DB_ACCESS.writeLock() : DB_ACCESS.readLock();
if (mod == 3) {
System.out.println("OUCH");
}
lock.lock();
try {
final long baseDelayNs = MICROSECONDS.toNanos(delays[mod]);
final int delayRangeNs = (int) (MICROSECONDS.toNanos(delays[mod + 1]) - baseDelayNs);
final long delayNs = baseDelayNs + random.nextInt(delayRangeNs);
final long deadline = System.nanoTime() + delayNs;
do {
LockSupport.parkNanos(deadline - System.nanoTime());
} while (System.nanoTime() < deadline && !Thread.interrupted());
} finally {
lock.unlock();
}
}
/**
* Initialize any state for this DB. Called once per DB instance; there is one DB instance per client thread.
*/
public void init() {
int i = 0;
for (String delay : getProperties().getProperty(SIMULATE_DELAY, SIMULATE_DELAY_DEFAULT).split(",")) {
delays[i++] = Long.parseLong(delay);
}
}
/**
* Read a record from the database. Each field/value pair from the result will be stored in a HashMap.
*
* @param table The name of the table
* @param key The record key of the record to read.
* @param fields The list of fields to read, or null for all of them
* @param result A HashMap of field/value pairs for the result
* @return Zero on success, a non-zero error code on error
*/
public Status read(String table, String key, Set<String> fields, Map<String, ByteIterator> result) {
delay();
return Status.OK;
}
/**
* Perform a range scan for a set of records in the database. Each field/value pair from the result will be stored
* in a HashMap.
*
* @param table The name of the table
* @param startkey The record key of the first record to read.
* @param recordcount The number of records to read
* @param fields The list of fields to read, or null for all of them
* @param result A Vector of HashMaps, where each HashMap is a set field/value pairs for one record
* @return Zero on success, a non-zero error code on error
*/
public Status scan(String table, String startkey, int recordcount, Set<String> fields,
Vector<HashMap<String, ByteIterator>> result) {
delay();
return Status.OK;
}
/**
* Update a record in the database. Any field/value pairs in the specified values HashMap will be written into the
* record with the specified record key, overwriting any existing values with the same field name.
*
* @param table The name of the table
* @param key The record key of the record to write.
* @param values A HashMap of field/value pairs to update in the record
* @return Zero on success, a non-zero error code on error
*/
public Status update(String table, String key, Map<String, ByteIterator> values) {
delay();
return Status.OK;
}
/**
* Insert a record in the database. Any field/value pairs in the specified values HashMap will be written into the
* record with the specified record key.
*
* @param table The name of the table
* @param key The record key of the record to insert.
* @param values A HashMap of field/value pairs to insert in the record
* @return Zero on success, a non-zero error code on error
*/
public Status insert(String table, String key, Map<String, ByteIterator> values) {
delay();
return Status.OK;
}
/**
* Delete a record from the database.
*
* @param table The name of the table
* @param key The record key of the record to delete.
* @return Zero on success, a non-zero error code on error
*/
public Status delete(String table, String key) {
delay();
return Status.OK;
}
}
/**
* Copyright (c) 2017 YCSB contributors. All rights reserved.
*
* Licensed under the Apache License, Version 2.0 (the "License"); you
* may not use this file except in compliance with the License. You
* may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
* implied. See the License for the specific language governing
* permissions and limitations under the License. See accompanying
* LICENSE file.
*/
package site.ycsb;
/**
* A byte iterator that handles encoding and decoding numeric values.
* Currently this iterator can handle 64 bit signed values and double precision
* floating point values.
*/
public class NumericByteIterator extends ByteIterator {
private final byte[] payload;
private final boolean floatingPoint;
private int off;
public NumericByteIterator(final long value) {
floatingPoint = false;
payload = Utils.longToBytes(value);
off = 0;
}
public NumericByteIterator(final double value) {
floatingPoint = true;
payload = Utils.doubleToBytes(value);
off = 0;
}
@Override
public boolean hasNext() {
return off < payload.length;
}
@Override
public byte nextByte() {
return payload[off++];
}
@Override
public long bytesLeft() {
return payload.length - off;
}
@Override
public void reset() {
off = 0;
}
public long getLong() {
if (floatingPoint) {
throw new IllegalStateException("Byte iterator is of the type double");
}
return Utils.bytesToLong(payload);
}
public double getDouble() {
if (!floatingPoint) {
throw new IllegalStateException("Byte iterator is of the type long");
}
return Utils.bytesToDouble(payload);
}
public boolean isFloatingPoint() {
return floatingPoint;
}
}
\ No newline at end of file
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment