[Git][java-team/libjavaewah-java][master] 10 commits: Update debian/watch file for github tag format
Tony Mancill (@tmancill)
gitlab at salsa.debian.org
Wed Jun 21 05:57:31 BST 2023
Tony Mancill pushed to branch master at Debian Java Maintainers / libjavaewah-java
Commits:
db931ebe by tony mancill at 2023-06-20T07:15:40-07:00
Update debian/watch file for github tag format
- - - - -
687a45f4 by tony mancill at 2023-06-20T07:15:55-07:00
New upstream version 1.2.3
- - - - -
a70e5f9f by tony mancill at 2023-06-20T07:15:56-07:00
Update upstream source from tag 'upstream/1.2.3'
Update to upstream version '1.2.3'
with Debian dir f20b012e548a1d98877aabe005c150a1b264848f
- - - - -
07021ebb by tony mancill at 2023-06-20T07:16:22-07:00
Refresh patches for upstream 1.2.3
- - - - -
927ff511 by tony mancill at 2023-06-20T07:17:23-07:00
Freshen years in debian/copyright
- - - - -
661834e2 by tony mancill at 2023-06-20T07:17:47-07:00
Bump Standards-Version to 4.6.2 (no changes)
- - - - -
ef0f1f5a by tony mancill at 2023-06-20T07:18:01-07:00
Set Rules-Requires-Root: no in debian/control
- - - - -
66ec8abe by tony mancill at 2023-06-20T21:38:19-07:00
Ignore moditect-maven-plugin (not packaged for Debian)
- - - - -
3bd4caf6 by tony mancill at 2023-06-20T21:48:14-07:00
whitespace
- - - - -
3df36f23 by tony mancill at 2023-06-20T21:48:14-07:00
Prepare changelog for upload
- - - - -
21 changed files:
- + .github/workflows/basic.yml
- .travis.yml
- README.md
- debian/changelog
- debian/control
- debian/copyright
- debian/maven.ignoreRules
- debian/patches/01-backward-compatibility.patch
- debian/rules
- debian/watch
- pom.xml
- src/main/java/com/googlecode/javaewah/EWAHCompressedBitmap.java
- src/main/java/com/googlecode/javaewah32/EWAHCompressedBitmap32.java
- + src/test/java/com/googlecode/javaewah/BackwardBitSetIterator.java
- + src/test/java/com/googlecode/javaewah/EWAHBitSetPair.java
- src/test/java/com/googlecode/javaewah/EWAHCompressedBitmapTest.java
- + src/test/java/com/googlecode/javaewah/ForwardBitSetIterator.java
- + src/test/java/com/googlecode/javaewah/FuzzEWAHTest.java
- + src/test/java/com/googlecode/javaewah32/EWAH32BitSetPair.java
- src/test/java/com/googlecode/javaewah32/EWAHCompressedBitmap32Test.java
- + src/test/java/com/googlecode/javaewah32/FuzzEWAH32Test.java
Changes:
=====================================
.github/workflows/basic.yml
=====================================
@@ -0,0 +1,17 @@
+name: Java CI
+
+on: [push, pull_request]
+
+jobs:
+ build:
+ runs-on: ubuntu-latest
+
+ steps:
+ - uses: actions/checkout at v2
+ - name: Set up JDK 16
+ uses: actions/setup-java at v2
+ with:
+ java-version: '16'
+ distribution: 'adopt'
+ - name: Build with Maven
+ run: mvn test
=====================================
.travis.yml
=====================================
@@ -13,3 +13,7 @@ branches:
script: mvn clean test
+
+cache:
+ directories:
+ - $HOME/.m2
=====================================
README.md
=====================================
@@ -1,31 +1,19 @@
JavaEWAH
==========================================================
-[](https://travis-ci.org/lemire/javaewah)
+[](https://github.com/lemire/javaewah/actions/workflows/basic.yml)
[![][maven img]][maven]
[![][license img]][license]
[![docs-badge][]][docs]
[](https://coveralls.io/r/lemire/javaewah?branch=master)
[](https://lgtm.com/projects/g/lemire/javaewah/context:java)
-(c) 2009-2016
-Daniel Lemire (http://lemire.me/en/),
-Cliff Moon,
-David McIntosh (https://github.com/mctofu),
-Robert Becho (https://github.com/RBecho),
-Colby Ranger (https://github.com/crangeratgoogle),
-Veronika Zenz (https://github.com/veronikazenz),
-Owen Kaser (https://github.com/owenkaser),
-Gregory Ssi-Yan-Kai (https://github.com/gssiyankai),
-and Rory Graves (https://github.com/rorygraves)
-
-
-This code is licensed under Apache License, Version 2.0 (ASL2.0).
-(GPL 2.0 derivatives are allowed.)
This is a word-aligned compressed variant of
the Java Bitset class. We provide both a 64-bit
and a 32-bit RLE-like compression scheme. It can
-be used to implement bitmap indexes.
+be used to implement bitmap indexes. The EWAH format
+it relies upon is used in the git implementation
+that runs GitHub.
The goal of word-aligned compression is not to
achieve the best compression, but rather to
@@ -96,7 +84,7 @@ When the bitset approach is applicable, it can be orders of
magnitude faster than other possible implementation of a set (e.g., as a hash set)
while using several times less memory.
-However, a bitset, even a compressed one is not always applicable. For example, if the
+However, a bitset, even a compressed one is not always applicable. For example, if
you have 1000 random-looking integers, then a simple array might be the best representation.
We refer to this case as the "sparse" scenario.
@@ -153,12 +141,8 @@ Data format
For more details regarding the compression format, please
see Section 3 of the following paper:
-Daniel Lemire, Owen Kaser, Kamel Aouiche, Sorting improves word-aligned bitmap indexes. Data & Knowledge Engineering 69 (1), pages 3-28, 2010.
- http://arxiv.org/abs/0901.3751
-
-
-
- (The PDF file is freely available on the arXiv site.)
+Daniel Lemire, Owen Kaser, Kamel Aouiche, [Sorting improves word-aligned bitmap indexes](http://arxiv.org/abs/0901.3751). Data & Knowledge Engineering 69 (1), pages 3-28, 2010.
+
Benchmark
---------
@@ -173,15 +157,22 @@ However, this is very naive. It is recommended that you run your own benchmarks.
Unit testing
------------
-As of October 2011, this packages relies on Maven. To
+As of October 2011, this package relies on Maven. To
test it:
+```
mvn test
+```
See
http://maven.apache.org/guides/introduction/introduction-to-the-lifecycle.html
for details.
+We support Java 8 and up, but to build the library, you need Java 9 and up
+with the default Maven setup, since we rely on the `--release` flag functionality
+(unavailable in Java 8) to sidestep the [Deaded-NoSuchMethodError issue](https://www.morling.dev/blog/bytebuffer-and-the-dreaded-nosuchmethoderror/).
+
+
Usage
-----
@@ -289,12 +280,6 @@ To install javaewah on Ubuntu, type:
sudo apt-get install libjavaewah-java
-Travis (Continuous integration)
--------------------------------
-
-You can check whether the latest version builds on your favorite version
-of Java using Travis: https://travis-ci.org/lemire/javaewah/builds/
-
Clojure
-------
@@ -343,13 +328,20 @@ https://groups.google.com/forum/#!forum/javaewah
Further reading
---------------
-Daniel Lemire, Owen Kaser, Kamel Aouiche, Sorting improves word-aligned bitmap indexes, Data & Knowledge Engineering 69 (1), 2010.
-http://arxiv.org/abs/0901.3751
+- Daniel Lemire, Owen Kaser, Kamel Aouiche, [Sorting improves word-aligned bitmap indexes](http://arxiv.org/abs/0901.3751), Data & Knowledge Engineering 69 (1), 2010.
+- Owen Kaser and Daniel Lemire, [Compressed bitmap indexes: beyond unions and intersections](http://arxiv.org/abs/1402.4466), Software: Practice and Experience 46 (2), 2016.
+
-Owen Kaser and Daniel Lemire, Compressed bitmap indexes: beyond unions and intersections, Software: Practice and Experience 46 (2), 2016.
-http://arxiv.org/abs/1402.4466
+Credit
+--------
+(c) 2009-2021
+[Daniel Lemire](http://lemire.me/en/), Cliff Moon, [David McIntosh](https://github.com/mctofu), [Robert Becho](https://github.com/RBecho), [Colby Ranger](https://github.com/crangeratgoogle), [Veronika Zenz](https://github.com/veronikazenz), [Owen Kaser](https://github.com/owenkaser), [Gregory Ssi-Yan-Kai](https://github.com/gssiyankai), and [Rory Graves](https://github.com/rorygraves)
+
+
+This code is licensed under Apache License, Version 2.0 (ASL2.0).
+(GPL 2.0 derivatives are allowed.)
Acknowledgement
---------------
=====================================
debian/changelog
=====================================
@@ -1,3 +1,16 @@
+libjavaewah-java (1.2.3-1) unstable; urgency=medium
+
+ * New upstream version 1.2.3
+ * Update debian/watch file for github tag format
+ * Refresh patches for upstream 1.2.3
+ * Freshen years in debian/copyright
+ * Bump Standards-Version to 4.6.2 (no changes)
+ * Set Rules-Requires-Root: no in debian/control
+ * Ignore moditect-maven-plugin (not yet packaged for Debian),
+ thus the resulting JAR file does not contain module-info.
+
+ -- tony mancill <tmancill at debian.org> Tue, 20 Jun 2023 21:35:57 -0700
+
libjavaewah-java (1.1.7-1) unstable; urgency=medium
* Team upload.
=====================================
debian/control
=====================================
@@ -11,10 +11,11 @@ Build-Depends:
junit4,
libmaven-bundle-plugin-java,
maven-debian-helper (>= 1.6.3)
-Standards-Version: 4.5.1
+Standards-Version: 4.6.2
Vcs-Git: https://salsa.debian.org/java-team/libjavaewah-java.git
Vcs-Browser: https://salsa.debian.org/java-team/libjavaewah-java
Homepage: https://github.com/lemire/javaewah
+Rules-Requires-Root: no
Package: libjavaewah-java
Architecture: all
=====================================
debian/copyright
=====================================
@@ -4,17 +4,17 @@ Upstream-Contact: JavaEWAH team
Source: https://github.com/lemire/javaewah
Files: *
-Copyright: 2009-2013, Daniel Lemire (http://lemire.me/en/)
- 2009-2013, Cliff Moon (https://github.com/cliffmoon)
- 2009-2013, David McIntosh (https://github.com/mctofu)
- 2009-2013, Robert Becho (https://github.com/RBecho)
- 2009-2013, Colby Ranger (https://github.com/crangeratgoogle)
- 2009-2013, Veronika Zenz (https://github.com/veronikazenz)
- 2009-2013, Google Inc.
+Copyright: 2009-2023, Daniel Lemire (http://lemire.me/en/)
+ 2009-2023, Cliff Moon (https://github.com/cliffmoon)
+ 2009-2023, David McIntosh (https://github.com/mctofu)
+ 2009-2023, Robert Becho (https://github.com/RBecho)
+ 2009-2023, Colby Ranger (https://github.com/crangeratgoogle)
+ 2009-2023, Veronika Zenz (https://github.com/veronikazenz)
+ 2009-2023, Google Inc.
License: Apache-2.0
Files: debian/*
-Copyright: 2013, Debian Java Maintainers <pkg-java-maintainers at lists.alioth.debian.org>
+Copyright: 2013-2023, Debian Java Maintainers <pkg-java-maintainers at lists.alioth.debian.org>
License: Apache-2.0
License: Apache-2.0
=====================================
debian/maven.ignoreRules
=====================================
@@ -1,6 +1,6 @@
-
* animal-sniffer-maven-plugin * * * *
* jacoco-maven-plugin * * * *
* maven-gpg-plugin * * * *
* maven-javadoc-plugin * * * *
* maven-source-plugin * * * *
+* moditect-maven-plugin * * * *
=====================================
debian/patches/01-backward-compatibility.patch
=====================================
@@ -3,7 +3,7 @@ Author: Emmanuel Bourg <ebourg at apache.org>
Forwarded: not-needed
--- a/src/main/java/com/googlecode/javaewah/EWAHCompressedBitmap.java
+++ b/src/main/java/com/googlecode/javaewah/EWAHCompressedBitmap.java
-@@ -2133,6 +2133,10 @@
+@@ -2170,6 +2170,10 @@
*/
public static final int WORD_IN_BITS = 64;
=====================================
debian/rules
=====================================
@@ -2,4 +2,3 @@
%:
dh $@
-
=====================================
debian/watch
=====================================
@@ -1,2 +1,2 @@
version=4
-https://github.com/lemire/javaewah/releases .*/JavaEWAH-([\d\.]+)\.tar\.gz
+https://github.com/lemire/javaewah/tags .*/JavaEWAH- at ANY_VERSION@\.tar\.gz
=====================================
pom.xml
=====================================
@@ -2,11 +2,10 @@
<modelVersion>4.0.0</modelVersion>
<groupId>com.googlecode.javaewah</groupId>
<artifactId>JavaEWAH</artifactId>
- <version>1.1.7</version>
+ <version>1.2.3</version>
<packaging>bundle</packaging>
<properties>
- <maven.compiler.source>1.8</maven.compiler.source>
- <maven.compiler.target>1.8</maven.compiler.target>
+ <maven.compiler.release>8</maven.compiler.release>
<encoding>UTF-8</encoding>
</properties>
<licenses>
@@ -45,7 +44,7 @@
<dependency>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
- <version>4.10</version>
+ <version>4.13.1</version>
<scope>test</scope>
</dependency>
</dependencies>
@@ -65,6 +64,8 @@
<artifactId>maven-surefire-plugin</artifactId>
<version>2.19.1</version>
<configuration>
+ <enableAssertions>true</enableAssertions>
+ <trimStackTrace>false</trimStackTrace>
<forkCount>3</forkCount>
<reuseForks>true</reuseForks>
<argLine>-Xmx1024m</argLine>
@@ -85,7 +86,7 @@
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-gpg-plugin</artifactId>
- <version>1.4</version>
+ <version>3.0.1</version>
<executions>
<execution>
<id>sign-artifacts</id>
@@ -99,7 +100,7 @@
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-javadoc-plugin</artifactId>
- <version>2.8</version>
+ <version>3.5.0</version>
<configuration>
<source>8</source>
</configuration>
@@ -125,6 +126,41 @@
</execution>
</executions>
</plugin>
+ <plugin>
+ <groupId>org.apache.maven.plugins</groupId>
+ <artifactId>maven-compiler-plugin</artifactId>
+ <version>3.11.0</version>
+ </plugin>
+ <plugin>
+ <groupId>org.moditect</groupId>
+ <artifactId>moditect-maven-plugin</artifactId>
+ <version>1.0.0.RC2</version>
+ <executions>
+ <execution>
+ <id>add-module-infos</id>
+ <phase>package</phase>
+ <goals>
+ <goal>add-module-info</goal>
+ </goals>
+ <configuration>
+ <jvmVersion>9</jvmVersion>
+ <overwriteExistingFiles>true</overwriteExistingFiles>
+ <module>
+ <moduleInfo>
+ <name>com.googlecode.javaewah</name>
+ <!-- export everything -->
+ <exports>*;</exports>
+ <!-- declare services consumed by the artifact -->
+ <addServiceUses>true</addServiceUses>
+ </moduleInfo>
+ </module>
+ <jdepsExtraArgs>
+ <arg>--multi-release=9</arg>
+ </jdepsExtraArgs>
+ </configuration>
+ </execution>
+ </executions>
+ </plugin>
</plugins>
</build>
<name>JavaEWAH</name>
=====================================
src/main/java/com/googlecode/javaewah/EWAHCompressedBitmap.java
=====================================
@@ -513,6 +513,7 @@ public final class EWAHCompressedBitmap implements Cloneable, Externalizable,
prey.discardFirstWords(predator.getRunningLength());
} else if (i_is_prey) {
final long index = prey.discharge(container, predator.getRunningLength());
+ // Todo: this may cause fragmentation whereas 0 literal words are inserted.
container.addStreamOfEmptyWords(false, predator.getRunningLength() - index);
} else {
final long index = prey.dischargeNegated(container, predator.getRunningLength());
@@ -1174,14 +1175,18 @@ public final class EWAHCompressedBitmap implements Cloneable, Externalizable,
}
nword += (int) rl;
long lw = RunningLengthWord.getNumberOfLiteralWords(this.buffer, pos);
- if(lw > 0) {
- long word = this.buffer.getWord(pos + 1);
- if(word != 0l) {
- long T = word & -word;
- return nword * WORD_IN_BITS + Long.bitCount(T - 1);
- }
+ for(int p = pos + 1 ; p <= pos + lw; p++) {
+ long word = this.buffer.getWord(p);
+ // In theory, words should never be zero. Unfortunately due to the
+ // design which requires us to support 'andnot' and effective universe
+ // sizes, we sometimes end up appending zero words.
+ if(word != 0l) {
+ long T = word & -word;
+ return nword * WORD_IN_BITS + Long.bitCount(T - 1);
+ }
+ nword++;
}
- }
+ }
return -1;
}
@@ -1257,6 +1262,12 @@ public final class EWAHCompressedBitmap implements Cloneable, Externalizable,
this.sizeInBits = i + 1;
if (value) {
if (dist > 0) {
+ // Let us trim the lone zero word if needed
+ if (this.rlw.getNumberOfLiteralWords() > 0 && this.buffer.getLastWord() == 0l) {
+ this.buffer.removeLastWord();
+ this.rlw.setNumberOfLiteralWords(this.rlw.getNumberOfLiteralWords() - 1);
+ insertEmptyWord(false);
+ }
if (dist > 1) {
fastaddStreamOfEmptyWords(false, dist - 1);
}
@@ -1609,28 +1620,42 @@ public final class EWAHCompressedBitmap implements Cloneable, Externalizable,
/**
* A more detailed string describing the bitmap (useful for debugging).
+ * A JSON output is produced.
*
* @return the string
*/
public String toDebugString() {
StringBuilder ans = new StringBuilder();
- ans.append(" EWAHCompressedBitmap, size in bits = ");
- ans.append(this.sizeInBits).append(" size in words = ");
- ans.append(this.buffer.sizeInWords()).append("\n");
+ ans.append("{\"size in bits\":");
+ ans.append(this.sizeInBits).append(", \"size in words\":");
+ ans.append(this.buffer.sizeInWords()).append(",");
final EWAHIterator i = this.getEWAHIterator();
+ ans.append(" \"content\": [");
+ boolean first = true;
while (i.hasNext()) {
RunningLengthWord localrlw = i.next();
+ if(!first) { ans.append(","); }
+ first = false;
+ ans.append("[");
+
if (localrlw.getRunningBit()) {
- ans.append(localrlw.getRunningLength()).append(" 1x11\n");
+ ans.append(localrlw.getRunningLength()).append(",").append(" \"1x11\", ");
} else {
- ans.append(localrlw.getRunningLength()).append(" 0x00\n");
+ ans.append(localrlw.getRunningLength()).append(",").append(" \"0x00\", ");
+ }
+ ans.append("[");
+ int j = 0;
+ for (; j + 1 < localrlw.getNumberOfLiteralWords(); ++j) {
+ long data = i.buffer().getWord(i.literalWords() + j);
+ ans.append("\"0x").append(Long.toHexString(data)).append("\",");
}
- ans.append(localrlw.getNumberOfLiteralWords()).append(" dirties\n");
- for (int j = 0; j < localrlw.getNumberOfLiteralWords(); ++j) {
+ if(j < localrlw.getNumberOfLiteralWords()) {
long data = i.buffer().getWord(i.literalWords() + j);
- ans.append("\t").append(data).append("\n");
+ ans.append("\"0x").append(Long.toHexString(data)).append("\"");
}
+ ans.append("]]");
}
+ ans.append("]}");
return ans.toString();
}
@@ -2070,8 +2095,20 @@ public final class EWAHCompressedBitmap implements Cloneable, Externalizable,
int fullwords = b / WORD_IN_BITS;
int shift = b % WORD_IN_BITS;
answer.addStreamOfEmptyWords(false, fullwords);
- if (shift == 0) {
- answer.buffer.push_back(this.buffer, 0, sz);
+ if (shift == 0) {
+ while (true) {
+ long rl = i.getRunningLength();
+ if (rl > 0) {
+ answer.addStreamOfEmptyWords(i.getRunningBit(), rl);
+ }
+ int x = i.getNumberOfLiteralWords();
+ for (int k = 0; k < x; ++k) {
+ answer.addWord(i.getLiteralWordAt(k));
+ }
+ if (!i.next()) {
+ break;
+ }
+ }
} else {
// whether the shift should justify a new word
final boolean shiftextension = ((this.sizeInBits + WORD_IN_BITS - 1) % WORD_IN_BITS) + shift >= WORD_IN_BITS;
=====================================
src/main/java/com/googlecode/javaewah32/EWAHCompressedBitmap32.java
=====================================
@@ -508,6 +508,7 @@ public final class EWAHCompressedBitmap32 implements Cloneable, Externalizable,
prey.discardFirstWords(predator
.getRunningLength());
} else if (i_is_prey) {
+ // Todo: this may cause fragmentation whereas 0 literal words are inserted.
final int index = prey.discharge(container,
predator.getRunningLength());
container.addStreamOfEmptyWords(false,
@@ -1178,13 +1179,17 @@ public final class EWAHCompressedBitmap32 implements Cloneable, Externalizable,
}
nword += rl;
int lw = RunningLengthWord32.getNumberOfLiteralWords(this.buffer, pos);
- if(lw > 0) {
- int word = this.buffer.getWord(pos + 1);
- if(word != 0) {
- int T = word & -word;
- return nword * WORD_IN_BITS + Integer.bitCount(T - 1);
- }
- }
+ for(int p = pos + 1 ; p <= pos + lw; p++) {
+ int word = this.buffer.getWord(p);
+ // In theory, words should never be zero. Unfortunately due to the
+ // design which requires us to support 'andnot' and effective universe
+ // sizes, we sometimes end up appending zero words.
+ if(word != 0l) {
+ int T = word & -word;
+ return nword * WORD_IN_BITS + Integer.bitCount(T - 1);
+ }
+ nword++;
+ }
}
return -1;
}
@@ -1262,6 +1267,12 @@ public final class EWAHCompressedBitmap32 implements Cloneable, Externalizable,
this.sizeInBits = i + 1;
if(value) {
if (dist > 0) {
+ // Let us trim the lone zero word if needed
+ if (this.rlw.getNumberOfLiteralWords() > 0 && this.buffer.getLastWord() == 0) {
+ this.buffer.removeLastWord();
+ this.rlw.setNumberOfLiteralWords(this.rlw.getNumberOfLiteralWords() - 1);
+ insertEmptyWord(false);
+ }
if (dist > 1) {
fastaddStreamOfEmptyWords(false, dist - 1);
}
@@ -1621,28 +1632,44 @@ public final class EWAHCompressedBitmap32 implements Cloneable, Externalizable,
/**
* A more detailed string describing the bitmap (useful for debugging).
+ * A JSON output is produced.
*
* @return the string
*/
+
public String toDebugString() {
- StringBuffer sb = new StringBuffer(" EWAHCompressedBitmap, size in bits = ");
- sb.append(this.sizeInBits).append(" size in words = ");
- sb.append(this.buffer.sizeInWords()).append("\n");
+ StringBuilder ans = new StringBuilder();
+ ans.append("{\"size in bits\":");
+ ans.append(this.sizeInBits).append(", \"size in words\":");
+ ans.append(this.buffer.sizeInWords()).append(",");
final EWAHIterator32 i = this.getEWAHIterator();
+ ans.append(" \"content\": [");
+ boolean first = true;
while (i.hasNext()) {
RunningLengthWord32 localrlw = i.next();
+ if(!first) { ans.append(","); }
+ first = false;
+ ans.append("[");
+
if (localrlw.getRunningBit()) {
- sb.append(localrlw.getRunningLength()).append(" 1x11\n");
+ ans.append(localrlw.getRunningLength()).append(",").append(" \"1x11\", ");
} else {
- sb.append(localrlw.getRunningLength()).append(" 0x00\n");
+ ans.append(localrlw.getRunningLength()).append(",").append(" \"0x00\", ");
+ }
+ ans.append("[");
+ int j = 0;
+ for (; j + 1 < localrlw.getNumberOfLiteralWords(); ++j) {
+ int data = i.buffer().getWord(i.literalWords() + j);
+ ans.append("\"0x").append(Integer.toHexString(data)).append("\",");
}
- sb.append(localrlw.getNumberOfLiteralWords()).append(" dirties\n");
- for (int j = 0; j < localrlw.getNumberOfLiteralWords(); ++j) {
+ if(j < localrlw.getNumberOfLiteralWords()) {
int data = i.buffer().getWord(i.literalWords() + j);
- sb.append("\t").append(data).append("\n");
+ ans.append("\"0x").append(Integer.toHexString(data)).append("\"");
}
+ ans.append("]]");
}
- return sb.toString();
+ ans.append("]}");
+ return ans.toString();
}
/**
@@ -2076,7 +2103,19 @@ public final class EWAHCompressedBitmap32 implements Cloneable, Externalizable,
int shift = b % WORD_IN_BITS;
answer.addStreamOfEmptyWords(false, fullwords);
if (shift == 0) {
- answer.buffer.push_back(this.buffer, 0, sz);
+ while (true) {
+ int rl = i.getRunningLength();
+ if (rl > 0) {
+ answer.addStreamOfEmptyWords(i.getRunningBit(), rl);
+ }
+ int x = i.getNumberOfLiteralWords();
+ for (int k = 0; k < x; ++k) {
+ answer.addWord(i.getLiteralWordAt(k));
+ }
+ if (!i.next()) {
+ break;
+ }
+ }
} else {
int w = 0;
while (true) {
=====================================
src/test/java/com/googlecode/javaewah/BackwardBitSetIterator.java
=====================================
@@ -0,0 +1,27 @@
+package com.googlecode.javaewah;
+
+import java.util.BitSet;
+import java.util.Iterator;
+// credit @svanmald
+public class BackwardBitSetIterator implements Iterator<Integer> {
+
+ private final BitSet bitSet;
+ private int next;
+
+ public BackwardBitSetIterator(BitSet bitSet) {
+ this.bitSet = bitSet;
+ this.next = bitSet.previousSetBit(bitSet.length());
+ }
+
+ @Override
+ public boolean hasNext() {
+ return next != -1;
+ }
+
+ @Override
+ public Integer next() {
+ int current = next;
+ next = bitSet.previousSetBit(current - 1);
+ return current;
+ }
+}
\ No newline at end of file
=====================================
src/test/java/com/googlecode/javaewah/EWAHBitSetPair.java
=====================================
@@ -0,0 +1,70 @@
+package com.googlecode.javaewah;
+
+import com.googlecode.javaewah.EWAHCompressedBitmap;
+import com.googlecode.javaewah.IntIterator;
+import java.util.BitSet;
+// credit @svanmald
+public class EWAHBitSetPair {
+
+ private EWAHCompressedBitmap bitmap;
+ private BitSet bitSet;
+
+ public EWAHBitSetPair() {
+ bitmap = new EWAHCompressedBitmap();
+ bitSet = new BitSet();
+ }
+
+ public void validate() {
+ assert bitmap.cardinality() == bitSet.cardinality();
+ ForwardBitSetIterator forwardBitSetIterator = new ForwardBitSetIterator(bitSet);
+ for (Integer current : bitmap) {
+ Integer next = forwardBitSetIterator.next();
+ assert bitmap.get(current);
+ assert next.equals(current);
+ }
+
+ BackwardBitSetIterator backwardBitSetIterator = new BackwardBitSetIterator(bitSet);
+ IntIterator reverseIterator = bitmap.reverseIntIterator();
+ while (reverseIterator.hasNext()) {
+ int nextBitMap = reverseIterator.next();
+ Integer nextBitSet = backwardBitSetIterator.next();
+ assert nextBitSet == nextBitMap;
+ }
+ assert !backwardBitSetIterator.hasNext();
+
+ EWAHCompressedBitmap result = new EWAHCompressedBitmap().or(bitmap);
+ assert result.equals(bitmap);
+ assert bitmap.equals(result);
+ assert bitmap.isEmpty() || bitmap.getFirstSetBit() == bitmap.iterator().next();
+ }
+
+ public void or(EWAHBitSetPair other) {
+ bitmap = bitmap.or(other.bitmap);
+ bitSet.or(other.bitSet);
+ }
+
+ public void and(EWAHBitSetPair other) {
+ bitmap = bitmap.and(other.bitmap);
+ bitSet.and(other.bitSet);
+ }
+
+ public void andNot(EWAHBitSetPair other) {
+ bitmap = bitmap.andNot(other.bitmap);
+ bitSet.andNot(other.bitSet);
+ }
+
+ public void xor(EWAHBitSetPair other) {
+ bitmap = bitmap.xor(other.bitmap);
+ bitSet.xor(other.bitSet);
+ }
+
+ public void set(int value) {
+ bitSet.set(value);
+ bitmap.set(value);
+ }
+
+ public void clear(int value) {
+ bitSet.clear(value);
+ bitmap.clear(value);
+ }
+}
=====================================
src/test/java/com/googlecode/javaewah/EWAHCompressedBitmapTest.java
=====================================
@@ -21,7 +21,103 @@ import static com.googlecode.javaewah.EWAHCompressedBitmap.WORD_IN_BITS;
*/
@SuppressWarnings("javadoc")
public class EWAHCompressedBitmapTest {
-
+ @Test
+ public void issue77() {
+ EWAHCompressedBitmap main = new EWAHCompressedBitmap();
+ main = main.shift(64);
+ main.set(63132);
+ Assert.assertEquals((long)main.intIterator().next(),63132);
+ EWAHCompressedBitmap other = new EWAHCompressedBitmap();
+ other.set(100);
+ other = other.shift(64);
+ other.clear(62514);
+ Assert.assertEquals((long)other.intIterator().next(),100+64);
+ }
+ @Test
+ public void issue72a() {
+ EWAHCompressedBitmap main = new EWAHCompressedBitmap();
+ EWAHCompressedBitmap other = new EWAHCompressedBitmap();
+ main.clear(70583);
+ other = other.xor(main);
+ other.set(43013);
+ other = other.xor(main);
+ Assert.assertEquals((long)other.intIterator().next(),43013);
+ Assert.assertEquals((long)other.reverseIntIterator().next(),43013);
+ }
+ @Test
+ public void issue72b() {
+ EWAHCompressedBitmap main = new EWAHCompressedBitmap();
+ EWAHCompressedBitmap other = new EWAHCompressedBitmap();
+ main.set(33209);
+ other = other.and(main);
+ other = other.xor(main);
+ Iterator<Integer> i = other.iterator();
+ Assert.assertEquals(i.hasNext(),true);
+ Assert.assertEquals((long)i.next(),(long)33209);
+ }
+ @Test
+ public void issue72c() {
+ EWAHCompressedBitmap main = new EWAHCompressedBitmap();
+ EWAHCompressedBitmap other = new EWAHCompressedBitmap();
+ main = main.and(other);
+ other.clear(96836);
+ main = main.andNot(other);
+ main = main.and(other);
+ main.set(96118);
+ other = other.and(main);
+ other = other.or(main);
+ IntIterator intIterator = other.reverseIntIterator();
+ Assert.assertEquals((long)intIterator.next(),96118);
+ }
+ @Test
+ public void issue68() {
+ EWAHCompressedBitmap one = new EWAHCompressedBitmap();
+ EWAHCompressedBitmap other = new EWAHCompressedBitmap();
+ one.set(18308);
+ other.set(24608);
+ other = other.and(one);
+ Assert.assertEquals((long)other.getFirstSetBit(),-1);
+ other.set(82764);
+ Assert.assertEquals((long)other.getFirstSetBit(),(long)other.iterator().next());
+ }
+ @Test
+ public void issue73() {
+ EWAHCompressedBitmap main = new EWAHCompressedBitmap();
+ EWAHCompressedBitmap other = new EWAHCompressedBitmap();
+ main.clear(10684);
+ other = other.andNot(main);
+ other = other.or(main);
+ new EWAHCompressedBitmap().or(other);
+ }
+ @Test
+ public void issue74() {
+ EWAHCompressedBitmap main = new EWAHCompressedBitmap();
+ EWAHCompressedBitmap other = new EWAHCompressedBitmap();
+ main = main.or(other);
+ other.set(7036);
+ main.set(44002);
+ other = other.and(main);
+ other = other.or(main);
+ Assert.assertEquals((long)other.iterator().next(),(long)44002);
+ }
+
+ @Test
+ public void issue70() {
+ EWAHCompressedBitmap one = new EWAHCompressedBitmap();
+ EWAHCompressedBitmap other = new EWAHCompressedBitmap();
+ one.set(16627);
+ other.set(52811);
+ other = other.and(one);
+ one = one.andNot(other);
+ one.set(16039);
+ other.set(78669);
+ other = other.or(one);
+ one = one.and(other);
+ other = other.andNot(one);
+ Assert.assertEquals((long)other.iterator().next(), 78669);
+ Assert.assertEquals((long)other.getFirstSetBit(), 78669);
+ }
+
@Test
public void swaptest() {
EWAHCompressedBitmap x = EWAHCompressedBitmap.bitmapOf(1,2,3);
@@ -383,7 +479,6 @@ public class EWAHCompressedBitmapTest {
public void testBug090b() throws Exception {
EWAHCompressedBitmap bm1 = new EWAHCompressedBitmap();
bm1.setSizeInBits(8, false); // Create a bitmap with no bit set
- System.out.println(bm1.toDebugString());
EWAHCompressedBitmap bm2 = new EWAHCompressedBitmap();
bm2.setSizeInBits(64, false); // Create a bitmap with no bit set
EWAHCompressedBitmap bm3 = new EWAHCompressedBitmap();
@@ -400,7 +495,6 @@ public class EWAHCompressedBitmapTest {
public void testBug090c() throws Exception {
EWAHCompressedBitmap bm1 = new EWAHCompressedBitmap();
bm1.setSizeInBits(8, false); // Create a bitmap with no bit set
- System.out.println(bm1.toDebugString());
EWAHCompressedBitmap bm2 = new EWAHCompressedBitmap();
bm2.setSizeInBits(64, false); // Create a bitmap with no bit set
EWAHCompressedBitmap bm3 = new EWAHCompressedBitmap();
@@ -1319,9 +1413,6 @@ public class EWAHCompressedBitmapTest {
EWAHCompressedBitmap and2 = new EWAHCompressedBitmap();
FastAggregation.bufferedandWithContainer(and2, 32, bitmaps[0],bitmaps[1],bitmaps[2]);
EWAHCompressedBitmap and3 = EWAHCompressedBitmap.and(bitmaps[0],bitmaps[1],bitmaps[2]);
- System.out.println(and1.sizeInBits());
- System.out.println(and2.sizeInBits());
- System.out.println(and3.sizeInBits());
assertEqualsPositions(and1, and2);
assertEqualsPositions(and2, and3);
}
@@ -1923,8 +2014,6 @@ public class EWAHCompressedBitmapTest {
for (int k = 1; k < ewah.length; ++k)
answer = answer.and(ewah[k]);
// result should be empty
- if (answer.toList().size() != 0)
- System.out.println(answer.toDebugString());
Assert.assertTrue(answer.toList().size() == 0);
Assert.assertTrue(EWAHCompressedBitmap.and(ewah).toList()
.size() == 0);
@@ -1990,9 +2079,6 @@ public class EWAHCompressedBitmapTest {
EWAHCompressedBitmap.or(ewah));
int k = 0;
for (int j : answer) {
- if (k != j)
- System.out.println(answer
- .toDebugString());
Assert.assertEquals(k, j);
k += 1;
}
@@ -2051,8 +2137,6 @@ public class EWAHCompressedBitmapTest {
}
int k = 0;
for (int j : answer) {
- if (k != j)
- System.out.println(answer.toDebugString());
Assert.assertEquals(k, j);
k += 1;
}
=====================================
src/test/java/com/googlecode/javaewah/ForwardBitSetIterator.java
=====================================
@@ -0,0 +1,28 @@
+package com.googlecode.javaewah;
+
+import java.util.BitSet;
+import java.util.Iterator;
+// credit @svanmald
+public class ForwardBitSetIterator implements Iterator<Integer> {
+
+ private final BitSet bitSet;
+ private int next;
+
+ public ForwardBitSetIterator(BitSet bitSet) {
+ this.bitSet = bitSet;
+ this.next = bitSet.nextSetBit(0);
+ }
+
+
+ @Override
+ public boolean hasNext() {
+ return next != -1;
+ }
+
+ @Override
+ public Integer next() {
+ int current = next;
+ next = bitSet.nextSetBit(next + 1);
+ return current;
+ }
+}
\ No newline at end of file
=====================================
src/test/java/com/googlecode/javaewah/FuzzEWAHTest.java
=====================================
@@ -0,0 +1,161 @@
+package com.googlecode.javaewah;
+
+import java.util.Random;
+import org.junit.Test;
+// credit @svanmald
+ at SuppressWarnings("javadoc")
+public class FuzzEWAHTest {
+ public static boolean areAssertsEnabled() {
+ boolean assertsEnabled = false;
+ assert assertsEnabled = true; // Intentional side effect!!!
+ return assertsEnabled;
+ }
+
+ @Test
+ public void testEwah() {
+ if(!areAssertsEnabled()) { throw new RuntimeException("asserts need to be enabled."); }
+ // ENABLE ASSERTS BEFORE EXECUTING TO ENABLE VALIDATION.
+ // if print = false and seed -1, the code will execute 10 random mutation to 2 bitmaps until infinity and each time validate the result
+ // Each time a set of 10 random mutations starts, a seed is printed (even if print = false).
+ // if one of the sets of 10 mutations fails validation, the printed seed can be used here together with print = true to reproduce the issue
+ System.out.println(" == Launching @svanmald's fuzzer! ");
+ testEwah(false, -1);
+ }
+
+ private void testEwah(boolean print, int seed) {
+ Random seedGenerator = new Random();
+ Mutation[] mutations = Mutation.values();
+ int times = 1000000;
+
+ while (times > 0) {
+ times --;
+ if((times % 10000) == 0) { System.out.print("."); System.out.flush(); }
+ int currentSeed = seed;
+ if (currentSeed == -1) {
+ currentSeed = seedGenerator.nextInt();
+ }
+ if (print) {
+ System.out.println("Seed " + currentSeed);
+ }
+ Random seededRandom = new Random(currentSeed);
+ EWAHBitSetPair main = new EWAHBitSetPair();
+ if (print) {
+ System.out.println("EWAHCompressedBitmap main = new EWAHCompressedBitmap();");
+ }
+ EWAHBitSetPair other = new EWAHBitSetPair();
+ if (print) {
+ System.out.println("EWAHCompressedBitmap other = new EWAHCompressedBitmap();");
+ }
+ for (int i = 0; i < 10; i++) {
+ Mutation mutation = mutations[seededRandom.nextInt(mutations.length)];
+ mutation.apply(print, seededRandom, main, other);
+ main.validate();
+ other.validate();
+ }
+ }
+ System.out.println();
+ }
+
+ public enum Mutation {
+ OR {
+ @Override
+ void apply(boolean print, Random random, EWAHBitSetPair main, EWAHBitSetPair other) {
+ if (random.nextDouble() < 0.5) {
+ if (print) {
+ System.out.println("main = main.or(other);");
+ }
+ main.or(other);
+ } else {
+ if (print) {
+ System.out.println("other = other.or(main);");
+ }
+ other.or(main);
+ }
+ }
+ },
+ AND {
+ @Override
+ void apply(boolean print, Random random, EWAHBitSetPair main, EWAHBitSetPair other) {
+ if (random.nextDouble() < 0.5) {
+ if (print) {
+ System.out.println("main = main.and(other);");
+ }
+ main.and(other);
+ } else {
+ if (print) {
+ System.out.println("other = other.and(main);");
+ }
+ other.and(main);
+ }
+ }
+ },
+ AND_NOT {
+ @Override
+ void apply(boolean print, Random random, EWAHBitSetPair main, EWAHBitSetPair other) {
+ if (random.nextDouble() < 0.5) {
+ if (print) {
+ System.out.println("main = main.andNot(other);");
+ }
+ main.andNot(other);
+ } else {
+ if (print) {
+ System.out.println("other = other.andNot(main);");
+ }
+ other.andNot(main);
+ }
+ }
+ },
+ XOR {
+ @Override
+ void apply(boolean print, Random random, EWAHBitSetPair main, EWAHBitSetPair other) {
+ if (random.nextDouble() < 0.5) {
+ if (print) {
+ System.out.println("main = main.xor(other);");
+ }
+ main.xor(other);
+ } else {
+ if (print) {
+ System.out.println("other = other.xor(main);");
+ }
+ other.xor(main);
+ }
+ }
+ },
+ SET {
+ @Override
+ void apply(boolean print, Random random, EWAHBitSetPair main, EWAHBitSetPair other) {
+ int value = random.nextInt(100_000);
+ if (random.nextDouble() < 0.5) {
+ if (print) {
+ System.out.println("main.set(" + value + ");");
+ }
+ main.set(value);
+ } else {
+ if (print) {
+ System.out.println("other.set(" + value + ");");
+ }
+ other.set(value);
+ }
+ }
+ },
+ CLEAR_RANDOM {
+ @Override
+ void apply(boolean print, Random random, EWAHBitSetPair main, EWAHBitSetPair other) {
+ int value = random.nextInt(100_000);
+ if (random.nextDouble() < 0.5) {
+ if (print) {
+ System.out.println("main.clear(" + value + ");");
+ }
+ main.clear(value);
+ } else {
+ if (print) {
+ System.out.println("other.clear(" + value + ");");
+ }
+ other.clear(value);
+ }
+ }
+ };
+
+ abstract void apply(boolean print, Random random, EWAHBitSetPair main, EWAHBitSetPair other);
+ }
+}
\ No newline at end of file
=====================================
src/test/java/com/googlecode/javaewah32/EWAH32BitSetPair.java
=====================================
@@ -0,0 +1,72 @@
+package com.googlecode.javaewah32;
+
+import com.googlecode.javaewah.EWAHCompressedBitmap;
+import com.googlecode.javaewah.IntIterator;
+import java.util.BitSet;
+import com.googlecode.javaewah.BackwardBitSetIterator;
+import com.googlecode.javaewah.ForwardBitSetIterator;
+// credit @svanmald
+public class EWAH32BitSetPair {
+
+ private EWAHCompressedBitmap bitmap;
+ private BitSet bitSet;
+
+ public EWAH32BitSetPair() {
+ bitmap = new EWAHCompressedBitmap();
+ bitSet = new BitSet();
+ }
+
+ public void validate() {
+ assert bitmap.cardinality() == bitSet.cardinality();
+ ForwardBitSetIterator forwardBitSetIterator = new ForwardBitSetIterator(bitSet);
+ for (Integer current : bitmap) {
+ Integer next = forwardBitSetIterator.next();
+ assert bitmap.get(current);
+ assert next.equals(current);
+ }
+
+ BackwardBitSetIterator backwardBitSetIterator = new BackwardBitSetIterator(bitSet);
+ IntIterator reverseIterator = bitmap.reverseIntIterator();
+ while (reverseIterator.hasNext()) {
+ int nextBitMap = reverseIterator.next();
+ Integer nextBitSet = backwardBitSetIterator.next();
+ assert nextBitSet == nextBitMap;
+ }
+ assert !backwardBitSetIterator.hasNext();
+
+ EWAHCompressedBitmap result = new EWAHCompressedBitmap().or(bitmap);
+ assert result.equals(bitmap);
+ assert bitmap.equals(result);
+ assert bitmap.isEmpty() || bitmap.getFirstSetBit() == bitmap.iterator().next();
+ }
+
+ public void or(EWAH32BitSetPair other) {
+ bitmap = bitmap.or(other.bitmap);
+ bitSet.or(other.bitSet);
+ }
+
+ public void and(EWAH32BitSetPair other) {
+ bitmap = bitmap.and(other.bitmap);
+ bitSet.and(other.bitSet);
+ }
+
+ public void andNot(EWAH32BitSetPair other) {
+ bitmap = bitmap.andNot(other.bitmap);
+ bitSet.andNot(other.bitSet);
+ }
+
+ public void xor(EWAH32BitSetPair other) {
+ bitmap = bitmap.xor(other.bitmap);
+ bitSet.xor(other.bitSet);
+ }
+
+ public void set(int value) {
+ bitSet.set(value);
+ bitmap.set(value);
+ }
+
+ public void clear(int value) {
+ bitSet.clear(value);
+ bitmap.clear(value);
+ }
+}
=====================================
src/test/java/com/googlecode/javaewah32/EWAHCompressedBitmap32Test.java
=====================================
@@ -23,7 +23,108 @@ import static com.googlecode.javaewah32.EWAHCompressedBitmap32.WORD_IN_BITS;
*/
@SuppressWarnings("javadoc")
public class EWAHCompressedBitmap32Test {
-
+ @Test
+ public void issue77() {
+ EWAHCompressedBitmap32 main = new EWAHCompressedBitmap32();
+ main = main.shift(64);
+ main.set(63132);
+ Assert.assertEquals((long)main.intIterator().next(),63132);
+ EWAHCompressedBitmap32 other = new EWAHCompressedBitmap32();
+ other.set(100);
+ other = other.shift(64);
+ other.clear(62514);
+ Assert.assertEquals((long)other.intIterator().next(),100+64);
+ }
+ @Test
+ public void issue72a() {
+ EWAHCompressedBitmap32 main = new EWAHCompressedBitmap32();
+ EWAHCompressedBitmap32 other = new EWAHCompressedBitmap32();
+ main.clear(70583);
+ other = other.xor(main);
+ other.set(43013);
+ other = other.xor(main);
+ Assert.assertEquals((long)other.intIterator().next(),43013);
+ Assert.assertEquals((long)other.reverseIntIterator().next(),43013);
+ }
+ @Test
+ public void issue72b() {
+ EWAHCompressedBitmap32 main = new EWAHCompressedBitmap32();
+ EWAHCompressedBitmap32 other = new EWAHCompressedBitmap32();
+ main.set(33209);
+ other = other.and(main);
+ other = other.xor(main);
+ System.out.println(other);
+ Iterator<Integer> i = other.iterator();
+ Assert.assertEquals(i.hasNext(),true);
+ Assert.assertEquals((long)i.next(),(long)33209);
+ }
+ @Test
+ public void issue72c() {
+ EWAHCompressedBitmap32 main = new EWAHCompressedBitmap32();
+ EWAHCompressedBitmap32 other = new EWAHCompressedBitmap32();
+ main = main.and(other);
+ other.clear(96836);
+ main = main.andNot(other);
+ main = main.and(other);
+ main.set(96118);
+ other = other.and(main);
+ other = other.or(main);
+ System.out.println(other);
+ IntIterator intIterator = other.reverseIntIterator();
+ Assert.assertEquals((long)intIterator.next(),96118);
+ }
+
+ @Test
+ public void issue73() {
+ EWAHCompressedBitmap32 main = new EWAHCompressedBitmap32();
+ EWAHCompressedBitmap32 other = new EWAHCompressedBitmap32();
+ main.clear(10684);
+ other = other.andNot(main);
+ other = other.or(main);
+ new EWAHCompressedBitmap32().or(other);
+ }
+
+ @Test
+ public void issue74() {
+ EWAHCompressedBitmap32 main = new EWAHCompressedBitmap32();
+ EWAHCompressedBitmap32 other = new EWAHCompressedBitmap32();
+ main = main.or(other);
+ other.set(7036);
+ main.set(44002);
+ other = other.and(main);
+ other = other.or(main);
+ Assert.assertEquals((long)other.iterator().next(),(long)44002);
+ }
+
+ @Test
+ public void issue68() {
+ EWAHCompressedBitmap32 one = new EWAHCompressedBitmap32();
+ EWAHCompressedBitmap32 other = new EWAHCompressedBitmap32();
+ one.set(18308);
+ other.set(24608);
+ other = other.and(one);
+ Assert.assertEquals((long)other.getFirstSetBit(),-1);
+ other.set(82764);
+ Assert.assertEquals((long)other.getFirstSetBit(),(long)other.iterator().next());
+ }
+
+ @Test
+ public void issue70() {
+ EWAHCompressedBitmap32 one = new EWAHCompressedBitmap32();
+ EWAHCompressedBitmap32 other = new EWAHCompressedBitmap32();
+ one.set(16627);
+ other.set(52811);
+ other = other.and(one);
+ one = one.andNot(other);
+ one.set(16039);
+ other.set(78669);
+ other = other.or(one);
+ one = one.and(other);
+ other = other.andNot(one);
+ Assert.assertEquals((long)other.iterator().next(), 78669);
+ Assert.assertEquals((long)other.getFirstSetBit(), 78669);
+ }
+
@Test
public void swaptest() {
EWAHCompressedBitmap32 x = EWAHCompressedBitmap32.bitmapOf(1,2,3);
@@ -328,7 +429,6 @@ public class EWAHCompressedBitmap32Test {
public void testBug090b() throws Exception {
EWAHCompressedBitmap32 bm1 = new EWAHCompressedBitmap32();
bm1.setSizeInBits(8, false); // Create a bitmap with no bit set
- System.out.println(bm1.toDebugString());
EWAHCompressedBitmap32 bm2 = new EWAHCompressedBitmap32();
bm2.setSizeInBits(32, false); // Create a bitmap with no bit set
EWAHCompressedBitmap32 bm3 = new EWAHCompressedBitmap32();
@@ -373,7 +473,6 @@ public class EWAHCompressedBitmap32Test {
public void testBug090c() throws Exception {
EWAHCompressedBitmap32 bm1 = new EWAHCompressedBitmap32();
bm1.setSizeInBits(8, false); // Create a bitmap with no bit set
- System.out.println(bm1.toDebugString());
EWAHCompressedBitmap32 bm2 = new EWAHCompressedBitmap32();
bm2.setSizeInBits(64, false); // Create a bitmap with no bit set
EWAHCompressedBitmap32 bm3 = new EWAHCompressedBitmap32();
@@ -1235,9 +1334,6 @@ public class EWAHCompressedBitmap32Test {
EWAHCompressedBitmap32 and2 = new EWAHCompressedBitmap32();
FastAggregation32.bufferedandWithContainer(and2, 32, bitmaps[0],bitmaps[1],bitmaps[2]);
EWAHCompressedBitmap32 and3 = EWAHCompressedBitmap32.and(bitmaps[0],bitmaps[1],bitmaps[2]);
- System.out.println(and1.sizeInBits());
- System.out.println(and2.sizeInBits());
- System.out.println(and3.sizeInBits());
assertEqualsPositions(and1, and2);
assertEqualsPositions(and2, and3);
}
@@ -1842,8 +1938,6 @@ public class EWAHCompressedBitmap32Test {
for (int k = 1; k < ewah.length; ++k)
answer = answer.and(ewah[k]);
// result should be empty
- if (answer.toList().size() != 0)
- System.out.println(answer.toDebugString());
Assert.assertTrue(answer.toList().size() == 0);
Assert.assertTrue(EWAHCompressedBitmap32.and(ewah)
.toList().size() == 0);
@@ -1934,9 +2028,6 @@ public class EWAHCompressedBitmap32Test {
EWAHCompressedBitmap32.or(ewah));
int k = 0;
for (int j : answer) {
- if (k != j)
- System.out.println(answer
- .toDebugString());
Assert.assertEquals(k, j);
k += 1;
}
@@ -1970,8 +2061,6 @@ public class EWAHCompressedBitmap32Test {
}
int k = 0;
for (int j : answer) {
- if (k != j)
- System.out.println(answer.toDebugString());
Assert.assertEquals(k, j);
k += 1;
}
=====================================
src/test/java/com/googlecode/javaewah32/FuzzEWAH32Test.java
=====================================
@@ -0,0 +1,161 @@
+package com.googlecode.javaewah32;
+
+import java.util.Random;
+import org.junit.Test;
+// credit @svanmald
+ at SuppressWarnings("javadoc")
+public class FuzzEWAH32Test {
+ public static boolean areAssertsEnabled() {
+ boolean assertsEnabled = false;
+ assert assertsEnabled = true; // Intentional side effect!!!
+ return assertsEnabled;
+ }
+
+ @Test
+ public void testEwah() {
+ if(!areAssertsEnabled()) { throw new RuntimeException("asserts need to be enabled."); }
+ // ENABLE ASSERTS BEFORE EXECUTING TO ENABLE VALIDATION.
+ // if print = false and seed -1, the code will execute 10 random mutation to 2 bitmaps until infinity and each time validate the result
+ // Each time a set of 10 random mutations starts, a seed is printed (even if print = false).
+ // if one of the sets of 10 mutations fails validation, the printed seed can be used here together with print = true to reproduce the issue
+ System.out.println(" == Launching @svanmald's fuzzer! ");
+ testEwah(false, -1);
+ }
+
+ private void testEwah(boolean print, int seed) {
+ Random seedGenerator = new Random();
+ Mutation[] mutations = Mutation.values();
+ int times = 1000000;
+
+ while (times > 0) {
+ times --;
+ if((times % 10000) == 0) { System.out.print("."); System.out.flush(); }
+ int currentSeed = seed;
+ if (currentSeed == -1) {
+ currentSeed = seedGenerator.nextInt();
+ }
+ if (print) {
+ System.out.println("Seed " + currentSeed);
+ }
+ Random seededRandom = new Random(currentSeed);
+ EWAH32BitSetPair main = new EWAH32BitSetPair();
+ if (print) {
+ System.out.println("EWAHCompressedBitmap main = new EWAHCompressedBitmap();");
+ }
+ EWAH32BitSetPair other = new EWAH32BitSetPair();
+ if (print) {
+ System.out.println("EWAHCompressedBitmap other = new EWAHCompressedBitmap();");
+ }
+ for (int i = 0; i < 10; i++) {
+ Mutation mutation = mutations[seededRandom.nextInt(mutations.length)];
+ mutation.apply(print, seededRandom, main, other);
+ main.validate();
+ other.validate();
+ }
+ }
+ System.out.println();
+ }
+
+ public enum Mutation {
+ OR {
+ @Override
+ void apply(boolean print, Random random, EWAH32BitSetPair main, EWAH32BitSetPair other) {
+ if (random.nextDouble() < 0.5) {
+ if (print) {
+ System.out.println("main = main.or(other);");
+ }
+ main.or(other);
+ } else {
+ if (print) {
+ System.out.println("other = other.or(main);");
+ }
+ other.or(main);
+ }
+ }
+ },
+ AND {
+ @Override
+ void apply(boolean print, Random random, EWAH32BitSetPair main, EWAH32BitSetPair other) {
+ if (random.nextDouble() < 0.5) {
+ if (print) {
+ System.out.println("main = main.and(other);");
+ }
+ main.and(other);
+ } else {
+ if (print) {
+ System.out.println("other = other.and(main);");
+ }
+ other.and(main);
+ }
+ }
+ },
+ AND_NOT {
+ @Override
+ void apply(boolean print, Random random, EWAH32BitSetPair main, EWAH32BitSetPair other) {
+ if (random.nextDouble() < 0.5) {
+ if (print) {
+ System.out.println("main = main.andNot(other);");
+ }
+ main.andNot(other);
+ } else {
+ if (print) {
+ System.out.println("other = other.andNot(main);");
+ }
+ other.andNot(main);
+ }
+ }
+ },
+ XOR {
+ @Override
+ void apply(boolean print, Random random, EWAH32BitSetPair main, EWAH32BitSetPair other) {
+ if (random.nextDouble() < 0.5) {
+ if (print) {
+ System.out.println("main = main.xor(other);");
+ }
+ main.xor(other);
+ } else {
+ if (print) {
+ System.out.println("other = other.xor(main);");
+ }
+ other.xor(main);
+ }
+ }
+ },
+ SET {
+ @Override
+ void apply(boolean print, Random random, EWAH32BitSetPair main, EWAH32BitSetPair other) {
+ int value = random.nextInt(100_000);
+ if (random.nextDouble() < 0.5) {
+ if (print) {
+ System.out.println("main.set(" + value + ");");
+ }
+ main.set(value);
+ } else {
+ if (print) {
+ System.out.println("other.set(" + value + ");");
+ }
+ other.set(value);
+ }
+ }
+ },
+ CLEAR_RANDOM {
+ @Override
+ void apply(boolean print, Random random, EWAH32BitSetPair main, EWAH32BitSetPair other) {
+ int value = random.nextInt(100_000);
+ if (random.nextDouble() < 0.5) {
+ if (print) {
+ System.out.println("main.clear(" + value + ");");
+ }
+ main.clear(value);
+ } else {
+ if (print) {
+ System.out.println("other.clear(" + value + ");");
+ }
+ other.clear(value);
+ }
+ }
+ };
+
+ abstract void apply(boolean print, Random random, EWAH32BitSetPair main, EWAH32BitSetPair other);
+ }
+}
\ No newline at end of file
View it on GitLab: https://salsa.debian.org/java-team/libjavaewah-java/-/compare/27a43a4156e20f2ce14ef05af982cce3f1fd76a9...3df36f231843c12c4c43f070c1589440e73a82c7
--
View it on GitLab: https://salsa.debian.org/java-team/libjavaewah-java/-/compare/27a43a4156e20f2ce14ef05af982cce3f1fd76a9...3df36f231843c12c4c43f070c1589440e73a82c7
You're receiving this email because of your account on salsa.debian.org.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://alioth-lists.debian.net/pipermail/pkg-java-commits/attachments/20230621/320a30fc/attachment.htm>
More information about the pkg-java-commits
mailing list