[Git][java-team/jruby-joni][debian/sid] 32 commits: [maven-release-plugin] prepare for next development iteration
Hideki Yamane
gitlab at salsa.debian.org
Sat Jan 30 13:08:25 GMT 2021
Hideki Yamane pushed to branch debian/sid at Debian Java Maintainers / jruby-joni
Commits:
de3148b4 by Charles Oliver Nutter at 2020-06-03T20:46:09-05:00
[maven-release-plugin] prepare for next development iteration
- - - - -
66ac9653 by Henry Thompson at 2020-06-08T21:59:52+01:00
Add ability to specify gpos in search
- - - - -
005a1251 by dependabot[bot] at 2020-10-13T08:55:56+00:00
Bump junit from 4.12 to 4.13.1
Bumps [junit](https://github.com/junit-team/junit4) from 4.12 to 4.13.1.
- [Release notes](https://github.com/junit-team/junit4/releases)
- [Changelog](https://github.com/junit-team/junit4/blob/main/doc/ReleaseNotes4.12.md)
- [Commits](https://github.com/junit-team/junit4/compare/r4.12...r4.13.1)
Signed-off-by: dependabot[bot] <support at github.com>
- - - - -
d1dc1882 by Charles Oliver Nutter at 2020-10-21T12:34:27-05:00
Merge pull request #50 from jruby/dependabot/maven/junit-junit-4.13.1
Bump junit from 4.12 to 4.13.1
- - - - -
a2044c2b by kishorkunal-raj at 2020-11-04T11:41:38+00:00
Adding ppc64le architecture support on travis-ci
- - - - -
b8fca38c by Marcin Mielżyński at 2020-11-04T23:53:20+01:00
Merge pull request #51 from kishorkunal-raj/master
Adding ppc64le architecture support on travis-ci
- - - - -
0cc4a140 by Jirka Marsik at 2020-11-23T17:20:07+01:00
Check bounds more carefully when parsing regular expressions
This is based on Onigmo changeset 29e7e6aedebafd5efbbd90655c8e0d495035d7b4,
which itself is based on a series of fixes introduced in oniguruma to remove
out-of-bounds reads identified by fuzz testing.
- - - - -
fead69c3 by Jirka Marsik at 2020-11-23T17:45:27+01:00
Handle the case when Unicode character property escapes are not terminated
- - - - -
c37ed338 by Marcin Mielżyński at 2020-11-23T19:26:24+01:00
Merge pull request #52 from jirkamarsik/out-of-bounds-in-parser
Fix out-of-bounds reads when parsing \p{ and other regular expressions
- - - - -
6b84cb31 by Marcin Mielzynski at 2020-11-23T23:31:57+01:00
move upwards
- - - - -
f7575010 by Jirka Marsik at 2020-11-25T09:19:35+01:00
Remove unsound quantifier reduction rules
Some of the quantifier reduction rules in oniguruma are invalid, leading to new
regular expressions with different semantics. This commit removes two such
reduction rules.
https://bugs.ruby-lang.org/issues/17341
https://github.com/ruby/ruby/pull/3808
- - - - -
51c97f05 by Charles Oliver Nutter at 2020-12-02T11:15:15-06:00
Merge pull request #53 from jirkamarsik/remove-unsound-quantifier-reductions
Remove unsound quantifier reduction rules.
- - - - -
832213f2 by Karol Bucek at 2020-12-03T15:17:46+01:00
[refactor] cleanup code analysis warnings
- - - - -
27f8fbd4 by Karol Bucek at 2020-12-03T15:17:46+01:00
[refactor] cleanup redundant (duplicate) code
- - - - -
e38b105b by Karol Bucek at 2020-12-03T15:17:46+01:00
[fix] missing exception throws
- - - - -
fcccd5c7 by Karol Bucek at 2020-12-03T15:17:47+01:00
[refactor] avoid string concat warnings
- - - - -
70ee48f1 by Karol Bucek at 2020-12-03T15:17:47+01:00
[refactor] simplify (thread) interrupt check
changes the this.interrupted flag to also be checked less often
- - - - -
35ddadc3 by Karol Bucek at 2020-12-03T16:23:07+01:00
[feat] check interrupts increasingly (till ~ previous 30k)
- - - - -
92d39c0c by Karol Bucek at 2020-12-03T16:24:21+01:00
reset loop check 'static' local on interrupt
- - - - -
b496985b by Karol Bucek at 2020-12-03T16:24:41+01:00
[build] fix (test) compilation on Java 13
- - - - -
91f03ebd by Karol Bucek at 2020-12-06T15:32:18+01:00
[refactor] rename constant-like i-var to camelCase
- - - - -
546ecfb4 by Karol Bucek at 2020-12-10T11:54:27+01:00
Merge pull request #54 from kares/cleanup+interrupt
[refactor] 'reduce' interrupt check on every iteration
- - - - -
e48842c8 by Marcin Mielżyński at 2021-01-26T20:26:11+01:00
Merge pull request #55 from henry-thompson/henry-thompson/global-position
Add ability to specify gpos in search
- - - - -
39d8c211 by Marcin Mielzynski at 2021-01-26T20:31:26+01:00
synchronize another label
- - - - -
77c0d35b by Marcin Mielzynski at 2021-01-26T20:40:05+01:00
[maven-release-plugin] prepare release joni-2.1.41
- - - - -
f91603df by Hideki Yamane at 2021-01-30T21:33:54+09:00
Merge tag 'joni-2.1.41' into debian/sid
[maven-release-plugin] copy for tag joni-2.1.41
- - - - -
7fb78ff9 by Hideki Yamane at 2021-01-30T21:51:40+09:00
Adjust junit4 dependency version to 4.13.1
- - - - -
1b44613f by Hideki Yamane at 2021-01-30T21:52:06+09:00
Set Standards-Version: 4.5.1
- - - - -
2b740025 by Hideki Yamane at 2021-01-30T21:52:43+09:00
new upstream release
- - - - -
497c97cb by Hideki Yamane at 2021-01-30T22:00:39+09:00
Add debian/lintian-overrides
It's intended for jruby
- - - - -
d46c1074 by Hideki Yamane at 2021-01-30T22:05:03+09:00
update changelog
- - - - -
7c98c712 by Hideki Yamane at 2021-01-30T22:05:14+09:00
upload to unstable
- - - - -
21 changed files:
- .travis.yml
- debian/changelog
- debian/control
- + debian/lintian-overrides
- pom.xml
- src/org/joni/Analyser.java
- src/org/joni/ArrayCompiler.java
- src/org/joni/ByteCodeMachine.java
- src/org/joni/ByteCodePrinter.java
- src/org/joni/Lexer.java
- src/org/joni/Matcher.java
- src/org/joni/MinMaxLen.java
- src/org/joni/OptAnchorInfo.java
- src/org/joni/OptExactInfo.java
- src/org/joni/Parser.java
- src/org/joni/Regex.java
- src/org/joni/UnsetAddrList.java
- src/org/joni/ast/QuantifierNode.java
- src/org/joni/exception/ErrorMessages.java
- test/org/joni/test/Test.java
- test/org/joni/test/TestA.java
Changes:
=====================================
.travis.yml
=====================================
@@ -2,3 +2,8 @@ language: java
jdk:
- openjdk9
+
+arch:
+ - amd64
+ - ppc64le
+
=====================================
debian/changelog
=====================================
@@ -1,3 +1,13 @@
+jruby-joni (2.1.41-1) unstable; urgency=medium
+
+ * New upstream release
+ * debian/control
+ - set Standards-Version: 4.5.1
+ - Adjust Build-Depends: junit version to 4.13.1
+ * Add debian/lintian-overrides
+
+ -- Hideki Yamane <henrich at debian.org> Sat, 30 Jan 2021 22:05:11 +0900
+
jruby-joni (2.1.40-1) unstable; urgency=medium
* New upstream release
=====================================
debian/control
=====================================
@@ -9,10 +9,10 @@ Build-Depends:
debhelper-compat (= 13),
default-jdk
Build-Depends-Indep:
- junit4 (>= 4.12),
+ junit4 (>= 4.13.1),
libjcodings-java (>= 1.0.55),
maven-debian-helper (>= 2.0)
-Standards-Version: 4.5.0
+Standards-Version: 4.5.1
Vcs-Git: https://salsa.debian.org/java-team/jruby-joni.git
Vcs-Browser: https://salsa.debian.org/java-team/jruby-joni
Homepage: https://github.com/jruby/joni/
=====================================
debian/lintian-overrides
=====================================
@@ -0,0 +1 @@
+libjruby-joni-java: breakout-link usr/lib/jruby/lib/joni.jar -> usr/share/java/joni.jar
=====================================
pom.xml
=====================================
@@ -4,7 +4,7 @@
<groupId>org.jruby.joni</groupId>
<artifactId>joni</artifactId>
<packaging>jar</packaging>
- <version>2.1.40</version>
+ <version>2.1.41</version>
<name>Joni</name>
<description>
Java port of Oniguruma: http://www.geocities.jp/kosako3/oniguruma
@@ -57,7 +57,7 @@
<dependency>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
- <version>4.12</version>
+ <version>4.13.1</version>
<scope>test</scope>
</dependency>
</dependencies>
@@ -76,12 +76,14 @@
<plugin>
<artifactId>maven-compiler-plugin</artifactId>
<version>3.8.0</version>
+ <configuration>
+ <source>1.7</source>
+ <target>1.7</target>
+ </configuration>
<executions>
<execution>
<id>default-compile</id>
<configuration>
- <source>1.7</source>
- <target>1.7</target>
<excludes>
<exclude>module-info.java</exclude>
</excludes>
=====================================
src/org/joni/Analyser.java
=====================================
@@ -114,7 +114,6 @@ final class Analyser extends Parser {
regex.captureHistory = env.captureHistory;
regex.btMemStart = env.btMemStart;
- regex.btMemEnd = env.btMemEnd;
if (isFindCondition(regex.options)) {
regex.btMemEnd = bsAll();
@@ -478,9 +477,6 @@ final class Analyser extends Parser {
break;
case NodeType.CTYPE:
- min = 1;
- break;
-
case NodeType.CCLASS:
case NodeType.CANY:
min = 1;
@@ -559,9 +555,6 @@ final class Analyser extends Parser {
break;
case NodeType.CTYPE:
- max = enc.maxLength();
- break;
-
case NodeType.CCLASS:
case NodeType.CANY:
max = enc.maxLength();
@@ -722,8 +715,6 @@ final class Analyser extends Parser {
break;
case NodeType.CTYPE:
- len = 1;
-
case NodeType.CCLASS:
case NodeType.CANY:
len = 1;
@@ -1305,12 +1296,6 @@ final class Analyser extends Parser {
switch(node.getType()) {
case NodeType.LIST:
- ListNode ln = (ListNode)node;
- do {
- setupSubExpCall(ln.value);
- } while ((ln = ln.tail) != null);
- break;
-
case NodeType.ALT:
ListNode can = (ListNode)node;
do {
=====================================
src/org/joni/ArrayCompiler.java
=====================================
@@ -652,7 +652,7 @@ final class ArrayCompiler extends Compiler {
}
}
- int modTLen = 0;
+ int modTLen;
if (emptyInfo != 0) {
modTLen = tlen + (OPSize.NULL_CHECK_START + OPSize.NULL_CHECK_END);
} else {
=====================================
src/org/joni/ByteCodeMachine.java
=====================================
@@ -26,7 +26,6 @@ import static org.joni.Option.isFindLongest;
import static org.joni.Option.isFindNotEmpty;
import static org.joni.Option.isNotBol;
import static org.joni.Option.isNotEol;
-import static org.joni.Option.isPosixRegion;
import org.jcodings.CodeRange;
import org.jcodings.Encoding;
@@ -37,8 +36,8 @@ import org.joni.exception.ErrorMessages;
import org.joni.exception.InternalException;
class ByteCodeMachine extends StackMachine {
- private static final int INTERRUPT_CHECK_EVERY = 30000;
- int interruptCheckCounter = 0; // we modulos this to occasionally check for interrupts
+ private static final int MAX_INTERRUPT_CHECK_EVERY = 256 << 7; // 32768
+ int interruptCheckEvery = 256; // << 1 after each check up to ^^^
volatile boolean interrupted = false;
private int bestLen; // return value
@@ -60,6 +59,9 @@ class ByteCodeMachine extends StackMachine {
public void interrupt() {
interrupted = true;
+ // might have no effect on the executing thread but worth a try
+ // we might not succeed interrupting on next loop but will eventually
+ synchronized (this) { interruptCheckEvery = 0; }
}
protected int stkp; // a temporary
@@ -167,14 +169,13 @@ class ByteCodeMachine extends StackMachine {
return enc.isSingleByte() || (msaOptions & Option.CR_7_BIT) != 0 ? executeSb(interrupt) : execute(interrupt);
}
- private final int execute(boolean interrupt) throws InterruptedException {
- Thread currentThread = Thread.currentThread();
- final int[]code = this.code;
+ private final int execute(final boolean checkThreadInterrupt) throws InterruptedException {
+ final int[] code = this.code;
+ int interruptCheckCounter = 0;
while (true) {
- if (interrupted ||
- (interrupt && interruptCheckCounter++ % INTERRUPT_CHECK_EVERY == 0 && currentThread.isInterrupted())) {
- currentThread.interrupted();
- throw new InterruptedException();
+ if (interruptCheckCounter++ >= interruptCheckEvery) {
+ handleInterrupted(checkThreadInterrupt);
+ interruptCheckCounter = 0;
}
if (Config.DEBUG_MATCH) debugMatchLoop();
@@ -237,7 +238,7 @@ class ByteCodeMachine extends StackMachine {
case OPCode.MEMORY_START_PUSH: opMemoryStartPush(); continue;
case OPCode.MEMORY_START: opMemoryStart(); continue;
case OPCode.MEMORY_END_PUSH: opMemoryEndPush(); continue;
- case OPCode.MEMORY_END: opMemoryEnd(); continue;
+ case OPCode.MEMORY_END: opMemoryEnd(); continue; //
case OPCode.KEEP: opKeep(); continue;
case OPCode.MEMORY_END_PUSH_REC: opMemoryEndPushRec(); continue;
case OPCode.MEMORY_END_REC: opMemoryEndRec(); continue;
@@ -303,14 +304,13 @@ class ByteCodeMachine extends StackMachine {
} // main while
}
- private final int executeSb(boolean interrupt) throws InterruptedException {
- Thread currentThread = Thread.currentThread();
- final int[]code = this.code;
+ private final int executeSb(final boolean checkThreadInterrupt) throws InterruptedException {
+ final int[] code = this.code;
+ int interruptCheckCounter = 0;
while (true) {
- if (interrupted ||
- (interrupt && interruptCheckCounter++ % INTERRUPT_CHECK_EVERY == 0 && currentThread.isInterrupted())) {
- currentThread.interrupted();
- throw new InterruptedException();
+ if (interruptCheckCounter++ >= interruptCheckEvery) {
+ handleInterrupted(checkThreadInterrupt);
+ interruptCheckCounter = 0;
}
if (Config.DEBUG_MATCH) debugMatchLoop();
@@ -442,6 +442,14 @@ class ByteCodeMachine extends StackMachine {
} // main while
}
+ private void handleInterrupted(final boolean checkThreadInterrupt) throws InterruptedException {
+ if (interrupted || (checkThreadInterrupt && Thread.currentThread().isInterrupted())) {
+ Thread.currentThread().interrupted();
+ throw new InterruptedException();
+ }
+ interruptCheckEvery = Math.min(interruptCheckEvery << 1, MAX_INTERRUPT_CHECK_EVERY);
+ }
+
private boolean opEnd() {
int n = s - sstart;
@@ -1309,7 +1317,7 @@ class ByteCodeMachine extends StackMachine {
}
private void opBeginPosition() {
- if (s != msaStart) opFail();
+ if (s != msaGpos) opFail();
}
private void opMemoryStartPush() {
=====================================
src/org/joni/ByteCodePrinter.java
=====================================
@@ -26,7 +26,7 @@ import org.joni.constants.internal.OPSize;
import org.joni.exception.InternalException;
class ByteCodePrinter {
- final int[]code;
+ final int[] code;
final int codeLength;
final byte[][] templates;
final Encoding enc;
@@ -43,19 +43,19 @@ class ByteCodePrinter {
}
private void pString(StringBuilder sb, int len, int s) {
- sb.append(":");
+ sb.append(':');
while (len-- > 0) sb.append(new String(new byte[]{(byte)code[s++]}));
}
private void pLenString(StringBuilder sb, int len, int mbLen, int s) {
int x = len * mbLen;
- sb.append(":" + len + ":");
+ sb.append(':').append(len).append(':');
while (x-- > 0) sb.append(new String(new byte[]{(byte)code[s++]}));
}
private void pLenStringFromTemplate(StringBuilder sb, int len, int mbLen, byte[]tm, int idx) {
int x = len * mbLen;
- sb.append(":T:" + len + ":");
+ sb.append(":T:").append(len).append(':');
while (x-- > 0) sb.append(new String(tm, idx++, 1));
}
@@ -64,7 +64,7 @@ class ByteCodePrinter {
BitSet bs;
int tm, idx;
- sb.append("[" + OPCode.OpCodeNames[code[bp]]);
+ sb.append('[').append(OPCode.OpCodeNames[code[bp]]);
int argType = OPCode.OpCodeArgTypes[code[bp]];
int ip = bp;
if (argType != Arguments.SPECIAL) {
@@ -74,32 +74,32 @@ class ByteCodePrinter {
break;
case Arguments.RELADDR:
- sb.append(":(" + code[bp] + ")");
+ sb.append(":(").append(code[bp]).append(')');
bp += OPSize.RELADDR;
break;
case Arguments.ABSADDR:
- sb.append(":(" + code[bp] + ")");
+ sb.append(":(").append(code[bp]).append(')');
bp += OPSize.ABSADDR;
break;
case Arguments.LENGTH:
- sb.append(":" + code[bp]);
+ sb.append(':').append(code[bp]);
bp += OPSize.LENGTH;
break;
case Arguments.MEMNUM:
- sb.append(":" + code[bp]);
+ sb.append(':').append(code[bp]);
bp += OPSize.MEMNUM;
break;
case Arguments.OPTION:
- sb.append(":" + code[bp]);
+ sb.append(':').append(code[bp]);
bp += OPSize.OPTION;
break;
case Arguments.STATE_CHECK:
- sb.append(":" + code[bp]);
+ sb.append(':').append(code[bp]);
bp += OPSize.STATE_CHECK;
break;
}
@@ -203,10 +203,10 @@ class ByteCodePrinter {
bp += OPSize.INDEX;
idx = code[bp];
bp += OPSize.INDEX;
- sb.append(":T:" + mbLen + ":" + len + ":");
+ sb.append(":T:").append(mbLen).append(":").append(len).append(":");
while (n-- > 0) sb.append(new String(templates[tm], idx++, 1));
} else {
- sb.append(":" + mbLen + ":" + len + ":");
+ sb.append(":").append(mbLen).append(":").append(len).append(":");
while (n-- > 0) sb.append(new String(new byte[]{(byte)code[bp++]}));
}
@@ -243,7 +243,7 @@ class ByteCodePrinter {
System.arraycopy(code, bp, bs.bits, 0, BitSet.BITSET_SIZE);
n = bs.numOn();
bp += BitSet.BITSET_SIZE;
- sb.append(":" + n);
+ sb.append(':').append(n);
break;
case OPCode.CCLASS_NOT:
@@ -251,7 +251,7 @@ class ByteCodePrinter {
System.arraycopy(code, bp, bs.bits, 0, BitSet.BITSET_SIZE);
n = bs.numOn();
bp += BitSet.BITSET_SIZE;
- sb.append(":" + n);
+ sb.append(':').append(n);
break;
case OPCode.CCLASS_MB:
@@ -261,7 +261,7 @@ class ByteCodePrinter {
cod = code[bp];
//bp += OPSize.CODE_POINT;
bp += len;
- sb.append(":" + cod + ":" + len);
+ sb.append(':').append(cod).append(':').append(len);
break;
case OPCode.CCLASS_MIX:
@@ -275,18 +275,18 @@ class ByteCodePrinter {
cod = code[bp];
//bp += OPSize.CODE_POINT;
bp += len;
- sb.append(":" + n + ":" + cod + ":" + len);
+ sb.append(':').append(n).append(':').append(cod).append(':').append(len);
break;
case OPCode.BACKREFN_IC:
mem = code[bp];
bp += OPSize.MEMNUM;
- sb.append(":" + mem);
+ sb.append(':').append(mem);
break;
case OPCode.BACKREF_MULTI_IC:
case OPCode.BACKREF_MULTI:
- sb.append(" ");
+ sb.append(' ');
len = code[bp];
bp += OPSize.LENGTH;
for (int i=0; i<len; i++) {
@@ -300,11 +300,11 @@ class ByteCodePrinter {
case OPCode.BACKREF_WITH_LEVEL: {
int option = code[bp];
bp += OPSize.OPTION;
- sb.append(":" + option);
+ sb.append(':').append(option);
int level = code[bp];
bp += OPSize.LENGTH;
- sb.append(":" + level);
- sb.append(" ");
+ sb.append(':').append(level);
+ sb.append(' ');
len = code[bp];
bp += OPSize.LENGTH;
for (int i=0; i<len; i++) {
@@ -322,14 +322,14 @@ class ByteCodePrinter {
bp += OPSize.MEMNUM;
addr = code[bp];
bp += OPSize.RELADDR;
- sb.append(":" + mem + ":" + addr);
+ sb.append(':').append(mem).append(':').append(addr);
break;
case OPCode.PUSH_OR_JUMP_EXACT1:
case OPCode.PUSH_IF_PEEK_NEXT:
addr = code[bp];
bp += OPSize.RELADDR;
- sb.append(":(" + addr + ")");
+ sb.append(":(").append(addr).append(')');
pString(sb, 1, bp);
bp++;
break;
@@ -337,7 +337,7 @@ class ByteCodePrinter {
case OPCode.LOOK_BEHIND:
len = code[bp];
bp += OPSize.LENGTH;
- sb.append(":" + len);
+ sb.append(':').append(len);
break;
case OPCode.PUSH_LOOK_BEHIND_NOT:
@@ -345,7 +345,7 @@ class ByteCodePrinter {
bp += OPSize.RELADDR;
len = code[bp];
bp += OPSize.LENGTH;
- sb.append(":" + len + ":(" + addr + ")");
+ sb.append(':').append(len).append(":(").append(addr).append(')');
break;
case OPCode.STATE_CHECK_PUSH:
@@ -354,7 +354,7 @@ class ByteCodePrinter {
bp += OPSize.STATE_CHECK_NUM;
addr = code[bp];
bp += OPSize.RELADDR;
- sb.append(":" + scn + ":(" + addr + ")");
+ sb.append(':').append(scn).append(":(").append(addr).append(')');
break;
case OPCode.CONDITION:
@@ -362,7 +362,7 @@ class ByteCodePrinter {
bp += OPSize.MEMNUM;
addr = code[bp];
bp += OPSize.RELADDR;
- sb.append(":" + mem + ":" + addr);
+ sb.append(':').append(mem).append(":").append(addr);
break;
default:
@@ -370,20 +370,20 @@ class ByteCodePrinter {
}
}
- sb.append("]");
- if (Config.DEBUG_COMPILE_BYTE_CODE_INFO) sb.append("@" + ip + "(" + (bp - ip) + ")");
+ sb.append(']');
+ if (Config.DEBUG_COMPILE_BYTE_CODE_INFO) sb.append('@').append(ip).append('(').append(bp - ip).append(')');
return bp;
}
private String compiledByteCodeListToString() {
- StringBuilder sb = new StringBuilder("code length: " + codeLength + "\n");
+ StringBuilder sb = new StringBuilder("code length: ").append(codeLength).append('\n');
int ncode = -1;
int bp = 0;
int end = codeLength;
while (bp < end) {
ncode++;
- sb.append(ncode % 5 == 0 ? "\n" : " ");
+ sb.append(ncode % 5 == 0 ? '\n' : ' ');
bp = compiledByteCodeToString(sb, bp);
}
sb.append("\n");
=====================================
src/org/joni/Lexer.java
=====================================
@@ -111,6 +111,7 @@ class Lexer extends ScannerSupport {
if (syntax.opEscBraceInterval()) {
if (c != syntax.metaCharTable.esc) return invalidRangeQuantifier(synAllow);
+ if (!left()) return invalidRangeQuantifier(synAllow);
fetch();
}
@@ -272,6 +273,7 @@ class Lexer extends ScannerSupport {
if (c == '+' || c == '-') {
int flag = c == '-' ? -1 : 1;
+ if (!left()) newValueException(INVALID_CHAR_IN_GROUP_NAME);
fetch();
if (!enc.isDigit(c)) newValueException(INVALID_GROUP_NAME, src, stop);
unfetch();
@@ -280,8 +282,10 @@ class Lexer extends ScannerSupport {
rlevel.p = level * flag;
existLevel = true;
- fetch();
- isEndCode = c == endCode;
+ if (left()) {
+ fetch();
+ isEndCode = c == endCode;
+ }
}
if (!isEndCode) {
@@ -490,7 +494,7 @@ class Lexer extends ScannerSupport {
int to = this.stop;
boolean inEsc = false;
- int i=0;
+ int i;
while(p < to) {
if (inEsc) {
inEsc = false;
@@ -532,7 +536,7 @@ class Lexer extends ScannerSupport {
token.type = TokenType.CHAR_PROPERTY;
token.setPropNot(c == 'P');
- if (syntax.op2EscPBraceCircumflexNot()) {
+ if (left() && syntax.op2EscPBraceCircumflexNot()) {
c2 = fetchTo();
if (c2 == '^') {
token.setPropNot(!token.getPropNot());
@@ -692,6 +696,7 @@ class Lexer extends ScannerSupport {
break;
case 'p':
case 'P':
+ if (!left()) break;
fetchTokenInCCFor_p();
break;
case 'x':
@@ -979,7 +984,7 @@ class Lexer extends ScannerSupport {
token.type = TokenType.CHAR_PROPERTY;
token.setPropNot(c == 'P');
- if (syntax.op2EscPBraceCircumflexNot()) {
+ if (left() && syntax.op2EscPBraceCircumflexNot()) {
fetch();
if (c == '^') {
token.setPropNot(!token.getPropNot());
@@ -1309,7 +1314,7 @@ class Lexer extends ScannerSupport {
throw new CharacterPropertyException(EncodingError.ERR_INVALID_CHAR_PROPERTY_NAME, bytes, _p, last);
}
}
- newInternalException(PARSER_BUG);
+ newValueException(PROPERTY_NAME_NEVER_TERMINATED, _p, stop);
return 0; // not reached
}
=====================================
src/org/joni/Matcher.java
=====================================
@@ -44,6 +44,7 @@ public abstract class Matcher extends IntHolder {
protected final Region msaRegion;
protected int msaBestLen;
protected int msaBestS;
+ protected int msaGpos;
protected int msaBegin;
protected int msaEnd;
@@ -81,9 +82,10 @@ public abstract class Matcher extends IntHolder {
return msaEnd;
}
- protected final void msaInit(int option, int start) {
+ protected final void msaInit(int option, int start, int gpos) {
msaOptions = option;
msaStart = start;
+ msaGpos = gpos;
if (Config.USE_FIND_LONGEST_SEARCH_ALL_OF_RANGE) msaBestLen = -1;
}
@@ -100,7 +102,7 @@ public abstract class Matcher extends IntHolder {
}
private final int matchCommon(int at, int range, int option, boolean interrupt) throws InterruptedException {
- msaInit(option, at);
+ msaInit(option, at, at);
if (Config.USE_CEC) {
int offset = at = str;
@@ -298,17 +300,29 @@ public abstract class Matcher extends IntHolder {
public final int search(int start, int range, int option) {
try {
- return searchCommon(start, range, option, false);
+ return searchCommon(start, start, range, option, false);
+ } catch (InterruptedException ex) {
+ return INTERRUPTED;
+ }
+ }
+
+ public final int search(int gpos, int start, int range, int option) {
+ try {
+ return searchCommon(gpos, start, range, option, false);
} catch (InterruptedException ex) {
return INTERRUPTED;
}
}
public final int searchInterruptible(int start, int range, int option) throws InterruptedException {
- return searchCommon(start, range, option, true);
+ return searchCommon(start, start, range, option, true);
+ }
+
+ public final int searchInterruptible(int gpos, int start, int range, int option) throws InterruptedException {
+ return searchCommon(gpos, start, range, option, true);
}
- private final int searchCommon(int start, int range, int option, boolean interrupt) throws InterruptedException {
+ private final int searchCommon(int gpos, int start, int range, int option, boolean interrupt) throws InterruptedException {
int s, prev;
int origStart = start;
int origRange = range;
@@ -325,7 +339,13 @@ public abstract class Matcher extends IntHolder {
/* search start-position only */
// !begin_position:!
if (range > start) {
- range = start + 1;
+ if (gpos > start) {
+ if (gpos < range) {
+ range = gpos + 1;
+ } else {
+ range = start + 1;
+ }
+ }
} else {
range = start;
}
@@ -369,7 +389,13 @@ public abstract class Matcher extends IntHolder {
} else if ((regex.anchor & AnchorType.ANYCHAR_STAR_ML) != 0) {
// goto !begin_position;!
if (range > start) {
- range = start + 1;
+ if (gpos > start) {
+ if (gpos < range) {
+ range = gpos + 1;
+ } else {
+ range = start + 1;
+ }
+ }
} else {
range = start;
}
@@ -382,7 +408,7 @@ public abstract class Matcher extends IntHolder {
if (regex.thresholdLength == 0) {
s = start = str;
prev = -1;
- msaInit(option, start);
+ msaInit(option, start, start);
if (Config.USE_CEC) stateCheckBuffClear();
@@ -394,7 +420,7 @@ public abstract class Matcher extends IntHolder {
if (Config.DEBUG_SEARCH) debugSearch(str, end, start, range);
- msaInit(option, origStart);
+ msaInit(option, origStart, gpos);
if (Config.USE_CEC) {
int offset = Math.min(start, range) - str;
stateCheckBuffInit(end - str, offset, regex.numCombExpCheck);
=====================================
src/org/joni/MinMaxLen.java
=====================================
@@ -54,9 +54,7 @@ final class MinMaxLen {
if (v2 > v1) return 1;
if (v2 < v1) return -1;
- if (other.min < min) return 1;
- if (other.min > min) return -1;
- return 0;
+ return Integer.compare(min, other.min);
}
boolean equal(MinMaxLen other) {
=====================================
src/org/joni/OptAnchorInfo.java
=====================================
@@ -79,7 +79,7 @@ final class OptAnchorInfo implements AnchorType {
}
static String anchorToString(int anchor) {
- StringBuffer s = new StringBuffer("[");
+ StringBuilder s = new StringBuilder("[");
if ((anchor & AnchorType.BEGIN_BUF) !=0 ) s.append("begin-buf ");
if ((anchor & AnchorType.BEGIN_LINE) !=0 ) s.append("begin-line ");
=====================================
src/org/joni/OptExactInfo.java
=====================================
@@ -73,7 +73,7 @@ final class OptExactInfo {
}
length = i;
- reachEnd = (p == end ? other.reachEnd : false);
+ reachEnd = (p == end && other.reachEnd);
OptAnchorInfo tmp = new OptAnchorInfo();
tmp.concat(anchor, other.anchor, 1, 1);
=====================================
src/org/joni/Parser.java
=====================================
@@ -469,6 +469,7 @@ class Parser extends Lexer {
} // USE_NAMED_GROUP
break;
case '<': /* look behind (?<=...), (?<!...) */
+ if (!left()) newSyntaxException(END_PATTERN_WITH_UNMATCHED_PARENTHESIS);
fetch();
if (c == '=') {
node = new AnchorNode(AnchorType.LOOK_BEHIND);
@@ -495,7 +496,7 @@ class Parser extends Lexer {
case '@':
if (syntax.op2AtMarkCaptureHistory()) {
if (Config.USE_NAMED_GROUP) {
- if (syntax.op2QMarkLtNamedGroup()) {
+ if (left() && syntax.op2QMarkLtNamedGroup()) {
fetch();
if (c == '<' || c == '\'') {
listCapture = true;
@@ -515,7 +516,7 @@ class Parser extends Lexer {
break;
case '(': /* conditional expression: (?(cond)yes), (?(cond)yes|no) */
- if (syntax.op2QMarkLParenCondition()) {
+ if (left() && syntax.op2QMarkLParenCondition()) {
int num = -1;
int name = -1;
fetch();
@@ -647,8 +648,7 @@ class Parser extends Lexer {
} // switch
if (c == ')') {
- EncloseNode en = EncloseNode.newOption(option);
- node = en;
+ node = EncloseNode.newOption(option);
returnCode = 2; /* option only */
return node;
} else if (c == ':') {
@@ -679,8 +679,7 @@ class Parser extends Lexer {
return node;
}
EncloseNode en = EncloseNode.newMemory(env.option, false);
- int num = env.addMemEntry();
- en.regNum = num;
+ en.regNum = env.addMemEntry();
node = en;
}
@@ -714,27 +713,25 @@ class Parser extends Lexer {
private Node parseEncloseNamedGroup2(boolean listCapture) {
int nm = p;
- int num = fetchName(c, false);
+ fetchName(c, false);
int nameEnd = value;
- num = env.addMemEntry();
+ int num = env.addMemEntry();
if (listCapture && num >= BitStatus.BIT_STATUS_BITS_NUM) newValueException(GROUP_NUMBER_OVER_FOR_CAPTURE_HISTORY);
regex.nameAdd(bytes, nm, nameEnd, num, syntax);
EncloseNode en = EncloseNode.newMemory(env.option, true);
en.regNum = num;
- Node node = en;
-
if (listCapture) env.captureHistory = bsOnAtSimple(env.captureHistory, num);
env.numNamed++;
- return node;
+ return en;
}
private int findStrPosition(int[]s, int n, int from, int to, Ptr nextChar) {
int x;
int q;
int p = from;
- int i = 0;
+ int i;
while (p < to) {
x = enc.mbcToCode(bytes, p, to);
q = p + enc.length(bytes, p, to);
@@ -925,7 +922,7 @@ class Parser extends Lexer {
case '+': lower = 1; break;
case '*': break;
case '2': lower = upper = 2; break;
- default : new InternalException(ErrorMessages.PARSER_BUG);
+ default : throw new InternalException(ErrorMessages.PARSER_BUG);
}
quantifierNode(nodes, np, lower, upper);
@@ -954,7 +951,7 @@ class Parser extends Lexer {
return np;
}
- private static int NODE_COMMON_SIZE = 16;
+ private static final int NODE_COMMON_SIZE = 16;
private Node parseExtendedGraphemeCluster() {
final Node[] nodes = new Node[NODE_COMMON_SIZE];
final int anyTargetPosition;
=====================================
src/org/joni/Regex.java
=====================================
@@ -257,7 +257,7 @@ public final class Regex {
if (nameTable != null) {
sb.append("name table\n");
for (NameEntry ne : nameTable) {
- sb.append(" " + ne + "\n");
+ sb.append(" ").append(ne).append("\n");
}
sb.append("\n");
}
=====================================
src/org/joni/UnsetAddrList.java
=====================================
@@ -50,7 +50,7 @@ public final class UnsetAddrList {
public void fix(Regex regex) {
for (int i=0; i<num; i++) {
EncloseNode en = targets[i];
- if (!en.isAddrFixed()) new InternalException(ErrorMessages.PARSER_BUG);
+ if (!en.isAddrFixed()) throw new InternalException(ErrorMessages.PARSER_BUG);
regex.code[offsets[i]] = en.callAddr; // is this safe ?
}
}
@@ -58,7 +58,9 @@ public final class UnsetAddrList {
public String toString() {
StringBuilder value = new StringBuilder();
if (num > 0) {
- for (int i = 0; i < num; i++) value.append("offset + " + offsets[i] + " target: " + targets[i].getAddressName());
+ for (int i = 0; i < num; i++) {
+ value.append("offset + ").append(offsets[i]).append(" target: ").append(targets[i].getAddressName());
+ }
}
return value.toString();
}
=====================================
src/org/joni/ast/QuantifierNode.java
=====================================
@@ -23,7 +23,6 @@ import static org.joni.ast.QuantifierNode.ReduceType.A;
import static org.joni.ast.QuantifierNode.ReduceType.AQ;
import static org.joni.ast.QuantifierNode.ReduceType.ASIS;
import static org.joni.ast.QuantifierNode.ReduceType.DEL;
-import static org.joni.ast.QuantifierNode.ReduceType.PQ_Q;
import static org.joni.ast.QuantifierNode.ReduceType.P_QQ;
import static org.joni.ast.QuantifierNode.ReduceType.QQ;
@@ -137,7 +136,6 @@ public final class QuantifierNode extends StateNode {
AQ, /* to '*?' */
QQ, /* to '??' */
P_QQ, /* to '+)??' */
- PQ_Q, /* to '+?)?' */
}
static final ReduceType[][]REDUCE_TABLE = {
@@ -146,7 +144,7 @@ public final class QuantifierNode extends StateNode {
{A, A, DEL, ASIS, P_QQ, DEL}, /* '+' */
{DEL, AQ, AQ, DEL, AQ, AQ}, /* '??' */
{DEL, DEL, DEL, DEL, DEL, DEL}, /* '*?' */
- {ASIS, PQ_Q, DEL, AQ, AQ, DEL} /* '+?' */
+ {ASIS, ASIS, ASIS, AQ, AQ, DEL} /* '+?' */
};
@@ -193,16 +191,6 @@ public final class QuantifierNode extends StateNode {
other.greedy = true;
return;
- case PQ_Q:
- setTarget(other);
- lower = 0;
- upper = 1;
- greedy = true;
- other.lower = 1;
- other.upper = REPEAT_INFINITE;
- other.greedy = false;
- return;
-
case ASIS:
setTarget(other);
return;
=====================================
src/org/joni/exception/ErrorMessages.java
=====================================
@@ -77,6 +77,7 @@ public interface ErrorMessages extends org.jcodings.exception.ErrorMessages {
final String UNDEFINED_GROUP_REFERENCE = "undefined group <%n> reference";
final String MULTIPLEX_DEFINED_NAME = "multiplex defined name <%n>";
final String MULTIPLEX_DEFINITION_NAME_CALL = "multiplex definition name <%n> call";
+ final String PROPERTY_NAME_NEVER_TERMINATED = "property name never terminated \\p{%n";
final String NEVER_ENDING_RECURSION = "never ending recursion";
final String GROUP_NUMBER_OVER_FOR_CAPTURE_HISTORY = "group number is too big for capture history";
final String NOT_SUPPORTED_ENCODING_COMBINATION = "not supported encoding combination";
=====================================
test/org/joni/test/Test.java
=====================================
@@ -99,6 +99,10 @@ public abstract class Test {
xx(pattern, str, from, to, mem, not, option());
}
+ public void xx(byte[] pattern, byte[] str, int gpos, int from, int to, int mem, boolean not) throws InterruptedException {
+ xx(pattern, str, gpos, 0, from, to, mem, not, option());
+ }
+
static boolean is7bit(byte[]bytes, int p, int end) {
for (int i = p; i < end; i++) {
if ((bytes[i] & 0xff) >= 0x80) return false;
@@ -107,6 +111,10 @@ public abstract class Test {
}
public int xx(byte[] pattern, byte[] str, int from, int to, int mem, boolean not, int option) throws InterruptedException {
+ return xx(pattern, str, 0, 0, from, to, mem, not, option);
+ }
+
+ public int xx(byte[] pattern, byte[] str, int gpos, int searchStart, int from, int to, int mem, boolean not, int option) throws InterruptedException {
Regex reg;
try {
@@ -126,18 +134,18 @@ public abstract class Test {
}
if ((!encoding().isSingleByte()) && encoding().isAsciiCompatible() && is7bit(str, 0, str.length)) {
- check(reg, pattern, str, option | Option.CR_7_BIT, from, to, mem, not);
+ check(reg, pattern, str, option | Option.CR_7_BIT, gpos, searchStart, from, to, mem, not);
}
- return check(reg, pattern, str, option, from, to, mem, not);
+ return check(reg, pattern, str, option, gpos, searchStart, from, to, mem, not);
}
- private int check(Regex reg, byte[]pattern, byte[]str, int option, int from, int to, int mem, boolean not) throws InterruptedException {
+ private int check(Regex reg, byte[]pattern, byte[]str, int option, int gpos, int searchStart, int from, int to, int mem, boolean not) throws InterruptedException {
Matcher m = reg.matcher(str, 0, length(str));
final Region region;
final int result;
try {
- result = m.searchInterruptible(0, length(str), option);
+ result = m.searchInterruptible(gpos, searchStart, length(str), option);
region = m.getEagerRegion();
} catch (JOniException je) {
Config.err.println("Pattern: " + reprTest(pattern, str, option));
@@ -160,7 +168,8 @@ public abstract class Test {
if (VERBOSE) Config.log.println("OK(NOT): " + reprTest(pattern, str, option));
nsucc++;
} else {
- Config.log.println("FAIL: " + reprTest(pattern, str, option));
+ Config.log.println("FAIL: " + reprTest(pattern, str, option) + " GPOS: "
+ + gpos + " Start: " + searchStart);
nfail++;
}
} else {
@@ -172,8 +181,9 @@ public abstract class Test {
if (VERBOSE) Config.log.println("OK: " + reprTest(pattern, str, option));
nsucc++;
} else {
- Config.log.println("FAIL: " + reprTest(pattern, str, option) + " Groups: [Exp " + from + "-" + to + ", Act "
- + region.beg[mem] + "-" + region.end[mem] + "]");
+ Config.log.println("FAIL: " + reprTest(pattern, str, option) + " GPOS: " + gpos + " Start: "
+ + searchStart + " Groups: [Exp " + from + "-" + to + ", Act " + region.beg[mem] + "-"
+ + region.end[mem] + "]");
nfail++;
}
}
@@ -201,6 +211,10 @@ public abstract class Test {
xx(pattern, str, 0, 0, 0, true, option);
}
+ protected void n(byte[] pattern, byte[] str, int gpos, int option) throws InterruptedException {
+ xx(pattern, str, gpos, 0, 0, 0, 0, true, option);
+ }
+
public void xxs(String pattern, String str, int from, int to, int mem, boolean not) throws InterruptedException {
xxs(pattern, str, from, to, mem, not, option());
}
@@ -227,6 +241,19 @@ public abstract class Test {
}
}
+ public int x2s(String pattern, String str, int gpos, int searchStart, int from, int to) throws InterruptedException {
+ return x2s(pattern, str, gpos, searchStart, from, to, option());
+ }
+
+ public int x2s(String pattern, String str, int gpos, int searchStart, int from, int to, int option) throws InterruptedException {
+ try {
+ return xx(pattern.getBytes(testEncoding()), str.getBytes(testEncoding()), gpos, searchStart, from, to, 0, false, option);
+ } catch (UnsupportedEncodingException uee) {
+ uee.printStackTrace();
+ return Matcher.FAILED;
+ }
+ }
+
public void x3s(String pattern, String str, int from, int to, int mem) throws InterruptedException {
x3s(pattern, str, from, to, mem, option());
}
@@ -244,8 +271,12 @@ public abstract class Test {
}
public void ns(String pattern, String str, int option) throws InterruptedException {
+ ns(pattern, str, 0, option);
+ }
+
+ public void ns(String pattern, String str, int gpos, int option) throws InterruptedException {
try {
- xx(pattern.getBytes(testEncoding()), str.getBytes(testEncoding()), 0, 0, 0, true, option);
+ xx(pattern.getBytes(testEncoding()), str.getBytes(testEncoding()), gpos, 0, 0, 0, 0, true, option);
} catch (UnsupportedEncodingException uee) {
uee.printStackTrace();
}
=====================================
test/org/joni/test/TestA.java
=====================================
@@ -542,5 +542,14 @@ public class TestA extends Test {
x2s("([.])", ".", 0, 1);
x2s("([a])", "a", 0, 1);
x2s("([\\w])", "a", 0, 1);
+
+ // gpos
+ ns("\\Gabc", "123abcdef", 2, Option.DEFAULT);
+ x2s("\\Gabc", "123abcdef", 3, 0, 3, 6);
+ x2s("\\Gabc", "123abcdef", 3, 3, 3, 6);
+ ns("\\Gabc", "123abcdef", 0, 3);
+
+ x2s("(?!\\G)", "abcd", 2, 3, 3, 3);
+ x2s("(?!\\G)", "abcd", 3, 3, 4, 4);
}
}
View it on GitLab: https://salsa.debian.org/java-team/jruby-joni/-/compare/bc27db5ab17964d4ebd6db185094b6d49d309405...7c98c7124f24ecc5fbc2ccda71f9166a64362666
--
View it on GitLab: https://salsa.debian.org/java-team/jruby-joni/-/compare/bc27db5ab17964d4ebd6db185094b6d49d309405...7c98c7124f24ecc5fbc2ccda71f9166a64362666
You're receiving this email because of your account on salsa.debian.org.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://alioth-lists.debian.net/pipermail/pkg-java-commits/attachments/20210130/49f7dc99/attachment.html>
More information about the pkg-java-commits
mailing list