[Git][java-team/groovycsv][master] 19 commits: New upstream version 1.3
Andrius Merkys
gitlab at salsa.debian.org
Tue Nov 5 15:37:03 GMT 2019
Andrius Merkys pushed to branch master at Debian Java Maintainers / groovycsv
Commits:
d80a08c2 by Andrius Merkys at 2019-11-04T16:48:36Z
New upstream version 1.3
- - - - -
3380a4db by Andrius Merkys at 2019-11-04T16:48:36Z
Update upstream source from tag 'upstream/1.3'
Update to upstream version '1.3'
with Debian dir 5e8d8f6889e09ecdb6eeabd082d8aed18b09010a
- - - - -
36c0fbac by Andrius Merkys at 2019-11-04T16:49:03Z
Preparing to package v1.3.
- - - - -
b61f65e8 by Andrius Merkys at 2019-11-05T05:47:58Z
Removing patches, as all of them seem to be integrated upstream.
- - - - -
11ca05fa by Andrius Merkys at 2019-11-05T05:49:00Z
Wrapping and sorting.
- - - - -
9006307f by Andrius Merkys at 2019-11-05T05:49:15Z
Bumping Standards-Version (no changes).
- - - - -
6df2212c by Andrius Merkys at 2019-11-05T05:49:56Z
Bumping debhelper compatibility level.
- - - - -
4e672b4a by Andrius Merkys at 2019-11-05T05:51:48Z
Updating Vcs-* tags in debian/control.
- - - - -
4c3c28c2 by Andrius Merkys at 2019-11-05T06:58:32Z
Updating changelog.
- - - - -
6ec19b7f by Andrius Merkys at 2019-11-05T13:56:20Z
Adding a patch to enable the compilation of tests.
- - - - -
e49c46bc by Andrius Merkys at 2019-11-05T13:56:55Z
Passing through the test dependencies.
- - - - -
8529bc00 by Andrius Merkys at 2019-11-05T14:57:25Z
Enabling build time tests.
- - - - -
994002d7 by Andrius Merkys at 2019-11-05T14:59:11Z
Adding a patch migrating the code to use opencsv v5.
- - - - -
0659dfe8 by Andrius Merkys at 2019-11-05T15:03:00Z
Recording the test dependencies.
- - - - -
248298cb by Andrius Merkys at 2019-11-05T15:10:54Z
Honoring DEB_BUILD_OPTIONS=nocheck. Removing 'get-orig-source' target.
- - - - -
3cc6ffa0 by Andrius Merkys at 2019-11-05T15:11:49Z
Switching URLs from HTTP to HTTPS.
- - - - -
657285fc by Andrius Merkys at 2019-11-05T15:18:44Z
Preparing for the upload.
- - - - -
5645e35c by Andrius Merkys at 2019-11-05T15:20:14Z
Adding patch descriptions.
- - - - -
5dd66a06 by Andrius Merkys at 2019-11-05T15:22:07Z
Making the code in migrate-to-opencsv-v5.patch prettier.
- - - - -
26 changed files:
- .gitignore
- LICENSE
- README.md
- build.gradle
- debian/changelog
- debian/compat
- debian/control
- debian/copyright
- debian/maven.ignoreRules
- − debian/maven.rules
- − debian/patches/01-ignore-gpg-plugin.patch
- − debian/patches/02-gradle-compatibility.patch
- + debian/patches/enable-tests.patch
- + debian/patches/migrate-to-opencsv-v5.patch
- debian/patches/series
- debian/rules
- gradle.properties.example
- src/com/xlson/groovycsv/AutoDetectHandler.groovy
- src/com/xlson/groovycsv/CsvIterator.groovy
- src/com/xlson/groovycsv/CsvParser.groovy
- src/com/xlson/groovycsv/PropertyMapper.groovy
- test/com/xlson/groovycsv/AutoDetectHandlerSpec.groovy
- test/com/xlson/groovycsv/ConfigurableColumnNamesSpec.groovy
- test/com/xlson/groovycsv/CsvIteratorSpec.groovy
- test/com/xlson/groovycsv/CsvParserSpec.groovy
- + test/com/xlson/groovycsv/PropertyMapperSpec.groovy
Changes:
=====================================
.gitignore
=====================================
@@ -3,3 +3,5 @@ build/*
lib/*
_site
gradle.properties
+.idea
+groovycsv.iml
\ No newline at end of file
=====================================
LICENSE
=====================================
@@ -1,4 +1,4 @@
- Copyright 2010 Leonard Axelsson
+ Copyright 2016 Leonard Gram
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
=====================================
README.md
=====================================
@@ -2,23 +2,18 @@
GroovyCSV is a library for Groovy which aims to make csv data
easier (and more idiomatically Groovy) to work with. The library was inspired by @[goeh's](http://twitter.com/goeh)
-[ExcelBuilder](http://www.technipelago.se/blog/?p=44) that lets you
+[ExcelBuilder](http://www.technipelago.se/blog/show/groovy-poi-excel) that lets you
iterate over rows in the excel file using `eachLine` and access values
using the column names.
-*Important*
-
-Package structure was changed from `com.xlson.csvparser` to
-`com.xlson.groovycsv` between release 0.1 and 0.2.
-
## Features
* Value-access by header name or position
* Iteration using the ordinary collection methods (`findAll`, `collect`
and so on)
-* Full support for OpenCSV's configurability
* Support for guessing separator and/or quote character
* Support for reading csv without headers
+* Support for skipping initial lines of the csv
## Example
@@ -26,13 +21,13 @@ The parse method returns an iterator over the rows in the csv. This
means we can use any of the default groovy ways to iterate, in this
example we see the for each loop in use.
- @Grab('com.xlson.groovycsv:groovycsv:1.0')
+ @Grab('com.xlson.groovycsv:groovycsv:1.3')
import static com.xlson.groovycsv.CsvParser.parseCsv
-
+
def csv = '''Name,Lastname
Mark,Andersson
Pete,Hansen'''
-
+
def data = parseCsv(csv)
for(line in data) {
println "$line.Name $line.Lastname"
@@ -47,8 +42,7 @@ The parse method takes a String or a Reader as argument.
## Getting GroovyCSV
-GroovyCSV is available in Maven Central. It is also available directly from GitHub
-(links below).
+GroovyCSV is available through Maven Central.
### Maven & Ivy configuration
@@ -56,27 +50,29 @@ GroovyCSV is available in Maven Central. It is also available directly from GitH
* *GroupId:* com.xlson.groovycsv
* *ArtifactId:* groovycsv
-* *Version:* 1.0
+* *Version:* 1.3
#### Latest snapshot
-* *Version:* 1.0-SNAPSHOT
+* *Version:* 1.3-SNAPSHOT
* *Repository:* https://oss.sonatype.org/content/groups/public/
### Downloads
-*GroovyCSV 1.0*
+*GroovyCSV 1.3*
+
-* [groovycsv-1.0.jar](https://github.com/downloads/xlson/groovycsv/groovycsv-0.2.jar)
-* [groovycsv-1.0-javadoc.jar](https://github.com/downloads/xlson/groovycsv/groovycsv-0.2-javadoc.jar)
-* [Javadoc Online](http://xlson.github.com/groovycsv/docs/1.0/javadoc/)
+* [groovycsv-1.3.jar](http://repo1.maven.org/maven2/com/xlson/groovycsv/groovycsv/1.3/groovycsv-1.3.jar)
+* [groovycsv-1.3-javadoc.jar](http://repo1.maven.org/maven2/com/xlson/groovycsv/groovycsv/1.3/groovycsv-1.3-javadoc.jar)
+* [Javadoc Online](http://xlson.github.com/groovycsv/docs/1.3/javadoc/)
## Dependencies
-* [Groovy 1.7.x](http://groovy.codehaus.org)
-* [OpenCSV 2.x](http://opencsv.sourceforge.net/)
+* [Groovy 1.8.x](http://groovy.codehaus.org) or later
+* [OpenCSV 4.x](http://opencsv.sourceforge.net/)
+
-Many thanks to Glen Smith and the other's in the OpenCSV team for
+Many thanks to everyone who's contributed to the project and everyone in the OpenCSV team for
doing all the heavy lifting.
## Building
=====================================
build.gradle
=====================================
@@ -2,7 +2,7 @@ buildscript {
repositories {
mavenCentral()
}
-
+
dependencies {
classpath 'de.huxhorn.gradle:de.huxhorn.gradle.pgp-plugin:0.0.4'
}
@@ -10,23 +10,23 @@ buildscript {
apply plugin: 'groovy'
apply plugin: 'maven'
-apply plugin: de.huxhorn.gradle.pgp.PgpPlugin
+apply plugin: 'signing'
apply plugin: 'idea'
repositories {
mavenCentral()
- mavenRepo urls: "http://m2repo.spockframework.org/snapshots"
+ maven { url "http://m2repo.spockframework.org/snapshots" }
}
dependencies {
- groovy group: 'org.codehaus.groovy', name: 'groovy-all', version: '1.7.3'
- compile 'net.sf.opencsv:opencsv:2.1'
- testCompile "org.spockframework:spock-core:0.4-groovy-1.7"
- testCompile "cglib:cglib-nodep:2.2"
- testCompile "org.objenesis:objenesis:1.2"
+ compile 'com.opencsv:opencsv:4.0'
+ compileOnly 'org.codehaus.groovy:groovy-all:1.8.8'
+ testCompile 'org.spockframework:spock-core:1.1-groovy-2.4-rc-2'
+ testCompile 'cglib:cglib-nodep:2.2'
+ testCompile 'org.objenesis:objenesis:1.2'
}
-version = '1.0'
+version = '1.3'
group = 'com.xlson.groovycsv'
sourceSets {
@@ -53,86 +53,102 @@ task sourcesJar(type: Jar) {
}
artifacts {
- archives groovydocJar
- archives sourcesJar
+ archives groovydocJar, sourcesJar
+}
+if(isSetupForDeployToMavenCentral()) {
+ signing {
+ sign configurations.archives
+ }
+}
+task wrapper(type: Wrapper) {
+ gradleVersion = '4.3.1'
+}
+
+task downloadDeps << {
+ def libDir = file('lib')
+ ant.delete(dir: libDir)
+ copy {
+ from configurations.testRuntime
+ into libDir
+ }
}
-def deployer = null
-def installer = install.repositories.mavenInstaller
-uploadArchives {
- // Checks if login information for the logon repo is set correctly.
- // Should be set in gradle.properties (check gradle.properties.example)
- if(project.hasProperty('repoUserName') && project.hasProperty('repoPassword')) {
- repositories {
- deployer = mavenDeployer {
- configureAuth = {
- authentication(userName: repoUserName, password: repoPassword)
- }
- snapshotRepository(url: "https://oss.sonatype.org/content/repositories/snapshots/", configureAuth)
- repository(url: "https://oss.sonatype.org/service/local/staging/deploy/maven2/", configureAuth)
- }
- }
+def isSetupForDeployToMavenCentral() {
+ def requiredProperties = ['ossrhUsername',
+ 'ossrhPassword',
+ 'signing.keyId',
+ 'signing.password',
+ 'signing.secretKeyRingFile']
+ for(prop in requiredProperties) {
+ if(!project.hasProperty(prop)) {
+ return false
}
+ }
+ return true
}
-[installer, deployer]*.pom*.whenConfigured {pom ->
- pom.project {
+uploadArchives {
+ repositories {
+ mavenDeployer {
+ beforeDeployment { MavenDeployment deployment -> signing.signPom(deployment) }
+
+ if(isSetupForDeployToMavenCentral()) {
+ repository(url: "https://oss.sonatype.org/service/local/staging/deploy/maven2/") {
+ authentication(userName: ossrhUsername, password: ossrhPassword)
+ }
+
+ snapshotRepository(url: "https://oss.sonatype.org/content/repositories/snapshots/") {
+ authentication(userName: ossrhUsername, password: ossrhPassword)
+ }
+ }
+
+ pom.project {
name 'GroovyCSV'
- packaging 'jar' // not working
+ packaging 'jar'
+ // optionally artifactId can be defined here
description 'Library for parsing csv in Groovy'
url 'http://github.com/xlson/groovycsv'
inceptionYear '2010'
-
+
scm {
- url 'scm:git:git at github.com:xlson/groovycsv.git'
- connection 'http://github.com/xlson/groovycsv'
+ connection 'scm:git:git at github.com:xlson/groovycsv.git'
+ developerConnection 'scm:git:git at github.com:xlson/groovycsv.git'
+ url 'http://github.com/xlson/groovycsv'
}
-
- licenses {
- license {
- name 'The Apache Software License, Version 2.0'
- url 'http://www.apache.org/licenses/LICENSE-2.0.txt'
- distribution 'repo'
- }
- }
-
- developers {
- developer {
- id 'xlson'
- name 'Leonard Axelsson'
- email 'leonard.axelsson at gmail.com'
- url 'http://xlson.com/'
- timezone '+1'
- }
- }
- }
-
- pom.withXml { XmlProvider xmlProvider ->
- def xml = xmlProvider.asString()
- def pomXml = new XmlParser().parseText(xml.toString())
-
- pomXml.version[0] + { packaging('jar') }
-
- def newXml = new StringWriter()
- def printer = new XmlNodePrinter(new PrintWriter(newXml))
- printer.preserveWhitespace = true
- printer.print(pomXml)
- xml.setLength(0)
- xml.append(newXml.toString())
- }
-}
-task wrapper(type: Wrapper) {
- gradleVersion = '1.0-milestone-3'
-}
+ licenses {
+ license {
+ name 'The Apache License, Version 2.0'
+ url 'http://www.apache.org/licenses/LICENSE-2.0.txt'
+ }
+ }
-task downloadDeps << {
- def libDir = file('lib')
- ant.delete(dir: libDir)
- copy {
- from configurations.testRuntime
- into libDir
+ developers {
+ developer {
+ id 'xlson'
+ name 'Leonard Gram'
+ email 'leo at xlson.com'
+ url 'http://xlson.com/'
+ timezone '+1'
+ }
+ }
+ }
+
+ pom.withXml { XmlProvider xmlProvider ->
+ def xml = xmlProvider.asString()
+ def pomXml = new XmlParser().parseText(xml.toString())
+
+ pomXml.version[0] + { packaging('jar') }
+
+ def newXml = new StringWriter()
+ def printer = new XmlNodePrinter(new PrintWriter(newXml))
+ printer.preserveWhitespace = true
+ printer.print(pomXml)
+ xml.setLength(0)
+ xml.append(newXml.toString())
+ }
}
+ }
}
=====================================
debian/changelog
=====================================
@@ -1,3 +1,18 @@
+groovycsv (1.3-1) unstable; urgency=medium
+
+ * Team upload.
+ * New upstream version 1.3
+ * Removing patches, as all of them seem to be integrated upstream.
+ * Bumping Standards-Version (no changes).
+ * Bumping debhelper compatibility level (no changes).
+ * Updating Vcs-* tags in debian/control.
+ * Enabling the build time tests.
+ * Adding a patch migrating the code to use opencsv v5.
+ * Removing 'get-orig-source' target.
+ * Switching URLs from HTTP to HTTPS.
+
+ -- Andrius Merkys <merkys at debian.org> Tue, 05 Nov 2019 10:17:55 -0500
+
groovycsv (1.0-2) unstable; urgency=medium
* Fixed the build failure caused by the transition from groovy2 to groovy
=====================================
debian/compat
=====================================
@@ -1 +1 @@
-9
+12
=====================================
debian/control
=====================================
@@ -2,20 +2,27 @@ Source: groovycsv
Section: java
Priority: optional
Maintainer: Debian Java Maintainers <pkg-java-maintainers at lists.alioth.debian.org>
-Uploaders: Emmanuel Bourg <ebourg at apache.org>
-Build-Depends: debhelper (>= 9),
- default-jdk,
- gradle-debian-helper,
- libopencsv-java,
- maven-repo-helper
-Standards-Version: 3.9.8
-Vcs-Git: https://anonscm.debian.org/git/pkg-java/groovycsv.git
-Vcs-Browser: https://anonscm.debian.org/cgit/pkg-java/groovycsv.git
-Homepage: http://github.com/xlson/groovycsv
+Uploaders:
+ Emmanuel Bourg <ebourg at apache.org>,
+Build-Depends:
+ debhelper (>= 12),
+ default-jdk,
+ gradle-debian-helper,
+ libcglib-nodep-java <!nocheck>,
+ libobjenesis-java <!nocheck>,
+ libopencsv-java,
+ libspock-java <!nocheck>,
+ maven-repo-helper,
+Standards-Version: 4.4.1
+Vcs-Git: https://salsa.debian.org/java-team/groovycsv.git
+Vcs-Browser: https://salsa.debian.org/java-team/groovycsv
+Homepage: https://github.com/xlson/groovycsv
Package: libgroovycsv-java
Architecture: all
-Depends: libopencsv-java, ${misc:Depends}
+Depends:
+ libopencsv-java,
+ ${misc:Depends},
Description: CSV parsing library for Groovy
GroovyCSV is a library to make CSV processing just a little bit
Groovier. The library uses opencsv behind the scenes and merely
=====================================
debian/copyright
=====================================
@@ -1,6 +1,6 @@
-Format: http://www.debian.org/doc/packaging-manuals/copyright-format/1.0/
+Format: https://www.debian.org/doc/packaging-manuals/copyright-format/1.0/
Upstream-Name: GroovyCSV
-Source: http://github.com/xlson/groovycsv
+Source: https://github.com/xlson/groovycsv
Files-Excluded: gradlew*
gradle/
=====================================
debian/maven.ignoreRules
=====================================
@@ -1,5 +1,2 @@
-cglib cglib-nodep * * * *
de.huxhorn.gradle de.huxhorn.gradle.pgp-plugin * * * *
-org.objenesis objenesis * * * *
-org.spockframework spock-core * * * *
=====================================
debian/maven.rules deleted
=====================================
@@ -1 +0,0 @@
-cglib cglib-nodep * s/.*/3.x/ * *
=====================================
debian/patches/01-ignore-gpg-plugin.patch deleted
=====================================
@@ -1,14 +0,0 @@
-Description: Ignore the GPG plugin (not needed)
-Author: Emmanuel Bourg <ebourg at apache.org>
-Forwarded: not-needed
---- a/build.gradle
-+++ b/build.gradle
-@@ -10,7 +10,7 @@
-
- apply plugin: 'groovy'
- apply plugin: 'maven'
--apply plugin: de.huxhorn.gradle.pgp.PgpPlugin
-+//apply plugin: de.huxhorn.gradle.pgp.PgpPlugin
- apply plugin: 'idea'
-
- repositories {
=====================================
debian/patches/02-gradle-compatibility.patch deleted
=====================================
@@ -1,18 +0,0 @@
-Description: Fixes the compatibility with the latest version of Gradle
-Origin: backport, https://github.com/xlson/groovycsv/commit/1e3d41e
---- a/build.gradle
-+++ b/build.gradle
-@@ -15,11 +15,11 @@
-
- repositories {
- mavenCentral()
-- mavenRepo urls: "http://m2repo.spockframework.org/snapshots"
-+ maven { url "http://m2repo.spockframework.org/snapshots" }
- }
-
- dependencies {
-- groovy group: 'org.codehaus.groovy', name: 'groovy-all', version: '1.7.3'
-+ compile group: 'org.codehaus.groovy', name: 'groovy-all', version: '1.7.3'
- compile 'net.sf.opencsv:opencsv:2.1'
- testCompile "org.spockframework:spock-core:0.4-groovy-1.7"
- testCompile "cglib:cglib-nodep:2.2"
=====================================
debian/patches/enable-tests.patch
=====================================
@@ -0,0 +1,13 @@
+Description: Use groovy-all to build test code.
+Author: Andrius Merkys <merkys at debian.org>
+--- a/build.gradle
++++ b/build.gradle
+@@ -20,7 +20,7 @@
+
+ dependencies {
+ compile 'com.opencsv:opencsv:4.0'
+- compileOnly 'org.codehaus.groovy:groovy-all:1.8.8'
++ compile 'org.codehaus.groovy:groovy-all:1.8.8'
+ testCompile 'org.spockframework:spock-core:1.1-groovy-2.4-rc-2'
+ testCompile 'cglib:cglib-nodep:2.2'
+ testCompile 'org.objenesis:objenesis:1.2'
=====================================
debian/patches/migrate-to-opencsv-v5.patch
=====================================
@@ -0,0 +1,44 @@
+Description: Migrate the code to use opencsv v5.
+Author: Andrius Merkys <merkys at debian.org>
+--- a/src/com/xlson/groovycsv/CsvParser.groovy
++++ b/src/com/xlson/groovycsv/CsvParser.groovy
+@@ -16,7 +16,11 @@
+
+ package com.xlson.groovycsv
+
++import com.opencsv.CSVParser
+ import com.opencsv.CSVReader
++import com.opencsv.ICSVParser
++import com.opencsv.validators.LineValidatorAggregator
++import com.opencsv.validators.RowValidatorAggregator
+
+ /**
+ * Helper class used to parse information from csv files using the column names
+@@ -152,11 +156,23 @@
+ quoteChar = args.quoteChar ?: '"'
+ }
+
+- if (escapeChar != null) {
+- return new CSVReader(reader, separator, quoteChar, escapeChar, skipLines)
+- } else {
+- return new CSVReader(reader, separator, quoteChar, skipLines)
++ if (escapeChar == null) {
++ escapeChar = ICSVParser.DEFAULT_ESCAPE_CHARACTER
+ }
++ CSVParser parser = new CSVParser(separator, quoteChar, escapeChar,
++ ICSVParser.DEFAULT_STRICT_QUOTES,
++ ICSVParser.DEFAULT_IGNORE_LEADING_WHITESPACE,
++ ICSVParser.DEFAULT_IGNORE_QUOTATIONS,
++ ICSVParser.DEFAULT_NULL_FIELD_INDICATOR,
++ Locale.getDefault())
++ return new CSVReader(reader, skipLines, parser,
++ CSVReader.DEFAULT_KEEP_CR,
++ CSVReader.DEFAULT_VERIFY_READER,
++ CSVReader.DEFAULT_MULTILINE_LIMIT,
++ Locale.getDefault(),
++ new LineValidatorAggregator(),
++ new RowValidatorAggregator(),
++ null)
+ }
+
+ /**
=====================================
debian/patches/series
=====================================
@@ -1,2 +1,2 @@
-01-ignore-gpg-plugin.patch
-02-gradle-compatibility.patch
+enable-tests.patch
+migrate-to-opencsv-v5.patch
=====================================
debian/rules
=====================================
@@ -5,5 +5,9 @@ export JAVA_HOME := /usr/lib/jvm/default-java
%:
dh $@ --buildsystem=gradle --with maven-repo-helper
-get-orig-source:
- uscan --download-current-version --force-download --no-symlink
+# Quick-and-dirty way to execute build time tests using gradle-debian-helper.
+# See #934729 for the description.
+override_dh_auto_test:
+ifeq (,$(filter nocheck,$(DEB_BUILD_OPTIONS)))
+ dh_auto_build -- test
+endif
=====================================
gradle.properties.example
=====================================
@@ -1,3 +1,7 @@
// This must be configured if you wanna deploy to the Nexus OSS Public Repo
-repoUserName=usernameHere
-repoPassword=passwordHere
+signing.keyId=
+signing.password=
+signing.secretKeyRingFile=
+
+ossrhUsername=
+ossrhPassword=
=====================================
src/com/xlson/groovycsv/AutoDetectHandler.groovy
=====================================
@@ -3,31 +3,31 @@ package com.xlson.groovycsv
class AutoDetectHandler {
List autoDetectSeparators = [",", ";", ":", "|"]
-
+
List autoDetectQuoteChars = ['"', "'", "%"]
-
+
String linesToInspect
-
-
+
+
String autoDetectQuoteChar() {
return mostFrequentChar(linesToInspect, autoDetectQuoteChars)
}
-
+
String autoDetectSeparator() {
return mostFrequentChar(linesToInspect, autoDetectSeparators)
}
-
- /**
- * Find the most frequent character in a string among a list of characters.
- * Falls back on the first character in the list if no character is found.
- *
- * @param sequence The string to search.
- * @param characters The list of characters to search.
- * @return The most frequent character.
- */
- private mostFrequentChar(String sequence, List characters) {
- characters.max{ sequence.count(it) }
- }
-
-
+
+ /**
+ * Find the most frequent character in a string among a list of characters.
+ * Falls back on the first character in the list if no character is found.
+ *
+ * @param sequence The string to search.
+ * @param characters The list of characters to search.
+ * @return The most frequent character.
+ */
+ private mostFrequentChar(String sequence, List characters) {
+ characters.max { sequence.count(it) }
+ }
+
+
}
=====================================
src/com/xlson/groovycsv/CsvIterator.groovy
=====================================
@@ -1,5 +1,5 @@
/*
- * Copyright 2010 Leonard Axelsson
+ * Copyright 2010 Leonard Gram
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
@@ -16,12 +16,12 @@
package com.xlson.groovycsv
-import au.com.bytecode.opencsv.CSVReader
+import com.opencsv.CSVReader
/**
* Iterates over the csv data in a non-synchronized way.
*
- * @author Leonard Axelsson
+ * @author Leonard Gram
* @since 0.1
*/
class CsvIterator implements Iterator {
@@ -63,14 +63,13 @@ class CsvIterator implements Iterator {
* @return true if there is more data in the iterator
*/
boolean hasNext() {
- if(isClosed()) {
+ if (isClosed()) {
return false
- }
- else if(nextValueIsRead()) {
+ } else if (nextValueIsRead()) {
return true
} else {
- readValue = csvReader.readNext()
- if(readValue == null) {
+ readValue = getNextValue()
+ if (readValue == null) {
close()
}
return readValue != null
@@ -92,15 +91,29 @@ class CsvIterator implements Iterator {
}
private def getNextValue() {
- if(nextValueIsRead()) {
+ if (nextValueIsRead()) {
def value = readValue
readValue = null
return value
} else {
- return csvReader.readNext()
+ def nextValue
+ for (; ;) {
+ nextValue = csvReader.readNext()
+ if (nextValue == null) {
+ break
+ } else if (isEmptyLine(nextValue)) {
+ // Continues the loop and reads another value
+ } else {
+ break
+ }
+ }
+ return nextValue
}
}
+ private boolean isEmptyLine(String[] nextValue) {
+ nextValue.size() == 1 && nextValue.first().isEmpty()
+ }
/**
* Gets the next row in the csv file.
=====================================
src/com/xlson/groovycsv/CsvParser.groovy
=====================================
@@ -1,5 +1,5 @@
/*
- * Copyright 2010 Leonard Axelsson
+ * Copyright 2010 Leonard Gram
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
@@ -16,7 +16,7 @@
package com.xlson.groovycsv
-import au.com.bytecode.opencsv.CSVReader
+import com.opencsv.CSVReader
/**
* Helper class used to parse information from csv files using the column names
@@ -30,16 +30,15 @@ import au.com.bytecode.opencsv.CSVReader
* Pete,Hansen'''
*
* def data = new CsvParser().parse(csv)
- * for(line in data) {
- * println "$line.Name $line.Lastname"
- * }</pre>
+ * for(line in data) {* println "$line.Name $line.Lastname"
+ *}</pre>
*
*
- * @author Leonard Axelsson
+ * @author Leonard Gram
* @since 0.1
*/
class CsvParser {
-
+
/**
* Number of characters used to provide to autodetection (in case auto
* detection is used.
@@ -91,10 +90,12 @@ class CsvParser {
* Arguments for configuration:
* <li>separator: configures the separator character to use (default: ,)
* <li>quoteChar: configures the quote character to use (default: ")
- * <li>escapeChar: configures the escape character for the separator and quoteChar (default:\
+ * <li>escapeChar: configures the escape character for the separator and quoteChar (default:\)
* <li>autoDetect: sets up autodetect that will honor other configurations you've done (default: false)
* <li>columnNames: set custom column names instead of using the first line
* <li>readFirstLine: reads the first line as csv instead of using it as headers
+ * <li>trimWhitespaceFromColumnNames: trims leading and trailing whitespace for column names when parsing them (default: false)
+ * <li>skipLines: skips the specified number of lines at the beginning (default: 0)
*
* <p>
* Usage:
@@ -106,8 +107,7 @@ class CsvParser {
* def data = new CsvParser().parse(csv, separator: '-')
*
* // Print all fruits that have a quantity higher than 3
- * data.findAll{ (it.Quantity as int) > 3 }.each{ println it }
- * </pre>
+ * data.findAll{ (it.Quantity as int) > 3 }.each{ println it }* </pre>
* </p>
* @param reader the csv to parse
* @param args the configuration arguments
@@ -129,14 +129,19 @@ class CsvParser {
if (args.columnNames) {
columnNames = args.columnNames
}
+
+ if (args.trimWhitespaceFromColumnNames) {
+ columnNames = columnNames.collect { it.toString().trim() }
+ }
return columnNames
}
private CSVReader createCSVReader(Map args = [:], Reader reader) {
- char separator
- char quoteChar
- char escapeChar = args.escapeChar
-
+ Character separator
+ Character quoteChar
+ Character escapeChar = args.escapeChar
+ Integer skipLines = args.skipLines ?: 0
+
if (args.autoDetect == true) {
reader = new PushbackReader(reader, autoDetectCharNumber)
doAutoDetection(args, reader)
@@ -147,24 +152,24 @@ class CsvParser {
quoteChar = args.quoteChar ?: '"'
}
- if(escapeChar) {
- return new CSVReader(reader, separator, quoteChar, escapeChar)
+ if (escapeChar != null) {
+ return new CSVReader(reader, separator, quoteChar, escapeChar, skipLines)
} else {
- return new CSVReader(reader, separator, quoteChar)
+ return new CSVReader(reader, separator, quoteChar, skipLines)
}
}
/**
* Performs automatic detection of separator and quote character.
- *
+ *
* It will search arguments for values 'auto' in separator and quoteChar. It
* will return a new version of the arguments modified with the values that
* were found.
- *
+ *
* If nothing is detected, the values are removed from the args.
- *
+ *
* Note that
- *
+ *
* @param args the configuration arguments.
* @param text the CSV as a String.
* @return modified args with detected.
@@ -174,8 +179,8 @@ class CsvParser {
def charsRead = reader.read(buf)
def linesToInspect = new String(buf)
reader.unread(buf, 0, charsRead)
-
-
+
+
def autoDetector = new AutoDetectHandler(linesToInspect: linesToInspect)
if (args.autoDetectQuoteChars) {
autoDetector.autoDetectQuoteChars = args.autoDetectQuoteChars
@@ -188,12 +193,12 @@ class CsvParser {
if (detectedSeparator) args.separator = detectedSeparator
else args.remove("separator")
}
- if(!args.quoteChar) {
+ if (!args.quoteChar) {
def detectedQuoteChar = autoDetector.autoDetectQuoteChar()
if (detectedQuoteChar) args.quoteChar = detectedQuoteChar
else args.remove("quoteChar")
}
return args
}
-
+
}
=====================================
src/com/xlson/groovycsv/PropertyMapper.groovy
=====================================
@@ -1,5 +1,5 @@
/*
- * Copyright 2010 Leonard Axelsson
+ * Copyright 2010 Leonard Gram
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
@@ -16,13 +16,16 @@
package com.xlson.groovycsv
+import groovy.transform.EqualsAndHashCode
+
/**
* Maps between column names and values in a list. Uses <code>propertyMissing</code>
* to allow for named access.
*
- * @author Leonard Axelsson
+ * @author Leonard Gram
* @since 0.1
*/
+ at EqualsAndHashCode
class PropertyMapper {
/**
@@ -30,8 +33,6 @@ class PropertyMapper {
*/
def values
-
-
/**
* The columns of the csv.
*/
@@ -66,4 +67,9 @@ class PropertyMapper {
String toString() {
columns.collect { key, index -> "$key: ${values[index]}" }.join(', ')
}
+
+ Map toMap() {
+ def sortedKeys = columns.keySet().sort { columns[it] }
+ [sortedKeys, values].transpose().collectEntries()
+ }
}
=====================================
test/com/xlson/groovycsv/AutoDetectHandlerSpec.groovy
=====================================
@@ -3,31 +3,31 @@ package com.xlson.groovycsv
import spock.lang.Specification;
class AutoDetectHandlerSpec extends Specification {
-
-
+
+
def "should auto detect quote character"() {
setup:
def adh = new AutoDetectHandler(linesToInspect: csvData)
-
+
expect:
adh.autoDetectQuoteChar() == quoteChar
-
+
where:
csvData | quoteChar
testDataWithColumnNamesAnd3Rows | '"'
testDataWithColumnNamesAnd2Rows | '"'
csvUsingDoubleQuoteAsQuoteChar | '"'
csvUsingPercentageAsQuoteChar | "%"
-
+
}
-
+
def "should auto detect separator"() {
setup:
def adh = new AutoDetectHandler(linesToInspect: csvData)
-
+
expect:
adh.autoDetectSeparator() == separator
-
+
where:
csvData | separator
testDataWithColumnNamesAnd3Rows | ","
@@ -36,7 +36,7 @@ class AutoDetectHandlerSpec extends Specification {
csvWithColonAsSeparator | ":"
}
-
+
def getTestDataWithColumnNamesAnd3Rows() {
'''Name,Lastname,Age,Email
Mark,Hamilton,45,mark at hamilton.com
@@ -56,7 +56,7 @@ Apple:5
Pear:10
Kiwi:200'''
}
-
+
def getCsvUsingDoubleQuoteAsQuoteChar() {
'''Typo,Desc
123,"text ,and more"'''
=====================================
test/com/xlson/groovycsv/ConfigurableColumnNamesSpec.groovy
=====================================
@@ -1,7 +1,7 @@
package com.xlson.groovycsv
+import com.opencsv.CSVReader
import spock.lang.Specification
-import au.com.bytecode.opencsv.CSVReader
class ConfigurableColumnNamesSpec extends Specification {
@@ -51,7 +51,7 @@ Jane,Doe,24"""
when:
def customColumnNames = ['1', '2', '3']
- def columnNames = new CsvParser().parseColumnNames([columnNames:customColumnNames], reader)
+ def columnNames = new CsvParser().parseColumnNames([columnNames: customColumnNames], reader)
then:
1 * reader.readNext()
@@ -64,9 +64,9 @@ Jane,Doe,24"""
when:
def customColumnNames = ['1', '2', '3']
- def columnNames = new CsvParser().parseColumnNames([columnNames:customColumnNames,
+ def columnNames = new CsvParser().parseColumnNames([columnNames : customColumnNames,
readFirstLine: true],
- reader)
+ reader)
then:
0 * reader.readNext()
columnNames == customColumnNames
=====================================
test/com/xlson/groovycsv/CsvIteratorSpec.groovy
=====================================
@@ -1,9 +1,9 @@
package com.xlson.groovycsv
+import com.opencsv.CSVReader
import spock.lang.Specification
-import au.com.bytecode.opencsv.CSVReader
-class CsvIteratorSpec extends Specification {
+class CsvIteratorSpec extends Specification {
def getCsvData() {
def csv = """a,b,c
@@ -31,7 +31,7 @@ class CsvIteratorSpec extends Specification {
setup:
CSVReader csvReader = Mock(CSVReader)
def iter = new CsvIterator(["a", "b"], csvReader)
- csvReader.readNext() >>> [["1","2"],["3","4"], null]
+ csvReader.readNext() >>> [["1", "2"], ["3", "4"], null]
when:
iter.next()
@@ -55,7 +55,7 @@ class CsvIteratorSpec extends Specification {
def csvIterator = new CsvIterator(*csvData)
when:
- csvIterator.each { }
+ csvIterator.each {}
then:
csvIterator.isClosed()
=====================================
test/com/xlson/groovycsv/CsvParserSpec.groovy
=====================================
@@ -1,7 +1,7 @@
package com.xlson.groovycsv
+import com.opencsv.CSVReader
import spock.lang.*
-import au.com.bytecode.opencsv.CSVReader
class CsvParserSpec extends Specification {
def getTestDataWithColumnNamesAnd3Rows() {
@@ -21,14 +21,14 @@ h,drink,60'''
'''Fruit:Count
Apple:5
Pear:10
-Kiwi:200'''
+Kiwi:200'''
}
- def getTestDataWithQuotedComma() {
- '''a,b
+ def getTestDataWithQuotedComma() {
+ '''a,b
-,abc-,4
abc,-4-'''
- }
+ }
def "Iterating over the parsed csv values are available by column name."() {
setup:
@@ -38,11 +38,11 @@ abc,-4-'''
data*."$columnName" == values
where:
- columnName | values
- "Name" | ['Mark', 'Bosse', "Peps"]
- "Lastname" | ['Hamilton', 'Bildoktorn', 'Persson']
- 'Age' | ['45', '50', '65']
- "Email" | ['mark at hamilton.com', 'bildoktorn at tv4.se', 'peps.persson at hotmail.com']
+ columnName | values
+ "Name" | ['Mark', 'Bosse', "Peps"]
+ "Lastname" | ['Hamilton', 'Bildoktorn', 'Persson']
+ 'Age' | ['45', '50', '65']
+ "Email" | ['mark at hamilton.com', 'bildoktorn at tv4.se', 'peps.persson at hotmail.com']
}
def "Functional collection methods are available on parsed object."() {
@@ -50,20 +50,7 @@ abc,-4-'''
def data = new CsvParser().parse(getTestDataWithColumnNamesAnd3Rows())
expect:
- data.findAll { (it.Age as int) > 46 }.size() == 2
- }
-
- def "PropertyMapper has a toString() that returns all the data in it's columns."() {
- setup:
- def pm = new PropertyMapper(values: values, columns: columns)
-
- expect:
- pm.toString() == toStringRepresentation
-
- where:
- columns | values | toStringRepresentation
- ['a': 0, 'b': 1, 'c': 2] | ['1', '2', '3'] | "a: 1, b: 2, c: 3"
- ['Name': 0, 'Age': 1] | ['Mark', '56'] | "Name: Mark, Age: 56"
+ data.findAll { (it.Age as int) > 46 }.size() == 2
}
def "readAll should never be called on the CSVReader instance used to parse the csv."() {
@@ -76,7 +63,8 @@ abc,-4-'''
when: "csv is parsed and looped through"
def data = partiallyMockedCsvParser.parse(getTestDataWithColumnNamesAnd2Rows())
- for(d in data) {}
+ for (d in data) {
+ }
then: "readAll() should not be called."
0 * csvReader.readAll()
@@ -90,9 +78,9 @@ abc,-4-'''
data*."$columnName" == values
where:
- columnName | values
- "Fruit" | ['Apple', 'Pear', 'Kiwi']
- "Count" | ["5", "10", "200"]
+ columnName | values
+ "Fruit" | ['Apple', 'Pear', 'Kiwi']
+ "Count" | ["5", "10", "200"]
}
def getCsvUsingDoubleQuoteAsQuoteChar() {
@@ -114,9 +102,9 @@ abc,-4-'''
csv*."$columnName" == values
where:
- csvData | quoteChar | values | columnName
- csvUsingDoubleQuoteAsQuoteChar | '"' | ['text ,and more'] | "Desc"
- csvUsingPercentageAsQuoteChar | "%" | ['bla, ha'] | "Desc"
+ csvData | quoteChar | values | columnName
+ csvUsingDoubleQuoteAsQuoteChar | '"' | ['text ,and more'] | "Desc"
+ csvUsingPercentageAsQuoteChar | "%" | ['bla, ha'] | "Desc"
}
def "Parse supports custom escape char."() {
@@ -130,6 +118,66 @@ abc,-4-'''
}
+ def "Parser quietly ignores one empty line."() {
+ setup:
+ def csvData = '''Fruit,Country
+
+Apple,Sweden
+'''
+ def csv = new CsvParser().parse(csvData)
+
+ expect:
+ for (line in csv) {
+ line.Fruit == 'Apple'
+ line.Country == 'Sweden'
+ }
+ }
+
+ def "Parser quitely ignores multiple empty lines."() {
+ setup:
+ def csvData = '''Color,Day
+Red,Monday
+
+
+
+Black,Friday'''
+
+ def csv = new CsvParser().parse(csvData)
+
+ expect:
+ def firstLine = csv.next()
+ firstLine.Color == 'Red'
+ firstLine.Day == 'Monday'
+
+ def secondLine = csv.next()
+ secondLine.Color == 'Black'
+ secondLine.Day == 'Friday'
+ csv.hasNext() == false
+ }
+
+ def "Parser quitely ignores multiple empty linessadadas."() {
+ setup:
+ def csvData = '''Color,Day
+
+
+
+Red,Monday
+
+
+
+Black,Friday
+
+
+'''
+
+ def csv = new CsvParser().parse(csvData)
+
+ expect:
+ for (line in csv) {
+ println line
+ }
+ }
+
def "Parse supports java.io.Reader as input."() {
when:
def csv = new CsvParser().parse(new StringReader(testDataWithColumnNamesAnd2Rows))
@@ -138,13 +186,13 @@ abc,-4-'''
csv*.Number == ['5', '60']
}
- def "CsvParser should auto detect separator and quote character"() {
+ def "CsvParser can auto detect separator and quote character"() {
when: "a CSV file is parsed with auto detection"
def csv = new CsvParser().parse(autoDetect: true, csvData)
then: "it should return the correct columns"
csv*."$property" == values
-
+
where:
csvData | property | values
testDataWithColumnNamesAnd3Rows | "Age" | ["45", "50", "65"]
@@ -154,59 +202,92 @@ abc,-4-'''
testDataWithColumnNamesAnd2Rows | "Word" | ["paris", "drink"]
testDataWithColumnNamesAnd3Rows | "Email" | ["mark at hamilton.com", "bildoktorn at tv4.se", "peps.persson at hotmail.com"]
}
-
+
def "should allow to override auto detection"() {
when: "autoDetect is active and a separator is provided"
def csv = new CsvParser().parse(autoDetect: true, separator: ',', csvWithColonAsSeparator)
-
+
then: "the separator provided is used"
csv*."Fruit:Count" == ["Apple:5", "Pear:10", "Kiwi:200"]
}
- def "The separator should be allowed in the csv data if its quoted"() {
- when:
- def csv = new CsvParser().parse(quoteChar:'-', testDataWithQuotedComma)
+ def "The separator should be allowed in the csv data if its quoted"() {
+ when:
+ def csv = new CsvParser().parse(quoteChar: '-', testDataWithQuotedComma)
+
+ then:
+ csv*.a == [',abc', 'abc']
+ }
+
+ def "Values in the csv can be obtained by using the index of the column."() {
+ when:
+ def csv = new CsvParser().parse(testDataWithColumnNamesAnd2Rows)
+ def line = csv.next()
+
+ then:
+ line[0] == 'a'
+ line[1] == 'paris'
+ line[2] == '5'
+ }
+
+ def csvWithoutHeaders = 'Joe,Doe,19'
+
+ def "Parsing csv without headers using the position of the values."() {
+ when:
+ def csv = new CsvParser().parse(readFirstLine: true, csvWithoutHeaders)
+ def line = csv.next()
- then:
- csv*.a == [',abc','abc']
- }
+ then:
+ line[0] == 'Joe'
+ line[1] == 'Doe'
+ line[2] == '19'
- def "Values in the csv can be obtained by using the index of the column."() {
- when:
- def csv = new CsvParser().parse(testDataWithColumnNamesAnd2Rows)
- def line = csv.next()
+ }
- then:
- line[0] == 'a'
- line[1] == 'paris'
- line[2] == '5'
- }
+ def "CsvParser.parseCsv can be used statically."() {
+ when:
+ def csv = CsvParser.parseCsv(csvData)
- def csvWithoutHeaders = 'Joe,Doe,19'
+ then:
+ csv*.Letter == letterValues
- def "Parsing csv without headers using the position of the values."() {
- when:
- def csv = new CsvParser().parse(readFirstLine: true, csvWithoutHeaders)
- def line = csv.next()
+ where:
+ csvData | letterValues
+ testDataWithColumnNamesAnd2Rows | ['a', 'h']
+ new StringReader(testDataWithColumnNamesAnd2Rows) | ['a', 'h']
+ }
+
+ def getTestDataWithWhitespaceSurroundingColumnNames() {
+ """ Name , Lastname,Country
+Leonard,Gram,Sweden"""
+ }
- then:
- line[0] == 'Joe'
- line[1] == 'Doe'
- line[2] == '19'
+ def "CsvParser can strip leading and trailing whitespaces on column names."() {
+ when:
+ def csv = CsvParser.parseCsv(testDataWithWhitespaceSurroundingColumnNames, trimWhitespaceFromColumnNames: true)
- }
+ then:
+ for(line in csv) {
+ line.Name == 'Leonard'
+ line.Lastname == 'Gram'
+ line.Country == 'Sweden'
+ }
+ }
- def "CsvParser.parseCsv can be used statically."() {
- when:
- def csv = CsvParser.parseCsv(csvData)
+ def "CsvParser can skip lines at the beginning of the file."() {
- then:
- csv*.Letter == letterValues
+ def csvWithSkipLines = '''skipped line1
+skipped line2
+Letter,Word,Number
+a,paris,5
+h,drink,60'''
+ when:
+ def csv = new CsvParser().parse(skipLines: 2, csvWithSkipLines)
+ def line = csv.next()
- where:
- csvData | letterValues
- testDataWithColumnNamesAnd2Rows | ['a', 'h']
- new StringReader(testDataWithColumnNamesAnd2Rows) | ['a', 'h']
- }
-
+ then:
+ line["Letter"] == "a"
+ line["Word"] == "paris"
+ line["Number"] == "5"
+ }
}
=====================================
test/com/xlson/groovycsv/PropertyMapperSpec.groovy
=====================================
@@ -0,0 +1,40 @@
+package com.xlson.groovycsv
+
+import spock.lang.Specification
+
+class PropertyMapperSpec extends Specification {
+
+ def getTestDataWithDuplicatedLines() {
+ '''a,b
+1,2
+1,2
+3,4'''
+ }
+
+ def "PropertyMapper has a toMap() that returns a map representation."() {
+ setup:
+ def pm = new PropertyMapper(values: values, columns: columns)
+
+ expect:
+ pm.toMap() == toMapRepresentation
+
+ where:
+ columns | values | toMapRepresentation
+ ['a': 0, 'b': 1, 'c': 2] | ['1', '2', '3'] | ['a': '1', 'b': '2', 'c': '3']
+ ['Name': 0, 'Age': 1] | ['Mark', '56'] | ['Name': 'Mark', 'Age': '56']
+ }
+
+ def "PropertyMapper has a toString() that returns all the data in it's columns."() {
+ setup:
+ def pm = new PropertyMapper(values: values, columns: columns)
+
+ expect:
+ pm.toString() == toStringRepresentation
+
+ where:
+ columns | values | toStringRepresentation
+ ['a': 0, 'b': 1, 'c': 2] | ['1', '2', '3'] | "a: 1, b: 2, c: 3"
+ ['Name': 0, 'Age': 1] | ['Mark', '56'] | "Name: Mark, Age: 56"
+ }
+
+}
View it on GitLab: https://salsa.debian.org/java-team/groovycsv/compare/c1957df531e6aaa99ba2fe66b5b7413d8f66ca6b...5dd66a062682e8dd35f12f79bda762636f4606dd
--
View it on GitLab: https://salsa.debian.org/java-team/groovycsv/compare/c1957df531e6aaa99ba2fe66b5b7413d8f66ca6b...5dd66a062682e8dd35f12f79bda762636f4606dd
You're receiving this email because of your account on salsa.debian.org.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://alioth-lists.debian.net/pipermail/pkg-java-commits/attachments/20191105/d7b5fdcb/attachment.html>
More information about the pkg-java-commits
mailing list