[Git][clojure-team/clj-yaml-clojure][upstream] New upstream version 0.7.2
Jérôme Charaoui (@lavamind)
gitlab at salsa.debian.org
Sun Feb 5 22:09:17 GMT 2023
Jérôme Charaoui pushed to branch upstream at Debian Clojure Maintainers / clj-yaml-clojure
Commits:
29400c2f by Jérôme Charaoui at 2023-02-05T16:31:35-05:00
New upstream version 0.7.2
- - - - -
9 changed files:
- + .circleci/config.yml
- .gitignore
- + CHANGELOG.md
- README.md
- project.clj
- − src/clj_yaml/core.clj
- + src/clojure/clj_yaml/core.clj
- + src/java/clj_yaml/MarkedConstructor.java
- test/clj_yaml/core_test.clj
Changes:
=====================================
.circleci/config.yml
=====================================
@@ -0,0 +1,17 @@
+version: 2.1
+jobs:
+ test:
+ docker:
+ - image: circleci/clojure:lein-2.8.1
+ steps:
+ - checkout
+ - run:
+ name: Ensure No Reflection Warnings
+ command: "! lein check 2>&1 | grep 'Reflection warning'"
+ - run: lein test
+
+
+workflows:
+ workflow:
+ jobs:
+ - test
=====================================
.gitignore
=====================================
@@ -1,5 +1,6 @@
+bin
lib
classes
*.jar
-.*
-pom.xml
+pom.xml*
+/target
=====================================
CHANGELOG.md
=====================================
@@ -0,0 +1,17 @@
+# Changelog
+All notable changes to this project will be documented in this file.
+
+The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/).
+
+## [0.7.0] - 2019-03-15
+### Changed
+- [Accept emoji](https://github.com/clj-commons/clj-yaml/pull/5).
+- [Update to SnakeYAML from 1.23 to 1.24](https://github.com/clj-commons/clj-yaml/pull/5).
+
+## [0.6.1] - 2019-02-06
+### Changed
+- [Restore default text-wrapping behavior of prior release](https://github.com/clj-commons/clj-yaml/pull/2).
+
+## [0.6.0] - 2019-01-04
+### Added
+- First release under [clj-commons](https://github.com/clj-commons) project.
=====================================
README.md
=====================================
@@ -1,33 +1,71 @@
-`clj-yaml` provides [YAML](http://yaml.org) encoding and decoding for Clojure via the [SnakeYAML](http://code.google.com/p/snakeyaml/) Java library.
+`clj-commons/clj-yaml` provides [YAML](http://yaml.org) encoding and
+decoding for Clojure via the [snakeyaml][] Java library.
+
+[SnakeYAML]: https://bitbucket.org/asomov/snakeyaml/
+
+
+[](https://clojars.org/clj-commons/clj-yaml) [](https://cljdoc.org/d/clj-commons/clj-yaml/CURRENT)
+ [](https://circleci.com/gh/clj-commons/clj-yaml)
+
+(This is a maintained fork of [the original][]).
+
+[the original]: https://github.com/lancepantz/clj-yaml
+
## Usage
- (require '[clj-yaml.core :as yaml])
-
- (yaml/generate-string
- [{:name "John Smith", :age 33}
- {:name "Mary Smith", :age 27}])
- "- {name: John Smith, age: 33}\n- {name: Mary Smith, age: 27}\n"
-
- (yaml/parse-string "
- - {name: John Smith, age: 33}
- - name: Mary Smith
- age: 27
- ")
- => ({:name "John Smith", :age 33}
- {:name "Mary Smith", :age 27})
+```clojure
+(require '[clj-yaml.core :as yaml])
+
+(yaml/generate-string
+ [{:name "John Smith", :age 33}
+ {:name "Mary Smith", :age 27}])
+"- {name: John Smith, age: 33}\n- {name: Mary Smith, age: 27}\n"
+
+(yaml/parse-string "
+- {name: John Smith, age: 33}
+- name: Mary Smith
+ age: 27
+")
+=> ({:name "John Smith", :age 33}
+ {:name "Mary Smith", :age 27})
+```
+
+By default, keys are converted to clojure keywords. To prevent this,
+add `:keywords false` parameters to the `parse-string` function:
+
+```clojure
+(yaml/parse-string "
+- {name: John Smith}
+" :keywords false)
+```
+
+Different flow styles (`:auto`, `:block`, `:flow`) allow customization of how YAML is rendered:
+
+
+```clojure
+(yaml/generate-string some-data :dumper-options {:flow-style :block})
+```
## Installation
-`clj-yaml` is available as a Maven artifact from [Clojars](http://clojars.org/clj-yaml):
+`clj-commons/clj-yaml` is available as a Maven artifact from [Clojars](http://clojars.org/clj-commons/clj-yaml).
+
+### Leiningen/Boot
+
+```clojure
+[clj-commons/clj-yaml "0.7.0"]
+```
+
+### Clojure CLI/`deps.edn`
- :dependencies
- [["clj-yaml" "0.4.0"]
- ...]
+```clojure
+clj-commons/clj-yaml {:mvn/version "0.7.0"}
+```
## Development
- $ git clone git://github.com/lancepantz/clj-yaml.git
+ $ git clone git://github.com/clj-commons/clj-yaml.git
$ lein deps
$ lein test
$ lein install
=====================================
project.clj
=====================================
@@ -1,6 +1,16 @@
-(defproject clj-yaml "0.4.0"
+(defproject clj-commons/clj-yaml "0.7.2"
:description "YAML encoding and decoding for Clojure using SnakeYAML"
- :url "http://github.com/lancepantz/clj-yaml"
+ :url "https://github.com/clj-commons/clj-yaml"
+ :license {:name "Eclipse Public License - v 1.0"
+ :url "http://www.eclipse.org/legal/epl-v10.html"
+ :distribution :repo
+ :comments "Same as Clojure"}
+ ;; Emit warnings on all reflection calls.
+ :global-vars {*warn-on-reflection* true}
+ :source-paths ["src/clojure"]
+ :java-source-paths ["src/java"]
+ :javac-options ["-target" "1.7" "-source" "1.7" "-Xlint:-options"]
:dependencies
- [[org.clojure/clojure "1.2.0"]
- [org.yaml/snakeyaml "1.5"]])
+ [[org.yaml/snakeyaml "1.26"]
+ [org.flatland/ordered "1.5.9"]]
+ :profiles {:provided {:dependencies [[org.clojure/clojure "1.10.1"]]}})
=====================================
src/clj_yaml/core.clj deleted
=====================================
@@ -1,78 +0,0 @@
-(ns clj-yaml.core
- (:import (org.yaml.snakeyaml Yaml DumperOptions DumperOptions$FlowStyle)))
-
-(def ^{:dynamic true} *keywordize* true)
-
-(def flow-styles
- {:auto DumperOptions$FlowStyle/AUTO
- :block DumperOptions$FlowStyle/BLOCK
- :flow DumperOptions$FlowStyle/FLOW})
-
-(defn make-dumper-options
- [& {:keys [flow-style]}]
- (doto (DumperOptions.)
- (.setDefaultFlowStyle (flow-styles flow-style))))
-
-(defn make-yaml
- [& {:keys [dumper-options]}]
- (if dumper-options
- (Yaml. (apply make-dumper-options
- (mapcat (juxt key val)
- dumper-options)))
- (Yaml.)))
-
-(defprotocol YAMLCodec
- (encode [data])
- (decode [data]))
-
-(defn decode-key [k]
- (if *keywordize* (keyword k) k))
-
-(extend-protocol YAMLCodec
-
- clojure.lang.IPersistentMap
- (encode [data]
- (into {}
- (for [[k v] data]
- [(encode k) (encode v)])))
-
- clojure.lang.IPersistentCollection
- (encode [data]
- (map encode data))
-
- clojure.lang.Keyword
- (encode [data]
- (name data))
-
- java.util.LinkedHashMap
- (decode [data]
- (into {}
- (for [[k v] data]
- [(decode-key k) (decode v)])))
-
- java.util.LinkedHashSet
- (decode [data]
- (into #{} data))
-
- java.util.ArrayList
- (decode [data]
- (map decode data))
-
- Object
- (encode [data] data)
- (decode [data] data)
-
- nil
- (encode [data] data)
- (decode [data] data))
-
-(defn generate-string [data & opts]
- (.dump (apply make-yaml opts)
- (encode data)))
-
-(defn parse-string
- ([string keywordize]
- (binding [*keywordize* keywordize]
- (parse-string string)))
- ([string]
- (decode (.load (make-yaml) string))))
=====================================
src/clojure/clj_yaml/core.clj
=====================================
@@ -0,0 +1,158 @@
+(ns clj-yaml.core
+ (:require [flatland.ordered.map :refer (ordered-map)]
+ [flatland.ordered.set :refer (ordered-set)])
+ (:import (org.yaml.snakeyaml Yaml DumperOptions DumperOptions$FlowStyle LoaderOptions)
+ (org.yaml.snakeyaml.constructor Constructor SafeConstructor BaseConstructor)
+ (org.yaml.snakeyaml.representer Representer)
+ (org.yaml.snakeyaml.error Mark)
+ (clj_yaml MarkedConstructor)
+ (java.util LinkedHashMap)))
+
+(def flow-styles
+ {:auto DumperOptions$FlowStyle/AUTO
+ :block DumperOptions$FlowStyle/BLOCK
+ :flow DumperOptions$FlowStyle/FLOW})
+
+(defn ^DumperOptions default-dumper-options
+ "clj-yaml 0.5.6 used SnakeYAML 1.13 which by default did *not* split long
+ lines. clj-yaml 0.6.0 upgraded to SnakeYAML 1.23 which by default *did* split
+ long lines. This ensures that generate-string uses the older behavior by
+ default, for the sake of stability, i.e. backwards compatibility."
+ []
+ (doto (DumperOptions.)
+ (.setSplitLines false)))
+
+(defn ^DumperOptions make-dumper-options
+ [& {:keys [flow-style]}]
+ (doto (default-dumper-options)
+ (.setDefaultFlowStyle (flow-styles flow-style))))
+
+(defn ^LoaderOptions default-loader-options
+ []
+ (LoaderOptions.))
+
+(defn ^LoaderOptions make-loader-options
+ [& {:keys [max-aliases-for-collections allow-recursive-keys allow-duplicate-keys]}]
+ (let [loader (default-loader-options)]
+ (when max-aliases-for-collections
+ (.setMaxAliasesForCollections loader max-aliases-for-collections))
+ (when allow-recursive-keys
+ (.setAllowRecursiveKeys loader allow-recursive-keys))
+ (when (instance? Boolean allow-duplicate-keys)
+ (.setAllowDuplicateKeys loader allow-duplicate-keys))
+ loader))
+
+(defn ^Yaml make-yaml
+ "Make a yaml encoder/decoder with some given options."
+ [& {:keys [dumper-options unsafe mark max-aliases-for-collections allow-recursive-keys allow-duplicate-keys]}]
+ (let [loader (make-loader-options :max-aliases-for-collections max-aliases-for-collections
+ :allow-recursive-keys allow-recursive-keys
+ :allow-duplicate-keys allow-duplicate-keys)
+ ^BaseConstructor constructor
+ (if unsafe (Constructor. loader)
+ (if mark
+ ;; construct2ndStep isn't implemented by MarkedConstructor,
+ ;; causing an exception to be thrown before loader options are
+ ;; used
+ (MarkedConstructor.)
+ (SafeConstructor. loader)))
+ ;; TODO: unsafe marked constructor
+ dumper (if dumper-options
+ (make-dumper-options :flow-style (:flow-style dumper-options))
+ (default-dumper-options))]
+ (Yaml. constructor (Representer.) dumper loader)))
+
+(defrecord Marked
+ [start end unmark])
+
+(defn mark
+ "Mark some data with start and end positions."
+ [start end marked]
+ (Marked. start end marked))
+
+(defn marked?
+ "Let us know whether this piece of data is marked with source positions."
+ [m]
+ (instance? Marked m))
+
+(defn unmark
+ "Strip the source information from this piece of data, if it exists."
+ [m]
+ (if (marked? m)
+ (:unmark m)
+ m))
+
+(defprotocol YAMLCodec
+ "A protocol for things that can be coerced to and from the types
+ that snakeyaml knows how to encode and decode."
+ (encode [data])
+ (decode [data keywords]))
+
+(extend-protocol YAMLCodec
+ clj_yaml.MarkedConstructor$Marked
+ (decode [data keywords]
+ (letfn [(from-Mark [^Mark mark]
+ {:line (.getLine mark)
+ :index (.getIndex mark)
+ :column (.getColumn mark)})]
+ ;; Decode the marked data and rewrap it with its source position.
+ (mark (-> data .start from-Mark)
+ (-> data .end from-Mark)
+ (-> data .marked
+ (decode keywords)))))
+
+ clojure.lang.IPersistentMap
+ (encode [data]
+ (let [lhm (LinkedHashMap.)]
+ (doseq [[k v] data]
+ (.put lhm (encode k) (encode v)))
+ lhm))
+
+ clojure.lang.IPersistentCollection
+ (encode [data]
+ (map encode data))
+
+ clojure.lang.Keyword
+ (encode [data]
+ (name data))
+
+ java.util.LinkedHashMap
+ (decode [data keywords]
+ (letfn [(decode-key [k]
+ (if keywords
+ ;; (keyword k) is nil for numbers etc
+ (or (keyword k) k)
+ k))]
+ (into (ordered-map)
+ (for [[k v] data]
+ [(-> k (decode keywords) decode-key) (decode v keywords)]))))
+
+ java.util.LinkedHashSet
+ (decode [data keywords]
+ (into (ordered-set) data))
+
+ java.util.ArrayList
+ (decode [data keywords]
+ (map #(decode % keywords) data))
+
+ Object
+ (encode [data] data)
+ (decode [data keywords] data)
+
+ nil
+ (encode [data] data)
+ (decode [data keywords] data))
+
+
+(defn generate-string [data & opts]
+ (.dump ^Yaml (apply make-yaml opts)
+ (encode data)))
+
+(defn parse-string
+ [^String string & {:keys [unsafe mark keywords max-aliases-for-collections allow-recursive-keys allow-duplicate-keys] :or {keywords true}}]
+ (decode (.load (make-yaml :unsafe unsafe
+ :mark mark
+ :max-aliases-for-collections max-aliases-for-collections
+ :allow-recursive-keys allow-recursive-keys
+ :allow-duplicate-keys allow-duplicate-keys)
+ string) keywords))
=====================================
src/java/clj_yaml/MarkedConstructor.java
=====================================
@@ -0,0 +1,61 @@
+package clj_yaml;
+
+import org.yaml.snakeyaml.constructor.Construct;
+import org.yaml.snakeyaml.constructor.Constructor;
+import org.yaml.snakeyaml.constructor.SafeConstructor;
+import org.yaml.snakeyaml.constructor.AbstractConstruct;
+import org.yaml.snakeyaml.nodes.Node;
+import org.yaml.snakeyaml.nodes.Tag;
+import org.yaml.snakeyaml.error.Mark;
+
+/* A subclass of SafeConstructor that wraps all the type-specific
+ constructors it defines with versions that mark the start and
+ end positions.
+*/
+public class MarkedConstructor extends SafeConstructor {
+ /* The types we want to wrap. */
+ public static Tag[] tags = {Tag.NULL, Tag.BOOL, Tag.INT, Tag.FLOAT,
+ Tag.BINARY, Tag.TIMESTAMP, Tag.OMAP,
+ Tag.PAIRS, Tag.SET, Tag.STR, Tag.SEQ, Tag.MAP};
+
+ public MarkedConstructor() {
+ // Make sure SafeConstructor's constructor is called first,
+ // so that we overwrite the keys that SafeConstructor sets.
+ super();
+ // Wrap all the constructors with Marking constructors.
+ for (Tag tag : tags) {
+ Construct old = this.yamlConstructors.get(tag);
+ this.yamlConstructors.put(tag, new Marking(old));
+ }
+ }
+ /* An intermediate representation of data marked with start and
+ end positions before we turn it into the nice clojure thing.
+ */
+ public static class Marked {
+ /* An object paired with start and end Marks. */
+ public Mark start;
+ public Mark end;
+ public Object marked;
+ public Marked(Mark start, Mark end, Object marked) {
+ this.start = start;
+ this.end = end;
+ this.marked = marked;
+ }
+ }
+
+ /* A wrapper around a Construct that marks source positions before calling
+ the original.
+ */
+ public class Marking extends AbstractConstruct {
+ public Construct constructor;
+ public Marking(Construct constructor) {
+ this.constructor = constructor;
+ }
+ public Object construct(Node node) {
+ return new Marked
+ (node.getStartMark(),
+ node.getEndMark(),
+ constructor.construct(node));
+ }
+ }
+}
=====================================
test/clj_yaml/core_test.clj
=====================================
@@ -1,6 +1,10 @@
(ns clj-yaml.core-test
- (:use clojure.test)
- (:use clj-yaml.core))
+ (:require [clojure.test :refer (deftest testing is)]
+ [clojure.string :as string]
+ [clj-yaml.core :refer [parse-string unmark generate-string]])
+ (:import [java.util Date]
+ (org.yaml.snakeyaml.error YAMLException)
+ (org.yaml.snakeyaml.constructor DuplicateKeyException)))
(def nested-hash-yaml
"root:\n childa: a\n childb: \n grandchild: \n greatgrandchild: bar\n")
@@ -24,8 +28,8 @@ items:
- Wicked Witch of the East
")
-(def inline-list-yaml "
---- # Shopping list
+(def inline-list-yaml
+"--- # Shopping list
[milk, pumpkin pie, eggs, juice]
")
@@ -48,6 +52,9 @@ women:
(def typed-data-yaml "
the-bin: !!binary 0101")
+(def io-file-typed-data-yaml "
+!!java.io.File")
+
(def set-yaml "
--- !!set
? Mark McGwire
@@ -58,6 +65,14 @@ the-bin: !!binary 0101")
(let [parsed (parse-string "foo: bar")]
(is (= "bar" (parsed :foo)))))
+(deftest parse-hash-with-numeric-key
+ (let [parsed (parse-string "123: 456")]
+ (is (= 456 (parsed 123)))))
+
+(deftest parse-hash-with-complex-key
+ (let [parsed (parse-string "[1, 2]: 3")]
+ (is (= 3 (parsed [1, 2])))))
+
(deftest parse-nested-hash
(let [parsed (parse-string nested-hash-yaml)]
(is (= "a" ((parsed :root) :childa)))
@@ -108,14 +123,110 @@ the-bin: !!binary 0101")
(let [parsed (parse-string typed-data-yaml)]
(is (= (Class/forName "[B") (type (:the-bin parsed))))))
+(deftest disallow-arbitrary-typed-data
+ (is (thrown? org.yaml.snakeyaml.error.YAMLException
+ (parse-string io-file-typed-data-yaml))))
+
(deftest keywordized
- (binding [*keywordize* false]
- (is (= "items" (-> hashes-lists-yaml parse-string ffirst))))
- (is (= "items" (-> hashes-lists-yaml (parse-string false) ffirst))))
+ (is (= "items"
+ (-> hashes-lists-yaml
+ (parse-string :keywords false)
+ ffirst))))
+
+(deftest not-keywordized-in-lists
+ (is (every? string?
+ (-> "[{b: c, c: d}]"
+ (parse-string :keywords false)
+ first
+ keys))))
+
+(deftest marking-source-position-works
+ (let [parsed (parse-string inline-list-yaml :mark true)]
+ ;; The list starts at the beginning of line 1.
+ (is (= 1 (-> parsed :start :line)))
+ (is (= 0 (-> parsed :start :column)))
+ ;; The first item starts at the second character of line 1.
+ (is (= 1 (-> parsed unmark first :start :line)))
+ (is (= 1 (-> parsed unmark first :start :column)))
+ ;; The first item ends at the fifth character of line 1.
+ (is (= 1 (-> parsed unmark first :end :line)))
+ (is (= 5 (-> parsed unmark first :end :column)))))
+
+(deftest text-wrapping
+ (let [data
+ {:description
+ "Big-picture diagram showing how our top-level systems and stakeholders interact"}]
+ (testing "long lines of text should not be wrapped"
+ ;; clj-yaml 0.5.6 used SnakeYAML 1.13 which by default did *not* split long lines.
+ ;; clj-yaml 0.6.0 upgraded to SnakeYAML 1.23 which by default *did* split long lines.
+ ;; This test ensures that generate-string uses the older behavior by default, for the sake
+ ;; of stability, i.e. backwards compatibility.
+ (is
+ (= "{description: Big-picture diagram showing how our top-level systems and stakeholders interact}\n"
+ (generate-string data))))))
(deftest dump-opts
(let [data [{:age 33 :name "jon"} {:age 44 :name "boo"}]]
(is (= "- age: 33\n name: jon\n- age: 44\n name: boo\n"
(generate-string data :dumper-options {:flow-style :block})))
(is (= "[{age: 33, name: jon}, {age: 44, name: boo}]\n"
- (generate-string data :dumper-options {:flow-style :flow})))))
\ No newline at end of file
+ (generate-string data :dumper-options {:flow-style :flow})))))
+
+(deftest parse-time
+ (testing "clj-time parses timestamps with more than millisecond precision correctly."
+ (let [timestamp "2001-11-23 15:02:31.123456 -04:00"
+ expected 1006542151123]
+ (is (= (.getTime ^Date (parse-string timestamp)) expected)))))
+
+(deftest maps-are-ordered
+ (let [parsed (parse-string hashes-lists-yaml)
+ [first second] (:items parsed)]
+ (is (= (keys first) '(:part_no :descrip :price :quantity)))
+ (is (= (keys second)'(:part_no :descrip :price :quantity :owners)))))
+
+
+(deftest nulls-are-fine
+ (testing "nil does not blow up"
+ (let [res (parse-string "- f:")]
+ (is (= [{:f nil}] res))
+ (is (str res)))))
+
+(deftest emoji-can-be-parsed
+ (let [yaml "{emoji: 💣}"]
+ (is (= yaml (-> yaml
+ (generate-string)
+ (parse-string)
+ (string/trim)))))
+
+ (testing "emoji in comments are OK too"
+ (let [yaml "# 💣 emoji in a comment\n42"]
+ (is (= 42 (parse-string yaml))))))
+
+(def too-many-aliases
+ (->> (range 51)
+ (map #(str "b" % ": *a"))
+ (cons "a: &a [\"a\",\"a\"]")
+ (string/join "\n")))
+
+(deftest max-aliases-for-collections-works
+ (is (thrown-with-msg? YAMLException #"Number of aliases" (parse-string too-many-aliases)))
+ (is (parse-string too-many-aliases :max-aliases-for-collections 51)))
+
+(def recursive-yaml "
+---
+&A
+- *A: *A
+")
+
+(deftest allow-recursive-works
+ (is (thrown-with-msg? YAMLException #"Recursive" (parse-string recursive-yaml)))
+ (is (parse-string recursive-yaml :allow-recursive-keys true)))
+
+(def duplicate-keys-yaml "
+a: 1
+a: 1
+")
+
+(deftest allow-recursive-works
+ (is (parse-string duplicate-keys-yaml))
+ (is (thrown-with-msg? DuplicateKeyException #"found duplicate key" (parse-string duplicate-keys-yaml :allow-duplicate-keys false))))
View it on GitLab: https://salsa.debian.org/clojure-team/clj-yaml-clojure/-/commit/29400c2fd3b8376fe4c02ef4a6481d3215577296
--
View it on GitLab: https://salsa.debian.org/clojure-team/clj-yaml-clojure/-/commit/29400c2fd3b8376fe4c02ef4a6481d3215577296
You're receiving this email because of your account on salsa.debian.org.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://alioth-lists.debian.net/pipermail/pkg-java-commits/attachments/20230205/11239eb6/attachment.htm>
More information about the pkg-java-commits
mailing list