Skip to content
Snippets Groups Projects
Unverified Commit 3ad532b9 authored by dpsutton's avatar dpsutton Committed by GitHub
Browse files

License improvements (#23120) 

* Switch from classpath to basis for license information

Previously we were chopping the classpath up and starting from there so
we just had a sequence of strings pointing at jars. Now using the basis
so we use a _much_ nicer map like

```clojure
{org.bouncycastle/bcprov-jdk15on
 {:mvn/version "1.70",
  :deps/manifest :mvn,
  :dependents
  [buddy/buddy-core
   org.bouncycastle/bcutil-jdk15on
   org.bouncycastle/bcpkix-jdk15on],
  :parents
  #{[buddy/buddy-sign buddy/buddy-core]
    [buddy/buddy-sign
     buddy/buddy-core
     org.bouncycastle/bcpkix-jdk15on
     org.bouncycastle/bcutil-jdk15on]
    [buddy/buddy-sign
     buddy/buddy-core
     org.bouncycastle/bcpkix-jdk15on]},
  :paths
  ["/Users/dan/.m2/repository/org/bouncycastle/bcprov-jdk15on/1.70/bcprov-jdk15on-1.70.jar"]}
 ...}

```

So we now have a true name for the dependency, a path to it, and a
version. No need to string munge on the classpath.

* Read pom with `maven-model`

rather than us trying to parse the xml, use a maven model on it. Note
there is another level we can go to that would be aware of parent poms
but we don't need the overkill here. That's far heavier than what we
need to do in this instance.

Note this also reorders the algo:
previously
- license in jar
- backfill
- license from pom

Now:
- license from jar
- license from pom
- backfill

Possible we will want to actually just skip the license from pom bit
since it only gives us a name and a url and not the full text. We could
match on these and identify them with the resources used from the
backfill if we like.

Another important change is that this no longer throws if it cannot find
a pom for a jar. This came up for the following lib:

```
;; deps.edn
com.google.cloud.sql/postgres-socket-factory
{:mvn/version "1.6.0"} ; Secure Google Cloud SQL postgres connections

;; override:
"com.github.jnr"             {"jffi$native" {:resource "apache2_0.txt"}}
```
Which was from a 3rd party PR.

* Way to generate libs that need overrides

* Remove unused fns, add tests, cleanup docstring

* Don't leave tap> in

Two poms blow up with BOM errors

- jakarta.activation-1.2.1.pom
- jakarta.activation-api-1.2.2.pom

> UTF-8 BOM plus xml decl of iso-8859-1 is incompatible
> (position: START_DOCUMENT seen <?xml version="1.0" encoding="iso-8859-1"... @1:41)

It has a parent pom and not a license so we wouldn't find it anyways.
parent cc8161cf
No related branches found
No related tags found
No related merge requests found
Hide whitespace changes
Inline Side-by-side
bin/build-mb/deps.edn
+ 7
− 4
View file @ 3ad532b9
{:paths ["src" "resources"]
:deps
{common/common {:local/root "../common"}
build-drivers/build-drivers {:local/root "../build-drivers"}
i18n/i18n {:local/root "../i18n"}
org.flatland/ordered {:mvn/version "1.15.10"}}
{common/common {:local/root "../common"}
build-drivers/build-drivers {:local/root "../build-drivers"}
i18n/i18n {:local/root "../i18n"}
org.flatland/ordered {:mvn/version "1.15.10"}
io.github.clojure/tools.build {:git/tag "v0.7.5" :git/sha "2526f58"}
;; value currently used in tools.build but top level since we directly depend on it
org.apache.maven/maven-model {:mvn/version "3.8.4"}}
:aliases
{:test {:extra-paths ["test"]
......
bin/build-mb/src/build.clj
+ 4
− 5
View file @ 3ad532b9
......@@ -5,6 +5,7 @@
[clojure.edn :as edn]
[clojure.java.io :as io]
[clojure.string :as str]
[clojure.tools.build.api :as b]
[environ.core :as env]
[flatland.ordered.map :as ordered-map]
[i18n.create-artifacts :as i18n]
......@@ -48,15 +49,13 @@
[edition]
{:pre [(#{:oss :ee} edition)]}
(u/step "Generate backend license information from jar files"
(let [[classpath] (u/sh {:dir u/project-root-directory
:quiet? true}
"clojure" (str "-A" edition) "-Spath")
(let [basis (b/create-basis {:project (u/filename u/project-root-directory "deps.edn")})
output-filename (u/filename u/project-root-directory
"resources"
"license-backend-third-party.txt")
{:keys [without-license]} (license/generate {:classpath classpath
{:keys [without-license]} (license/generate {:basis basis
:backfill (edn/read-string
(slurp (io/resource "overrides.edn")))
(slurp (io/resource "overrides.edn")))
:output-filename output-filename
:report? false})]
(when (seq without-license)
......
bin/build-mb/src/build/licenses.clj
+ 114
− 106
View file @ 3ad532b9
......@@ -27,57 +27,24 @@
(:require [clojure.data.xml :as xml]
[clojure.edn :as edn]
[clojure.java.io :as io]
[clojure.string :as str])
(:import (java.nio.file Files FileSystem FileSystems FileVisitOption LinkOption OpenOption Path Paths)))
[clojure.string :as str]
[clojure.tools.build.api :as b])
(:import (java.nio.file Files FileSystem FileSystems FileVisitOption LinkOption OpenOption Path Paths)
(org.apache.maven.model License Model)
(org.apache.maven.model.io.xpp3 MavenXpp3Reader)
(java.io FileReader)))
(set! *warn-on-reflection* true)
(def classpath-separator (System/getProperty "path.separator"))
(defn jar-file? [filename]
(str/ends-with? filename "jar"))
(def tag-name (comp keyword (fnil name "") :tag))
(def ^:private tag-content (juxt tag-name (comp first :content)))
(defn pom->coordinates [pom-xml]
(let [coords (->> pom-xml
:content
(filter #(#{:groupId :artifactId :version} (tag-name %)))
(map tag-content)
(into {}))
parent (->> pom-xml
:content
(filter #(#{:parent} (tag-name %)))
first
:content
(keep tag-content)
(into {}))]
{:group (or (:groupId coords) (:groupId parent))
:artifact (:artifactId coords)
:version (or (:version coords) (:version parent))}))
(defn pom->licenses [pom-xml]
(let [licenses (some->> pom-xml
:content
(filter #(#{:licenses} (tag-name %)))
first
:content
first
:content
(map tag-content)
(into {}))]
licenses))
(defn ->BiPredicate [f]
(defn- ->BiPredicate [f]
(reify java.util.function.BiPredicate
(test [_ x y]
(f x y))))
(def path-options (into-array String []))
(def filevisit-options (into-array FileVisitOption []))
(def link-options (into-array LinkOption []))
(def open-options (into-array OpenOption []))
(def ^:private path-options (into-array String []))
(def ^:private filevisit-options (into-array FileVisitOption []))
(def ^:private link-options (into-array LinkOption []))
(def ^:private open-options (into-array OpenOption []))
(defn determine-pom
"Produce a pom path from a jar. Looks first for a pom adjacent to the jar, and then finds all files that end with
......@@ -97,8 +64,7 @@
(str/ends-with? (str path) "pom.xml")))]
(.. (Files/find jar-root Integer/MAX_VALUE pred filevisit-options)
findFirst
(orElse nil)))
(throw (ex-info "Cannot locate pom" {:jar jar-filename}))))
(orElse nil)))))
(defn do-with-path-is
"Open an inputstream on `path` and call `f` with the inputstream as an argument. Function `f` should not be lazy."
......@@ -106,6 +72,27 @@
(with-open [is (Files/newInputStream path open-options)]
(f is)))
(defn license-from-pom
"Read license information from a pom. This reads only the local pom and does not trace parent poms. Clojure.core.async includes a parent pom:
<parent>
<groupId>org.clojure</groupId>
<artifactId>pom.contrib</artifactId>
<version>1.1.0</version>
</parent>
which would specify a license. To trace this would require setting up way more machinery and so just let the
overrides catch this scenario."
[^Path pom-path]
(try
(let [reader (MavenXpp3Reader.)
model (.read reader (FileReader. (.toFile pom-path)))
license ^License (first (.getLicenses model))]
(when license
{:name (.getName license)
:url (.getUrl license)}))
(catch Exception _e nil)))
(def ^:private license-file-names
["LICENSE" "LICENSE.txt" "META-INF/LICENSE"
"META-INF/LICENSE.txt" "license/LICENSE"])
......@@ -120,55 +107,55 @@
(defn license-from-backfill
"Look in the backfill information for license information."
[{:keys [group artifact]} backfill]
(if-let [license (some #(get-in backfill %)
[[group artifact]
[:override/group group]])]
(if (string? license)
license
(if-let [resource (io/resource (:resource license))]
(slurp resource)
(throw (ex-info (str "Missing license for " artifact)
{:group group
:artifact artifact
:backfill license}))))))
[lib-name backfill]
(let [[group artifact] ((juxt namespace name) lib-name)]
(when-let [license (some #(get-in backfill %)
[[group artifact]
[:override/group group]])]
(if (string? license)
license
(if-let [resource (io/resource (:resource license))]
(slurp resource)
(throw (ex-info (str "Missing license for " artifact)
{:group group
:artifact artifact
:backfill license})))))))
(defn discern-license-and-coords
"Returns a tuple of [jar-filename {:coords :license [:error]}. Error is optional. License will be a string of license
text, coords a map with group and artifact."
[^String jar-filename backfill]
(let [jar-path ^Path (Paths/get jar-filename path-options)
classloader ^ClassLoader (ClassLoader/getSystemClassLoader)]
[[lib-name info] backfill]
(let [jar-filename (-> info :paths first)
jar-path (Paths/get ^String jar-filename path-options)
classloader (ClassLoader/getSystemClassLoader)]
(if-not (Files/exists jar-path link-options)
[jar-filename {:error "Jar does not exist"}]
[lib-name {:error "Jar does not exist"}]
(try
(with-open [jar-fs (FileSystems/newFileSystem jar-path classloader)]
(let [pom-path (determine-pom jar-filename jar-fs)
license-path (license-from-jar jar-fs)
[coords pom-license] (do-with-path-is pom-path
(comp (juxt pom->coordinates pom->licenses)
#(xml/parse % :skip-whitespace true)))
license (or (when license-path
(with-open [is (Files/newInputStream license-path open-options)]
(slurp is)))
(license-from-backfill coords backfill)
(let [{:keys [name url]} pom-license]
(when name (str name ": " url))))]
[jar-filename (cond-> {:coords coords :license license}
(not (and license coords))
(assoc :error "Error determining license or coords"))]))
(let [license (or (when-let [license-path (license-from-jar jar-fs)]
(with-open [is (Files/newInputStream license-path open-options)]
(slurp is)))
(when-let [pom-path (determine-pom jar-filename jar-fs)]
(let [{:keys [name url]} (license-from-pom pom-path)]
(when name (str name ": " url))))
(license-from-backfill lib-name backfill))]
[lib-name (cond-> {:coords {:group (namespace lib-name)
:artifact (name lib-name)
:version (:mvn/version info)}
:license license}
(not license)
(assoc :error "Error determining license"))]))
(catch Exception e
[jar-filename {:error e}])))))
(defn write-license [success-os [_jar {:keys [coords license]}]]
(let [{:keys [group artifact]} coords]
(binding [*out* success-os]
(println "The following software may be included in this product: "
group ":" artifact
". This software contains the following license and notice below:")
(println "\n")
(println license)
(println "\n\n----------\n"))))
[lib-name {:error e}])))))
(defn write-license [success-os [lib {:keys [coords license]}]]
(binding [*out* success-os]
(println "The following software may be included in this product:"
(str lib ": " (:version coords) ".")
"This software contains the following license and notice below:")
(println "\n")
(println license)
(println "\n\n----------\n")))
(defn report-missing [error-os [jar {:keys [coords]}]]
(let [{:keys [group artifact]} coords
......@@ -178,24 +165,26 @@
(binding [*out* error-os]
(println dep-name " : No license information found."))))
(defn process* [{:keys [classpath-entries backfill]}]
(let [info (map #(discern-license-and-coords % backfill) classpath-entries)
(defn process*
"Returns a map of `:with-license` and `:without-license`."
[{:keys [libs backfill]}]
(let [info (map #(discern-license-and-coords % backfill) libs)
categorized (group-by (comp (complement #(contains? % :error)) second) info)]
{:with-license (categorized true)
:without-license (categorized false)}))
(defn jar-entries
"Returns a seq of jar entries on the classpath"
[classpath]
(->> (str/split classpath (re-pattern classpath-separator))
(filter jar-file?)))
"Filters the `:libs` map of a basis to just `:mvn` based (jar based) libs. Keys are symbols of the lib and the value
is a map of `:mvn/version` and `:paths` amont others."
[basis]
(into {} (filter (comp #{:mvn} :deps/manifest val) (:libs basis))))
(defn generate
"Process a classpath, creating a file of all license information, writing to `:output-filename`. `classpath-entries`
should be a seq of classpath roots. Split a classpath on the classpath separator. Backfill is a clojure data
structure or a filename of an edn file of a clojure datastructure providing for backfilling license information if
it is not discernable from the jar. Should be of the form (note keys are strings not symbols)
"Process a basis from clojure.tools.build.api/create-basis, creating a file of all license information, writing to
`:output-filename`. Backfill is a clojure data structure or a filename of an edn file of a clojure datastructure
providing for backfilling license information if it is not discernable from the jar. Should be of the form (note
keys are strings not symbols)
{\"group\" {\"artifact\" \"license text\"}
\"group\" {\"artifact\" {:resource \"filename-of-license\"}}
......@@ -206,23 +195,24 @@
Algorithm is:
- check jar for license file at a few different standard paths. If present keep this text.
- Look in pom file next to jar or in jar for license information. If found this information is used,
it is not expanded into a full license text.
- look in provided backfill information for license text or a resource containing the license text
- Look in pom file next to jar or in jar for license information. If found this information is used, it is not
expanded into a full license text.
Reports if `:report?` is true (the default). Writes missing license information to *err* and summary of identified licenses to *out*.
Reports if `:report?` is true (the default). Writes missing license information to *err* and summary of identified
licenses to *out*.
Returns a map
{:with-license [ [jar-filename {:coords {:group :artifact :version} :license <text>}] ...]
:without-license [ [jar-filename {:coords {:group :artifact :version} :error <text>}] ... ]}"
[{:keys [classpath backfill output-filename report?] :or {report? true}}]
{:with-license [ [lib-name {:coords {:group :artifact :version} :license <text>}] ...]
:without-license [ [lib-name {:coords {:group :artifact :version} :error <text>}] ... ]}"
[{:keys [basis backfill output-filename report?] :or {report? true}}]
(let [backfill (if (string? backfill)
(edn/read-string (slurp (io/resource backfill)))
(or backfill {}))
entries (jar-entries classpath)
entries (jar-entries basis)
{:keys [with-license without-license] :as license-info}
(process* {:classpath-entries entries
:backfill backfill})]
(process* {:libs entries
:backfill backfill})]
(when (seq with-license)
(with-open [os (io/writer output-filename)]
(run! #(write-license os %) with-license)))
......@@ -236,3 +226,21 @@
;; best defaults. Want to make sure we never kill our build script
#_(System/exit (if (seq without-license) 1 0))))
license-info))
(comment
(def basis (b/create-basis {:project "path-to-metabase/deps.edn"}))
(def libs (process* {:libs (jar-entries basis)
:backfill (edn/read-string
(slurp (io/resource "overrides.edn")))}))
(process* {:libs (-> basis :libs (select-keys '[org.clojure/clojure
org.clojure/core.async]))})
(get-in basis [:libs 'org.clojure/clojure])
(->> libs
:without-license
(map first))
(def libs-no-overrides (process* {:libs (jar-entries basis)}))
(->> libs-no-overrides :without-license (map first))
)
bin/build-mb/test/build/licenses_test.clj
+ 67
− 100
View file @ 3ad532b9
......@@ -5,24 +5,12 @@
[clojure.java.io :as io]
[clojure.string :as str]
[clojure.test :refer [deftest is run-tests testing]]
[clojure.tools.build.api :as b]
[metabuild-common.core :as u])
(:import (java.io StringReader StringWriter)
(java.nio.file Files FileSystems LinkOption Path Paths)))
(def classpath-urls (str/split (System/getProperty "java.class.path")
(re-pattern lic/classpath-separator)))
(defn jar-filename-from-cp
[jar-filename classpath-urls]
(let [[x & rst :as found] (filter #(when (str/includes? % jar-filename) %) classpath-urls)]
(cond (seq rst) (throw (ex-info (str "Multiple jars found for " jar-filename)
{:filename jar-filename
:found found}))
(not x) (throw (ex-info (str "No results found for " jar-filename)
{:filename jar-filename}))
x x)))
(defn parse [rdr] (xml/parse rdr :skip-whitespace true))
(defn- parse [rdr] (xml/parse rdr :skip-whitespace true))
(def clojure-xml "<?xml version=\"1.0\" encoding=\"UTF-8\"?>
<project xmlns=\"http://maven.apache.org/POM/4.0.0\" xmlns:xsi=\"http://www.w3.org/2001/XMLSchema-instance\" xsi:schemaLocation=\"http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd\">
......@@ -43,7 +31,8 @@
</project>"
)
(def clojure-jdbc-xml "<?xml version=\"1.0\" encoding=\"utf-8\"?>
(def ^:private clojure-jdbc-xml
"<?xml version=\"1.0\" encoding=\"utf-8\"?>
<project xmlns=\"http://maven.apache.org/POM/4.0.0\" xmlns:xsi=\"http://www.w3.org/2001/XMLSchema-instance\" xsi:schemaLocation=\"http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd\">
<modelVersion>4.0.0</modelVersion>
<artifactId>java.jdbc</artifactId>
......@@ -58,59 +47,40 @@
</project>"
)
(deftest pom->coordinates-test
(testing "Parses version information from pom"
(testing "When group information is top level"
(let [xml (parse (StringReader. clojure-xml))]
(is (= {:group "org.clojure"
:artifact "clojure"
:version "1.10.3"}
(lic/pom->coordinates xml)))))
(testing "When group informatoin is under the parent tag"
(let [xml (parse (StringReader. clojure-jdbc-xml))]
(is (= {:group "org.clojure"
:artifact "java.jdbc"
:version "0.7.11"}
(lic/pom->coordinates xml)))))))
(defn jar->path [filename]
(defn jar->path
"Return a Path for a jar."
^Path [filename]
(let [path (Paths/get filename (into-array String []))]
(if (Files/exists path (into-array LinkOption []))
path
(throw (ex-data (str "Jar " filename " not found") {:filename filename})))))
(defn jar [filename]
(jar->path (jar-filename-from-cp filename classpath-urls)))
(deftest pom->license-test
(testing "Can find license information from the pom"
(testing "If present"
(let [xml (parse (StringReader. clojure-xml))]
(is (= {:name "Eclipse Public License 1.0"
:url "http://opensource.org/licenses/eclipse-1.0.php"
:distribution "repo"}
(lic/pom->licenses xml))))
(let [jar-filename (jar-filename-from-cp "org/clojure/clojure" classpath-urls)
jar-path (Paths/get jar-filename (into-array String []))
^ClassLoader classloader nil]
(with-open [jar-fs (FileSystems/newFileSystem jar-path classloader)]
(let [pom-path (lic/determine-pom jar-filename jar-fs)]
(is (= {:name "Eclipse Public License 1.0"
:url "http://opensource.org/licenses/eclipse-1.0.php"
:distribution "repo"}
(lic/do-with-path-is pom-path (comp lic/pom->licenses parse))))))))
(testing "Returning nil if not present"
(let [xml (parse (StringReader. clojure-jdbc-xml))]
(is (nil? (lic/pom->licenses xml)))))))
(throw (ex-info (str "Jar " filename " not found") {:filename filename})))))
(def ^:private basis
"A basis for this project suitable for testing."
(b/create-basis {:project (u/filename u/project-root-directory "deps.edn")}))
(defn- jar ^Path [lib-name]
(jar->path (-> basis :libs (get lib-name) :paths first)))
(deftest license-from-pom-test
(let [clojure-jar (-> basis :libs (get 'org.clojure/clojure) :paths first)
jar-path (Paths/get ^String clojure-jar (into-array String []))]
(with-open [jar-fs (FileSystems/newFileSystem jar-path
(ClassLoader/getSystemClassLoader))]
(let [clojure-pom (lic/determine-pom clojure-jar jar-fs)]
(is (some? clojure-pom) "Clojure pom not found")
(is (= {:name "Eclipse Public License 1.0"
:url "http://opensource.org/licenses/eclipse-1.0.php"}
(lic/license-from-pom clojure-pom))))))
(let [libs (-> basis :libs (select-keys '[org.clojure/clojure]) first)]
(is (= "Eclipse Public License 1.0: http://opensource.org/licenses/eclipse-1.0.php"
(-> (lic/discern-license-and-coords libs {}) second :license)))))
(deftest license-from-jar-test
(letfn [(license-path [j f]
(let [cl ^ClassLoader (ClassLoader/getSystemClassLoader)]
;; java 16 has a single arity that just takes a path, but older versions need a classloader. It can be
;; nil but felt weird typehinting the nil in a binding
(with-open [jar-fs (FileSystems/newFileSystem ^Path (jar j) cl)]
(some-> (lic/license-from-jar jar-fs)
f))))
(with-open [jar-fs (FileSystems/newFileSystem (jar j) (ClassLoader/getSystemClassLoader))]
(some-> (lic/license-from-jar jar-fs)
f)))
(first-line [path]
(lic/do-with-path-is path (fn [is]
(->> (slurp is)
......@@ -120,11 +90,11 @@
str/trim))))]
(testing "Can find license information bundled in the jar"
(is (= "META-INF/LICENSE.txt"
(license-path "commons/commons-math3" str)))
(license-path 'org.apache.commons/commons-math3 str)))
(is (= "Apache License"
(license-path "commons/commons-math3" first-line))))
(license-path 'org.apache.commons/commons-math3 first-line))))
(testing "Nil if not present"
(is (nil? (license-path "cronparser/cron-parser-core" identity))))))
(is (nil? (license-path 'net.redhogs.cronparser/cron-parser-core identity))))))
(deftest license-from-backfill-test
(let [backfill {"a" {"a" "License Information"}
......@@ -133,20 +103,20 @@
{"group-y" "License Information"
"group-z" {:resource "EPL.txt"}}}]
(doseq [[coords expected-license]
[[{:group "a" :artifact "a"} "License Information"]
[{:group "b" :artifact "b"} "Permission is hereby granted"]
[{:group "group-y" :artifact "literally-any-package"} "License Information"]
[{:group "group-z" :artifact "literally-any-package"} "\nEclipse Public License"]]]
[['a/a "License Information"]
['b/b "Permission is hereby granted"]
['group-y/literally-anything "License Information"]
['group-z/literally-anything "\nEclipse Public License"]]]
(let [license (lic/license-from-backfill coords backfill)]
(is (not (str/blank? license)))
(is (str/starts-with? license expected-license))))))
(deftest process*-test
(testing "categorizes jar entries"
(let [jars (map #(jar-filename-from-cp % classpath-urls)
["org/clojure/clojure"
"commons/commons-math3"
"cronparser/cron-parser-core"])
(let [jar-libs (select-keys (:libs basis)
'[org.clojure/clojure
org.apache.commons/commons-math3
net.redhogs.cronparser/cron-parser-core])
normalize-entry (fn [[jar {:keys [coords license error]}]]
[((juxt :group :artifact) coords)
(cond-> {:license (not (str/blank? license))}
......@@ -162,9 +132,9 @@
["org.apache.commons" "commons-math3"] {:license true}}
:without-license
{["net.redhogs.cronparser" "cron-parser-core"]
{:license false :error "Error determining license or coords"}}}
(normalize (lic/process* {:classpath-entries jars
:backfill {}})))))
{:license false :error "Error determining license"}}}
(normalize (lic/process* {:libs jar-libs
:backfill {}})))))
(testing "with backfill by group and artifact"
(is (= {:with-license
{["org.clojure" "clojure"] {:license true}
......@@ -172,7 +142,7 @@
["net.redhogs.cronparser" "cron-parser-core"] {:license true}}
:without-license
{}}
(normalize (lic/process* {:classpath-entries jars
(normalize (lic/process* {:libs jar-libs
:backfill
{"net.redhogs.cronparser"
{"cron-parser-core" "license"}}})))))
......@@ -183,24 +153,23 @@
["net.redhogs.cronparser" "cron-parser-core"] {:license true}}
:without-license
{}}
(normalize (lic/process* {:classpath-entries jars
(normalize (lic/process* {:libs jar-libs
:backfill
{:override/group
{"net.redhogs.cronparser" "license"}}}))))))))
(deftest write-license-test
(is (= (str "The following software may be included in this product: a : a . "
(is (= (str "The following software may be included in this product: a/a: 1.0. "
"This software contains the following license and notice below:\n\n\n"
"license text\n\n\n----------\n\n")
(let [os (StringWriter.)]
(lic/write-license os ["jar" {:coords {:group "a" :artifact "a"}
:license "license text"}])
(lic/write-license os ['a/a {:coords {:group "a" :artifact "a" :version "1.0"}
:license "license text"}])
(str os)))))
(defn- loop-until-success [f max step-name]
(loop [n 0]
(let [success? (try
(do (f) true)
(let [success? (try (f) true
(catch Exception _ false))]
(cond success? ::done
(and (not success?) (< n max)) (recur (inc n))
......@@ -209,21 +178,19 @@
(deftest all-deps-have-licenses
(testing "All deps on the classpath have licenses"
(loop-until-success #(u/sh {:dir u/project-root-directory} "clojure" "-A:ee" "-P") 3 "download deps")
(let [classpath (u/sh {:dir u/project-root-directory
:quiet? true}
"clojure"
"-A:ee"
"-Spath")
classpath-entries (->> (str/split (last classpath) (re-pattern lic/classpath-separator))
(filter lic/jar-file?))]
(let [results (lic/process* {:classpath-entries classpath-entries
:backfill (edn/read-string
(slurp (io/resource "overrides.edn")))})]
(is (nil? (:without-license results))
(str "Deps without license information:\n"
(str/join "\n" (map first (:without-license results)))))
(is (= (set classpath-entries)
(into #{} (->> results :with-license (map first))))))
(let [jar-libs (lic/jar-entries basis)
results (lic/process* {:libs jar-libs
:backfill (edn/read-string
(slurp (io/resource "overrides.edn")))})]
(is (nil? (:without-license results))
(str "Deps without license information:\n"
(str/join "\n" (map first (:without-license results)))))
(is (= (set (keys jar-libs))
(into #{} (->> results :with-license (map first)))))
(is (some? (:without-license
(lic/process* {:classpath-entries classpath-entries
:backfill {}})))))))
(lic/process* {:libs jar-libs
:backfill {}})))))))
(comment
(run-tests)
)
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment