CVE-2016-4434

Debian Bug report logs - #825501
CVE-2016-4434

Toggle useless messages

Message #5 received at submit@bugs.debian.org (full text, mbox, reply):

From: Moritz Muehlenhoff <jmm@debian.org>

To: Debian Bug Tracking System <submit@bugs.debian.org>

Subject: CVE-2016-4434

Date: Fri, 27 May 2016 11:58:33 +0200

Source: tika
Severity: grave
Tags: security

Hi,
please see http://seclists.org/oss-sec/2016/q2/413  for details.

Cheers,
        Moritz

Message #10 received at 825501@bugs.debian.org (full text, mbox, reply):

From: Emmanuel Bourg <ebourg@apache.org>

To: Moritz Muehlenhoff <jmm@debian.org>, 825501@bugs.debian.org

Subject: Re: Bug#825501: CVE-2016-4434

Date: Fri, 27 May 2016 12:26:53 +0200

Thank you for the notice Moritz. Tika isn't really used in Debian yet, I
packaged it as a dependency of Apache JMeter but didn't enable it. I'll
fix it in unstable, but I don't think it's worth fixing in Jessie.

Emmanuel Bourg

Message #21 received at 825501@bugs.debian.org (full text, mbox, reply):

From: Faidon Liambotis <paravoid@debian.org>

To: Moritz Muehlenhoff <jmm@debian.org>, 825501@bugs.debian.org

Subject: Re: Bug#825501: CVE-2016-4434

Date: Thu, 11 Jan 2018 14:03:23 +0200

On Fri, May 27, 2016 at 11:58:33AM +0200, Moritz Muehlenhoff wrote:
> please see http://seclists.org/oss-sec/2016/q2/413  for details.

That link says:
  Versions Affected: 
  Apache Tika 0.10 to 1.12

So perhaps 1.5 isn't affected after all? I tried to find the relevant
commit in the upstream git but failed :(

Regards,
Faidon

Message #26 received at 825501@bugs.debian.org (full text, mbox, reply):

From: Moritz Muehlenhoff <jmm@inutil.org>

To: Faidon Liambotis <paravoid@debian.org>

Cc: 825501@bugs.debian.org

Subject: Re: Bug#825501: CVE-2016-4434

Date: Fri, 12 Jan 2018 19:54:58 +0100

On Thu, Jan 11, 2018 at 02:03:23PM +0200, Faidon Liambotis wrote:
> On Fri, May 27, 2016 at 11:58:33AM +0200, Moritz Muehlenhoff wrote:
> > please see http://seclists.org/oss-sec/2016/q2/413  for details.
> 
> That link says:
>   Versions Affected: 
>   Apache Tika 0.10 to 1.12
> 
> So perhaps 1.5 isn't affected after all? I tried to find the relevant
> commit in the upstream git but failed :(

Commit https://github.com/apache/tika/commit/f444fd784b99b181cd7bd54cdec9fbd132b4ef93
in 1.17 added a test case, so this might be related to changes in Xerces/J
which are possibly bundled by Tika downloads? Might be worth clarifying with
Tim Allison <tallison@apache.org>.

Cheers,
        Moritz

Message #31 received at 825501@bugs.debian.org (full text, mbox, reply):

From: Salvatore Bonaccorso <carnil@debian.org>

To: Faidon Liambotis <paravoid@debian.org>

Cc: Moritz Muehlenhoff <jmm@inutil.org>, 825501@bugs.debian.org

Subject: Re: Bug#825501: CVE-2016-4434

Date: Thu, 18 Jan 2018 22:36:24 +0100

Hi Faidon,

On Fri, Jan 12, 2018 at 07:54:58PM +0100, Moritz Muehlenhoff wrote:
> On Thu, Jan 11, 2018 at 02:03:23PM +0200, Faidon Liambotis wrote:
> > On Fri, May 27, 2016 at 11:58:33AM +0200, Moritz Muehlenhoff wrote:
> > > please see http://seclists.org/oss-sec/2016/q2/413  for details.
> > 
> > That link says:
> >   Versions Affected: 
> >   Apache Tika 0.10 to 1.12
> > 
> > So perhaps 1.5 isn't affected after all? I tried to find the relevant
> > commit in the upstream git but failed :(
> 
> Commit https://github.com/apache/tika/commit/f444fd784b99b181cd7bd54cdec9fbd132b4ef93
> in 1.17 added a test case, so this might be related to changes in Xerces/J
> which are possibly bundled by Tika downloads? Might be worth clarifying with
> Tim Allison <tallison@apache.org>.

Above, you said "so perhaps 1.5 isn't affected after all?". But why
this conclusion? 1.5 as currently in unstable and oldstable present
falls within the affected range of 0.15 and 1.12.

The issue is claimed to be fixed in upstream 1.13 (and as Moritz
pointed out a test was added. Comparing commits between 1.12 and 1.13
I was unable to isolate the relevant commit(s), but there are some
touching the code for "OOXML files and XMP in PDF and other file
formats".

So yes, maybe Tim Allison can help identify which are the required
commits, but best course might just to try to update to the newest
upstream version for unstable.

Regards,
Salvatore

Message #36 received at 825501@bugs.debian.org (full text, mbox, reply):

From: Faidon Liambotis <paravoid@debian.org>

To: Salvatore Bonaccorso <carnil@debian.org>

Cc: Moritz Muehlenhoff <jmm@inutil.org>, 825501@bugs.debian.org

Subject: Re: Bug#825501: CVE-2016-4434

Date: Thu, 18 Jan 2018 23:44:48 +0200

On Thu, Jan 18, 2018 at 10:36:24PM +0100, Salvatore Bonaccorso wrote:
> > > That link says:
> > >   Versions Affected: 
> > >   Apache Tika 0.10 to 1.12
> > > 
> > > So perhaps 1.5 isn't affected after all? I tried to find the relevant
> > > commit in the upstream git but failed :(
> > 
> > Commit https://github.com/apache/tika/commit/f444fd784b99b181cd7bd54cdec9fbd132b4ef93
> > in 1.17 added a test case, so this might be related to changes in Xerces/J
> > which are possibly bundled by Tika downloads? Might be worth clarifying with
> > Tim Allison <tallison@apache.org>.
> 
> Above, you said "so perhaps 1.5 isn't affected after all?". But why
> this conclusion? 1.5 as currently in unstable and oldstable present
> falls within the affected range of 0.15 and 1.12.

s/0.15/0.10/ in what you said just above, but yes, you're obviously
right and I misread the range. Apologies for the confusion -- I guess I
was too enthusiastic in trying to figure out an easy way out of this :)

> So yes, maybe Tim Allison can help identify which are the required
> commits, but best course might just to try to update to the newest
> upstream version for unstable.

Indeed! (but note that I'm not the maintainer)

Thanks,
Faidon

Message #41 received at 825501@bugs.debian.org (full text, mbox, reply):

From: Cyril Brulebois <kibi@debian.org>

To: Salvatore Bonaccorso <carnil@debian.org>, 825501@bugs.debian.org

Cc: Faidon Liambotis <paravoid@debian.org>, Moritz Muehlenhoff <jmm@inutil.org>, pantomime-clojure@packages.debian.org, puppetdb@packages.debian.org, pkg-puppet-devel@lists.alioth.debian.org

Subject: Re: Bug#825501: CVE-2016-4434

Date: Mon, 31 Dec 2018 01:13:51 +0100

[Message part 1 (text/plain, inline)]

Heya,

Not the maintainer either, just joining the fun to see if I can help get
stuff to move; my main motivation behind this is trying to get the
puppetdb → pantomime-clojure → tika dependency chain in a suitable state
for buster (other *-clojure packages need fixing, but FTBFSes have
patches/MRs now, and uploads should be happening soon enough; but
there's still comidi-clojure's #889125 to keep me busy anyway…)


Salvatore Bonaccorso <carnil@debian.org> (2018-01-18):
> The issue is claimed to be fixed in upstream 1.13 (and as Moritz
> pointed out a test was added. Comparing commits between 1.12 and 1.13
> I was unable to isolate the relevant commit(s), but there are some
> touching the code for "OOXML files and XMP in PDF and other file
> formats".

Right, I haven't been able to pinpoint the exact changes, but those
could be “hidden” in things like pdfbox version bumps, etc. Even if a
specific fix for 1.5 would be identified, it seems hard to get it to
build; I've tried that just to see what was feasible, and it doesn't
look good anyway:

  https://bugs.debian.org/850798#12

Not being a Java expert, I've then moved to giving the latest upstream
release (1.20) a shot, but there were too many red things, so I've tried
to aim at 1.13 “only”, to get this CVE addressed.

My WIP is available there:
  https://salsa.debian.org/kibi/tika
  https://salsa.debian.org/kibi/tika/commits/master

Downloaded and imported 1.13 with uscan, then failed to apply patches,
(almost) all of which I've disabled. I've number mine 90+ for easy
identification.

First failure was missing version for junit dependencies:
| [ERROR] [ERROR] Some problems were encountered while processing the POMs:
| […]
| [ERROR]   
| [ERROR]   The project org.apache.tika:tika-serialization:1.13 (/home/kibi/hack/bsp/puppetdb-builds/tika.git/tika-serialization/pom.xml) has 1 error
| [ERROR]     'dependencies.dependency.version' for junit:junit:jar is missing. @ org.apache.tika:tika-serialization:[unknown-version], /home/kibi/hack/bsp/puppetdb-builds/tika.git/tika-serialization/pom.xml, line 59, column 17
| [ERROR]   
| [ERROR]   The project org.apache.tika:tika-batch:1.13 (/home/kibi/hack/bsp/puppetdb-builds/tika.git/tika-batch/pom.xml) has 1 error
| [ERROR]     'dependencies.dependency.version' for junit:junit:jar is missing. @ org.apache.tika:tika-batch:[unknown-version], /home/kibi/hack/bsp/puppetdb-builds/tika.git/tika-batch/pom.xml, line 85, column 17
| [ERROR]   
| [ERROR]   The project org.apache.tika:tika-translate:1.13 (/home/kibi/hack/bsp/puppetdb-builds/tika.git/tika-translate/pom.xml) has 1 error
| [ERROR]     'dependencies.dependency.version' for junit:junit:jar is missing. @ org.apache.tika:tika-translate:[unknown-version], /home/kibi/hack/bsp/puppetdb-builds/tika.git/tika-translate/pom.xml, line 66, column 17
| [ERROR]   
| [ERROR]   The project org.apache.tika:tika-langdetect:1.13 (/home/kibi/hack/bsp/puppetdb-builds/tika.git/tika-langdetect/pom.xml) has 1 error
| [ERROR]     'dependencies.dependency.version' for junit:junit:jar is missing. @ org.apache.tika:tika-langdetect:[unknown-version], /home/kibi/hack/bsp/puppetdb-builds/tika.git/tika-langdetect/pom.xml, line 64, column 17
| [ERROR]   
| [ERROR]   The project org.apache.tika:tika-example:1.13 (/home/kibi/hack/bsp/puppetdb-builds/tika.git/tika-example/pom.xml) has 1 error
| [ERROR]     'dependencies.dependency.version' for junit:junit:jar is missing. @ org.apache.tika:tika-example:[unknown-version], /home/kibi/hack/bsp/puppetdb-builds/tika.git/tika-example/pom.xml, line 114, column 17

Hence debian/patches/90-add-junit-version.patch


Next failure:
| [ERROR] Error resolving version for plugin 'org.apache.maven.plugins:maven-javadoc-plugin' from the repositories [local (/home/kibi/hack/bsp/puppetdb-builds/tika.git/debian/maven-repo), central (https://repo.maven.apache.org/maven2)]: Plugin not found in any plugin repository -> [Help 1]

so I've added libmaven-javadoc-plugin-java to B-D-I.


Next failure, an unknown package:
| [INFO] Reactor Summary for Apache Tika 1.13:
| [INFO] 
| [INFO] Apache Tika parent ................................. FAILURE [  0.011 s]
| [INFO] Apache Tika core ................................... SKIPPED
| [INFO] Apache Tika parsers ................................ SKIPPED
| [INFO] Apache Tika XMP .................................... SKIPPED
| [INFO] Apache Tika serialization .......................... SKIPPED
| [INFO] Apache Tika batch .................................. SKIPPED
| [INFO] Apache Tika language detection ..................... SKIPPED
| [INFO] Apache Tika translate .............................. SKIPPED
| [INFO] Apache Tika examples ............................... SKIPPED
| [INFO] Apache Tika Java-7 Components ...................... SKIPPED
| [INFO] Apache Tika ........................................ SKIPPED
| [INFO] ------------------------------------------------------------------------
| [INFO] BUILD FAILURE
| [INFO] ------------------------------------------------------------------------
| [INFO] Total time:  1.033 s
| [INFO] Finished at: 2018-12-30T23:56:45Z
| [INFO] ------------------------------------------------------------------------
| [ERROR] Plugin de.thetaphi:forbiddenapis:2.0 or one of its dependencies could not be resolved: Cannot access central (https://repo.maven.apache.org/maven2) in offline mode and the artifact de.thetaphi:forbiddenapis:jar:2.0 has not been downloaded from it before. -> [Help 1]

so I've patched it out, esp. given we have these comments:
|       <!-- The Tika Bundle has no java code of its own, so no need to do -->
|       <!--  any forbidden API checking against it (it gets confused...) -->

and it's marked skip=true, which made it like optional enough…

Hence debian/patches/91-drop-forbiddenapis-dependency.patch


Next issue:
| [INFO] Reactor Summary for Apache Tika 1.13:
| [INFO] 
| [INFO] Apache Tika parent ................................. SUCCESS [  0.004 s]
| [INFO] Apache Tika core ................................... SUCCESS [  4.768 s]
| [INFO] Apache Tika parsers ................................ FAILURE [  0.007 s]
| [INFO] Apache Tika XMP .................................... SKIPPED
| [INFO] Apache Tika serialization .......................... SKIPPED
| [INFO] Apache Tika batch .................................. SKIPPED
| [INFO] Apache Tika language detection ..................... SKIPPED
| [INFO] Apache Tika translate .............................. SKIPPED
| [INFO] Apache Tika examples ............................... SKIPPED
| [INFO] Apache Tika Java-7 Components ...................... SKIPPED
| [INFO] Apache Tika ........................................ SKIPPED
| [INFO] ------------------------------------------------------------------------
| [INFO] BUILD FAILURE
| [INFO] ------------------------------------------------------------------------
| [INFO] Total time:  5.829 s
| [INFO] Finished at: 2018-12-31T00:01:51Z
| [INFO] ------------------------------------------------------------------------
| [ERROR] Error resolving version for plugin 'org.codehaus.gmaven:groovy-maven-plugin' from the repositories [local (/home/kibi/hack/bsp/puppetdb-builds/tika.git/debian/maven-repo), central (https://repo.maven.apache.org/maven2)]: Plugin not found in any plugin repository -> [Help 1]

so I've patched it out, as it appears in a profile with the “testSetup”
id, which I thought might not be entirely needed.

Hence debian/patches/92-drop-groovy-maven-plugin-dependency.patch


Next issue:
| [INFO] Reactor Summary for Apache Tika 1.13:
| [INFO] 
| [INFO] Apache Tika parent ................................. SUCCESS [  0.002 s]
| [INFO] Apache Tika core ................................... SUCCESS [  4.163 s]
| [INFO] Apache Tika parsers ................................ FAILURE [  0.127 s]
| [INFO] Apache Tika XMP .................................... SKIPPED
| [INFO] Apache Tika serialization .......................... SKIPPED
| [INFO] Apache Tika batch .................................. SKIPPED
| [INFO] Apache Tika language detection ..................... SKIPPED
| [INFO] Apache Tika translate .............................. SKIPPED
| [INFO] Apache Tika examples ............................... SKIPPED
| [INFO] Apache Tika Java-7 Components ...................... SKIPPED
| [INFO] Apache Tika ........................................ SKIPPED
| [INFO] ------------------------------------------------------------------------
| [INFO] BUILD FAILURE
| [INFO] ------------------------------------------------------------------------
| [INFO] Total time:  5.366 s
| [INFO] Finished at: 2018-12-31T00:06:02Z
| [INFO] ------------------------------------------------------------------------
| [ERROR] Failed to execute goal on project tika-parsers: Could not resolve dependencies for project org.apache.tika:tika-parsers:jar:1.13: The following artifacts could not be resolved: org.apache.tika:tika-core:jar:tests:debian, org.gagravarr:vorbis-java-tika:jar:debian, com.healthmarketscience.jackcess:jackcess:jar:debian, com.healthmarketscience.jackcess:jackcess-encrypt:jar:debian, net.sourceforge.jmatio:jmatio:jar:debian, org.apache.pdfbox:pdfbox-tools:jar:debian, com.rometools:rome:jar:debian, org.codelibs:jhighlight:jar:debian, com.pff:java-libpst:jar:debian, com.github.junrar:junrar:jar:debian, org.apache.cxf:cxf-rt-rs-client:jar:debian, org.xerial:sqlite-jdbc:jar:debian, org.apache.opennlp:opennlp-tools:jar:debian, org.apache.commons:commons-exec:jar:debian, com.googlecode.json-simple:json-simple:jar:debian, org.json:json:jar:debian, com.google.code.gson:gson:jar:debian, com.github.jai-imageio:jai-imageio-core:jar:debian, edu.ucar:netcdf4:jar:debian, edu.ucar:grib:jar:debian, edu.ucar:cdm:jar:debian, edu.ucar:httpservices:jar:debian, org.apache.commons:commons-csv:jar:debian, org.apache.sis.core:sis-utility:jar:debian, org.apache.sis.storage:sis-netcdf:jar:debian, org.apache.sis.core:sis-metadata:jar:debian, org.opengis:geoapi:jar:debian, org.apache.ctakes:ctakes-core:jar:debian, com.fasterxml.jackson.core:jackson-core:jar:debian: Cannot access central (https://repo.maven.apache.org/maven2) in offline mode and the artifact org.apache.tika:tika-core:jar:tests:debian has not been downloaded from it before. -> [Help 1]

As I've seen other patches marking similar dependencies as optional in
tika-parsers/pom.xml, I've tried to mimick that; unfortunately without
any changes in the output.

Anyway, this is debian/patches/93-mark-parsers-dependencies-as-optional.patch


Some advice on where to go from here would be welcome: does it make
sense to try and get the right hammer to get 1.13 in a buildable state?
Should one try to package 1.20 instead anyway? Please note I haven't even
checked yet what version could work for pantomime-clojure.

(I've cc'ed the Puppet Package Maintainers on this mail for wider reach.)


Cheers,
-- 
Cyril Brulebois (kibi@debian.org)            <https://debamax.com/>
D-I release manager -- Release team member -- Freelance Consultant

[signature.asc (application/pgp-signature, inline)]

Message #46 received at 825501@bugs.debian.org (full text, mbox, reply):

From: Salvatore Bonaccorso <carnil@debian.org>

To: Cyril Brulebois <kibi@debian.org>

Cc: 825501@bugs.debian.org, Faidon Liambotis <paravoid@debian.org>, Moritz Muehlenhoff <jmm@inutil.org>, pantomime-clojure@packages.debian.org, puppetdb@packages.debian.org, pkg-puppet-devel@lists.alioth.debian.org, team@security.debian.org

Subject: Re: Bug#825501: CVE-2016-4434

Date: Mon, 31 Dec 2018 08:04:18 +0100

Hi Cyril,

[I have not looked in detail on your poposal this is mainly focusing on
one item below]

On Mon, Dec 31, 2018 at 01:13:51AM +0100, Cyril Brulebois wrote:
> Heya,
> 
> Not the maintainer either, just joining the fun to see if I can help get
> stuff to move; my main motivation behind this is trying to get the
> puppetdb → pantomime-clojure → tika dependency chain in a suitable state
> for buster (other *-clojure packages need fixing, but FTBFSes have
> patches/MRs now, and uploads should be happening soon enough; but
> there's still comidi-clojure's #889125 to keep me busy anyway…)
> 
> 
> Salvatore Bonaccorso <carnil@debian.org> (2018-01-18):
> > The issue is claimed to be fixed in upstream 1.13 (and as Moritz
> > pointed out a test was added. Comparing commits between 1.12 and 1.13
> > I was unable to isolate the relevant commit(s), but there are some
> > touching the code for "OOXML files and XMP in PDF and other file
> > formats".
> 
> Right, I haven't been able to pinpoint the exact changes, but those
> could be “hidden” in things like pdfbox version bumps, etc. Even if a
> specific fix for 1.5 would be identified, it seems hard to get it to
> build; I've tried that just to see what was feasible, and it doesn't
> look good anyway:
> 
>   https://bugs.debian.org/850798#12
> 
> Not being a Java expert, I've then moved to giving the latest upstream
> release (1.20) a shot, but there were too many red things, so I've tried
> to aim at 1.13 “only”, to get this CVE addressed.

I think though that would not be sensible in the following way: the
mentioned CVE is not the only one affecting, currently there are the
following open (some have associated Debian BTS bugreports, other have
not yet):

https://security-tracker.debian.org/tracker/source-package/tika

Furthermore if we only update to 1.13 there are likely some of the
currently <not-affected> CVEs which will make tika affected, because
the issue was introduced post 1.5. One example of this is for instance
CVE-2016-6809, where the Matlab file parser was only introduced in 1.6
and the issue fixed in 1.14. Or CVE-2018-17197 which affects 1.8 to
1.19.1. CVE-2018-1338, which was introduced in 1.7. CVE-2018-1335,
present from 1.7 to 1.17.

There might be others, so I think the new upstream version fixing all
known current CVE is actually needed.

Regards,
Salvatore

Message #51 received at 825501@bugs.debian.org (full text, mbox, reply):

From: Moritz Mühlenhoff <jmm@inutil.org>

To: Salvatore Bonaccorso <carnil@debian.org>

Cc: Cyril Brulebois <kibi@debian.org>, 825501@bugs.debian.org, Faidon Liambotis <paravoid@debian.org>, Moritz Muehlenhoff <jmm@inutil.org>, pantomime-clojure@packages.debian.org, puppetdb@packages.debian.org, pkg-puppet-devel@lists.alioth.debian.org, team@security.debian.org

Subject: Re: Bug#825501: CVE-2016-4434

Date: Mon, 31 Dec 2018 12:55:28 +0100

On Mon, Dec 31, 2018 at 08:04:18AM +0100, Salvatore Bonaccorso wrote:
> Hi Cyril,
> 
> 
> https://security-tracker.debian.org/tracker/source-package/tika
> 
> Furthermore if we only update to 1.13 there are likely some of the
> currently <not-affected> CVEs which will make tika affected, because
> the issue was introduced post 1.5. One example of this is for instance
> CVE-2016-6809, where the Matlab file parser was only introduced in 1.6
> and the issue fixed in 1.14. Or CVE-2018-17197 which affects 1.8 to
> 1.19.1. CVE-2018-1338, which was introduced in 1.7. CVE-2018-1335,
> present from 1.7 to 1.17.
> 
> There might be others, so I think the new upstream version fixing all
> known current CVE is actually needed.

Agreed. Also 1.13 was released in May 2016, so by the time buster gets
released it would be ~ 5 years old.

Cheers,
        Moritz

Message #56 received at 825501@bugs.debian.org (full text, mbox, reply):

From: Cyril Brulebois <kibi@debian.org>

To: Moritz Mühlenhoff <jmm@inutil.org>

Cc: Salvatore Bonaccorso <carnil@debian.org>, 825501@bugs.debian.org, Faidon Liambotis <paravoid@debian.org>, pantomime-clojure@packages.debian.org, puppetdb@packages.debian.org, pkg-puppet-devel@lists.alioth.debian.org, team@security.debian.org

Subject: Re: Bug#825501: CVE-2016-4434

Date: Thu, 3 Jan 2019 16:24:39 +0100

[Message part 1 (text/plain, inline)]

Hi everyone,

Security team: thanks for your input.
PuppetDB/Clojure maintainers: draft plan in this mail, feedback welcome.

Moritz Mühlenhoff <jmm@inutil.org> (2018-12-31):
> On Mon, Dec 31, 2018 at 08:04:18AM +0100, Salvatore Bonaccorso wrote:
> > Furthermore if we only update to 1.13 there are likely some of the
> > currently <not-affected> CVEs which will make tika affected, because
> > the issue was introduced post 1.5. One example of this is for
> > instance CVE-2016-6809, where the Matlab file parser was only
> > introduced in 1.6 and the issue fixed in 1.14. Or CVE-2018-17197
> > which affects 1.8 to 1.19.1. CVE-2018-1338, which was introduced in
> > 1.7. CVE-2018-1335, present from 1.7 to 1.17.
> > 
> > There might be others, so I think the new upstream version fixing all
> > known current CVE is actually needed.

There goes my vague idea of trying to handle a small(er) diff; of course
that makes a lot of sense…

> Agreed. Also 1.13 was released in May 2016, so by the time buster gets
> released it would be ~ 5 years old.

s/buster/bookworm/ I suppose but I see your point.


So, looking at current upstreams:
 - 4.4.x branch of puppetdb seems a bit inactive (since 2018-02), even
   if it has a few commits on top of the version currently sitting in
   unstable; it still documents a dependency on pantomime(-clojure)
   2.1.0, which itself documents a dependency on tika 1.5.
 - pantomime upstream is at 2.10.0, released early 2018; it documents
   a dependency on tika 1.19.1
 - tika upstream is at 1.20

Keeping in mind my knowledge of Clojure, Java, and their respective
ecosystems is rather limited, I'd like to share some initial hunches
anyway.

puppetdb seems to only reference pantomime once (outside top-level
project.clj), in src/puppetlabs/puppetdb/middleware.clj:
| […] :require […] [pantomime.media :as media]
                    ^^^^^^^^^^^^^^^^^^^^^^^^^
| […]
| (defn verify-content-type
|   "Verification for the specified list of content-types."
|   [app content-types]
|   {:pre [(coll? content-types)
|          (every? string? content-types)]}
|   (fn [{:keys [headers] :as req}]
|     (if (= (:request-method req) :post)
|       (let [content-type (headers "content-type")
|             mediatype (if (nil? content-type) nil
|                           (str (media/base-type content-type)))]
                                  ^^^^^^^^^^^^^^^
|         (if (or (nil? mediatype) (some #{mediatype} content-types))
|           (app req)
|           (http/error-response (tru "content type {0} not supported" mediatype)
|                                http/status-unsupported-type)))
|       (app req))))

and hopefully that didn't change too much between pantomime 2.1.0 and
2.10.0 as that is likely the crux of pantomime?

With that in mind, but without having checked code changes in pantomime,
I hope it should be possible to bump the pantomime dependency from 2.1.0
to 2.10.0 on the puppetdb side (better catch up with upstream?).


On the pantomime side, it seems it should work fine with tika 1.19.1, as
documented in dependencies in the master branch (3 commits on top of the
v2.10.0 tag). That should help us consider tika 1.20, as possible
breaking changes should be manageable between those two versions?

Changes in dependencies in pantomime (project.clj) seem rather limited
here's an excerpt between the v2.1.0 and v2.10.0 tags (excluding changes
in :profiles and :aliases):
| -  :dependencies [[org.clojure/clojure "1.5.1"]
| -                 [org.apache.tika/tika-core "1.5"]]
| +  :dependencies [[org.clojure/clojure "1.9.0"]
| +                 [org.apache.tika/tika-parsers "1.17"]
| +                 [org.apache.commons/commons-compress "1.15"]]

This should be OK? We have clojure 1.9 in unstable, tika-parsers even
becomes 1.19.1 (as mentioned above, in the master branch), and we have
those versions for commons-compress:
| libcommons-compress-java | 1.13-1        | stable             | source, all
| libcommons-compress-java | 1.18-1        | testing            | source, all
| libcommons-compress-java | 1.18-1        | unstable           | source, all


So maybe a way forward would be:
 - keep puppetdb at the current version (or maybe taking an upstream
   snapshot or suggesting a new upstream release with those few commits),
   and leave switching to 5.x or 6.x branches for later
 - bump pantomime-clojure to latest upstream (2.10)
 - bump tika to latest upstream (1.20)

I doubt I would be able to deal with tika 1.20 alone (see the issues I
had to deal with when trying my luck with 1.15 in my previous mail),
even if we were to try and trim it down to the bare set of features that
pantomime needs.


Thoughts about the plan? If that doesn't look too crazy, anyone with
some availability to help me get tika 1.20 in shape?


Cheers,
-- 
Cyril Brulebois (kibi@debian.org)            <https://debamax.com/>
D-I release manager -- Release team member -- Freelance Consultant

[signature.asc (application/pgp-signature, inline)]

Message #63 received at 825501-submitter@bugs.debian.org (full text, mbox, reply):

From: Emmanuel Bourg <ebourg@apache.org>

To: 825501-submitter@bugs.debian.org

Subject: Bug #825501 in tika marked as pending

Date: Sat, 19 Jan 2019 23:10:55 +0000

Control: tag -1 pending

Hello,

Bug #825501 in tika reported by you has been fixed in the
Git repository and is awaiting an upload. You can see the commit
message below and you can check the diff of the fix at:

https://salsa.debian.org/java-team/tika/commit/da8f564f02250bc904da8c03c45d2fa4c8f9c147

------------------------------------------------------------------------
The new release fixes CVE-2016-4434 (Closes: #825501)
------------------------------------------------------------------------

(this message was generated automatically)
-- 
Greetings

https://bugs.debian.org/825501

Message #70 received at 825501-close@bugs.debian.org (full text, mbox, reply):

From: Emmanuel Bourg <ebourg@apache.org>

To: 825501-close@bugs.debian.org

Subject: Bug#825501: fixed in tika 1.18-1

Date: Sat, 19 Jan 2019 23:35:47 +0000

Source: tika
Source-Version: 1.18-1

We believe that the bug you reported is fixed in the latest version of
tika, which is due to be installed in the Debian FTP archive.

A summary of the changes between this version and the previous one is
attached.

Thank you for reporting the bug, which will now be closed.  If you
have further comments please address them to 825501@bugs.debian.org,
and the maintainer will reopen the bug report if appropriate.

Debian distribution maintenance software
pp.
Emmanuel Bourg <ebourg@apache.org> (supplier of updated tika package)

(This message was generated automatically at their request; if you
believe that there is a problem with it please contact the archive
administrators by mailing ftpmaster@ftp-master.debian.org)


-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA512

Format: 1.8
Date: Sun, 20 Jan 2019 00:08:04 +0100
Source: tika
Binary: libtika-java
Architecture: source
Version: 1.18-1
Distribution: unstable
Urgency: medium
Maintainer: Debian Java Maintainers <pkg-java-maintainers@lists.alioth.debian.org>
Changed-By: Emmanuel Bourg <ebourg@apache.org>
Description:
 libtika-java - Apache Tika - content analysis toolkit
Closes: 825501 900000
Changes:
 tika (1.18-1) unstable; urgency=medium
 .
   * New upstream release
     - Fixes CVE-2016-4434: XML External Entity vulnerability (Closes: #825501)
     - Fixes CVE-2018-1339: Infinite loop in the CHM parser (Closes: #900000)
     - Refreshed the patches
     - Ignore the new dl, eval, langdetect and nlp modules
     - New dependencies on libcommons-exec-java, libjackson2-annotations-java,
       libjackson2-core-java, libjackson2-databind-java, libhttpmime-java,
       libjsoup-java, libuima-core-java, libandroid-json-org-java
       and libjson-simple-java
     - Depend on libpdfbox2-java instead of libpdfbox-java
     - Depend on librome-java (>= 1.6)
     - Depend on libapache-mime4j-java (>= 0.8.1)
     - Depend on libapache-poi-java (>= 3.17)
     - Ignore the new parsers with missing dependencies
   * Enabled the mp4 parser
   * Fixed the build failure with Java 11
Checksums-Sha1:
 575e3b998aa917b405dc0d860d9254055ed35a9b 2668 tika_1.18-1.dsc
 5e3296e786017f6c48e5b037119e67def2b7b108 2460536 tika_1.18.orig.tar.xz
 7fa2e01a7b678acd0028c6f39ca57fc6ac76366c 7320 tika_1.18-1.debian.tar.xz
 1921c9105f2727a8094a9d2d906b0334f3598307 16596 tika_1.18-1_source.buildinfo
Checksums-Sha256:
 64eaa3dedec4a74f16b9d4b753aff226f671fa3399817e137dceb74e1828b84b 2668 tika_1.18-1.dsc
 b107c1519f69cc041185984a765cc210d84063a77376ff7d726b504284be24d7 2460536 tika_1.18.orig.tar.xz
 ef44ba42e64edd844bc4c410039278a2e49e904026d979f23a07e9e9f0c5a676 7320 tika_1.18-1.debian.tar.xz
 b4c7d09d00afdb25f263a7f9fa78e4a911d2009e4bd79a5f1bd033615ce81284 16596 tika_1.18-1_source.buildinfo
Files:
 76ece2170ee72d09da4f0a28e64b7156 2668 java optional tika_1.18-1.dsc
 8a059ae0583ee590437b70cbddcb1473 2460536 java optional tika_1.18.orig.tar.xz
 36aadb1079dfa3e34c4bb767b39eed56 7320 java optional tika_1.18-1.debian.tar.xz
 3fd0aaefd99a576a824db645f2da4892 16596 java optional tika_1.18-1_source.buildinfo

-----BEGIN PGP SIGNATURE-----

iQJGBAEBCgAwFiEEuM5N4hCA3PkD4WxA9RPEGeS50KwFAlxDrmESHGVib3VyZ0Bh
cGFjaGUub3JnAAoJEPUTxBnkudCs2MYQAKv03Susi/IF0UkPZ6noCfvtzDewzewe
dNG7p6izgZd01FNPc+Tv4glLHbsN0LpU/OGVZgf4UJFld6pwmVPQr0msy171tjmr
A+UkQKeARq4zMPAAbs0tbtENEkTo2tNK6KfYil/Zu6fWkC4fy0aDgCwgubux0xoW
p1wrSxQ4IiSioNkI0abE/P5QguwSCiuX89rj0Y+VGcKuIrTm8tu2l33bvpgHDHXJ
m2iyB7EtvwlaEdSjkH8sPusAuutb/1d4gofCNwV5OoVsKBKv8RN73yxExeDf7K9e
mdFDMh4LnLx+7tCRwH1kEN1vwvKviuv60SLKcnfp9JyQ8g76fZn0WQCpo6HGMrij
lIgXvDE94O2+R09CqpJJVl7fe6GjVJo7DUnHOLnqHX7wx+qtuxoSGavQptSoOvGf
/6ITBu6TljTutEWNENcVYqZ1vEoDiODa2cmbYGkxHpjrLl1esnfQdrgGRh/1vKU1
20AvYyFwHDdfqC6T3Cg+cMLDZtnsAiyPkrkrjmCFbMF4u1LJYZ8Luon8uC+vm0Db
a03aSS5Y4FY7xDlNPa5iY9ZZWpiAuadezw11riUIqeXPIBur0xJQvLuW4/cWMGAt
mGRkHXeN44M1F+Qy4EPhX/t13GPkJ4tF4eiEoNWd0LTbrDPgBP6tXdRoOoY7xc4D
D1rUqW6jxCq8
=l2eo
-----END PGP SIGNATURE-----

Send a report that this bug log contains spam.