diffoscope: CVE-2017-0359: writes to arbitrary locations on disk based on the contents of an untrusted archive

Debian Bug report logs - #854723
diffoscope: CVE-2017-0359: writes to arbitrary locations on disk based on the contents of an untrusted archive

Toggle useless messages

Message #5 received at submit@bugs.debian.org (full text, mbox, reply):

From: Ximin Luo <infinity0@debian.org>

To: Debian Bug Tracking System <submit@bugs.debian.org>

Subject: diffoscope writes to arbitrary locations on disk based on the contents of an untrusted archive

Date: Thu, 09 Feb 2017 22:14:12 +0100

Package: diffoscope
Version: 67
Severity: grave
Tags: patch security
Justification: user security hole

Dear Maintainer,

5fdfe91e71f1c520d902350b18f793b8c69d9118 introduced a security hole where
diffoscope may write to arbitrary locations on disk depending on the contents
of an untrusted archive. For example, comparing the following two files:

https://bugs.debian.org/cgi-bin/bugreport.cgi?att=1;bug=843811;filename=libBrokenLocale.a.0;msg=5
https://bugs.debian.org/cgi-bin/bugreport.cgi?att=2;bug=843811;filename=libBrokenLocale.a.1;msg=5

Traceback (most recent call last):
  File "/home/infinity0/xx/diffoscope/diffoscope/main.py", line 281, in main
    sys.exit(run_diffoscope(parsed_args))
[..]
  File "/home/infinity0/xx/diffoscope/diffoscope/comparators/utils/libarchive.py", line 174, in extract
    self.ensure_unpacked()
  File "/home/infinity0/xx/diffoscope/diffoscope/comparators/utils/libarchive.py", line 219, in ensure_unpacked
    os.makedirs(os.path.dirname(dst), exist_ok=True)
  File "/usr/lib/python3.5/os.py", line 241, in makedirs
    mkdir(name, mode)
PermissionError: [Errno 13] Permission denied: '/SYM64'

Note that this could easily have been something like /home/infinity0/.profile.

I have pushed a nearly-complete fix to git (after version 75 was just released)
which prevents the writes. However reads are still done using the uncleaned
names, but this is a much less severe issue. So, if I don't supply a fix for
the second lesser issue soon, the existing fix should be released ASAP.

X

-- System Information:
Debian Release: 9.0
  APT prefers testing
  APT policy: (990, 'testing'), (500, 'unstable-debug'), (500, 'testing-debug'), (300, 'unstable'), (200, 'experimental'), (1, 'experimental-debug')
Architecture: amd64 (x86_64)
Foreign Architectures: i386

Kernel: Linux 4.8.0-2-amd64 (SMP w/4 CPU cores)
Locale: LANG=en_GB.utf8, LC_CTYPE=en_GB.utf8 (charmap=UTF-8)
Shell: /bin/sh linked to /bin/dash
Init: systemd (via /run/systemd/system)

Versions of packages diffoscope depends on:
ii  python3-libarchive-c   2.1-3.1
ii  python3-magic          1:5.29-3
ii  python3-pkg-resources  33.1.1-1
pn  python3:any            <none>

Versions of packages diffoscope recommends:
ii  acl                        2.2.52-3
ii  apktool                    2.2.1+dfsg-2
ii  binutils-multiarch         2.27.90.20170124-2
ii  bzip2                      1.0.6-8.1
ii  caca-utils                 0.99.beta19-2+b1
ii  colord                     1.3.3-2
ii  cpio                       2.11+dfsg-6
ii  default-jdk [java-sdk]     2:1.8-58
ii  default-jdk-headless       2:1.8-58
ii  enjarify                   1:1.0.3-3
ii  fontforge-extras           0.3-4
ii  fp-utils                   3.0.0+dfsg-10
ii  fp-utils-3.0.0 [fp-utils]  3.0.0+dfsg-10
ii  genisoimage                9:1.1.11-3
ii  gettext                    0.19.8.1-2
ii  ghc                        8.0.1-17
ii  ghostscript                9.20~dfsg-2
ii  gnupg                      2.1.18-3
ii  jsbeautifier               1.6.4-6
ii  llvm                       1:3.8-34+b1
ii  mono-utils                 4.6.2.7+dfsg-1
ii  openjdk-8-jdk [java-sdk]   8u121-b13-2
ii  openssh-client             1:7.4p1-6
ii  pdftk                      2.02-4+b1
ii  poppler-utils              0.48.0-2
ii  python3-argcomplete        1.8.1-1
ii  python3-debian             0.1.30
ii  python3-guestfs            1:1.34.3-7
ii  python3-progressbar        2.3-4
ii  python3-rpm                4.12.0.2+dfsg1-1
ii  python3-tlsh               3.4.4+20151206-1+b1
ii  rpm2cpio                   4.12.0.2+dfsg1-1
ii  sng                        1.1.0-1+b1
ii  sqlite3                    3.16.2-2
ii  squashfs-tools             1:4.3-3
ii  unzip                      6.0-21
ii  vim-common                 2:8.0.0197-1
ii  xxd                        2:8.0.0197-1
ii  xz-utils                   5.2.2-1.2

Versions of packages diffoscope suggests:
ii  libjs-jquery  3.1.1-2

-- no debconf information

Message #10 received at 854723@bugs.debian.org (full text, mbox, reply):

From: Chris Lamb <lamby@debian.org>

To: 854723@bugs.debian.org

Subject: Re: Bug#854723: diffoscope writes to arbitrary locations on disk based on the contents of an untrusted archive

Date: Fri, 10 Feb 2017 11:07:22 +1300

tags 854723 + pending
thanks

> diffoscope may write to arbitrary locations on disk depending on the contents
> of an untrusted archive

We can actually avoid all edge-cases of sanitisation by simply not using
the supplied filename and maintaining our own mapping.

Given this is both safer (and has far less code) I've gone ahead and committed
that here:

  https://anonscm.debian.org/git/reproducible/diffoscope.git/commit/?id=632a40828a54b399787c25e7fa243f732aef7e05


Regards,

-- 
      ,''`.
     : :'  :     Chris Lamb
     `. `'`      lamby@debian.org / chris-lamb.co.uk
       `-

Message #17 received at 854723@bugs.debian.org (full text, mbox, reply):

From: Ximin Luo <infinity0@debian.org>

To: Chris Lamb <lamby@debian.org>, 854723@bugs.debian.org

Subject: Re: Bug#854723: diffoscope writes to arbitrary locations on disk based on the contents of an untrusted archive

Date: Thu, 09 Feb 2017 23:10:00 +0000

Chris Lamb:
> tags 854723 + pending
> thanks
> 
>> diffoscope may write to arbitrary locations on disk depending on the contents
>> of an untrusted archive
> 
> We can actually avoid all edge-cases of sanitisation by simply not using
> the supplied filename and maintaining our own mapping.
> 
> Given this is both safer (and has far less code) I've gone ahead and committed
> that here:
> 
>   https://anonscm.debian.org/git/reproducible/diffoscope.git/commit/?id=632a40828a54b399787c25e7fa243f732aef7e05
> 

Thanks, this is better.

However this particular scheme might not work so well with large archives with lots and lots of members (>many thousands), depending on what filesystem the tempdir contained in. I'd suggest to use names like $x/$y where $x = idx // 4096, $y = idx % 4096.

X

-- 
GPG: ed25519/56034877E1F87C35
GPG: rsa4096/1318EFAC5FBBDBCE
https://github.com/infinity0/pubkeys.git

Message #22 received at 854723@bugs.debian.org (full text, mbox, reply):

From: Ximin Luo <infinity0@debian.org>

To: Chris Lamb <lamby@debian.org>, 854723@bugs.debian.org

Subject: Re: Bug#854723: diffoscope writes to arbitrary locations on disk based on the contents of an untrusted archive

Date: Thu, 09 Feb 2017 23:14:00 +0000

Ximin Luo:
> Chris Lamb:
>> tags 854723 + pending
>> thanks
>>
>>> diffoscope may write to arbitrary locations on disk depending on the contents
>>> of an untrusted archive
>>
>> We can actually avoid all edge-cases of sanitisation by simply not using
>> the supplied filename and maintaining our own mapping.
>>
>> Given this is both safer (and has far less code) I've gone ahead and committed
>> that here:
>>
>>   https://anonscm.debian.org/git/reproducible/diffoscope.git/commit/?id=632a40828a54b399787c25e7fa243f732aef7e05
>>
> 
> Thanks, this is better.
> 
> However this particular scheme might not work so well with large archives with lots and lots of members (>many thousands), depending on what filesystem the tempdir contained in. I'd suggest to use names like $x/$y where $x = idx // 4096, $y = idx % 4096.
> 

Also, are you sure this doesn't interfere with the detection of order-only differences, or the ability to match up similar-member-names?

X

-- 
GPG: ed25519/56034877E1F87C35
GPG: rsa4096/1318EFAC5FBBDBCE
https://github.com/infinity0/pubkeys.git

Message #27 received at 854723@bugs.debian.org (full text, mbox, reply):

From: Chris Lamb <lamby@debian.org>

To: Ximin Luo <infinity0@debian.org>, 854723@bugs.debian.org

Subject: Re: Bug#854723: diffoscope writes to arbitrary locations on disk based on the contents of an untrusted archive

Date: Fri, 10 Feb 2017 12:20:22 +1300

Ximin Luo wrote:

> this particular scheme might not work so well with large archives
> with lots and lots of members

Mm although unlikely to be a serious problem as we aren't iterating
over the directory. 

> Also, are you sure this doesn't interfere with the detection of
> order-only differences, or the ability to match up
> similar-member-names?

We still use the archive's member name throughout diffoscope; the
unpacked path shouldn't leak outside of that comparator. Also, the
tests pass… *g*


Regards,

-- 
      ,''`.
     : :'  :     Chris Lamb
     `. `'`      lamby@debian.org / chris-lamb.co.uk
       `-

Message #32 received at 854723@bugs.debian.org (full text, mbox, reply):

From: Moritz Mühlenhoff <jmm@inutil.org>

To: Chris Lamb <lamby@debian.org>

Cc: 854723@bugs.debian.org

Subject: Re: Bug#854723: diffoscope writes to arbitrary locations on disk based on the contents of an untrusted archive

Date: Sat, 11 Feb 2017 12:45:44 +0100

On Fri, Feb 10, 2017 at 11:07:22AM +1300, Chris Lamb wrote:
> tags 854723 + pending
> thanks
> 
> > diffoscope may write to arbitrary locations on disk depending on the contents
> > of an untrusted archive

Please use CVE-2017-0359

Cheers,
        Moritz

Message #39 received at 854723-close@bugs.debian.org (full text, mbox, reply):

From: Mattia Rizzolo <mattia@debian.org>

To: 854723-close@bugs.debian.org

Subject: Bug#854723: fixed in diffoscope 77

Date: Mon, 13 Feb 2017 16:33:35 +0000

Source: diffoscope
Source-Version: 77

We believe that the bug you reported is fixed in the latest version of
diffoscope, which is due to be installed in the Debian FTP archive.

A summary of the changes between this version and the previous one is
attached.

Thank you for reporting the bug, which will now be closed.  If you
have further comments please address them to 854723@bugs.debian.org,
and the maintainer will reopen the bug report if appropriate.

Debian distribution maintenance software
pp.
Mattia Rizzolo <mattia@debian.org> (supplier of updated diffoscope package)

(This message was generated automatically at their request; if you
believe that there is a problem with it please contact the archive
administrators by mailing ftpmaster@ftp-master.debian.org)


-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA512

Format: 1.8
Date: Mon, 13 Feb 2017 16:25:02 +0100
Source: diffoscope
Binary: diffoscope
Architecture: source
Version: 77
Distribution: unstable
Urgency: medium
Maintainer: Reproducible builds folks <reproducible-builds@lists.alioth.debian.org>
Changed-By: Mattia Rizzolo <mattia@debian.org>
Description:
 diffoscope - in-depth comparison of files, archives, and directories
Closes: 854723 854745 854783
Changes:
 diffoscope (77) unstable; urgency=medium
 .
   [ Chris Lamb ]
   * tests/comparators/utils:
     + Correct logic of module_exists, ensuring we correctly skip in case of
       modules containing a dot in their name.  Closes: #854745
   * comparators/utils/libarchive:
     + No need to track archive directory locations.
   * Add --exclude option.  Closes: #854783
   * Add PyPI badge to README.rst.
   * Update .travis.yml from http://travis.debian.net.
 .
   [ Mattia Rizzolo ]
   * Add CVE reference to the changelog of v76.
   * Add my key to debian/upstream/signing-key.asc.
 .
   [ Ximin Luo ]
   * comparators/utils/libarchive:
     + When extracting archives, try to keep directory sizes small.
 .
 diffoscope (76) unstable; urgency=medium
 .
   [ Chris Lamb ]
   * Extract archive members using an auto-incrementing integer, avoiding the
     need to sanitise filenames and avoiding writes to arbitrary locations.
     (Closes: #854723 - CVE-2017-0359)
 .
   [ Ximin Luo ]
   * Simplify call to subprocess.Popen
Checksums-Sha1:
 88ab09a8ecf57244ee21bd5c2f19a39b0f1c5062 2972 diffoscope_77.dsc
 b0c72453546afd30364c36aa2a86355d712ad55f 349436 diffoscope_77.tar.xz
 619ab27596d84ee53ebe2e8924c3ad662e1deea8 16138 diffoscope_77_amd64.buildinfo
Checksums-Sha256:
 964f94d42f970ba32d73770e9d0c151fe149633cfb9054333bafe7df3f0271ee 2972 diffoscope_77.dsc
 c9adeb0bfb0c92a3501df04b6ea4300c3896f15a9008803e4e12c1f312528499 349436 diffoscope_77.tar.xz
 3e10be4a12c432443536830551d536e73dbb4de8f1374cf7ec6c5a033104a793 16138 diffoscope_77_amd64.buildinfo
Files:
 853b57d21d18fafb72701114b189a315 2972 devel optional diffoscope_77.dsc
 13f5d4623bfd49a3787a3d03c9f4f076 349436 devel optional diffoscope_77.tar.xz
 dc24dbcee5c0028bc590f98a97504d14 16138 devel optional diffoscope_77_amd64.buildinfo

-----BEGIN PGP SIGNATURE-----

iQIzBAEBCgAdFiEEi3hoeGwz5cZMTQpICBa54Yx2K60FAlih0y4ACgkQCBa54Yx2
K61wdBAAlmhCj1QjMML0C7SgOprVNYmKhIwo+MzW0DHD4Mn3FaTU6v2QdtCO3Sev
/kFVQDf2Tv+qAZhJ4h67a0hF710myt5faZVkMCqtCUsiVaNs/Q3py5wZAkJgpXMO
C6gEP1rCQctDGKricbLasYhv71iUU6MNQym4G1sohJxi74fQ+CM1+STvCV3MIfIm
SivzqSLja/lnyuRdHFjSIhPwR0tSoePmwQaNR7omd4EhC8seFS7vVcnRqO9bwSWz
khZpLPKvSEdLfzLDjNGo4trFi6EEyMkdOh+Oqy2RQNYmVr1YQYkmNTn284FUQduv
YxTG2/iFbdCWnYp+94RZI22TTJwBUsiAssAGFJD3e89ID6aPrjX/nq0ojHvz8mLF
NWt/KySh4tkFzMCRP3d61hVm0QASPp1oXWHhxSBkXt1DPOQ6ujtsbme62bLZAh0r
xco5c+JbRPMcjlNd4dZMy9qG0kjs+pRP6gF6qMbh8DXyPRAJrDQ8Y13U0dNKYzR+
ND9+dsVuFn/crrvjJqkowr4PqiQNbAa1zGQJtsxVXpw5Y8HtRPHNSWR976orR2Cp
QmqzEXCy9L+jN9oU9l/dKtRSBqvDckUZVfO6g177uasPAMdC3h4V/2WsFqXylDP6
AQxDrS/qia2YczKWytJ0LgluQsvmQn5j+93IhpyJNCJPaTwa5U0=
=QV/2
-----END PGP SIGNATURE-----

Send a report that this bug log contains spam.