CVE-2023-2088 / OSSA-2023-003: Unauthorized volume access through deleted volume attachments

Debian Bug report logs - #1035932
CVE-2023-2088 / OSSA-2023-003: Unauthorized volume access through deleted volume attachments

Reply or subscribe to this bug.

Toggle useless messages

Message #5 received at submit@bugs.debian.org (full text, mbox, reply):

From: Thomas Goirand <zigo@debian.org>

To: Debian Bug Tracking System <submit@bugs.debian.org>

Subject: CVE-2023-2088 / OSSA-2023-003: Unauthorized volume access through deleted volume attachments

Date: Thu, 11 May 2023 13:37:25 +0200

Source: python-os-brick
Version: 6.2.0-1
Severity: grave

============================================================================
OSSA-2023-003: Unauthorized volume access through deleted volume attachments
============================================================================

:Date: May 10, 2023
:CVE: CVE-2023-2088


Affects
~~~~~~~
- Cinder: <20.2.1, >=21.0.0 <21.2.1, ==22.0.0
- Glance_store: <3.0.1, >=4.0.0 <4.1.1, >=4.2.0 <4.3.1
- Nova: <25.1.2, >=26.0.0 <26.1.2, ==27.0.0
- Os-brick: <5.2.3, >=6.0.0 <6.1.1, >=6.2.0 <6.2.2


Description
~~~~~~~~~~~
An unauthorized access to a volume could occur when an iSCSI or FC
connection from a host is severed due to a volume being unmapped on
the storage system and the device is later reused for another volume
on the same host.

**Scope:** Only deployments with iSCSI or FC volumes are affected.
However, the fix for this issue includes a configuration change in
Nova and Cinder that may impact you on your next upgrade regardless
of what backend storage technology you use. See the *Configuration
change* section below, and item 4(B) in the *Patches and Associated
Deployment Changes* for details.

This data leak can be triggered by two different situations.

**Accidental case:** If there is a problem with network connectivity
during a normal detach operation, OpenStack may fail to clean the
situation up properly. Instead of force-detaching the compute node
device, Nova ignores the error, assuming the instance has already
been deleted. Due to this incomplete operation OpenStack may end up
selecting the wrong multipath device when connecting another volume
to an instance.

**Intentional case:** A regular user can create an instance with a
volume, and then delete the volume attachment directly in Cinder,
which neglects to notify Nova. The compute node SCSI plumbing (over
iSCSI/FC) will continue trying to connect to the original
host/port/LUN, not knowing the attachment has been deleted. If a
subsequent volume attachment re-uses the host/port/LUN for a
different instance and volume, the original instance will gain
access to it once the SCSI plumbing reconnects.

Configuration Change
--------------------
To prevent the intentional case, the Block Storage API provided by
Cinder must only accept attachment delete requests from Nova for
instance-attached volumes. A complicating factor is that Nova
deletes an attachment by making a call to the Block Storage API on
behalf of the user (that is, by passing the user's token), which
makes the request indistinguishable from the user making this
request directly. The solution is to have Nova include a service
token along with the user's token so that Cinder can determine that
the detach request is coming from Nova. The ability for Nova to pass
a service token has been supported since Ocata, but has not been
required until now. Thus, deployments that are not currently sending
service user credentials from Nova will need to apply the relevant
code changes and also make configuration changes to solve the
problem.

Patches and Associated Deployment Changes
-----------------------------------------
Given the above analysis, a thorough fix must include the following
elements:

1. The os-brick library must implement the ``force`` option for
   fibre channel, which which has only been available for iSCSI
   until now (covered by the linked patches).

2. Nova must call os-brick with the ``force`` option when
   disconnecting volumes from deleted instances (covered by the
   linked patches).

3. In deployments where Glance uses the cinder glance_store driver,
   glance must call os-brick with the ``force`` option when
   disconnecting volumes (covered by the linked patches).

4. Cinder must distinguish between safe and unsafe attachment delete
   requests and reject the unsafe ones. This part of the fix has two
   components:

   a. The Block Storage API will return a 409 (Conflict) for a
      request to delete an attachment if there is an instance
      currently using the attachment, **unless** the request is
      being made by a service (for example, Nova) on behalf of a
      user (covered by the linked patches).

   b. In order to recognize that a request is being made by a
      service on behalf of a user, Nova must be configured to send a
      service token along with the user token. If this configuration
      change is not made, the cinder change will reject **any**
      request to delete an attachment associated with a volume that
      is attached to an instance. Nova must be configured to send a
      service token to Cinder, and Cinder must be configured to
      accept service tokens. This is described in the following
      document and **IS NOT AUTOMATICALLY APPLIED BY THE LINKED
      PATCHES:** (Using service tokens to prevent long-running job
      failures)
      https://docs.openstack.org/cinder/latest/configuration/block-storage/service-token.html
      The Nova patch mentioned in step 2 includes a similar document
      more focused on Nova:
      doc/source/admin/configuration/service-user-token.rst

5. The cinder glance_store driver does not attach volumes to
   instances; instead, it attaches volumes directly to the Glance
   node. Thus, the Cinder change in step 4 will recognize an
   attachment-delete request coming from Glance as safe and allow
   it. (Of course, we expect that you will have applied the patches
   in steps 1 and 3 to your Glance nodes.)




Errata
~~~~~~
An additional nova patch is required to fix a minor regression in periodic tasks and some nova-manage actions (errata 1). Also a patch to tempest is needed to account for behavior changes with fixes in place (errata 2).



Patches
~~~~~~~
- https://review.opendev.org/882836 (2023.1/antelope cinder)
- https://review.opendev.org/882851 (2023.1/antelope glance_store)
- https://review.opendev.org/882858 (2023.1/antelope nova)
- https://review.opendev.org/882859 (2023.1/antelope nova errata 1)
- https://review.opendev.org/882843 (2023.1/antelope os-brick)
- https://review.opendev.org/882835 (2023.2/bobcat cinder)
- https://review.opendev.org/882834 (2023.2/bobcat glance_store)
- https://review.opendev.org/882847 (2023.2/bobcat nova)
- https://review.opendev.org/882852 (2023.2/bobcat nova errata 1)
- https://review.opendev.org/882840 (2023.2/bobcat os-brick)
- https://review.opendev.org/882876 (2023.2/bobcat tempest errata 2)
- https://review.opendev.org/882869 (Wallaby nova)
- https://review.opendev.org/882870 (Wallaby nova errata 1)
- https://review.opendev.org/882839 (Xena cinder)
- https://review.opendev.org/882855 (Xena glance_store)
- https://review.opendev.org/882867 (Xena nova)
- https://review.opendev.org/882868 (Xena nova errata 1)
- https://review.opendev.org/882848 (Xena os-brick)
- https://review.opendev.org/882838 (Yoga cinder)
- https://review.opendev.org/882854 (Yoga glance_store)
- https://review.opendev.org/882863 (Yoga nova)
- https://review.opendev.org/882864 (Yoga nova errata 1)
- https://review.opendev.org/882846 (Yoga os-brick)
- https://review.opendev.org/882837 (Zed cinder)
- https://review.opendev.org/882853 (Zed glance_store)
- https://review.opendev.org/882860 (Zed nova)
- https://review.opendev.org/882861 (Zed nova errata 1)
- https://review.opendev.org/882844 (Zed os-brick)


Credits
~~~~~~~
- Jan Wasilewski from Atman (CVE-2023-2088)
- Gorka Eguileor from Red Hat (CVE-2023-2088)


References
~~~~~~~~~~
- https://launchpad.net/bugs/2004555
- http://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2023-2088


Notes
~~~~~
- Limited Protection Against Accidents... If you are only concerned with
  protecting against the accidental case described earlier in this document,
  steps 1-3 above should be sufficient. Note, however, that only applying steps
  1-3 leaves your cloud wide open to the intentional exploitation of this
  vulnerability. Therefore, we recommend that the full fix be applied to all
  deployments.
- Using Configuration as a Short-Term Mitigation... An alternative approach to
  mitigation can be found in OSSN-0092
  https://wiki.openstack.org/wiki/OSSN/OSSN-0092
- The stable/xena and stable/wallaby branches are under extended maintenance
  and will receive no new point releases, but patches for them are provided as
  a courtesy where available.


OSSA History
~~~~~~~~~~~~
- 2023-05-10 - Errata 2
- 2023-05-10 - Errata 1
- 2023-05-10 - Original Version

-- 
Jeremy Stanley
OpenStack Vulnerability Management Team

Message #8 received at 1035932-submitter@bugs.debian.org (full text, mbox, reply):

From: Thomas Goirand <zigo@debian.org>

To: 1035932-submitter@bugs.debian.org

Subject: Bug#1035932 marked as pending in python-os-brick

Date: Thu, 11 May 2023 11:42:51 +0000

Control: tag -1 pending

Hello,

Bug #1035932 in python-os-brick reported by you has been fixed in the
Git repository and is awaiting an upload. You can see the commit
message below and you can check the diff of the fix at:

https://salsa.debian.org/openstack-team/libs/python-os-brick/-/commit/0a3dc82a70a202620c39043261614c6fbc636fa1

------------------------------------------------------------------------
* New upstream point release:
    - Fixes CVE-2023-2088: Unauthorized volume access through deleted volume
      attachments. (Closes: #1035932).
------------------------------------------------------------------------

(this message was generated automatically)
-- 
Greetings

https://bugs.debian.org/1035932

Message #13 received at 1035932-submitter@bugs.debian.org (full text, mbox, reply):

From: Thomas Goirand <zigo@debian.org>

To: 1035932-submitter@bugs.debian.org

Subject: Bug#1035932 marked as pending in python-os-brick

Date: Thu, 11 May 2023 11:51:37 +0000

Control: tag -1 pending

Hello,

Bug #1035932 in python-os-brick reported by you has been fixed in the
Git repository and is awaiting an upload. You can see the commit
message below and you can check the diff of the fix at:

https://salsa.debian.org/openstack-team/libs/python-os-brick/-/commit/766b5e22b7f98ce01dbb1279460572ed887d324f

------------------------------------------------------------------------
* CVE-2023-30861: Unauthorized volume access through deleted volume
    attachments. Added upstream patch: Support force disconnect for FC
    (Closes: #1035932).
------------------------------------------------------------------------

(this message was generated automatically)
-- 
Greetings

https://bugs.debian.org/1035932

Message #16 received at 1035932-submitter@bugs.debian.org (full text, mbox, reply):

From: Thomas Goirand <zigo@debian.org>

To: 1035932-submitter@bugs.debian.org

Subject: Bug#1035932 marked as pending in python-os-brick

Date: Thu, 11 May 2023 11:53:25 +0000

Control: tag -1 pending

Hello,

Bug #1035932 in python-os-brick reported by you has been fixed in the
Git repository and is awaiting an upload. You can see the commit
message below and you can check the diff of the fix at:

https://salsa.debian.org/openstack-team/libs/python-os-brick/-/commit/96cca7de67c2e76ca91b09117d316699cb69879c

------------------------------------------------------------------------
* CVE-2023-30861: Unauthorized volume access through deleted volume
    attachments. Added upstream patch: Support force disconnect for FC
    (Closes: #1035932).
------------------------------------------------------------------------

(this message was generated automatically)
-- 
Greetings

https://bugs.debian.org/1035932

Message #19 received at 1035932-submitter@bugs.debian.org (full text, mbox, reply):

From: Thomas Goirand <zigo@debian.org>

To: 1035932-submitter@bugs.debian.org

Subject: Bug#1035932 marked as pending in python-os-brick

Date: Thu, 11 May 2023 12:14:07 +0000

Control: tag -1 pending

Hello,

Bug #1035932 in python-os-brick reported by you has been fixed in the
Git repository and is awaiting an upload. You can see the commit
message below and you can check the diff of the fix at:

https://salsa.debian.org/openstack-team/libs/python-os-brick/-/commit/6c0e61af9c3f22e7cd4c0083fd6bc35708cbff2a

------------------------------------------------------------------------
* CVE-2023-30861: Unauthorized volume access through deleted volume
    attachments. Added upstream patch: Support force disconnect for FC
    (Closes: #1035932).
------------------------------------------------------------------------

(this message was generated automatically)
-- 
Greetings

https://bugs.debian.org/1035932

Message #22 received at 1035932-submitter@bugs.debian.org (full text, mbox, reply):

From: Thomas Goirand <zigo@debian.org>

To: 1035932-submitter@bugs.debian.org

Subject: Bug#1035932 marked as pending in python-os-brick

Date: Thu, 11 May 2023 12:17:35 +0000

Control: tag -1 pending

Hello,

Bug #1035932 in python-os-brick reported by you has been fixed in the
Git repository and is awaiting an upload. You can see the commit
message below and you can check the diff of the fix at:

https://salsa.debian.org/openstack-team/libs/python-os-brick/-/commit/6c9c154a057d9bbb6af916ad1a2f23e2987cfbea

------------------------------------------------------------------------
* CVE-2023-30861: Unauthorized volume access through deleted volume
    attachments. Added upstream patch: Support force disconnect for FC
    (Closes: #1035932).
------------------------------------------------------------------------

(this message was generated automatically)
-- 
Greetings

https://bugs.debian.org/1035932

Message #25 received at 1035932-submitter@bugs.debian.org (full text, mbox, reply):

From: Thomas Goirand <zigo@debian.org>

To: 1035932-submitter@bugs.debian.org

Subject: Bug#1035932 marked as pending in python-os-brick

Date: Thu, 11 May 2023 12:19:48 +0000

Control: tag -1 pending

Hello,

Bug #1035932 in python-os-brick reported by you has been fixed in the
Git repository and is awaiting an upload. You can see the commit
message below and you can check the diff of the fix at:

https://salsa.debian.org/openstack-team/libs/python-os-brick/-/commit/dfeb4ee8526e8bddb4764c7d64f21b27e3de821d

------------------------------------------------------------------------
* CVE-2023-30861: Unauthorized volume access through deleted volume
    attachments. Added upstream patch: Support force disconnect for FC
    (Closes: #1035932).
------------------------------------------------------------------------

(this message was generated automatically)
-- 
Greetings

https://bugs.debian.org/1035932

Message #30 received at 1035932-close@bugs.debian.org (full text, mbox, reply):

From: Debian FTP Masters <ftpmaster@ftp-master.debian.org>

To: 1035932-close@bugs.debian.org

Subject: Bug#1035932: fixed in python-os-brick 6.1.0-3

Date: Thu, 11 May 2023 12:21:36 +0000

Source: python-os-brick
Source-Version: 6.1.0-3
Done: Thomas Goirand <zigo@debian.org>

We believe that the bug you reported is fixed in the latest version of
python-os-brick, which is due to be installed in the Debian FTP archive.

A summary of the changes between this version and the previous one is
attached.

Thank you for reporting the bug, which will now be closed.  If you
have further comments please address them to 1035932@bugs.debian.org,
and the maintainer will reopen the bug report if appropriate.

Debian distribution maintenance software
pp.
Thomas Goirand <zigo@debian.org> (supplier of updated python-os-brick package)

(This message was generated automatically at their request; if you
believe that there is a problem with it please contact the archive
administrators by mailing ftpmaster@ftp-master.debian.org)


-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

Format: 1.8
Date: Thu, 11 May 2023 13:46:37 +0200
Source: python-os-brick
Architecture: source
Version: 6.1.0-3
Distribution: unstable
Urgency: high
Maintainer: Debian OpenStack <team+openstack@tracker.debian.org>
Changed-By: Thomas Goirand <zigo@debian.org>
Closes: 1035932
Changes:
 python-os-brick (6.1.0-3) unstable; urgency=high
 .
   * CVE-2023-30861: Unauthorized volume access through deleted volume
     attachments. Added upstream patch: Support force disconnect for FC
     (Closes: #1035932).
Checksums-Sha1:
 07f585c69f218831992fb159c231ac68fd420a57 3092 python-os-brick_6.1.0-3.dsc
 beebf0fcb1388d8bafb876835123892b772176dc 8740 python-os-brick_6.1.0-3.debian.tar.xz
 7f62f1b2c23ce3365f438fb0076121559da8067e 12900 python-os-brick_6.1.0-3_amd64.buildinfo
Checksums-Sha256:
 2e6353bdb9dde3f22599daea603b523bbcbbef7a5872f172e924acb0b1a6d836 3092 python-os-brick_6.1.0-3.dsc
 35d9322d45668777ad276148fe0aa9465765d4d3f8da9b08a96ca75826ce1c2b 8740 python-os-brick_6.1.0-3.debian.tar.xz
 20ab74e3f76515ba14b7d040af78bbdfbcf3a376da905d6b9b085c16e7492375 12900 python-os-brick_6.1.0-3_amd64.buildinfo
Files:
 731438c92fbc4c9327b53401803ddd47 3092 python optional python-os-brick_6.1.0-3.dsc
 808d241f4dfd5700a1b7c02a1597d152 8740 python optional python-os-brick_6.1.0-3.debian.tar.xz
 ebe924715ed9aef8269ed420053bf4a2 12900 python optional python-os-brick_6.1.0-3_amd64.buildinfo

-----BEGIN PGP SIGNATURE-----

iQIzBAEBCAAdFiEEoLGp81CJVhMOekJc1BatFaxrQ/4FAmRc1soACgkQ1BatFaxr
Q/48Qg//dvDhsD6ITt/C4NuNsomMXxwrXg9j9w0T5tFKPE5+urAE5BR5upLJ/RM7
YbE1QvZC4/4olXgN/UVxN+FIQldagkeZJlPHln05DOFC9sIesR+TextU93RsPp6a
LkYKX4PVWVmI4jtQhoncW2z6yercFZDd+YCZ2CJiEmDf8hjz3MNYL493eOOWIXj5
ljF0fibskgx9O0ooMDCxh5MC5kizIS1U8MJAKLRKjfh4IJshNKxPoufNC4QBBlD8
ZOIhw2bwTOWxd77lOuyrxEyL7k5kXAQ3QiQH20dtPv+4KKj562we0verAmgNfpMK
Ij+pBxk0c0Cti/fNB5rSgq+NHjLHvLwKdqe/YaSfd+RxxERlX2Jv1sE5XfLC5YpL
vl9tuDITovQr+3h+0LnLXt1CYJgynQx9ZwoFX6AP6teWYBwl22CV9b0/8RXhbg3e
17TQ3pIctC4OeoLo7b006+p+YO/VTBUyNWozdjYGSJlUSK8WG+st91NS1a+O81s6
auREvHiuQJobvRZbncP12AF0zz3sJJXbaHPKoAaTJ25nLAaHBpNi4IYnBLtjSSu0
s4dzmq7856j1y7Ri/PKbED/0HqUxN7ggltvt5SQ5Y7hZxHwyPUM8gCVXS5pCMS8X
7g9pfPQAJM5PhLOvG6z/dQ9hXOLu4NsJMzNEyuhsxOUSlSEBu30=
=+NRe
-----END PGP SIGNATURE-----

Message #35 received at 1035932-close@bugs.debian.org (full text, mbox, reply):

From: Debian FTP Masters <ftpmaster@ftp-master.debian.org>

To: 1035932-close@bugs.debian.org

Subject: Bug#1035932: fixed in python-os-brick 6.2.2-1

Date: Thu, 11 May 2023 12:21:47 +0000

Source: python-os-brick
Source-Version: 6.2.2-1
Done: Thomas Goirand <zigo@debian.org>

We believe that the bug you reported is fixed in the latest version of
python-os-brick, which is due to be installed in the Debian FTP archive.

A summary of the changes between this version and the previous one is
attached.

Thank you for reporting the bug, which will now be closed.  If you
have further comments please address them to 1035932@bugs.debian.org,
and the maintainer will reopen the bug report if appropriate.

Debian distribution maintenance software
pp.
Thomas Goirand <zigo@debian.org> (supplier of updated python-os-brick package)

(This message was generated automatically at their request; if you
believe that there is a problem with it please contact the archive
administrators by mailing ftpmaster@ftp-master.debian.org)


-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

Format: 1.8
Date: Thu, 11 May 2023 13:34:21 +0200
Source: python-os-brick
Architecture: source
Version: 6.2.2-1
Distribution: experimental
Urgency: medium
Maintainer: Debian OpenStack <team+openstack@tracker.debian.org>
Changed-By: Thomas Goirand <zigo@debian.org>
Closes: 1035932
Changes:
 python-os-brick (6.2.2-1) experimental; urgency=medium
 .
   * New upstream point release:
     - Fixes CVE-2023-2088: Unauthorized volume access through deleted volume
       attachments. (Closes: #1035932).
Checksums-Sha1:
 37cad0c18be3dc69c7f3e7228772c794d6b567a4 2912 python-os-brick_6.2.2-1.dsc
 141c6bd51fbcf84ff3ba7e7c582c7761f9e28ee4 176276 python-os-brick_6.2.2.orig.tar.xz
 ec9000e382176fb0ef0adf30036014db3e0955a6 4704 python-os-brick_6.2.2-1.debian.tar.xz
 6346770d2625b8714fbf985042ad7dd6c9c9f87b 12890 python-os-brick_6.2.2-1_amd64.buildinfo
Checksums-Sha256:
 f4c1aca67f62647659dd5976976d3cb7b12ca857fe7908c51998fa1ec911a563 2912 python-os-brick_6.2.2-1.dsc
 9891864e3e3f91035b0f95646ad30b53c693f2c4e8a080ea31924d70ea14389f 176276 python-os-brick_6.2.2.orig.tar.xz
 86896323189d9a7ce964e78feee587a0f8572eeceba2a2bc08ca941c254e50a3 4704 python-os-brick_6.2.2-1.debian.tar.xz
 4fbd70e686f0ae0b5a767a8642b21bfd22d43983e01582b2c18fd9cff1289726 12890 python-os-brick_6.2.2-1_amd64.buildinfo
Files:
 bfbb2df7e11caaa16e9c63177373e4da 2912 python optional python-os-brick_6.2.2-1.dsc
 b856ab305026f5b4a1e506181d9c5fee 176276 python optional python-os-brick_6.2.2.orig.tar.xz
 a14f91e25d9032bf257ade41b39402cf 4704 python optional python-os-brick_6.2.2-1.debian.tar.xz
 1dc451a2c00cda37958e528c9954b958 12890 python optional python-os-brick_6.2.2-1_amd64.buildinfo

-----BEGIN PGP SIGNATURE-----

iQIzBAEBCAAdFiEEoLGp81CJVhMOekJc1BatFaxrQ/4FAmRc1L4ACgkQ1BatFaxr
Q/5sYQ//Uvg+hn6Bbhxx2SW6En1WyfJx1mRX7wDhcsboGEytrM5ne7RNl+BQVv0V
RezINwaWW4AkKIpGTfTe6ju05dxsTKuzR+0xmtX5K1cvfeUROiqFr9p9+lImkwNR
LkycvTILR0tftMeww19TwvxcXrXVGH6nRPLl+8IdiFHkRc9Q39aUhVShin29eToe
OP+6DRkH6lfoZWkkJ2tl93elCnLzrEZ7MZdS/Ak6G0HAvkYScJ4ikT/WCqLAw1l2
ttFvgJ395eTlHA8SkOfmeGMdCqDZnrLZnhyzwxuxRrRGvvyKzrXgbkt4ojexTxo5
r9Ym7HW7Uhdz9aODLKz4AO45A9qYTgpE2CHJR1VDMpMwIowREPMll2hLBdeZgGYK
K4osZxut5EMjLGEaUYSBuemE7zAL2A5oCBPXshp+3Lv4dHEPiggAqqtIDDv2FbUp
A3jtJ+wiusCbxnaTINWy/gQZDwC3gb1dp8rr0K4PwRiPUpIbTpWiWoE6Gc9sR+J1
ztt7RmQsrTrgJ/AK/1IBPRGYHop49LzUxUnWbx4Go3davdlgZfHiYNVqYcEkWwJi
9lTkqGLyLozbQjOqmfSu8y4w/tknXSCITB0BfRle3Sy1pJYJK6AIyC6D3NCG2al5
a+spMtG5HsObGOLNkSUOb9eUaDRFJwXX5oSldkBLs0dLW9tUBDg=
=RbJ+
-----END PGP SIGNATURE-----

Send a report that this bug log contains spam.