request(..., preload_content=False).read(-1) raises an unexpected Exception #3122
Closed
Description
Subject
With preload_content=False, calling read(-1) on the body raises RuntimeError("buffer is empty") while read() or read(None) return the whole request body as expected.
Environment
import platform
import ssl
import urllib3
print("OS", platform.platform())
print("Python", platform.python_version())
print(ssl.OPENSSL_VERSION)
print("urllib3", urllib3.__version__)prints:
OS macOS-13.5.1-arm64-arm-64bit
Python 3.11.4
OpenSSL 3.1.1 30 May 2023
urllib3 2.0.4
Steps to Reproduce
A simple and isolated way to reproduce the issue. A code snippet would be great.
Bad case:
import urllib3
http = urllib3.PoolManager()
http.request("GET", "https://httpbin.org/robots.txt", preload_content=False).read(-1)
raises: RuntimeError: buffer is empty.
while these work:
http.request("GET", "https://httpbin.org/robots.txt", preload_content=False).read()
or
http.request("GET", "https://httpbin.org/robots.txt", preload_content=False).read(None)
Expected Behavior
The whole buffer should be returned, as -1 is the expected default argument for fileobj according to the python stdlib.
Actual Behavior
it raises
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/Users/kosta/Library/Caches/pypoetry/virtualenvs/urllib3-read-IiCHxn1z-py3.11/lib/python3.11/site-packages/urllib3/response.py", line 877, in read
return self._decoded_buffer.get(amt)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/kosta/Library/Caches/pypoetry/virtualenvs/urllib3-read-IiCHxn1z-py3.11/lib/python3.11/site-packages/urllib3/response.py", line 255, in get
raise RuntimeError("buffer is empty")
Note: urllib3 1.x raises:
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/Users/kosta/Library/Caches/pypoetry/virtualenvs/uai-snaky-path-TevYsdGm-py3.9/lib/python3.9/site-packages/urllib3/response.py", line 567, in read
data = self._fp_read(amt) if not fp_closed else b""
File "/Users/kosta/Library/Caches/pypoetry/virtualenvs/uai-snaky-path-TevYsdGm-py3.9/lib/python3.9/site-packages/urllib3/response.py", line 533, in _fp_read
return self._fp.read(amt) if amt is not None else self._fp.read()
File "/opt/homebrew/Cellar/python@3.9/3.9.17_1/Frameworks/Python.framework/Versions/3.9/lib/python3.9/http/client.py", line 462, in read
b = bytearray(amt)
ValueError: negative count