This is indeed something worth mentioning in the docs. Would you be interested in creating a PR? ;)

This is still true at this time of writing, and this is a major documentation issue, isn't it ?

Is this going to be worked on at some point?

This seems a really strange interaction, but it's just bitten me. I'm using axios to upload large files to presigned S3 URLs.

const stream = fs.createReadStream(file);

const config: AxiosRequestConfig = {
  onUploadProgress: (event) => progressCallback(Math.round((100 * event.loaded) / stats.size)),
  headers: { 'Content-Length': stats.size, 'Content-Type': contentType },
  validateStatus: () => true,

  // Without this, we load the whole file into memory (which can be several GB).
  maxRedirects: 0,
};

const signedUrl = await this.signPutUrl(key, stats.size);
const rsp = await axios.put(signedUrl, stream, config);

Without the maxRedirects bit I have the RAM usage of NodeJS skyrocket, and in some cases bring down the whole process. I struggled to find any reference online to explain what I'm doing wrong till I stumbled on this thread.

Am I missing anything here? This seems really odd behaviour. Would be great to have this documented.

@aza547 Yes, you need to disable redirects if you are uploading large data, since stream buffering is enabled to be able to resend the data in case of a redirect.
There is a warning about this ​Axios/follow-redirects specificity in the docs, although it is written in the "progress capturing" section.

⚠️ Warning It is recommended to disable redirects by setting maxRedirects: 0 to upload the stream in the node.js environment, as follow-redirects package will buffer the entire stream in RAM without following the "backpressure" algorithm.

In the long term, we hope to handle redirects ourselves and buffer the upstream only for a short time and buffer size, but for now, we have what we have.

closing this its relatively old and @DigitalBrainJS has mentioned a future proposal to handle this

@jasonsaayman, a future proposal does not resolve the issue. Can this be open until this is fixed without maxRedirects: 0?

@realies for sure, i am trying to see what we can and cannot close so will gladly re-open

I discovered that when the stream is passed as compressed using gzip, you do not need to set maxRedirects: 0 to prevent it from being fully buffered. I thought this might be helpful.

This works as expected:

import { Readable } from 'node:stream';
import axios from 'axios';
import zlib from 'node:zlib';

const readable = Readable.from(genData(1e9));
const gzip = zlib.createGzip();
const compressed = readable.pipe(gzip);

await axios.post('http://localhost:3000', compressed, {
	headers: {
		'Content-Type': 'text/csv',
	},
});

async function* genData(max) {
	let i = 0;
	yield 'username,password,email\n';
	while (i < max) {
		yield `user${i},password${i},user${i}@gmail.com\n`;
		i++;
	}
}

This required maxRedirects: 0 to work as expected:

import { Readable } from 'node:stream';
import axios from 'axios';

const readable = Readable.from(genData(1e9));

await axios.post('http://localhost:3000', readable, {
	headers: {
		'Content-Type': 'text/csv',
	},
	maxRedirects: 0, // this is required; otherwise, a memory leak will occur
});

async function* genData(max) {
	let i = 0;
	yield 'username,password,email\n';
	while (i < max) {
		yield `user${i},password${i},user${i}@gmail.com\n`;
		i++;
	}
}