Closed
Description
opened on Feb 7, 2024
Prerequisites
- I have written a descriptive issue title
- I have searched existing issues to ensure the bug has not already been reported
Mongoose version
8.1.1
Node.js version
18.18.2
MongoDB server version
5
Typescript version (if applicable)
No response
Description
Hello, me again!
- A document has a subdocument array.
- The array is modified (namely, push, pull or splice is used).
- The document is saved inside a transaction.
- The transaction is retried at least one (I mean the whole transaction as opposed to just the commit).
Result: the whole array is saved incorrectly and can even become corrupted, in turn ruining the whole document.
I noticed that in more recent versions document arrays are actually proxy objects, and my guess is that this might be causing problems, especially when using .pull()
/.splice()
.
Steps to Reproduce
import mongoose from "mongoose";
import { MongoServerError, MongoErrorLabel } from "mongodb";
console.log("connecting...");
// Connecting to a *ReplicaSet* specifically to make sure that transactions work correctly.
// Nevertheless, whether or not an actual ReplicaSet is used has no effect on the issue.
await mongoose.connect(
"mongodb://mongo-node1:27017,mongo-node2:27018,mongo-node3:27019/xxx?replicaSet=eof"
);
console.log("connected!");
// Define some schemas...
const subItemSchema = new mongoose.Schema(
{
name: { type: String, required: true },
},
{ _id: false }
);
const itemSchema = new mongoose.Schema(
{
name: { type: String, required: true },
subItems: { type: [subItemSchema], required: true },
},
{ _id: false }
);
const schema = new mongoose.Schema({
items: { type: [itemSchema], required: true },
});
// ...and a model
const Model = mongoose.model("MyModel", schema);
// Clear the collection...
await Model.deleteMany({});
// ...and create one document
await Model.create({
items: [
{ name: "test1", subItems: [{ name: "x1" }] },
{ name: "test2", subItems: [{ name: "x2" }] },
],
});
const doc = await Model.findOne();
// Array modification - choose one...
// Works okay, but I guess that's because it's an idempotent operation
doc.items.addToSet({ name: "test3", subItems: [{ name: "x3" }] });
// Adds unexpected extra entries
doc.items.push({ name: "test3", subItems: [{ name: "x3" }] });
// Corrupts the document. It seems that moving this inside the `.transaction()` callback solves the problem
doc.items.pull(doc.items[0]);
// Corrupts the document
doc.items.splice(0, 1);
let attempt = 0;
await mongoose.connection.transaction(async (session) => {
console.log(`Attempt: ${attempt}`);
await doc.save({ session });
// This is the important bit. Uncomment this to trigger a transaction retry.
// if (attempt === 0) {
// attempt += 1;
// throw new MongoServerError({
// message: "Test transient transaction failures & retries",
// errorLabels: [MongoErrorLabel.TransientTransactionError],
// });
// }
});
const updatedDoc = await Model.findOne();
// See the structure here. Depending on the array operation, the structure may be different,
// but it's always wrong.
console.log(updatedDoc.items);
// Bye bye!
await mongoose.disconnect();
Expected Behavior
If .push()
is used
The total number of documents added to the array grows exponentially with every retry - it is 2^n, where n is the number of retries (forgive me, can't go without a lol here :)).
Expectation: only one document is added.
If .pull()
or .splice()
is used
The document that is saved in the database looks like this:
{
"_id": {
"$oid": "65c40f6c0975fbad509f0aa8"
},
"items": {
"0": {
"name": "test2",
"subItems": [
{
"name": "x2"
}
]
}
},
"__v": 0
}
As you can see, the array got converted into an object, and that's why I'm throwing suspicious looks at the Proxy object used for document array.
Activity