MongoDB 16MB Document Limit Exhaustion: Unbounded Array Anti-Patterns, Outlier Handling & Bucket Architecture
Identifying documents exceeding the strict 16MB BSON barrier with $bsonSize aggregations, refactoring to Bucket/Subset patterns, and GridFS migration.
Operational Verification & Reliability Metadata
The error string is a symptom, not the root cause. Do not restart the server blindly. Run the diagnostic queries below to pinpoint the actual failure mode.
▸Symptom
Mutations pushing elements into existing documents fail with:
MongoServerError: BSONObj size: 16793600 is invalid. Size must be between 0 and 16793600▸What You See in Production
Specific power-users cannot save data; $push updates abort with size boundary limits.
▸Root Cause Matrix
| Probability | Root Cause | Description |
|---|---|---|
| %70 | Unbounded Array Antipattern | Appending activity logs or telemetry directly into embedded arrays. |
| %20 | Embedded Media Files | Storing binary payloads directly in documents rather than GridFS. |
| %10 | Denormalization Overflow | Injudicious 1-to-N embedding without capping. |
▸EMERGENCY RUNBOOK (0-5 min)
// 1. Locate oversized documents
db.users.aggregate([
{ $project: { sizeBytes: { $bsonSize: "$$ROOT" } } },
{ $sort: { sizeBytes: -1 } },
{ $limit: 5 }
]);
// 2. Slice array to retain latest 100 entries
db.users.updateOne(
{ _id: ObjectId("...") },
{ $push: { audit_logs: { $each: [], $slice: -100 } } }
);▸PERMANENT BLUEPRINT
- 1Bucket & Subset Pattern: Decouple unbounded historical events into satellite collections.
- 2Adopt GridFS: Store binary assets larger than 1MB in GridFS storage.
▸Verification
db.users.aggregate([
{ $project: { sizeMB: { $divide: [{ $bsonSize: "$$ROOT" }, 1048576] } } },
{ $sort: { sizeMB: -1 } },
{ $limit: 1 }
]);Other Database Error Triage Playbooks
Other battle-tested diagnostic guides for high-severity database errors.
MongoDB "not master" / NotWritablePrimary Fix
Troubleshooting MongoDB "not master" (legacy < 5.0) and NotWritablePrimary (code 10107 >= 5.0) during replica set failovers, connection string topology URIs, retryWrites, and oplog sizing.
MongoDB E11000: Duplicate Key Error Triage & Mitigation
Diagnosing E11000 errors caused by concurrent insert races, index verification, and implementing atomic upsert with $setOnInsert.
MongoDB MongoNetworkError & HostUnreachable Triage
Addressing driver network timeouts, topology change failovers, invalid SRV connection strings, and serverSelectionTimeoutMS tuning.