error while loading database file

[dotob]

i have a database file which i load using your sample code:

var Datastore = require('nedb')
  , db = new Datastore({ filename: 'p.nedb' });
db.loadDatabase(function (err) {
  // Callback is optional
  // Now commands will be executed
    console.log("finished", err);
});

then i get this error:

⇒  node nedbtest.js
finished Error: "toString()" failed
    at Buffer.toString (buffer.js:528:11)
    at tryToString (fs.js:445:15)
    at FSReqWrap.readFileAfterClose [as oncomplete] (fs.js:436:12)

any ideas? i could send you the file if it helps (it is 35mb zipped, 639mb unzipped, but i previously had the error with smaller files as well). the error is not catchable...

thanks for looking into this

[martpie]

I got the same error. With a 300mb library.

[lukasbestle]

NeDB loads the full database into memory. Such large databases are not going to work. Please use Mongo or another non-embedded database.

[martpie]

NeDB is indeed not meant for this amount of data, maybe a note could be added about the max recommended size of a database in the README ?

[kaiquewdev]

Using nedb on an application the following error occurs when the serving process was triggered:

Feathers application started on localhost:3030
/Users/kaiquesilva/pieces/feathers-scrape-it/node_modules/nedb/lib/datastore.js:77
if (err) { throw err; }
^

Error: ENOENT: no such file or directory, rename '/Users/kaiquesilva/pieces/feathers-scrape-it/data/users.db~' -> '/Users/kaiquesilva/pieces/feathers-scrape-it/data/users.db'
at Error (native)

[kaiquewdev]

I'm nothing using emacs, the current file mention on that error does not exists.

[jeffrey008]

@kaiquewdev, I am having the same issue as you right now, did you find any solution to that? Thanks.

[Danielv123]

I have a small DB with roughly 400 small items, around a hundred characters each. I am using

db.update(doc, object, {multi:true})

where doc is the item from db.find({name:object.name}) and object is an slightly edited version of that object.

I am guessing because of the way nedb works, it has appended the edit to the bottom of the database.db file without erasing previous version of the item, getting me a 285 mb database that prevents my application from launching.

Is there any fix for this?

[martpie]

@Danielv123

I am guessing because of the way nedb works, it has appended the edit to the bottom of the database.db file without erasing previous version of the item

It should not, you should check that you call db.update correctly.

If you're looking for a more scalable Library, I switched to LinvoDB3 some time ago, slower but far more scalable in terms of amount of data.

[Danielv123]

@KeitIG I call update at line 60 in this file. I can confirm that the items does update and are registered with the same ID in the database file, with older versions towards the top of the file and the newer copy on the bottom.

[JamesMGreene]

This would probably be relieved with PR #463.