Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

NODE_MEM doesn't work #3

Open
SimonBin opened this issue Apr 4, 2019 · 1 comment
Open

NODE_MEM doesn't work #3

SimonBin opened this issue Apr 4, 2019 · 1 comment
Labels

Comments

@SimonBin
Copy link

SimonBin commented Apr 4, 2019

The gtfs crashes with a Node out of memory error

info: >> Exporting to GTFS

<--- Last few GCs --->

 1379495 ms: Mark-sweep 1288.8 (1376.1) -> 1288.5 (1377.1) MB, 2426.2 / 0.0 ms [allocation failure] [GC in old space requested].
 1381833 ms: Mark-sweep 1288.5 (1377.1) -> 1288.5 (1340.1) MB, 2337.4 / 0.0 ms [last resort gc].
 1384150 ms: Mark-sweep 1288.5 (1340.1) -> 1288.5 (1337.1) MB, 2317.7 / 0.0 ms [last resort gc].


<--- JS stacktrace --->

==== JS stack trace =========================================

Security context: 0x22dc1f7cfb51 <JS Object>
    1: writeForStop(aka writeForStop) [/node_modules/podigg/lib/gtfs/GtfsBuilder.js:~147] [pc=0x3ce6ba1895ae] (this=0x22dc1f704381 <undefined>,stop=0x22c6585160f9 <an Object with map 0x5568686ac71>,i=5)
    2: /* anonymous */(aka /* anonymous */) [/node_modules/podigg/lib/gtfs/GtfsBuilder.js:~146] [pc=0x3ce6ba192c54] (this=0x22dc1f704381 <undefined>,connection=0x25ec5956efe9 <an Object with map...

FATAL ERROR: CALL_AND_RETRY_LAST Allocation failed - JavaScript heap out of memory
 1: node::Abort() [node]
 2: 0x109838c [node]

despite running with -e NODE_MEM=50000 as indicated in the docs. I believe the issue is that the parameter does not extend into the childProcess.spawn calls

As a dirty fix I replaced /usr/local/bin/node with a script that calls node --max_old_space_size=50000

@rubensworks
Copy link
Member

I believe the issue is that the parameter does not extend into the childProcess.spawn calls

Hmmm, you're probably right.

A fix that would pass the active environment variables would probably be a clean solution: https://stackoverflow.com/questions/33593522/overcome-node-js-memory-limit-with-fork

I don't have the bandwidth to look into this in the near future, but I'm happy to accept a PR on this.

Thanks for reporting!

@rubensworks rubensworks added the bug label Apr 4, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

2 participants