env LD_PRELOAD not effect on Cluster Mode

[Cow258]

#3684
https://github.com/Unitech/pm2/issues/2618
lovell/sharp#1803

What's going wrong?

Unable to load env LD_PRELOAD under Cluster Mode

How could we reproduce this issue?

Full detail here
lovell/sharp#1803

Supporting information

$ pm2 report
--- PM2 report ----------------------------------------------------------------
Date                 : Tue Jul 30 2019 04:04:02 GMT+0000 (Coordinated Universal Time)
===============================================================================
--- Daemon -------------------------------------------------
pm2d version         : 3.5.1
node version         : 12.6.0
node path            : not found
argv                 : /usr/bin/node,/usr/lib/node_modules/pm2/lib/Daemon.js
argv0                : node
user                 : ubuntu
uid                  : 1000
gid                  : 1000
uptime               : 40min
===============================================================================
--- CLI ----------------------------------------------------
local pm2            : 3.5.1
node version         : 12.6.0
node path            : /usr/bin/pm2
argv                 : /usr/bin/node,/usr/bin/pm2,report
argv0                : node
user                 : ubuntu
uid                  : 1000
gid                  : 1000
===============================================================================
--- System info --------------------------------------------
arch                 : x64
platform             : linux
type                 : Linux
cpus                 : Intel(R) Xeon(R) CPU E5-2676 v3 @ 2.40GHz
cpus nb              : 1
freemem              : 395948032
totalmem             : 1031700480
home                 : /home/ubuntu
===============================================================================

[Cow258]

@Unitech This issue still here for a long time, could you please take a look at this? Thanks

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[Cow258]

The issue still here today

[tommyjs007]

Facing the same issue. any suggestions?

[Unitech]

Cluster mode is tightly linked on how PM2 has been started.
So if you want to load a specific LD_PRELOAD, you need to restart PM2 with that env variable.

So:

pm2 kill
LD_PRELOAD=xxxx pm2 ls
# Now if you start an app in cluster mode, LD_PRELOAD will be injected in that app
pm2 start app.js -i max

[Cow258]

@Unitech How to ensure pm2 used LD_PRELOAD when system boot up?

[Cow258]

@lokoArt
Currently,
I am using fork mode to prevent memory leak.
But that cause unable to provide zero-downtime service,
all update required between scheduled maintenance period like 02:00-03:00,
or accept the downtime when you need update immediate.

If set env LD_PRELOAD=xxxx before pm2 start that is work,
I will change all instance into cluster mode.

[Cow258]

Successful to add LD_PRELOAD=xxxx into Cluster mode!
🎉🎉🎉🎉🎉🎉🎉🎉🎉🎉🎉🎉

Add LD_PRELOAD=/usr/lib/x86_64-linux-gnu/libjemalloc.so.1
Into /env/environment
Also do
export LD_PRELOAD=/usr/lib/x86_64-linux-gnu/libjemalloc.so.1
Then
sudo reboot

The cluster mode will working with LD_PRELOAD=xxxx,
Not issue when processing many big size images with sharp.js,
Thanks to @Unitech provide some tips for this issue.\

[jordygroote]

Successful to add LD_PRELOAD=xxxx into Cluster mode!
🎉🎉🎉🎉🎉🎉🎉🎉🎉🎉🎉🎉

Add LD_PRELOAD=/usr/lib/x86_64-linux-gnu/libjemalloc.so.1
Into /env/environment
Also do
export LD_PRELOAD=/usr/lib/x86_64-linux-gnu/libjemalloc.so.1
Then
sudo reboot

The cluster mode will working with LD_PRELOAD=xxxx,
Not issue when processing many big size images with sharp.js,
Thanks to @Unitech provide some tips for this issue.\

Where to place LD_PRELOAD=/usr/lib/x86_64-linux-gnu/libjemalloc.so.1 , /env/environment isn't a valid directory?

[matthijs88]

@jordygroote I ran into the same problem. It's /etc/enviroment

[Cow258]

Update

Install latest jemalloc 5.2.1 on Ubuntu 18

Refer: https://gist.github.com/diginfo/be7347e6e6c4f05375c51bca90f220e8

sudo apt-get -y install autoconf libxslt-dev xsltproc docbook-xsl
git clone https://github.com/jemalloc/jemalloc.git
cd jemalloc
autoconf
./configure
make dist
sudo make install

You can found jemalloc at /usr/local/lib/libjemalloc.so.2

sudo su
vi /etc/enviroment

# Then add this line
LD_PRELOAD=/usr/local/lib/libjemalloc.so.2

Then

export LD_PRELOAD=/usr/local/lib/libjemalloc.so.2

Then optional reboot sudo reboot

[Cow258]

Update

I faced this problem again in a week, and it took me 2 days to solve it
After seeing this comment, I got the answer
nodejs/help#1518 (comment)

Somehow, the LD_PRELOAD environment variable may not work
You need to modify /etc/ld.so.preload to solve this problem

# make install version
sudo echo "/usr/local/lib/libjemalloc.so.2" >> /etc/ld.so.preload

# apt-get version
sudo echo "/usr/lib/x86_64-linux-gnu/libjemalloc.so" >> /etc/ld.so.preload

Then restart all node processes to use jemalloc for allocation:

# If you are using pm2
pm2 kill
pm2 resurrect

Then check the PID of your running node process and plug it into the command below to verify it is using jemalloc:

ps aux | grep node
cat /proc/<PID>/smaps | grep jemalloc

[hypo-thesis]

@Cow258 Thanks mate. I owe you a debt.

[rayanchemin]

Update

I faced this problem again in a week, and it took me 2 days to solve it After seeing this comment, I got the answer nodejs/help#1518 (comment)

Somehow, the LD_PRELOAD environment variable may not work You need to modify /etc/ld.so.preload to solve this problem

# make install version
sudo echo "/usr/local/lib/libjemalloc.so.2" >> /etc/ld.so.preload

# apt-get version
sudo echo "/usr/lib/x86_64-linux-gnu/libjemalloc.so" >> /etc/ld.so.preload

Then restart all node processes to use jemalloc for allocation:

# If you are using pm2
pm2 kill
pm2 resurrect

Then check the PID of your running node process and plug it into the command below to verify it is using jemalloc:

ps aux | grep node
cat /proc/<PID>/smaps | grep jemalloc

Thank you so much!