davide/purple-explorer
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Cluster updates + add new start/stop options

Browse Source 
-More graceful shutdown of node cluster on 'npm stop' with better cleanup of resources on exit
-Added new stop_explorer.sh script which looks up the explorer port # via settings file and closes the application running on that port # instead of saving and killing the process by pid as it did before
-Added support for pm2 and forever using 'npm run start-pm2' and 'npm run start-forever' respectively
-pm2 is automatically installed when starting with 'npm run start-pm2' if it is not already installed
-forever is automatically installed when starting with 'npm run start-forever' if it is not already installed
-Updated existing npm commands in package.json by replacing hardcoded 'node' with '$(which node)'
-/path/to/nodejs changed to /path/to/node in the /settings.json.template, /lib/settings.js and /scripts/sync.js files
-README updates:
 -Added a new 'Start/Stop the Explorer' section
 -Added PM2 instructions to the 'Start/Stop the Explorer' section
 -Moved Start/Stop Explorer instructions to the 'Start/Stop the Explorer' section
 -Moved Forever instructions to the 'Start/Stop the Explorer' section
 -/path/to/nodejs changed to /path/to/node
 -Some additional small misc fixes
This commit is contained in:
joeuhren
2021-04-17 13:26:05 -06:00
parent 24cd111e0c
commit f948e29085
9 changed files with 236 additions and 88 deletions
Download Patch File Download Diff File Expand all files Collapse all files
README.md
+125 -51
View File
@@ -36,8 +36,14 @@ Table of Contents
- [Download Source Code](#download-source-code)
- [Install Node Modules](#install-node-modules)
- [Configure Explorer Settings](#configure-explorer-settings)
- [Starting the Explorer](#starting-the-explorer)
- [Stopping the Explorer](#stopping-the-explorer)
- [Start/Stop the Explorer](#startstop-the-explorer)
- [Start Explorer (Use for Testing)](#start-explorer-use-for-testing)
- [Stop Explorer (Use for Testing)](#stop-explorer-use-for-testing)
- [Start Explorer Using PM2 (Recommended for Production)](#start-explorer-using-pm2-recommended-for-production)
- [Start Explorer Using PM2 and Log Viewer](#start-explorer-using-pm2-and-log-viewer)
- [Stop Explorer Using PM2 (Recommended for Production)](#stop-explorer-using-pm2-recommended-for-production)
- [Start Explorer Using Forever (Alternate Production Option)](#start-explorer-using-forever-alternate-production-option)
- [Stop Explorer Using Forever (Alternate Production Option)](#stop-explorer-using-forever-alternate-production-option)
- [Syncing Databases with the Blockchain](#syncing-databases-with-the-blockchain)
- [Sample Crontab](#sample-crontab)
- [Wallet Settings](#wallet-settings)
@@ -48,10 +54,6 @@ Table of Contents
- [Prerequisites](#prerequisites)
- [Manually Link TLS/SSL Certificates to the Explorer](#manually-link-tlsssl-certificates-to-the-explorer)
- [Use Nginx as a Reverse Proxy](#use-nginx-as-a-reverse-proxy)
- [Forever](#forever)
- [Install Forever](#install-forever)
- [Starting the Explorer Using Forever](#starting-the-explorer-using-forever)
- [Stopping the Explorer Using Forever](#stopping-the-explorer-using-forever)
- [CORS Support](#cors-support)
- [What is CORS?](#what-is-cors)
- [How to Benefit From Using CORS?](#how-to-benefit-from-using-cors)
@@ -266,36 +268,142 @@ cp ./settings.json.template ./settings.json
*Make required changes in settings.json*
##### Starting the Explorer
### Start/Stop the Explorer
You can launch the explorer in a terminal window that will output all warnings and error msgs with the following cmd (be sure to run from within the explorer directory):
#### Start Explorer (Use for Testing)
You can launch the explorer in a terminal window that will output all warnings and error msgs with one of the following cmds (be sure to run from within the explorer directory):
```
npm start
```
or (useful for crontab):
```
npm run prestart && /path/to/node --stack-size=10000 ./bin/cluster
```
**NOTE:** mongod must be running to start the explorer.
The explorer defaults to cluster mode, forking an instance of its process to each cpu core. This results in increased performance and stability. Load balancing gets automatically taken care of and any instances that for some reason die, will be restarted automatically. If desired, a single instance can be launched with:
The explorer defaults to cluster mode by forking an instance of its process to each cpu core, which results in increased performance and stability. Load balancing gets automatically taken care of and any instances that for some reason die, will be restarted automatically. If desired, a single instance can be launched with:
```
node --stack-size=10000 bin/instance
npm run start-instance
```
##### Stopping the Explorer
or (useful for crontab):
To stop the explorer running with `npm start` you can end the process with the key combination `CTRL+C` in the terminal that is running the explorer or from another terminal you can use the following cmd (be sure to run from within the explorer directory):
```
npm run prestart && /path/to/node --stack-size=10000 ./bin/instance
```
#### Stop Explorer (Use for Testing)
To stop the explorer running with `npm start` you can end the process with the key combination `CTRL+C` in the terminal that is running the explorer, or from another terminal you can use one of the following cmds (be sure to run from within the explorer directory):
```
npm stop
```
or (useful for crontab):
```
sh ./scripts/stop_explorer.sh
```
#### Start Explorer Using PM2 (Recommended for Production)
[PM2](https://www.npmjs.com/package/pm2) is a process manager for Node.js applications with a built-in load balancer that allows you to always keep the explorer alive and running even if it crashes. Once you have configured the explorer to work properly in a production environment, it is recommended to use PM2 to start and stop the explorer instead of `npm start` and `npm stop` to keep the explorer constantly running without the need to always keep a terminal window open.
You can start the explorer using PM2 with one of the following terminal cmds (be sure to run from within the explorer directory):
```
npm run start-pm2
```
or (useful for crontab):
```
npm run prestart "pm2" && /path/to/pm2 start ./bin/instance -i 0 --node-args="--stack-size=10000"
```
**NOTE:** Use the following cmd to find the install path for PM2:
```
which pm2
```
#### Start Explorer Using PM2 and Log Viewer
Alternatively, you can start the explorer using PM2 and automatically open the log viewer which will allow for viewing all warnings and error msgs as they come up by using one of the following terminal cmds (be sure to run from within the explorer directory):
```
npm run start-pm2-debug
```
or (useful for crontab):
```
npm run prestart "pm2" && /path/to/pm2 start ./bin/instance -i 0 --node-args="--stack-size=10000" && /path/to/pm2 logs
```
#### Stop Explorer Using PM2 (Recommended for Production)
To stop the explorer when it is running via PM2 you can use one of the following terminal cmds (be sure to run from within the explorer directory):
```
npm run stop-pm2
```
or (useful for crontab):
```
/path/to/pm2 stop ./bin/instance
```
#### Start Explorer Using Forever (Alternate Production Option)
[Forever](https://www.npmjs.com/package/forever) is an alternative to PM2 which is another useful Node.js module that is used to always keep the explorer alive and running even if the explorer crashes or stops. Once you have configured the explorer to work properly in a production environment, forever can be used as an alternative to PM2 to start and stop the explorer instead of `npm start` and `npm stop` to keep the explorer constantly running without the need to always keep a terminal window open.
You can start the explorer using forever with one of the following terminal cmds (be sure to run from within the explorer directory):
```
npm run start-forever
```
or (useful for crontab):
```
npm run prestart && /path/to/node /path/to/forever start ./bin/cluster
```
**NOTE:** Use the following cmd to find the install path for forever:
```
which forever
```
#### Stop Explorer Using Forever (Alternate Production Option)
To stop the explorer when it is running via forever you can use one of the following terminal cmds (be sure to run from within the explorer directory):
```
npm run stop-forever
```
or (useful for crontab):
```
/path/to/node /path/to/forever stop ./bin/cluster
```
### Syncing Databases with the Blockchain
sync.sh (located in scripts/) is used for updating the local databases. This script must be called from the explorers root directory.
```
Usage: scripts/sync.sh /path/to/nodejs [mode]
Usage: scripts/sync.sh /path/to/node [mode]
Mode: (required)
update Updates index from last sync to current block
@@ -322,10 +430,10 @@ Notes:
*Example crontab; update index every minute, market data every 2 minutes, peers and masternodes every 5 minutes*
```
*/1 * * * * /path/to/explorer/scripts/sync.sh /path/to/nodejs update > /dev/null 2>&1
*/2 * * * * /path/to/explorer/scripts/sync.sh /path/to/nodejs market > /dev/null 2>&1
*/5 * * * * /path/to/explorer/scripts/sync.sh /path/to/nodejs peers > /dev/null 2>&1
*/5 * * * * /path/to/explorer/scripts/sync.sh /path/to/nodejs masternodes > /dev/null 2>&1
*/1 * * * * /path/to/explorer/scripts/sync.sh /path/to/node update > /dev/null 2>&1
*/2 * * * * /path/to/explorer/scripts/sync.sh /path/to/node market > /dev/null 2>&1
*/5 * * * * /path/to/explorer/scripts/sync.sh /path/to/node peers > /dev/null 2>&1
*/5 * * * * /path/to/explorer/scripts/sync.sh /path/to/node masternodes > /dev/null 2>&1
```
### Wallet Settings
@@ -510,40 +618,6 @@ Certbot will ask a few simple questions and generate the necessary TLS/SSL certi
3. If all went well, you should now be able to start up the explorer and browse to it using a secure https connection like [https://example.com](https://example.com).
### Forever
[Forever](https://www.npmjs.com/package/forever) is a useful node.js module that is used to always keep the explorer alive and running even if the explorer crashes or stops. Once you have configured the explorer to work properly in a production environment, it is recommended to use forever to start and stop the explorer instead of `npm start` and `npm stop` to keep the explorer constantly running without the need to always keep a terminal window open.
#### Install Forever
Run the following cmd in a terminal to install forever:
```
sudo npm install forever -g
```
#### Starting the Explorer Using Forever
You can start the explorer using forever with the following terminal cmd (be sure to run from within the explorer directory):
```
npm run prestart && /path/to/nodejs /path/to/forever start bin/cluster
```
**NOTE:** Use the following cmd to find the install path for forever:
```
which forever
```
#### Stopping the Explorer Using Forever
To stop the explorer when it is running via forever you can use the following terminal cmd (be sure to run from within the explorer directory):
```
/path/to/nodejs /path/to/forever stop bin/cluster
```
### CORS Support
eIquidus has basic CORS support which is useful to prevent other sites from consuming public APIs while still allowing specific websites whitelisted access.
bin/cluster
+33 -29
View File
@@ -2,42 +2,46 @@ var cluster = require('cluster');
var fs = require('fs');
if (cluster.isMaster) {
fs.writeFile('./tmp/cluster.pid', process.pid.toString(), function (err) {
if (err) {
console.log('Error: unable to create cluster.pid');
process.exit(1);
} else {
console.log('Starting cluster with pid: ' + process.pid);
console.log('Starting cluster with pid: ' + process.pid);
// ensure workers exit cleanly
process.on('SIGINT', function() {
console.log('Cluster shutting down..');
// ensure workers exit cleanly
process.on('SIGINT', () => {
console.log('Cluster shutting down..');
for (var id in cluster.workers) {
console.log('Worker shutting down (' + id + ')');
cluster.workers[id].kill();
}
// send kill cmd to all workers
for (var id in cluster.workers) {
console.log('Worker (' + id + ') shutting down...');
process.kill(cluster.workers[id]['process']['pid'], 'SIGINT');
}
// exit the master process
function waitForWorkerShutdown () {
if (Object.keys(cluster.workers).length > 0) {
// continue waiting since worker threads are still open
setTimeout(waitForWorkerShutdown, 100);
} else {
// all worker threads have closed
// now exit the master process
process.exit(0);
});
}
}
// count the machine's CPUs
var cpuCount = require('os').cpus().length;
waitForWorkerShutdown();
});
// create a worker for each CPU
for (var i = 0; i < cpuCount; i += 1)
cluster.fork();
// count the machine's CPUs
var cpuCount = require('os').cpus().length;
// listen for dying workers
cluster.on('exit', function (worker, code, signal) {
if (worker['process']['exitCode'] === 0)
console.log('Worker shut down.');
else if ((signal != 'SIGINT') && (worker['process']['exitCode'] !== 0) && (worker.exitedAfterDisconnect !== true)) {
console.log('Cluster restarting...');
cluster.fork();
}
});
// create a worker for each CPU
for (var i = 0; i < cpuCount; i += 1)
cluster.fork();
// listen for dying workers
cluster.on('exit', function (worker, code, signal) {
if (worker['process']['exitCode'] === 0) {
console.log('Worker (' + worker['id'] + ') shutdown complete');
} else if ((signal != 'SIGINT') && (worker['process']['exitCode'] !== 0) && (worker.exitedAfterDisconnect !== true)) {
console.log('Cluster restarting...');
cluster.fork();
}
});
} else
bin/instance
+11
View File
@@ -76,4 +76,15 @@ db.connect(dbString, function() {
var server = app.listen(app.get('port'), '::', function() {
debug('Express server listening on port ' + server.address().port);
});
process.on('SIGINT', () => {
server.close(() => {
var mongoose = require('mongoose');
mongoose.connection.close(false, () => {
// close the main process now that all http and database connections have closed
process.exit(0);
});
});
});
});
lib/settings.js
+2 -2
View File
@@ -965,9 +965,9 @@ exports.sync = {
// If you sync using more than 1 parallel task, then historical balance data for wallet addresses can possibly be saved out-of-order and there is currently no workaround for this.
// Therefore, it is recommended to keep this setting to 1 parallel task for now until a proper solution can be procured for the historical balance issue.
"block_parallel_tasks": 1,
// update_timeout: The amount of time to wait (in milliseconds) before moving to the next block or transaction during blockchain sync or reindex (scripts/sync.sh /path/to/nodejs update or scripts/sync.sh /path/to/nodejs reindex)
// update_timeout: The amount of time to wait (in milliseconds) before moving to the next block or transaction during blockchain sync or reindex (scripts/sync.sh /path/to/node update or scripts/sync.sh /path/to/node reindex)
"update_timeout": 10,
// check_timeout: The amount of time to wait (in milliseconds) before moving to the next block or transaction during a check sync (scripts/sync.sh /path/to/nodejs check)
// check_timeout: The amount of time to wait (in milliseconds) before moving to the next block or transaction during a check sync (scripts/sync.sh /path/to/node check)
"check_timeout": 250,
// save_stats_after_sync_blocks: During index syncronization, stats are saved after processing this many blocks to save time by not having to save after each block
"save_stats_after_sync_blocks": 100,
package.json
+9 -3
View File
@@ -3,9 +3,15 @@
"version": "1.99.0",
"private": true,
"scripts": {
"start": "npm run prestart && node --stack-size=10000 ./bin/cluster",
"stop": "kill -2 $(cat tmp/cluster.pid)",
"test": "node ./node_modules/jasmine/bin/jasmine.js test/*Spec.js",
"start": "npm run prestart && $(which node) --stack-size=10000 ./bin/cluster",
"stop": "sh ./scripts/stop_explorer.sh",
"start-instance": "npm run prestart && $(which node) --stack-size=10000 ./bin/instance",
"start-forever": "npm run prestart \"forever\" && $(which node) $(which forever) start ./bin/cluster",
"stop-forever": "$(which node) $(which forever) stop ./bin/cluster",
"start-pm2": "npm run prestart \"pm2\" && $(which pm2) start ./bin/instance -i 0 --node-args=\"--stack-size=10000\"",
"start-pm2-debug": "npm run prestart \"pm2\" && $(which pm2) start ./bin/instance -i 0 --node-args=\"--stack-size=10000\" && $(which pm2) logs",
"stop-pm2": "$(which pm2) stop ./bin/instance",
"test": "$(which node) ./node_modules/jasmine/bin/jasmine.js test/*Spec.js",
"prestart": "sh ./scripts/prestart.sh"
},
"dependencies": {
scripts/prestart.sh
+21
View File
@@ -6,6 +6,27 @@
# npm update
#fi
# Check if an argument was passed into this script
if [ -n "${1}" ]; then
# Determine which argument was passed
case "${1}" in
"pm2")
# Check if pm2 is installed
if [ -z "$(which pm2)" ]; then
# Install pm2
npm install pm2@latest -g
fi
;;
"forever")
# Check if forever is installed
if [ -z "$(which forever)" ]; then
# Install forever
npm install forever -g
fi
;;
esac
fi
# Ensure that selected theme is properly installed
sh ./scripts/sass_theme_reader.sh
scripts/stop_explorer.sh
+32
View File
@@ -0,0 +1,32 @@
#!/bin/bash
validate_port() {
if [ -z "$1" ] || ([ -n "$1" ] && ((echo $1 | egrep -q '^[0-9]+$' && ([ $1 -lt 1 ] || [ $1 -gt 65535 ])) || ! test "$1" -gt 0 2> /dev/null)); then
echo "err"
fi
}
# Check if the settings.json file exists
if [ -f ./settings.json ]; then
# Read the webserver.port value from settings.json file
readonly SERVER_PORT="$(./node_modules/.bin/strip-json-comments settings.json | ./node_modules/.bin/json -q webserver.port)"
# Check if the webserver.port value is a valid port #
if [ -z "$(validate_port $SERVER_PORT)" ]; then
# Check if the server is currently running
if [ -n "$(lsof -t -i:${SERVER_PORT} 2> /dev/null)" ]; then
# Send a SIGINT kill signal to the process that is currently using the explorer's server port
kill -2 $(lsof -t -i:${SERVER_PORT})
# Show shutdown msg
echo "Explorer shutting down... Please wait..."
else
# Webserver is not running
echo "Error: Cannot stop explorer because it is not currently running"
fi
else
# Invalid port number
echo "Error: webserver.port value not found in settings.json. webserver.port value is missing or file is not valid json."
fi
else
# Missing settings file
echo "Error: Cannot find settings.json"
fi
scripts/sync.js
+1 -1
View File
@@ -12,7 +12,7 @@ var database = 'index';
// displays usage and exits
function usage() {
console.log('Usage: scripts/sync.sh /path/to/nodejs [mode]');
console.log('Usage: scripts/sync.sh /path/to/node [mode]');
console.log('');
console.log('Mode: (required)');
console.log('update Updates index from last sync to current block');
settings.json.template
+2 -2
View File
@@ -1045,9 +1045,9 @@
// If you sync using more than 1 parallel task, then historical balance data for wallet addresses can possibly be saved out-of-order and there is currently no workaround for this.
// Therefore, it is recommended to keep this setting to 1 parallel task for now until a proper solution can be procured for the historical balance issue.
"block_parallel_tasks": 1,
// update_timeout: The amount of time to wait (in milliseconds) before moving to the next block or transaction during blockchain sync or reindex (scripts/sync.sh /path/to/nodejs update or scripts/sync.sh /path/to/nodejs reindex)
// update_timeout: The amount of time to wait (in milliseconds) before moving to the next block or transaction during blockchain sync or reindex (scripts/sync.sh /path/to/node update or scripts/sync.sh /path/to/node reindex)
"update_timeout": 10,
// check_timeout: The amount of time to wait (in milliseconds) before moving to the next block or transaction during a check sync (scripts/sync.sh /path/to/nodejs check)
// check_timeout: The amount of time to wait (in milliseconds) before moving to the next block or transaction during a check sync (scripts/sync.sh /path/to/node check)
"check_timeout": 250,
// save_stats_after_sync_blocks: During index syncronization, stats are saved after processing this many blocks to save time by not having to save after each block
"save_stats_after_sync_blocks": 100,