-A number of functions have been rewritten to be more optimized and faster: calculate_total, is_unique, convert_to_satoshi, get_input_addresses, processVoutAddresses, prepare_vout, prepare_vin
-Txes are now written to database via bulk writes which helps improve the sync speed and also controls memory usage with batching to write data once a certain threshold is reached
-update_address function changed to update_addresses since it now bulk writes the addresses in batches to improve sync speed and also controls memory usage with batching to write data once a certain threshold is reached
-The syncLoop function has been completely removed from the project and replaced with async library loops or even normal "for" loops in some cases which greatly improves sync speeds over large batches of data
-Fixed an issue with the flattened count of txes that is saved to the coinstats collection which could save incorrectly when using more than 1 thread
-Fixed an issue with the block sync which caused an unwanted delay when syncing less blocks than the amount of threads used to sync the data
-Fixed an issue with vout data processing that could sometimes populate data out of order
-Added a new sync.batch_size setting used to determine how many records (txes, addresses, addresstxes) should be saved in a single database transaction
-Added a new wait_for_bulk_database_save setting used to increase the block sync speed at the cost of not returning any error msgs for data that failed to save
-get_input_addresses function no longer returns in the exports section of the explorer.js file since it is only referenced in that file
-Updated explorerspec tests to use the newest function changes for any tests that needed to be updated
Special thanks to Karzo from Pepecoin for help with the bulkwrite code changes!
-The "Maximum call stack size exceeded" error is now handled internally by the block sync script in a way which will capture the error and re-launch the sync using a larger stack size and have the sync resume from where it left off. If the re-launch still doesn't have enough memory it will continue re-launching with more and more memory until the sync can finish without errors and then it will return to sync with a lower memory footprint for future syncs
-Added a new option for sync.elastic_stack_size which is used to determine how much memory should be used to increase the stack size for the block sync after encountering the "Maximum call stack size exceeded" error
-Fixed an issue with the block sync when using more than 1 thread that could sometimes cause the flattened txes value in the coinstats database to be written incorrectly (Use `npm run reindex-txcount` to fix this issue without needing to reindex the entire database)
-Updated the benchmark script so that it can also benefit from being able to capture the "Maximum call stack size exceeded" error even though the timing will be off so it outputs a new warning in that scenario which instructs to run the benchmark again with a higher stack size to properly capture the benchmark time
-Removed the "Maximum call stack size exceeded" error notes from the "Known Issues" section of the README
-Plugins can now be enabled via settings.json after dropping the plugin files into the new plugins directory
-Enabling plugins will allow extending the normal functionality of the explorer with new database collections, menus, pages and apis + open up a new url for data to be sent from the plugin to the explorer
-A new plugins section was added to the settings with a definition for the generic-snapshots plugin
-Locale strings are now loaded and shared out via the settings so there is generally no more need to explicitly include the locale.js file
-The locale object has been updated to localization within the explorer
-A number of new locale strings have been added and their values replaced with the locale string within the explorer
-Added plugin support verbiage and a link to the generic-snapshots crowdfunding task to the README
-This fix prevents an error from being thrown that crashes the explorer when trying to use a masternode count that is a sinlge number. A single number masternode count is not valid or usable since it cannot differentiate the number of good and bad nodes, but the explorer will no longer crash when given this data
-Added support for a couple masternode list and count formats that do not have a separate fieldname for each value
-Masternode grid columns are now shown or hidden based on whether there is any data
-The previous market price calculation setting was hardcoded to only display market and USD prices for a single exchange and trading pair which was not very accurate for coins listed on multiple exchanges or with multiple trading pairs. The new default is to average the market prices for all supported exchanges and trading pairs
-The coingecko market price option was added to allow fetching the market price directly from the coingecko api instead of calculating it via supported exchanges known to the explorer
-Added a new root setting option for default_coingecko_ids which allows presetting symbols to their associated internal coingecko id to help prevent matching to the wrong currency with same symbol via coingecko api calls
-Fixed an issue where the explorer would fail to start with an enabled exchange that had no defined trading pairs
-Javascript in the layout.pug has been moved around in an attempt to only load code into the client based on certain settings such as the type of menu (side or top) in order to slim down the loading of pages by not including unnecessary code
-Fixed a bug on the richlist page where the last updated date and logo spinning animation would fail to load if the distribution chart was disabled in the settings
-The .ext/getsummary api has been modified to only return the connection and block counts when necessary
-The getmasternodecount rpc api will no longer fetch data unless it is enabled in settings
-Mongoose and mongodb dependencies have been updated to the most recent versions in package.json
-Updated the project for use with the newest mongoose major version which required fixing many breaking changes to a number of files in the explorer
-Removed the db.find_address_tx() function as it is unused
NOTE: This update requires mongoose 6.0.10 or higher. If you are getting errors running the explorer after upgrading, it is recommended to run an 'npm update' or manually update mongoose to a more modern version which should fix the problem
-The block sync will now remove orphaned block data from the txes collecton, address balances/sent/received data as well as addresstx data and now stores limited info in a new orphans collection
-Added a new optional page for viewing orphaned block data
-The coinbase address now updates sent totals from POS rewards and other transactions that have vout but no vin addresses
-Block and transaction pages now display a warning when viewing an orphaned block or tx
-Added a couple new fields to the coinstats collection for tracking orphaned blocks
-Added new locale strings for the orphaned block feature
-Some blockchains store certain vin or vout addresses within an array instead of as a simple string which would cause them to be read improperly by the explorer and result in transactions that were not attributed to the proper address. It also caused multiple inputs/outputs from the same address in a single transaction to not be grouped together like other transactions in the explorer
-If the explorer is installed to a directory that has a space character somewhere in the absolute path, the locking mechanism will not work correctly on non-windows computers and will cause undesired behavior without this fix
-The update_explorer.js script has been improved with better spacing and the ability to restart the explorer automatically to ensure new changes take effect immidiately (works with npm start, pm2 and forever)
-The code to compile scss to css has been moved from the prestart script into its own compile_css.js script which is now called from the update-explorer.js script to apply css changes after update
-The cluster code now handles a custom restart msg which is used to restart the explorer from the update explorer process
-Pm2 and Forever are now referenced by the name 'explorer' instead of ./bin/instance or ./bin/cluster [SEE IMPORTANT NOTE BELOW]
-Added reload/restart scripts to the package.json for pm2 and forever
-Pm2 and forever now write a pid file to the tmp directory when started. NOTE: Forever is now started from the prestart script due to a bug in forever that prevents the pid from being written to a different directory without the absolute path
-Fixed a bug which caused the prestart script to be run twice when starting the explorer with `npm start`
-The cluster code now accepts a numeric argument to force a specific number of instances to be loaded
-The `npm run start-instance` cmd now loads using the cluster code with a single instance
-The is_locked function now accepts an optional 'silent' argument to prevent displaying msgs while checking for pid/lock files
-Added some process.exit statements to the stop_explorer.js file
-Updated the README with cmd changes from package.json and updated description of the "Update Explorer Script"
IMPORTANT NOTE: It is strongly recommended to stop the explorer before performing this update. If the explorer is running while you perform this update, you will need to stop and restart the explorer for this update to fully take effect. Because of the changes in this commit, stopping the explorer using the built-in pm2 and/or forever stop cmds will not work and you will need to type out the full stop cmd this one time only, and going forward from now on you should no longer need to even stop the explorer for any update as it is now built into the update cmd.
If running using pm2 and you cannot stop the explorer, you can use stop using the following full cmd syntax:
Windows:
pm2 stop ./bin/instance
Linux and other OS's:
node node_modules/pm2/bin/pm2 stop ./bin/instance
If running using forever and you cannot stop the explorer, you can use stop using the following full cmd syntax:
All OS's:
node node_modules/forever/bin/forever stop ./bin/cluster
-freegeoip.app now requires an api key so the peer sync has been updated to use reallyfreegeoip.org instead
-Added extra error checking to the peer sync so that it will error out when the geolocation service does not return a proper object
-Added locks to all sync processes (blocks, markets, peers, masternodes) as well as "create backup", "restore backup" and "delete database" functions. This helps prevent problems with syncing data while a backup is in progress for example
-The code to initialize certain database collections on startup was moved into database.js and is now called from restore_backup.js and delete_database.js. This effectively allows the database to be deleted or restored to a completely different backup while the explorer is still running
-Lock functions (create_lock, remove_lock, is_locked) were moved into explorer.js for better reusability and rewriten to be synchronous
-is_locked function now accepts an array of lock files to be able to check for multiple locks in a single call
-remove_sync_message() function was moved into database.js so that restore_backup.js and delete_database.js can also check for and remove the sync msg if it exists
-Useful Scripts section updated in the README to make it clear that the explorer no longer needs to be stopped for these scripts to be run
-Most if not all log messages now start with a capitlal letter
-Adds a tx_type field to the tx model which is typically null for "normal" transaction types, but can also display 'p2pk' for bitcoin txes which require addtional encoding to reveal the P2PKH address as well as 'zksnarks' for transactions with hidden sender or receiver data
-Additional fixes for how data is displayed when a valid wallet address cannot be found
-Includes some small updates to how zksnarks transactions display hidden sender/receiver data
-Replace tabs with double-spaces
-Betting spacing and lining up of code functions
-Add missing semi-colons
-Remove extra characters and spaces where applicable
-Remove commented-out code fragments
-Add missing 2021 date to LICENSE
-Small touchups and other misc nitpickings
-Restructured the settings.json.template and settings.js files with better groupings
-Added better comments for improved explanations of all settings
-Better handling of page length options on all datatables; Page length options are now configurable and also only display options based on limits set by certain items_per_page and max_items_per_query settings
-Markets have been extended to support unlimited trading pairs for each exchange
-Added coin_symbol and pair_symbol to the markets schema to support multiple trading pairs
-Added a burned field to the richlist schema for tracking burned coin addresses
-Added a locale string for the new richlist wealth distribution "Top 1-100 Total" table row
-Updated a couple locale strings that were incomplete or needed fixing (api_getblockhash and api_getnextrewardwhenstr)
-Added a new css class to remove some redundant inline styles for the richlist wealth distribution color boxes
-Richlist page now allows better separation of the top list(s) and the wealth distribution table and chart (you can now hide or show sections as desired)
-Richlist page now allows for better burned coin support (Hide burned coin addresses from lists and totals even when the burned coin amounts are still included in the total coin supply value)
-Block page now only displays the raw block link if the api_page.public_apis.rpc.getblock.enabled settings is set to true
-Transaction page now only displays the raw tx link if the api_page.public_apis.rpc.getrawtransaction.enabled setting is set to true
-Rate limiting class has been made global and is now used to limit market requests
-getnetworkhashps rpc call now returns a '-' if shared_pages.show_hashrate is not set to true
-Searching for unsycned blocks/txs no longer saves the data locally but instead still shows the data on screen. This helps prevent syncing data out-of-order (This eliminates the need for db_index.pid which has been removed)
-Coin supply is now always taken from the stats collection database instead of from the wallet via rpc command in all cases except when syncing
-Lots of misc fixes and code cleanup changes
-List of changed settings:
-title -> shared_pages.page_title
-coin -> coin.name
-symbol -> coin.symbol
-logo -> shared_pages.logo
-headerlogo -> shared_pages.page_header.home_link_logo
-favicon -> shared_pages.favicon
-homelink -> shared_pages.page_header.home_link
-logoheight -> shared_pages.page_header.home_link_logo_height
-sticky_header -> shared_pages.page_header.sticky_header
-sticky_footer -> shared_pages.page_footer.sticky_footer
-footer_height_desktop -> shared_pages.page_footer.footer_height_desktop
-footer_height_tablet -> shared_pages.page_footer.footer_height_tablet
-footer_height_mobile -> shared_pages.page_footer.footer_height_mobile
-social_link_percent_height_desktop -> shared_pages.page_footer.social_link_percent_height_desktop
-social_link_percent_height_tablet -> shared_pages.page_footer.social_link_percent_height_tablet
-social_link_percent_height_mobile -> shared_pages.page_footer.social_link_percent_height_mobile
-theme -> shared_pages.theme
-port -> webserver.port
-update_timeout -> sync.update_timeout
-check_timeout -> sync.check_timeout
-block_parallel_tasks -> sync.block_parallel_tasks
-use_rpc -> api_cmds.use_rpc
-confirmations -> shared_pages.confirmations
-display.api -> api_page.enabled
-display.markets -> markets_page.enabled
-display.richlist -> richlist_page.enabled
-display.search -> shared_pages.page_header.show_search
-display.movement -> movement_page.enabled
-display.network -> network_page.enabled
-display.masternodes -> masternodes_page.enabled
-display.claim_address -> claim_address_page.enabled
-display.claim_address_header_menu -> claim_address_page.show_header_menu
-display.page_header_bgcolor -> shared_pages.page_header.bgcolor
-display.page_footer_bgcolor -> shared_pages.page_footer.bgcolor
-display.table_header_bgcolor -> shared_pages.table_header_bgcolor
-display.networkpnl -> shared_pages.page_header.panels.network_panel.display_order
-display.difficultypnl -> shared_pages.page_header.panels.difficulty_panel.display_order
-display.masternodespnl -> shared_pages.page_header.panels.masternodes_panel.display_order
-display.coinsupplypnl -> shared_pages.page_header.panels.coin_supply_panel.display_order
-display.pricepnl -> shared_pages.page_header.panels.price_panel.display_order
-display.marketcappnl -> shared_pages.page_header.panels.market_cap_panel.display_order
-display.logopnl -> shared_pages.page_header.panels.logo_panel.display_order
-index.show_last_updated -> index_page.show_last_updated
-index.show_hashrate -> shared_pages.show_hashrate
-index.difficulty -> shared_pages.difficulty
-index.last_txs -> api_page.public_apis.ext.getlasttxs.max_items_per_query
-index.txs_per_page -> index_page.transaction_table.items_per_page (for index page) AND movement_page.movement_table.items_per_page (for movement page)
-reward_page.show_last_updated -> blockchain_specific.heavycoin.reward_page.show_last_updated
-api.blockindex -> api_page.sample_data.blockindex
-api.blockhash -> api_page.sample_data.blockhash
-api.txhash -> api_page.sample_data.txhash
-api.address -> api_page.sample_data.address
-markets.exchange -> markets_page.default_exchange.trading_pair
-markets.default -> markets_page.default_exchange.exchange_name
-markets.market_dropdown_menu -> markets_page.show_market_dropdown_menu
-markets.market_select_visible -> markets_page.show_market_select
-richlist.distribution -> richlist_page.wealth_distribution.show_distribution_chart
-richlist.received -> richlist_page.show_received_coins
-richlist.balance -> richlist_page.show_current_balance
-movement.min_amount -> movement_page.movement_table.min_amount
-movement.low_flag -> movement_page.movement_table.low_warning_flag
-movement.high_flag -> movement_page.movement_table.high_warning_flag
-social_links -> shared_pages.page_footer.social_links
-social_links[x].image_url -> shared_pages.page_footer.social_links[x].image_path
-genesis_tx -> transaction_page.genesis_tx
-genesis_block -> block_page.genesis_block
-heavy -> blockchain_specific.heavycoin.enabled
-save_stats_after_sync_blocks -> sync.save_stats_after_sync_blocks
-txcount -> api_page.public_apis.ext.getaddresstxs.max_items_per_query
-txcount_per_page -> address_page.history_table.items_per_page
-show_sent_received -> address_page.show_sent_received
-supply -> sync.supply
-nethash -> shared_pages.page_header.panels.network_panel.nethash
-nethash_units -> shared_pages.page_header.panels.network_panel.nethash_units
-usecors -> webserver.cors.enabled
-corsorigin -> webserver.cors.corsorigin
-burned_coins -> richlist_page.burned_coins.addresses
-public_api.rpc.getdifficulty -> api_page.public_apis.rpc.getdifficulty.enabled
-public_api.rpc.getconnectioncount -> api_page.public_apis.rpc.getconnectioncount.enabled
-public_api.rpc.getblockcount -> api_page.public_apis.rpc.getblockcount.enabled
-public_api.rpc.getblockhash -> api_page.public_apis.rpc.getblockhash.enabled
-public_api.rpc.getblock -> api_page.public_apis.rpc.getblock.enabled
-public_api.rpc.getrawtransaction -> api_page.public_apis.rpc.getrawtransaction.enabled
-public_api.rpc.getnetworkhashps -> api_page.public_apis.rpc.getnetworkhashps.enabled
-public_api.rpc.getvotelist -> api_page.public_apis.rpc.getvotelist.enabled
-public_api.rpc.getmasternodecount -> api_page.public_apis.rpc.getmasternodecount.enabled
-public_api.rpc.getmaxmoney -> blockchain_specific.heavycoin.public_apis.getmaxmoney.enabled
-public_api.rpc.getmaxvote -> blockchain_specific.heavycoin.public_apis.getmaxvote.enabled
-public_api.rpc.getvote -> blockchain_specific.heavycoin.public_apis.getvote.enabled
-public_api.rpc.getphase -> blockchain_specific.heavycoin.public_apis.getphase.enabled
-public_api.rpc.getreward -> blockchain_specific.heavycoin.public_apis.getreward.enabled
-public_api.rpc.getsupply -> blockchain_specific.heavycoin.public_apis.getsupply.enabled
-public_api.rpc.getnextrewardestimate -> blockchain_specific.heavycoin.public_apis.getnextrewardestimate.enabled
-public_api.rpc.getnextrewardwhenstr -> blockchain_specific.heavycoin.public_apis.getnextrewardwhenstr.enabled
-public_api.ext.getmoneysupply -> api_page.public_apis.ext.getmoneysupply.enabled
-public_api.ext.getdistribution -> api_page.public_apis.ext.getdistribution.enabled
-public_api.ext.getaddress -> api_page.public_apis.ext.getaddress.enabled
-public_api.ext.getaddresstxs -> api_page.public_apis.ext.getaddresstxs.enabled
-public_api.ext.gettx -> api_page.public_apis.ext.gettx.enabled
-public_api.ext.getbalance -> api_page.public_apis.ext.getbalance.enabled
-public_api.ext.getlasttxs -> api_page.public_apis.ext.getlasttxs.enabled
-public_api.ext.getcurrentprice -> api_page.public_apis.ext.getcurrentprice.enabled
-public_api.ext.getbasicstats -> api_page.public_apis.ext.getbasicstats.enabled
-public_api.ext.getsummary -> api_page.public_apis.ext.getsummary.enabled
-public_api.ext.getnetworkpeers -> api_page.public_apis.ext.getnetworkpeers.enabled
-public_api.ext.getmasternodelist -> api_page.public_apis.ext.getmasternodelist.enabled
-public_api.ext.getmasternoderewards -> api_page.public_apis.ext.getmasternoderewards.enabled
-public_api.ext.getmasternoderewardstotal -> api_page.public_apis.ext.getmasternoderewardstotal.enabled
-api_cmds.heavies.getmaxmoney -> blockchain_specific.heavycoin.api_cmds.getmaxmoney
-api_cmds.heavies.getmaxvote -> blockchain_specific.heavycoin.api_cmds.getmaxvote
-api_cmds.heavies.getvote -> blockchain_specific.heavycoin.api_cmds.getvote
-api_cmds.heavies.getphase -> blockchain_specific.heavycoin.api_cmds.getphase
-api_cmds.heavies.getreward -> blockchain_specific.heavycoin.api_cmds.getreward
-api_cmds.heavies.getnextrewardestimate -> blockchain_specific.heavycoin.api_cmds.getnextrewardestimate
-api_cmds.heavies.getnextrewardwhenstr -> blockchain_specific.heavycoin.api_cmds.getnextrewardwhenstr
-api_cmds.heavies.getsupply -> blockchain_specific.heavycoin.api_cmds.getsupply
-List of new settings:
-shared_pages.page_header.panels.network_panel.enabled: allow enabling/disabling of the network panel
-shared_pages.page_header.panels.difficulty_panel.enabled: allow enabling/disabling of the difficulty panel
-shared_pages.page_header.panels.masternodes_panel.enabled: allow enabling/disabling of the masternodes panel
-shared_pages.page_header.panels.coin_supply_panel.enabled: allow enabling/disabling of the coin supply panel
-shared_pages.page_header.panels.price_panel.enabled: allow enabling/disabling of the price panel
-shared_pages.page_header.panels.market_cap_panel.enabled: allow enabling/disabling of the market cap panel
-shared_pages.page_header.panels.logo_panel.enabled: allow enabling/disabling of the logo panel
-index_page.transaction_table.page_length_options: specify the page length options that determine how many items/records to display in the table at any given time
-index_page.transaction_table.reload_table_seconds: the time in seconds to automatically reload the table data from the server
-address_page.history_table.page_length_options: specify the page length options that determine how many items/records to display in the table at any given time
-masternodes_page.masternode_table.page_length_options: specify the page length options that determine how many items/records to display in the table at any given time
-masternodes_page.masternode_table.items_per_page: the default amount of items/records to display in the table at any given time
-movement_page.movement_table.page_length_options: specify the page length options that determine how many items/records to display in the table at any given time
-movement_page.movement_table.reload_table_seconds: the time in seconds to automatically reload the table data from the server
-network_page.connections_table.page_length_options: specify the page length options that determine how many items/records to display in the table at any given time
-network_page.connections_table.items_per_page: the default amount of items/records to display in the table at any given time
-network_page.addnodes_table.page_length_options: specify the page length options that determine how many items/records to display in the table at any given time
-network_page.addnodes_table.items_per_page: the default amount of items/records to display in the table at any given time
-network_page.onetry_table.page_length_options: specify the page length options that determine how many items/records to display in the table at any given time
-network_page.onetry_table.items_per_page: the default amount of items/records to display in the table at any given time
-richlist_page.wealth_distribution.show_distribution_table: show/hide the wealth distribution summary table
-richlist_page.wealth_distribution.colors: a list of html color codes to represent the top 100 groupings in the wealth distribution table and pie chart
-richlist_page.burned_coins.include_burned_coins_in_distribution: determine whether to include burned coins in the wealth distribution section or not
-markets_page.exchanges.altmarkets.enabled: enable/disable the altmarkets exchange
-markets_page.exchanges.altmarkets.trading_pairs: a list of market trading pair symbols
-markets_page.exchanges.bittrex.enabled: enable/disable the bittrex exchange
-markets_page.exchanges.bittrex.trading_pairs: a list of market trading pair symbols
-markets_page.exchanges.bleutrade.enabled: enable/disable the bleutrade exchange
-markets_page.exchanges.bleutrade.trading_pairs: a list of market trading pair symbols
-markets_page.exchanges.crex.enabled: enable/disable the crex exchange
-markets_page.exchanges.crex.trading_pairs: a list of market trading pair symbols
-markets_page.exchanges.poloniex.enabled: enable/disable the poloniex exchange
-markets_page.exchanges.poloniex.trading_pairs: a list of market trading pair symbols
-markets_page.exchanges.stex.enabled: enable/disable the stex exchange
-markets_page.exchanges.stex.trading_pairs: a list of market trading pair symbols
-markets_page.exchanges.yobit.enabled: enable/disable the yobit exchange
-markets_page.exchanges.yobit.trading_pairs: a list of market trading pair symbols
-claim_address_page.enable_bad_word_filter: enable/disable the "bad word" filter for claimed addresses, so that trying to claim an address with a bad word will fail
-sync.show_sync_msg_when_syncing_more_than_blocks: show the sync msg at the top of all pages during index syncronization if there are more than this many blocks to process
-labels[x].enabled: allow enabling/disabling of specific wallet address labels
-blockchain_specific.heavycoin.reward_page.enabled: enable/disable the reward page
-blockchain_specific.zksnarks.enabled: enable/disable Zcash zk-SNARKs private transaction support (WIP - 90% complete)
-List of deleted settings
-address: unnecessary setting has been replaced by dynamic values via http request
-markets.coin: replaced by a richer set of settings that allow choosing the coin for each market
-markets.enabled -> replaced by a richer set of settings that allow enabling/disabling each market specifically
-lock_during_index: no longer possible to save unsynced blocks/txs via search so this settings is now obsolete/unused
-Added a new "Masternodes" page which displays the current list of masternodes on the network
-/api/getmasternodelist is no longer publicly accessible and has been replaced by /ext/getmasternodelist which returns the masternode list from local collection instead of directly from wallet
-Added new masternode sync options to sync.js and sync.sh
-Added new masternodes_last_updated field to the Stats collection
-Updated delete_database.sh and restore_backup.sh to include support for the new masternodes collection
-Network header menu icon changed to allow the new Masternodes menu item to use the old network icon
Joe Uhren