-An optional "Extracted By" column can now be added to the homepage tx data, the block page, tx page and/or in the /ext/getlasttxs api
-Added 4 new settings to allow displaying the "Extracted By" data on the homepage, block page, transaction page and/or in the /ext/getlasttxs api
-Fixed an issue with the get_txs function where it wasn't properly searching by txid
-The rl_labels.pug file has been updated to consolidate similar code without being duplicated
-Updated the README with new verbiage for the extracted by data
-A number of functions have been rewritten to be more optimized and faster: calculate_total, is_unique, convert_to_satoshi, get_input_addresses, processVoutAddresses, prepare_vout, prepare_vin
-Txes are now written to database via bulk writes which helps improve the sync speed and also controls memory usage with batching to write data once a certain threshold is reached
-update_address function changed to update_addresses since it now bulk writes the addresses in batches to improve sync speed and also controls memory usage with batching to write data once a certain threshold is reached
-The syncLoop function has been completely removed from the project and replaced with async library loops or even normal "for" loops in some cases which greatly improves sync speeds over large batches of data
-Fixed an issue with the flattened count of txes that is saved to the coinstats collection which could save incorrectly when using more than 1 thread
-Fixed an issue with the block sync which caused an unwanted delay when syncing less blocks than the amount of threads used to sync the data
-Fixed an issue with vout data processing that could sometimes populate data out of order
-Added a new sync.batch_size setting used to determine how many records (txes, addresses, addresstxes) should be saved in a single database transaction
-Added a new wait_for_bulk_database_save setting used to increase the block sync speed at the cost of not returning any error msgs for data that failed to save
-get_input_addresses function no longer returns in the exports section of the explorer.js file since it is only referenced in that file
-Updated explorerspec tests to use the newest function changes for any tests that needed to be updated
Special thanks to Karzo from Pepecoin for help with the bulkwrite code changes!
-The "Maximum call stack size exceeded" error is now handled internally by the block sync script in a way which will capture the error and re-launch the sync using a larger stack size and have the sync resume from where it left off. If the re-launch still doesn't have enough memory it will continue re-launching with more and more memory until the sync can finish without errors and then it will return to sync with a lower memory footprint for future syncs
-Added a new option for sync.elastic_stack_size which is used to determine how much memory should be used to increase the stack size for the block sync after encountering the "Maximum call stack size exceeded" error
-Fixed an issue with the block sync when using more than 1 thread that could sometimes cause the flattened txes value in the coinstats database to be written incorrectly (Use `npm run reindex-txcount` to fix this issue without needing to reindex the entire database)
-Updated the benchmark script so that it can also benefit from being able to capture the "Maximum call stack size exceeded" error even though the timing will be off so it outputs a new warning in that scenario which instructs to run the benchmark again with a higher stack size to properly capture the benchmark time
-Removed the "Maximum call stack size exceeded" error notes from the "Known Issues" section of the README
-The block_parallel_tasks feature has been improved and fixed so that it is now safe to cancel (Ctrl+C) or kill (kill cmd not kill -9) the task and resume the sync later without missing transactions. The new block_parallel_tasks default is 8 threads which seems to be the sweet spot for any type of cpu
-Numerous improvements to the benchamark script to utilize new benchmark settings, auto-add credentials to the benchmark database, reuse the same sync code as the regular block sync instead of using a copy of the code and more
-Added a new cmd to run the benchmark script `npm run benchmark`
-README updated to include the new benchmark script instrutions + include multi-threaded sync as a feature
-Chart.js has been updated to v4.4.7
-The chartjs-plugin-crosshair chart plugin has been updated to v2.0.5 via a forked version that has a working sync feature which is now available as a new setting option for use with the network charts
-Added a new max_hours setting to display chart data for a certain number of hours instead of a fixed set of records which can help reveal holes in the sync process for the explorer and/or blockchain
-Added a new timestamp field to the network history collection for use with the max_hours setting chart data
-Added a number of new network chart settings to control display of the chart title, legend, a new vertical block line option, chart height, an option to force 2 charts to appear on their own row or beside each other, and an option to force a chart to take up all available space in the chart box without extra padding
-Added a new dependency chartjs-plugin-annotation v3.1.0 to display block lines in new hourly charts
-Plugins can now be enabled via settings.json after dropping the plugin files into the new plugins directory
-Enabling plugins will allow extending the normal functionality of the explorer with new database collections, menus, pages and apis + open up a new url for data to be sent from the plugin to the explorer
-A new plugins section was added to the settings with a definition for the generic-snapshots plugin
-Locale strings are now loaded and shared out via the settings so there is generally no more need to explicitly include the locale.js file
-The locale object has been updated to localization within the explorer
-A number of new locale strings have been added and their values replaced with the locale string within the explorer
-Added plugin support verbiage and a link to the generic-snapshots crowdfunding task to the README
-Supported captchas include Google reCaptcha v3 (score-based), Google reCaptcha v2 (checkbox and invisible) and hCaptcha ("Always Challenge" mode)
-Captcha options are global to the explorer even though the only form submission page is the "Claim Address" feature which takes full advantage of the new captcha options
-New settings allow reading of the hash algorithm used to mine a particular block for coins that support this feature and have the algorithm data stored in the raw block data
-An "Algorithm" column has been added to the block page and main transaction homepage when multi-algo data is enabled
-The /ext/getlasttxs api will now return the hash algorithm if reading of the multi-algo data is enabled
-As of Feb 2024 the free "keyless" coingecko api will be deprecated and will likely stop working. There is a free demo api key that can be used which can now be plugged into the explorer settings to allow the coingecko api to continue working. Read more here: https://support.coingecko.com/hc/en-us/articles/21880397454233
-The previous market price calculation setting was hardcoded to only display market and USD prices for a single exchange and trading pair which was not very accurate for coins listed on multiple exchanges or with multiple trading pairs. The new default is to average the market prices for all supported exchanges and trading pairs
-The coingecko market price option was added to allow fetching the market price directly from the coingecko api instead of calculating it via supported exchanges known to the explorer
-Added a new root setting option for default_coingecko_ids which allows presetting symbols to their associated internal coingecko id to help prevent matching to the wrong currency with same symbol via coingecko api calls
-Fixed an issue where the explorer would fail to start with an enabled exchange that had no defined trading pairs
-jQuery has been updated to v3.7.1
-Luxon has been updated to v3.4.3. Also updated the link to the documentation showing how to format dates in the settings.json.template and settings.js
-Datatables has been updated to v1.13.6
-Font Awesome has been updated to v6.4.2 and all icons in the project have been updated to use the newest v6.x codes
-OverlayScrollbars has been updated to v2.3.2
-flag-icons (previously called flag-icon-css) has been updated to v6.11.1
-Updated the main style.scss file with better color support for the new datatable loading animation color
-Exor theme has been updated to v3.0.2 with a small fix to support the new datatable loading animation color
-Added a new "powered_by_text" option to the page_footer settings which can be used to override the text in the middle of the footer without the need to modify the source code
-The word "Explorer" was removed from the powered by text for new installs
-The block sync will now remove orphaned block data from the txes collecton, address balances/sent/received data as well as addresstx data and now stores limited info in a new orphans collection
-Added a new optional page for viewing orphaned block data
-The coinbase address now updates sent totals from POS rewards and other transactions that have vout but no vin addresses
-Block and transaction pages now display a warning when viewing an orphaned block or tx
-Added a couple new fields to the coinstats collection for tracking orphaned blocks
-Added new locale strings for the orphaned block feature
-The port_filter settings for all 3 network page tables now support a -1 value which will completely hide the port from the associated table. If there are/were multiple ip addresses in the peers list with connections from different ports, this new option will consolidate those multiple records into a single entry since the port is no longer visible
-A new side menu option was added for displaying menu items vertically on the left-hand side of the page
-Added a new script called OverlayScrollbars v1.13.3 for displaying custom scrollbars on the mobile menu (both top and side menu) in the event that they extend beyond the visible screen
-Updated the market menu dropdown colors to better match the base menu colors (default, light and dark colors only)
-Updated the Exor theme to v3.0.1 with a few small fixes for displaying menu items in the side menu and updated market dropdown colors
-Updated how the richlist pie chart is sized to prevent it from creating unnecessary scrollbars because of the side menu
-Search field and button are now hidden in the top menu when the mobile menu is expanded to make it easier to scroll through menu items
-Added a tooltip title to the top menu search and menu toggler buttons
-Added a new port_filter setting to the 3 network_page tables which allows showing only results from peers on the selected port
-Added a new hide_protocols setting to the 3 network_page tables which allows hiding results from peers on the selected protocols
-Added a Port column to the data table on the Network page Connections tab
-The data table on the Network page Connections tab is now sortable
-The peer sync now refreshes data for peers that existed since last sync
-Adding and updating peers now displays the port # in the log output
-The /ext/getnetworkpeers api is now sorted by ip4/ip6, address and port which also means all data on the Network page is sorted this way by default as well
-The /ext/getnetworkpeers api no longer requires the internal argument for calls originating from the Network page
-The Network page now only makes a single call to the /ext/getnetworkpeers api instead of 3 calls
-The find_peer and drop_peer functions now requiresa port to distinguish between multiple connections to the same peer on different ports
-This exchange has 2 separate sites using the same api, so new functionality was added to the explorer to handle alt markets like this
-market files now allow for new field names: "market_name_alt" was added for the alternate market name, "market_logo_alt" was added for displaying an alternate logo, a "market_url" function was added that will return a url string to replace the {url_prefix} code in the market_url_template and an "isAlt" function was added to determine if the alt name/logo should be used
-A new freiexchange entry was added to the markets_page.exchanges setting
-Updated the README with the updated market info
-Difficulty chart would fail to populate if the shared_pages.difficulty setting was 'Hybrid' and is now handled properly. The chart now supports 2 datasets: POW and POS
-NetworkHistory.difficulty field was renamed to difficulty_pow and a new difficulty_pos field was added
-The shared_pages.page_header.network_charts.difficulty_chart.line_color setting was renamed to pow_line_color
-The shared_pages.page_header.network_charts.difficulty_chart.fill_color setting was renamed to pow_fill_color
NOTE: If you have existing NetworkHistory data from before this update, it will be automatically assumed to be POW. If it was actualy POS, you will need to either manually fix the data in mongo or simply wait a couple hours while syncing to eventually push the old data out of the chart
-Instead of using a single file for the favicon, there are now 4 size options for a more modern favicon experience as inspired by this article: https://www.emergeinteractive.com/insights/detail/The-Essentials-of-FavIcons/
-Old default favicon file has been replaced with 4 different sized pngs
-Not all sizes need to be set. If any of the new settings are left blank or the filename does not exist, that favicon size will be skipped
-All shell scripts have been removed and replaced with javascript equivalents which allows for better platform independence
-All scripts have been improved over the older shell scripts to be more functional and dynamic
-Updated all applicable cmds in the package.json to target the new js scripts + added the backup, restore and delete database functions to the list
-Removed the json and strip-json-comments-cli packages as they are no longer needed
-Added a new package realine-sync
-Updated cluster code with better Windows support when shutting down the explorer
-Backup and Restore scripts now use mongo's own encryption instead of tar.gz by default. Older tar.gz backups can still be restored as long as the tar.gz suffix is explicitly added
-Backup and Restore scripts now support connecting to remote database based on the mongo details in settings.json
-Updated README to include a new line for the 'platform independence' feature, updated script cmds and notes about certain steps being Linux only
-Added new setting 'show_exchange_url' which hides/shows the exchange link
-Updated all exchanges with a trade url template which can be used to automatically determine the correct url for each trading pair
-Hashrate chart is populated from get_hashrate
-Difficulty chart is populated from get_difficulty
-Added a networkhistories collection to hold the new network data
-Index sync and resync functions now check for and update network history data at the end of the sync process
-restore_backup.sh and delete_database.sh scripts now drop the new networkhistories collection
-Added a new javascript library chartjs-plugin-crosshair which adds crosshair functionality to the new chart.js line charts
-Added new settings to settings.json for controlling things like collecting network_history data, independently enabling/disabling the network charts and changing colors of various aspects of the charts
-Updated README to introduce the network charts and added a line for the chartjs-plugin-crosshair library
-Bootstrap was upgraded to latest version (v5.1.3) which required tons of changes to the UI. Many little UI problems have been resolved
-Bootswatchs themes have all been upgraded to support bootstrap v5.1.3 + 4 new themes were added (Morph, Quartz, Vapor and Zephyr)
-Exor theme upgraded to v3.0.0 with support for bootstrap v5.1.3
-Many layout and UI improvements such as more consistent hover effects on most tables and more readable text across all themes
-Market sync now sets the last USD price based on the default market pair even if it is not BTC
-Removed the coindesk api which was only really useful for getting the USD price of BTC
-Added a couple coingecko apis which are used to get the current USD value of the default market pair
-Updated the readme, settings file comments and info/api page with better verbiage regarding the market price which was previously only measured in BTC
-More graceful shutdown of node cluster on 'npm stop' with better cleanup of resources on exit
-Added new stop_explorer.sh script which looks up the explorer port # via settings file and closes the application running on that port # instead of saving and killing the process by pid as it did before
-Added support for pm2 and forever using 'npm run start-pm2' and 'npm run start-forever' respectively
-pm2 is automatically installed when starting with 'npm run start-pm2' if it is not already installed
-forever is automatically installed when starting with 'npm run start-forever' if it is not already installed
-Updated existing npm commands in package.json by replacing hardcoded 'node' with '$(which node)'
-/path/to/nodejs changed to /path/to/node in the /settings.json.template, /lib/settings.js and /scripts/sync.js files
-README updates:
-Added a new 'Start/Stop the Explorer' section
-Added PM2 instructions to the 'Start/Stop the Explorer' section
-Moved Start/Stop Explorer instructions to the 'Start/Stop the Explorer' section
-Moved Forever instructions to the 'Start/Stop the Explorer' section
-/path/to/nodejs changed to /path/to/node
-Some additional small misc fixes
Joe Uhren