ci: add linter task "ban unicode" to protect against malicious unicode

This script scans the whole codebase for unicode characters and
errors if it finds any, unless the character is specifically whitelisted.

The motivation is to protect against homoglyph attacks, invisible unicode characters,
bidirectional and other control characters, and other malicious unicode usage.

Given that we mostly expect to use ASCII characters in the source code,
the most robust and generic fix seems to be to just ban all unicode usage.

see https://trojansource.codes/ :
> Compilers, interpreters, and build pipelines supporting Unicode should throw errors or warnings
> for unterminated bidirectional control characters in comments or string literals,
> and for identifiers with mixed-script confusable characters.
> Language specifications should formally disallow unterminated bidirectional
> control characters in comments and string literals.
> Code editors and repository frontends should make bidirectional control characters
> and mixed-script confusable characters perceptible with visual symbols or warnings.

also https://github.com/maltfield/detect-malicious-unicode

This commit is contained in:

SomberNight

2025-05-09 16:54:23 +00:00

parent 351cc6abd9

commit 96f861a570

2 changed files with 66 additions and 0 deletions

									
										.cirrus.yml
									
		+8
		
												View File
												
				@@ -174,6 +174,14 @@ task:

				        ELECTRUM_LINTERS_IGNORE: ""

				      allow_failures: true

				task:

				  name: "linter: ban unicode"

				  container:

				    image: python:3.10

				    cpu: 1

				    memory: 1G

				  main_script:

				    - contrib/ban_unicode.py

				# Cron jobs configured in https://cirrus-ci.com/settings/...

				# - job "nightly" on branch "master" at "0 30 2 * * ?"  (every day at 02:30Z)