This script scans the whole codebase for unicode characters and
errors if it finds any, unless the character is specifically whitelisted.
The motivation is to protect against homoglyph attacks, invisible unicode characters,
bidirectional and other control characters, and other malicious unicode usage.
Given that we mostly expect to use ASCII characters in the source code,
the most robust and generic fix seems to be to just ban all unicode usage.
see https://trojansource.codes/ :
> Compilers, interpreters, and build pipelines supporting Unicode should throw errors or warnings
> for unterminated bidirectional control characters in comments or string literals,
> and for identifiers with mixed-script confusable characters.
> Language specifications should formally disallow unterminated bidirectional
> control characters in comments and string literals.
> Code editors and repository frontends should make bidirectional control characters
> and mixed-script confusable characters perceptible with visual symbols or warnings.
also https://github.com/maltfield/detect-malicious-unicode
fixes https://github.com/spesmilo/electrum/pull/9726/files#r2057787097 :
> The version of readlink installed on macOS does not support the -e flag:
> ```
> 💬 INFO: preparing electrum-locale.
> readlink: illegal option -- e
> usage: readlink [-fn] [file ...]
> ```
>
> On a mac you can install a version of readlink that argbash expects:
> Using homebrew, `brew install coreutils` will install `greadlink` which supports the `-e` flag.
I don't think we actually need to resolve symlinks here.
There are already some examples of similar usage with realpath vs grealpath. Let's just do that.
The Qt lupdate tool that extracts translatable strings from .qml files
writes paths relative to its output .ts file into the .ts file.
These paths are then retained as-is when converted to .pot format.
The last few commits moved around the working directory of the lupdate tool
(from electrum/locale to electrum/locale/build), which resulted in a change
of all relative paths in the final messages.pot we upload to crowdin.
E.g. from:
```
#: ../gui/qml/components/Addresses.qml:64
```
to:
```
#: ../../gui/qml/components/Addresses.qml:64
```
I think a change like this does not invalidate the translations. Still, it is annoying.
This commit adds an extra processing step to "fix" these strings to:
```
#: electrum/gui/qml/components/Addresses.qml:64
```
- this merges `contrib/deterministic-build/locale` and `electrum/locale`
- it is now once again possible have translations when running from a local git clone
- which was already possible in the past before crowdin removed their unauthenticated APIs
- see https://github.com/spesmilo/electrum/issues/9531
- however, the translations available are the often-old frozen strings from electrum-locale
- while previously one could just download the latest strings from crowdin
note: these sdists are now reproducibly built.
Ideally the person updating the pinned hash should be different from the one who uploaded the dep to PyPI and reproduce and verify the hash. This prevents hiding a backdoor in just the sdist uploaded to PyPI.
but people can try to reproduce the PyPI sdists at any time and ring alarm bells if they can't
- debian 11 only has python 3.9, deb12 has py3.11
- pip install pip is no longer needed, atm apt has new enough pip
- and on deb12, started getting "error: externally-managed-environment"
- faketime does not seem to work properly on debian 12
(getting reproducibility issues for the tarball)
- so instead we untar, fix the timestamps manually, and re-tar
Haven't checked if electrum-ecc compiles libsecp reproducibly.
For now let's just keep the old flow.
(but if we spent time on making that compilation reproducible,
the appimage and the macos builds could use it directly)
SomberNight