Commit Graph

79 Commits

Author SHA1 Message Date
SomberNight c23fca37d4 build: clarify which locale files are included
related: 2cb4e56be3
2021-03-31 17:00:39 +02:00
Axel Gembe 6e1caf7e88 AppImage build: Include libxcb into the image
Ubuntu 20.04 needs `libxcb-xinerama.so.0`
and Debian 10 needs `libxcb-util.so.1` for the Qt `xcb` plugin.

-----

ported from https://github.com/Electron-Cash/Electron-Cash/commit/380f04a8054fe9fd9169721089c5da360f9d0807

see https://github.com/Electron-Cash/Electron-Cash/issues/2196

-----

In particular, see this comment:

>> I confirmed that both Ubuntu 20.04 and Debian 10.8 clean install have an issue with the 4.2.4 AppImage.
>
> Any idea what broke it? I assume it used to work.

Yes, it did work in 4.2.3 which used PyQt5 5.13.2. I just had a look at the xcb plugin and it definitely has less dependencies:
```
ago@ubuntu2004vm:~/src/Electron-Cash/dist$ ldd squashfs-root/usr/lib/python3.6/site-packages/PyQt5/Qt/plugins/platforms/libqxcb.so|grep xcb
        libX11-xcb.so.1 => /lib/x86_64-linux-gnu/libX11-xcb.so.1 (0x00007f0ec9d07000)
        libxcb.so.1 => /lib/x86_64-linux-gnu/libxcb.so.1 (0x00007f0ec9cdd000)
        libxcb-xkb.so.1 => /lib/x86_64-linux-gnu/libxcb-xkb.so.1 (0x00007f0ec6ead000)
```

compared to PyQt5 5.15.2 in 4.2.4:
```
ago@ubuntu2004vm:~/src/Electron-Cash/dist$ ldd squashfs-root/usr/lib/python3.8/site-packages/PyQt5/Qt/plugins/platforms/libqxcb.so|grep xcb
        libX11-xcb.so.1 => /lib/x86_64-linux-gnu/libX11-xcb.so.1 (0x00007fcb5715f000)
        libxcb-icccm.so.4 => /lib/x86_64-linux-gnu/libxcb-icccm.so.4 (0x00007fcb57158000)
        libxcb-image.so.0 => /lib/x86_64-linux-gnu/libxcb-image.so.0 (0x00007fcb56f51000)
        libxcb-shm.so.0 => /lib/x86_64-linux-gnu/libxcb-shm.so.0 (0x00007fcb56f4c000)
        libxcb-util.so.1 => /lib/x86_64-linux-gnu/libxcb-util.so.1 (0x00007fcb56d46000)
        libxcb-keysyms.so.1 => /lib/x86_64-linux-gnu/libxcb-keysyms.so.1 (0x00007fcb56d41000)
        libxcb-randr.so.0 => /lib/x86_64-linux-gnu/libxcb-randr.so.0 (0x00007fcb56d2e000)
        libxcb-render-util.so.0 => /lib/x86_64-linux-gnu/libxcb-render-util.so.0 (0x00007fcb56d27000)
        libxcb-render.so.0 => /lib/x86_64-linux-gnu/libxcb-render.so.0 (0x00007fcb56d16000)
        libxcb-shape.so.0 => /lib/x86_64-linux-gnu/libxcb-shape.so.0 (0x00007fcb56d11000)
        libxcb-sync.so.1 => /lib/x86_64-linux-gnu/libxcb-sync.so.1 (0x00007fcb56d07000)
        libxcb-xfixes.so.0 => /lib/x86_64-linux-gnu/libxcb-xfixes.so.0 (0x00007fcb56cfd000)
        libxcb-xinerama.so.0 => /lib/x86_64-linux-gnu/libxcb-xinerama.so.0 (0x00007fcb56cf8000)
        libxcb-xkb.so.1 => /lib/x86_64-linux-gnu/libxcb-xkb.so.1 (0x00007fcb56cda000)
        libxcb.so.1 => /lib/x86_64-linux-gnu/libxcb.so.1 (0x00007fcb56cae000)
```
2021-03-27 21:41:09 +01:00
SomberNight 472d112433 sdist build: (minor) exclude more stuff from tar.gz, rm git clean log
The `git clean --dry-run` became redundant with "fresh clone" builds.
The new exclude in MANIFEST.in should not affect production builds (also due to fresh clone),
but they are nice when building from dirty local dir.
2021-03-27 03:39:10 +01:00
SomberNight eaffced6dd binaries: bump python version 2021-03-13 16:52:19 +01:00
SomberNight 6e6e956de4 appimage build: adapt to new PyQt (5.15.3+) layout 2021-03-13 16:52:15 +01:00
gruve-p abc155b688 Appimage build: update git=1:2.7.4-0ubuntu1.10 15b5d9d (#7091) 2021-03-09 23:35:02 +00:00
SomberNight dded25f398 windows build: cache pip downloads 2021-03-07 22:12:26 +01:00
ThomasV 9b7f9219a3 fix travis builds (update openssl) 2021-02-19 15:49:37 +01:00
SomberNight 3d640dfc1f windows binaries: bump python version (3.7.9->3.8.7)
Had to also bump ubuntu version 18.04->20.04,
as was getting errors running the self-compiled pyinstaller otherwise (weird...):

```
from .utils.git import get_repo_revision
ModuleNotFoundError: No module named 'PyInstaller.utils'
```
(similar to https://github.com/pyinstaller/pyinstaller/issues/4403 )

-----

Note re appimage: when trying to compile python 3.8.x on ubuntu 16.04, I am getting:

./Modules/posixmodule.c: In function ‘os_copy_file_range_impl’:
./Modules/posixmodule.c:10351:15: error: implicit declaration of function ‘copy_file_range’ [-Werror=implicit-function-declaration]
         ret = copy_file_range(src, p_offset_src, dst, p_offset_dst, count, flags);

This is because ubuntu 16.04 has too old glibc.
2021-01-12 02:28:35 +01:00
SomberNight 19f806ddf4 build: don't allow setuptools to sneakily install build-time deps
see https://pip.pypa.io/en/stable/reference/pip_install/#controlling-setup-requires
> Setuptools offers the setup_requires setup() keyword for specifying
> dependencies that need to be present in order for the setup.py
> script to run. Internally, Setuptools uses easy_install to
> fulfill these dependencies.
> pip has no way to control how these dependencies are located.
> None of the package index options have an effect.

With these changes, we will now instead hard fail if this were to happen.

related: https://github.com/spesmilo/electrum/issues/5859#issuecomment-743621898
2020-12-12 02:52:38 +01:00
SomberNight 4ca2a5cf3e appimage build: build most of our python dependencies from source
instead of using pre-built binary wheels from PyPI
2020-12-09 16:38:03 +01:00
SomberNight 37a124fa1c appimage: update package in dockerfile 2020-12-09 15:00:42 +01:00
SomberNight a4e342ac58 requirements: rename some files 2020-11-14 04:30:48 +01:00
Jin Eguchi 56f380a62c appimage: update openssl & libudev-dev (#6599) 2020-09-17 14:39:20 +00:00
SomberNight 9204102663 binaries: pip install build requirements first
I no longer trust pip to install packages from a requirements.txt file in the correct order.
For reproducibility, let's install pip/setuptools/wheels/cython first.

see https://github.com/pypa/pip/issues/2362#issuecomment-418423458
see #5859 and #6382
2020-09-08 16:44:35 +02:00
SomberNight 4f46741c52 binaries: bump python version (3.7.7->3.7.9) 2020-09-08 16:44:24 +02:00
SomberNight 995250948a appimage build: pin glibc version in docker image, for reproducibility
fixes #6357
2020-07-08 23:54:54 +02:00
SomberNight 3c6b049f9a appimage: update package in dockerfile 2020-06-17 00:33:36 +02:00
SomberNight f5f3394552 git sanity: enforce "git checkout commithash" actually pulls commit
If there is a collision between a branch name and a commit hash, git
will choose the branch, even if the full 40-hex-long commit hash is
given. GitHub disallows branches/tags with such a name but git itself
does not. By adding the `^{commit}` syntax sugar after a ref name,
we can tell git that we want the commit hash to be preferred,
and hence we don't need to trust GitHub (only git).

see https://security.stackexchange.com/questions/225411/
2020-06-16 19:55:17 +02:00
SomberNight dacc61a41d sdist build: update message about reproducibility 2020-06-13 03:12:33 +02:00
SomberNight c5c8ea15bb sdist build: stop making .zip distributables as they are not deterministic
see https://bugs.python.org/issue40963
2020-06-12 19:48:33 +02:00
SomberNight 901a900ec5 sdist build: when building docker image, no interactive prompts!
see https://askubuntu.com/questions/909277/
2020-06-12 19:48:29 +02:00
SomberNight a06f5da7c2 sdist build: bump base image to ubuntu 20.04 2020-06-12 19:48:26 +02:00
SomberNight 891390f9a1 sdist build: umask should be specified for git clone
(not nice to change umask of host :/)
2020-06-12 19:48:23 +02:00
SomberNight 24a007840f sdist build: use modern pip
the one in apt refused to install certain package versions (that were pinned by hash!!)
and installed different versions instead... e.g.:

Collecting wheel==0.34.2 (from -r /opt/electrum/contrib/build-linux/sdist/../../../contrib/deterministic-build/requirements.txt (line 112))
  Downloading https://files.pythonhosted.org/packages/75/28/521c6dc7fef23a68368efefdcd682f5b3d1d58c2b90b06dc1d0b805b51ae/wheel-0.34.2.tar.gz (58kB)
    100% |████████████████████████████████| 61kB 3.8MB/s
  Requested wheel==0.34.2 from https://files.pythonhosted.org/packages/75/28/521c6dc7fef23a68368efefdcd682f5b3d1d58c2b90b06dc1d0b805b51ae/wheel-0.34.2.tar.gz#sha256=8788e9155fe14f54164c1b9eb0a319d98ef02c160725587ad60f14ddc57b6f96 (from -r /opt/electrum/contrib/build-linux/sdist/../../../contrib/deterministic-build/requirements.txt (line 112)), but installing version 0.30.0
2020-06-12 19:48:19 +02:00
SomberNight e12bc4817a attempt at reproducible tarballs (sdist) 2020-06-12 19:48:15 +02:00
SomberNight b0230f6a4b build: fix win/appimage binaries following jsonrpc dep-removal
follow-up #6220
2020-06-11 05:37:52 +02:00
SomberNight 2dfef9dde6 appimage: update package in dockerfile
Ubuntu no longer serves old version
2020-06-11 02:52:09 +02:00
Jin Eguchi 6339afee3b appimage: update openssl (#6186) 2020-05-29 00:30:20 +00:00
SomberNight 587f8df8ad binaries: update base docker image for wine/appimage 2020-05-14 20:24:17 +02:00
SomberNight 7143e9199f binaries: bump python version (3.7.6->3.7.7) 2020-05-14 20:24:07 +02:00
ThomasV 594f13b6f7 appimage: update libudev-dev in Dockerfile 2020-05-13 10:55:08 +02:00
Jin Eguchi 1846154ca3 build: update git in dockerfiles (#6107) 2020-04-21 22:48:01 +00:00
SomberNight c2d6a902dd build: update some packages in dockerfiles
Ubuntu no longer serves old version
2020-04-15 18:06:59 +02:00
SomberNight e5e512df8c appimage: update package in dockerfile
Ubuntu no longer serves old version
2020-03-10 18:20:46 +01:00
SomberNight 4cec098d2d build: create a standalone build script for libsecp256k1
heavily based on Electron-Cash/Electron-Cash@eda015908e
2020-02-11 16:48:24 +01:00
Jin Eguchi 4313bde4c2 appimage: update libudev-dev (#5936) 2020-02-07 11:41:04 +00:00
wakiyamap 1237134339 Fix travis appimage 2020-01-29 12:55:23 +09:00
SomberNight a5cd34dc08 follow-up prev (oops, only committed part of the changes) 2020-01-22 18:26:29 +01:00
Axel Gembe d3385e49bb Build: Install libxkbcommon-x11 in AppImage
Newer distributions do not install libxkbcommon-x11 by default
anymore and Qt depends on it.

-----

taken from https://github.com/Electron-Cash/Electron-Cash/commit/ca3e4501cdc42332f4d7079ef9392c529c9d97b8
2020-01-22 18:16:53 +01:00
Axel Gembe 4406eebbfe Build: Uninstall Cython from AppImage
Cython is not needed at runtime.

-----

taken from https://github.com/Electron-Cash/Electron-Cash/commit/c64910055dad6bdcbd01b01121352d2fa3fedb40

related #5859
2020-01-22 12:27:17 +01:00
SomberNight 80025a3af4 requirements-hw: re-add Cython
this reverts ec496a8222
Cython must be pinned down for reproducible builds
related #5859
2020-01-22 12:08:30 +01:00
SomberNight 1c4728ecc6 appimage binary: bump python version (3.6.8->3.7.6) 2020-01-16 19:12:24 +01:00
SomberNight 7c090f92ce binaries: use "--no-dependencies" option for pip install
All (incl indirect) dependencies are already listed in deterministic-build/requirements*.txt.
This option makes it easier to manually rm a dependency from that list for e.g. testing.
2020-01-10 19:01:32 +01:00
SomberNight b3c0231b2b appimage build: add notes re investigating reproducibility failure 2019-12-24 03:30:26 +01:00
Axel Gembe 880bd16883 AppImage: Improve binary stripping
Slightly reduces file size, improves build speed and makes build more
reproducible.

The .comment section contained GCC version information which could cause
different build output from just a minor update in GCC. The information is not
needed so we strip this.

The strip command was invoked using xargs, spawning a new process for each file.
This is inefficient as xargs can correctly run the strip command with multiple
file names.

-----

taken from https://github.com/Electron-Cash/Electron-Cash/commit/43aaf9572f822b977ecffe68f9cce428e5d27c18
2019-12-17 21:41:17 +01:00
SomberNight 255bf7caf4 build: update some packages in dockerfiles
Ubuntu no longer serves old version
2019-12-10 23:54:45 +01:00
SomberNight 9f9b0954e2 appimage: update package in dockerfile
Ubuntu no longer serves old version
2019-12-06 22:02:17 +01:00
SomberNight 69720946c1 appimage: update package in dockerfile
Ubuntu no longer serves old version
2019-12-04 20:53:48 +01:00
SomberNight b381fd84fb build: when building libsecp256k1, patch Makefile.am before autogen.sh
apparently this could have caused issues on MacOS

based on Electron-Cash/Electron-Cash@69f6cd0aa0
2019-09-08 18:47:30 +02:00