CI: reduce parallelism for pytest.
In November 2022 we seemed to increase parallelism from 2 and 3 to 10! That is a huge load for these CI boxes, and does explain some of our flakes. We only run in parallel because some tests sleep, but it's diminishing returns (GH runners have 4 VCPUs, 16GB RAM). This reduces it so: - Normal runs are -n 4 - Valgrind runs are -n 2 - Sanitizer runs are -n 3 If I use my beefy build box (64BG RAM) but reduce it to 4 CPUs: Time for pytest -n 5: Time for pytest -n 4: Time for pytest -n 3: Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
This commit is contained in:
13
.github/workflows/ci.yaml
vendored
13
.github/workflows/ci.yaml
vendored
@@ -312,7 +312,7 @@ jobs:
|
||||
- name: Test
|
||||
env:
|
||||
SLOW_MACHINE: 1
|
||||
PYTEST_PAR: 10
|
||||
PYTEST_PAR: 4
|
||||
TEST_DEBUG: 1
|
||||
TEST_DB_PROVIDER: ${{ matrix.TEST_DB_PROVIDER }}
|
||||
TEST_NETWORK: ${{ matrix.TEST_NETWORK }}
|
||||
@@ -430,7 +430,7 @@ jobs:
|
||||
COMPAT: 1
|
||||
CFG: ${{ matrix.CFG }}
|
||||
SLOW_MACHINE: 1
|
||||
PYTEST_PAR: 10
|
||||
PYTEST_PAR: 4
|
||||
TEST_DEBUG: 1
|
||||
TEST_DB_PROVIDER: ${{ matrix.TEST_DB_PROVIDER }}
|
||||
TEST_NETWORK: ${{ matrix.TEST_NETWORK }}
|
||||
@@ -524,8 +524,9 @@ jobs:
|
||||
env:
|
||||
SLOW_MACHINE: 1
|
||||
TEST_DEBUG: 1
|
||||
PYTEST_PAR: 2
|
||||
run: |
|
||||
VALGRIND=1 uv run eatmydata pytest tests/ -n 3 ${PYTEST_OPTS} ${{ matrix.PYTEST_OPTS }}
|
||||
VALGRIND=1 uv run eatmydata pytest tests/ -n ${PYTEST_PAR} ${PYTEST_OPTS} ${{ matrix.PYTEST_OPTS }}
|
||||
- name: Upload test results
|
||||
if: always()
|
||||
uses: actions/upload-artifact@v4
|
||||
@@ -613,8 +614,10 @@ jobs:
|
||||
run: tar -xvjf cln-compile-clang-sanitizers.tar.bz2
|
||||
|
||||
- name: Test
|
||||
env:
|
||||
PYTEST_PAR: 2
|
||||
run: |
|
||||
uv run eatmydata pytest tests/ -n 2 ${PYTEST_OPTS} ${{ matrix.PYTEST_OPTS }}
|
||||
uv run eatmydata pytest tests/ -n ${PYTEST_PAR} ${PYTEST_OPTS} ${{ matrix.PYTEST_OPTS }}
|
||||
- name: Upload test results
|
||||
if: always()
|
||||
uses: actions/upload-artifact@v4
|
||||
@@ -737,7 +740,7 @@ jobs:
|
||||
COMPAT: 1
|
||||
CFG: ${{ matrix.CFG }}
|
||||
SLOW_MACHINE: 1
|
||||
PYTEST_PAR: 10
|
||||
PYTEST_PAR: 4
|
||||
TEST_DEBUG: 1
|
||||
TEST_DB_PROVIDER: ${{ matrix.TEST_DB_PROVIDER }}
|
||||
TEST_NETWORK: ${{ matrix.TEST_NETWORK }}
|
||||
|
||||
Reference in New Issue
Block a user