Since APK support was added, the CONTROL directory is created only after
the checksum step. With IPK_FILES_CHECKSUMS enabled, redirecting output
to CONTROL/files-sha256sum fails for packages that do not create CONTROL
themselves. The error is hidden by "|| true", so these packages are built
without checksums and pkg_check skips them. Only base-files still gets
the file.
Create the directory before writing the checksums, as in 23.05, and
drop the "|| true", so that a failure to write the file stops the build
instead of going unnoticed.
Tested with a test package in the main, 25.12 and 24.10 SDKs. The IPKs
contain files-sha256sum matching data.tar.gz, and without the mkdir the
build now fails.
Fixes: d788ab376f ("build: add APK package build capabilities")
Signed-off-by: Josef Schlehofer <pepe.schlehofer@gmail.com>
Link: https://github.com/openwrt/openwrt/pull/26257
Signed-off-by: Christian Marangi <ansuelsmth@gmail.com>
RUST_SCCACHE builds sccache 0.18.0 from a vendor tarball and makes it
the RUSTC_WRAPPER of the cargo builds of host tools, target packages
and libstd-rust. For target packages, scripts/rust-target-wrapper.sh
runs it in turn. The option sits next to CCACHE and is available with
any Rust toolchain of the build system. The packages feed defines
RUST_SCCACHE as well, without a dependency, and Kconfig merges both
definitions, so the tools and the cargo variables check for a Rust
toolchain once more.
sccache is built without default features: the local disk cache stays,
the remote storage backends and their HTTP and TLS stacks are left out,
so the build needs no host library. sccache needs rustc to build, so
the rustc build from source and the sccache build run unwrapped.
sccache keeps its cache and configuration in $(TOPDIR)/.sccache, or in
RUST_SCCACHE_DIR when set, and its server listens on a socket in
$(TMP_DIR), so that each tree has a server of its own. distclean
removes .sccache, and the SCCACHE_* variables of the environment are
unexported.
Signed-off-by: Daniel Golle <daniel@makrotopia.org>
Pass the Rust target, selected compiler, Cargo executable and
Corrosion module location to CMake packages using rust-values.mk, told
by its include guard. Cargo goes as Rust_CARGO as well, which FindRust
of corrosion reads, as projects such as fish carry it; it looks in PATH
and in ~/.cargo/bin otherwise. Carry the target Cargo environment
through configure, build and install for both Ninja and Makefiles,
keeping native probes on the host sysroot.
Register the libstd build dependency and Rust configuration inputs for
target consumers. Package CMAKE_OPTIONS can override generated options.
Signed-off-by: Daniel Golle <daniel@makrotopia.org>
Add target and native Rust compiler entries when a package includes
rust-values.mk. Its include guard tells, as the packages feed has a
rust-values.mk of its own, which python3-host.mk includes for every
Python package. The target entry carries the triple, CPU flags,
target-owned sysroot and standard-library linkage choice; native probes
use the host sysroot.
Supply bindgen with the toolchain sysroot and selected LLVM paths.
Register the target libstd build dependency and configuration inputs
for Rust consumers.
Signed-off-by: Daniel Golle <daniel@makrotopia.org>
With CONFIG_KERNEL_RUST pass RUSTC, BINDGEN, RUST_LIB_SRC and
LIBCLANG_PATH to the kernel build. These are the variables
scripts/rust_is_available.sh checks.
KERNEL_RUST is enabled by default with any Rust toolchain, selects
bindgen and follows the kernel's architecture requirements: ARM32 needs
little-endian ARMv7, and ARM32 and LoongArch require KASAN to be
disabled.
CONFIG_RUST makes further symbols visible, which kconfig asks for
because the generic configuration does not answer them. Answer them in
the generic configuration so that targets need no changes. Filter the
rustc and bindgen version texts out of the kernel configuration
comparison.
Suggested-by: Andre Heider <a.heider@gmail.com>
Signed-off-by: Daniel Golle <daniel@makrotopia.org>
Add rust-package.mk for target packages written in Rust. The Cargo
environment carries the target triple, CPU flags, the C and C++ flags
for cc-rs, the linker and the ABI-specific sysroot of libstd-rust,
while native build scripts and proc macros keep the host sysroot and
linker. Builds run in the crate's working directory with an explicit
target and offline from the vendor tarball; with a source tree override
(git-src, USE_SOURCE_DIR), which has none, Cargo fetches the locked
crates. Target packages depend on libstd-rust at build time and track
its configuration inputs for rebuilds.
rustc runs the linker with arguments of its own, without TARGET_LDFLAGS.
The linker selection and the -z options that hardening.mk and
package.mk set for every package are passed to rustc, for packages and
for libstd-rust.
When the target triple is the build host's, cc-rs takes the host
compiler for target C code as well. For that triple the C and C++
compiler of cc-rs is scripts/rust-cc-wrapper.sh, linked into
$(STAGING_DIR_HOST)/bin by tools/rust. It runs the target compiler with
the target flags for build scripts whose OUT_DIR lies in the target
triple's directory, and the host compiler for the others.
RUST_PREFER_DYNAMIC is available with any Rust toolchain, controls the
default userspace standard-library linkage and defaults to shared
linking. Packages can override it. Release builds use LTO either way;
rustc runs LTO next to -Cprefer-dynamic only with the unstable
-Zdylib-lto. RUST_PKG_DEPENDS carries the architecture dependencies
and, with shared linking, the libstd-rust runtime dependency. For
packages that keep the default, that dependency is conditional on
RUST_PREFER_DYNAMIC, as .config is not read while package metadata is
collected.
Release builds emit no unwind tables and take the size options of
unstable rustc that libstd-rust is built with: panics without location,
derived Debug output reduced to type names and, with
RUST_PANIC_IMMEDIATE_ABORT, -Cpanic=immediate-abort. Permitting unstable
options for every crate with RUSTC_BOOTSTRAP=1 changes the code of
dependencies: the build scripts of anyhow, thiserror and proc-macro2
then detect unstable features and use them.
scripts/rust-target-wrapper.sh, the RUSTC_WRAPPER of package builds,
adds the options, and -Zdylib-lto with a shared libstd, only to the
target crates cargo compiles, recognised by --target, -C metadata and
--error-format=json, and permits unstable options for that crate alone.
Build scripts and their probes see neither. A package keeps panic
locations with RUST_PKG_LOCATION_DETAIL:=1 and selects the Debug output
with RUST_PKG_FMT_DEBUG. None of this applies with CONFIG_DEBUG.
Packages build the release profile, as maturin, CMake and autotools
projects do, and CONFIG_DEBUG gives it the settings of the dev profile,
ahead of the profile settings of the package. Build paths are mapped out
of packages as out of libstd-rust.
Without a Rust toolchain of the build system, Rust packages build with
rust/host of the packages feed, which installs to $(STAGING_DIR)/host,
so that packages can move from the Rust support of the packages feed
one at a time. RUSTC and CARGO point there, there is neither a sysroot
of libstd-rust nor a shared libstd, the mips64 triples are those its
std is built for, and on the build host's triple the target linker is
set through the Cargo variable. The build dependencies, on libstd-rust
or rust/host, follow USE_RUST_TOOLCHAIN in the package metadata, also
for host builds through rust-host-build.mk. After a switch from None to
a Rust toolchain, the cargo of rust/host still comes first in PATH, so
the help of the toolchain choice names its clean-up.
The file is derived from lang/rust in the packages feed by Luca Barbato
and Donald Hoskins.
Reported-by: Andre Heider <a.heider@gmail.com>
Co-developed-by: Brian Redbeard <redbeard@dead-city.org>
Signed-off-by: Brian Redbeard <redbeard@dead-city.org>
Co-developed-by: Aditya Nugraha <vortexilation@gmail.com>
Signed-off-by: Aditya Nugraha <vortexilation@gmail.com>
Signed-off-by: Daniel Golle <daniel@makrotopia.org>
Build the sysroot crate with the selected Cargo and rustc, using the
source tarball's vendored crates and shared target CPU flags. Stage the
rlibs and shared library through normal package-owned target staging,
and include that sysroot in the SDK.
Separate Cargo output by ABI identity to keep dependency artefacts from
different configurations in their own build directories.
Name the shared library and package ABI from compiler, source, target
and build configuration inputs, with configuration changes triggering
a rebuild. Declare libgcc as a runtime dependency for libgcc_s.so.1.
Release builds optimise for size, abort on panic and emit no unwind
tables. They also take the size options of unstable rustc, which the
sysroot build permits anyway: panics carry no location
(-Zlocation-detail=none) and derived Debug output is reduced to type
names (-Zfmt-debug=shallow). RUST_PANIC_IMMEDIATE_ABORT, on by default,
builds with -Cpanic=immediate-abort and without the panic-unwind
feature: a panic stops the program at once, without formatting a
message. rustc requires the same panic strategy for all crates of a
program, the standard library included, so this is a build-wide option.
Debug builds retain unwinding, panic messages and full Debug output.
Build paths are mapped out of the library as iremap in rules.mk does
for C, unless CONFIG_DEBUG is set without
CONFIG_REPRODUCIBLE_DEBUG_INFO. The map stays out of the inputs of the
ABI identity, which would otherwise depend on the build path.
rust-values.mk maps the OpenWrt target to its Rust target triple and CPU
flags; the mapping is derived from lang/rust in the packages feed by
Luca Barbato and Donald Hoskins. The mips64 ABI comes from
CONFIG_MIPS64_ABI, which every mips64 configuration sets, and n64 maps
to the muslabi64 and gnuabi64 triples. These are hard-float, so
soft-float mips64 with musl uses mips64-openwrt-linux-musl, the
soft-float n64 target of rustc, and the other soft-float mips64 and
mips64el configurations, which have no Rust target, get no Rust
packages. The e500v2 of mpc85xx has no FPU but shares the hard-float
powerpc triple, whose hard-float feature is turned off with
CONFIG_SOFT_FLOAT. When the target triple is the build host's, as for
x86_64 and aarch64 glibc targets, Cargo links build scripts with the
target linker as well but keeps target rustflags off them, so the target
linker is passed as a rustflag in that case.
include/scan.mk rescans the package metadata when an include/rust-*.mk
file changes, as for include/package*.mk.
Co-developed-by: Brian Redbeard <redbeard@dead-city.org>
Signed-off-by: Brian Redbeard <redbeard@dead-city.org>
Co-developed-by: Aditya Nugraha <vortexilation@gmail.com>
Signed-off-by: Aditya Nugraha <vortexilation@gmail.com>
Signed-off-by: Daniel Golle <daniel@makrotopia.org>
Build bindgen-cli 0.73.2 from a vendor tarball. bindgen loads libclang
from the BPF LLVM toolchain: the prebuilt one unpacked into the source
tree, the one built by tools/llvm-bpf or the host LLVM. Versions before
0.72.1 generate opaque structures together with libclang 22 or newer,
which tools/llvm-bpf provides.
BUILD_BINDGEN is enabled by default with any Rust toolchain and selects
the built LLVM when neither the host nor the prebuilt LLVM is in use.
Signed-off-by: Daniel Golle <daniel@makrotopia.org>
Add rust-host-build.mk for host tools written in Rust.
Host/Compile/Cargo installs the crate with the staged rustc and cargo,
offline and with the locked dependencies of its vendor tarball by
default, and RustBinHostBuild installs the binaries into the host
package staging directory.
The file is derived from lang/rust in the packages feed by Luca Barbato
and Donald Hoskins.
Co-developed-by: Brian Redbeard <redbeard@dead-city.org>
Signed-off-by: Brian Redbeard <redbeard@dead-city.org>
Signed-off-by: Daniel Golle <daniel@makrotopia.org>
Add download methods which try the OpenWrt mirror, then verify an
upstream source and generate an offline vendor archive with
scripts/gen-cargo-vendor-tarball.sh. cargo-vendor-crate takes a .crate
and cargo-vendor-github a GitHub tag archive. cargo-vendor-archive
takes any other archive tar unpacks: release archives in other
compressions, PyPI source archives, crates published with build
metadata in their version. meson-vendor-wraps puts the files of the
selected [wrap-file] wraps of a meson project into
subprojects/packagecache, as mesa fetches its crates through meson
wraps at configure time.
The GitHub method serves the Rust packages of the packages feed, which
are released as tag archives, some with the Cargo project in a
subdirectory. It fetches the archive from codeload.github.com as
$(PKG_NAME)-$(PKG_VERSION).tar.gz. The top directory of every vendor
archive is the one PKG_BUILD_DIR unpacks to, and an existing
.cargo/config.toml of the project stays in front of the vendor
configuration. With the Cargo project in a subdirectory, both are in
the top directory, where cargo finds them wherever a build runs it.
Canonical tar metadata, checksum normalisation and the staged zstd
compressor keep generated archive contents stable. The archive has mode
0644, which the buildbots keep when they copy new files of dl/ to the
source mirror.
Verify PKG_HASH before atomically publishing the archive. Parse cache
options before selecting Cargo, and use the existing locked, verified
downloader for the pinned official Cargo fallback. The generator keeps
its Cargo home in staging_dir/hostpkg/cargo and its work directory in
tmp/, clears inherited CARGO_* settings and takes the staged cargo or
the pinned official one.
Co-developed-by: Brian Redbeard <redbeard@dead-city.org>
Signed-off-by: Brian Redbeard <redbeard@dead-city.org>
Signed-off-by: Daniel Golle <daniel@makrotopia.org>
Provide rustc, cargo, the host rust-std, rust-src and llvm-tools in
$(STAGING_DIR_HOST) in one of three ways. RUST_TOOLCHAIN_BUILD builds
them from the rustc source tarball with x.py. RUST_TOOLCHAIN_PREBUILT
installs the pinned components from static.rust-lang.org. For
RUST_TOOLCHAIN_HOST the components come from a relocatable toolchain on
the build host, which has to match the Rust version and is checked after
installation. The pinned components cover Linux hosts with glibc or
musl and macOS hosts, on x86_64 and aarch64; other hosts get an error
that names the host.
The choice sits in a "Rust toolchain options" menu of the developer
options, next to the external toolchain options and independent of
them. EXPERIMENTAL defaults to the prebuilt toolchain. As a Kconfig
choice resolves to none while its menu is hidden, USE_RUST_PREBUILT
carries that default in this case.
The xz2 crate of bootstrap is linked statically so that the compiler
needs no host liblzma at run time.
host-build.mk empties MAKEFLAGS for the build; x.py gets the jobserver
of make in MAKEFLAGS, so that bootstrap leaves the job count of ninja
and cargo to it, and one job when make runs without a jobserver.
The LLVM of a Rust release sinks the leading fence of a weak cmpxchg
into the reservation. On the e500v2 of mpc85xx the sync there clears
the reservation, so compare_exchange_weak loops never end. tools/rust
carries the LLVM change of rust/host in the packages feed, and on
mpc85xx the source build is the only choice and the default.
All three ways install into a prefix in one build directory, whose
prepared stamp depends on the choice, and the prefix is copied to
$(STAGING_DIR_HOST). As all three enable the same tool, the stamp of
the tools stage depends on the choice too. The installed files are
recorded in $(STAGING_DIR_HOST)/stamp and removed on clean and before
the next installation. The rust-installer metadata is removed from the
prefix: its manifests name absolute paths, which its uninstall.sh and
a later install.sh into the same prefix delete, those of the build
host's toolchain included.
Cargo reads .cargo/config.toml in every parent directory of its working
directory, and cargo and rustc take settings from the environment. The
build stops when such a file exists in TOPDIR or a parent directory,
and the CARGO, CARGO_* and RUST* variables of the environment are
unexported.
rust-values.mk holds the Rust version, the host triple, the hashes of
the pinned components and the Cargo home. The Cargo home is
$(STAGING_DIR_HOSTPKG)/cargo, shared by all targets, so that $(DL_DIR)
holds downloads only.
The xz2 and LLVM patches, the Rust build recipe and rust-values.mk are
derived from lang/rust in the packages feed by Luca Barbato and Donald
Hoskins.
Suggested-by: Andre Heider <a.heider@gmail.com>
Co-developed-by: Brian Redbeard <redbeard@dead-city.org>
Signed-off-by: Brian Redbeard <redbeard@dead-city.org>
Co-developed-by: Aditya Nugraha <vortexilation@gmail.com>
Signed-off-by: Aditya Nugraha <vortexilation@gmail.com>
Signed-off-by: Daniel Golle <daniel@makrotopia.org>
Host builds install libtool archives into staging_dir/host/lib, and
libssl.la, libtls.la, libgmpxx.la, libmpfr.la, libmpc.la and libisl.la
name their dependencies by absolute path in dependency_libs. A prebuilt
staging_dir/host used from another directory, as with scripts/ext-tools.sh
or the tools container in the CI of a fork, then fails to link any tool
that is rebuilt against them:
libtool: error: '/__w/openwrt/openwrt/openwrt/staging_dir/host/lib/libcrypto.la' is not a valid libtool archive
The .la files are now deleted after each host install, as they already
are for target staging. Consumers link through -L/-l and pkg-config.
Assisted-by: Claude:claude-opus-5-5
Signed-off-by: Julius Bairaktaris <julius@bairaktaris.de>
Signed-off-by: Nick Hainke <vincent@systemli.org>
Link: https://github.com/openwrt/openwrt/pull/25361
Signed-off-by: Robert Marko <robimarko@gmail.com>
Add target/linux/dtbs_check for manually checking selected OpenWrt
DTBs against the patched kernel binding schemas. It runs only when
explicitly requested; normal builds do not invoke it.
DT_SCHEMA_FILES can limit validation to specific bindings.
Link: https://github.com/openwrt/openwrt/pull/26240
Signed-off-by: Robert Marko <robimarko@gmail.com>
- NF_TABLES_SET, NFT_COUNTER, NFT_META, NFT_OBJREF: these are built
into nf_tables.ko and are no longer separate options or modules
- BRIDGE_EBT_NFQUEUE: there is no ebt_nfqueue module in the kernel
- IP_NF_TARGET_ROUTE: commented out, the out-of-tree ROUTE target is
long gone
None of these produced a module, so the package contents do not change.
Signed-off-by: Sander van Deijck <sander@vandeijck.com>
Link: https://github.com/openwrt/openwrt/pull/26067
Signed-off-by: Robert Marko <robimarko@gmail.com>
Since Linux 5.13, IPv4 and IPv6 packet logging are handled by
nf_log_syslog, which is part of kmod-nf-log. nf_log_ipv4, nf_log_ipv6
and nf_log_common no longer exist, so kmod-nf-log6 is an empty package.
Remove it, and drop the entries for the old modules from
include/netfilter.mk. CONFIG_NF_LOG_IPV4 and CONFIG_NF_LOG_IPV6 are
backwards-compatibility options that only select NF_LOG_SYSLOG, which
kmod-nf-log already enables.
CONFIG_NF_LOG_IPV6 was only configured through the KCONFIG of
kmod-nf-log6. Add it to the generic config as not set, next to
CONFIG_NF_LOG_IPV4, as it would otherwise be left unconfigured when
nf_tables is enabled and abort the kernel configuration. It is still
enabled when selected by NETFILTER_XT_TARGET_LOG for ip6tables.
kmod-nf-ipt6 now depends on kmod-nf-log directly, which it previously
got through kmod-nf-log6.
Signed-off-by: Sander van Deijck <sander@vandeijck.com>
Link: https://github.com/openwrt/openwrt/pull/26067
Signed-off-by: Robert Marko <robimarko@gmail.com>
uclibc++ was removed in 2021, and include/uclibc++.mk has only been
kept as a deprecated shim since. Nothing includes it anymore, and its
deprecation message was never shown because $(warn) is not a make
function. Remove it.
Also drop the ld-uClibc.so.* pattern from the mklibs dynamic linker
lookup, as uClibc is no longer supported.
Signed-off-by: Sander van Deijck <sander@vandeijck.com>
Link: https://github.com/openwrt/openwrt/pull/26068
Signed-off-by: Christian Marangi <ansuelsmth@gmail.com>
CONFIG_GENERIC_GPIO and CONFIG_KALLSYMS_EXTRA_PASS were removed from the
kernel long ago. The gpio feature is detected by CONFIG_GPIOLIB alone,
and disabling KALLSYMS_EXTRA_PASS has no effect.
Signed-off-by: Sander van Deijck <sander@vandeijck.com>
Link: https://github.com/openwrt/openwrt/pull/26068
Signed-off-by: Christian Marangi <ansuelsmth@gmail.com>
The workaround for empty Module.symvers.tmp files is needed on kernel
6.18, which is now the only kernel version, so always use it.
Fixes: 666fee6f48 ("kernel.mk: add 6.18 workaround for empty Module.symvers.tmp")
Signed-off-by: Sander van Deijck <sander@vandeijck.com>
Link: https://github.com/openwrt/openwrt/pull/26052
Signed-off-by: Robert Marko <robimarko@gmail.com>
Prepare for the addition of a 'container' feed which is going to use
a Make function called 'BuildContainer' to declare packages. Include
that pattern in scan.mk, so `scripts/feeds` is able to find and install
such packages.
Signed-off-by: Daniel Golle <daniel@makrotopia.org>
The /lib/apk/packages/<pkg>.list file is read by the default_postinst
and default_prerm helpers to find init scripts and uci-defaults of a
package. A layer 1 package keeps its installed record on the uvol
.meta volume and ships volume payloads, not root filesystem files, so
the list would sit on the overlay away from the record that describes
it. Do not generate it when APK_INFO_ selects layer:1; the helpers
treat a missing list as empty.
Signed-off-by: Daniel Golle <daniel@makrotopia.org>
APK_TAGS_ and APK_SCRIPTS_ are the only per-package hooks into the apk
mkpkg command line, so a package that needs another pkginfo field has
to smuggle it through one of them. Add APK_INFO_<pkg>, spliced into
the mkpkg invocation after the tags, for fields such as "layer:1".
Signed-off-by: Daniel Golle <daniel@makrotopia.org>
legacy_sdcard_do_upgrade needs partx after a full image write, but only
some targets listed it in DEFAULT_PACKAGES, where it could be deselected.
Select it from the feature and add it to DEFAULT_PACKAGES for the
ImageBuilder, which also covers qoriq, sifiveu and mvebu/cortexa9.
Signed-off-by: John Audia <therealgraysky@proton.me>
Many image recipes write header fields with printf or echo piped into
dd bs=N count=1 conv=sync. Bash writes the output of both line
buffered, so a field holding a 0x0a byte reaches the pipe in two writes.
If dd reads between them, it gets a short block, pads it with zeros due
to conv=sync and stops due to count=1, so the rest of the field is
lost. This depends on the scheduling of the builder, and the rebuilder
has caught it in a ZNET header of the Zyxel WSM20:
-00000000: 5a4e 4554 80cd 7900 530a 4132 530a 0000 ZNET..y.S.A2S...
+00000000: 5a4e 4554 80cd 7900 530a 4132 530a 4132 ZNET..y.S.A2S.A2
Fields holding sizes, CRCs or MD5 sums hit this at random, e.g. in
elx-header, iodata-mstc-header2, seil-header or nec-fw. Text fields
only hit it if they contain a newline, but they use the same pattern.
dongwon-header had the same problem and no longer uses dd since
commit f6ef30a35e ("ath79: dongwon-header: don't read the CRC from a
pipe with dd").
Pass iflag=fullblock to every dd that reads such a pipe with count= or
conv=sync, so dd reads until the block is full or the input ends.
dd with bs=1 is left alone, as a read always returns a full block there.
iflag=fullblock is supported by GNU and uutils dd, like
oflag=seek_bytes already used in ramips. Build dd in tools/coreutils as
well, so hosts other than Linux get GNU dd next to the GNU date and
stat the images already need.
Link: https://github.com/openwrt/openwrt/pull/25897
Signed-off-by: Paul Spooren <mail@aparcar.org>
Commit 4071398b13 replaced $(VERSION_DIST) in the netgear-dni and
netgear-encrypted-factory recipes with $(shell cat $(VERSION_DIST) | sed
...) to turn whitespace into dashes. VERSION_DIST holds the distribution
name such as "OpenWrt" and not a file name, so cat fails with "No such
file or directory" and the expansion is always empty.
The change was meant to let a distribution name with a space, such as
"ColVisTec AG", pass through the argument parsing of
netgear-encrypted-factory.py as "ColVisTec-AG". What it produced instead
was an empty string for every distribution name, including the one it
was written for. The build error went away only because an empty string
contains no whitespace.
Since then, the DNI header of the Netgear images carries
"version:V.r12345" instead of "version:VOpenWrt.r12345", and the
encrypted factory images get "V1.0.0.0..r12345" with an empty field
between two dots instead of "V1.0.0.0.OpenWrt.r12345".
The distribution name has been part of the Netgear version string since
commit 6d2fa1c896 added the first WNDR3700 factory image in 2009, and
the encrypted factory recipe used it from its introduction in commit
efca76ffce.
Use $(subst $(space),-,$(VERSION_DIST)) instead, which restores the
distribution name and still replaces spaces with dashes as intended.
Fixes: 4071398b13 ("build: escape whitespaces in VERSION_DIST for Netgear images")
Signed-off-by: Grische <github@grische.xyz>
Assisted-by: Claude:claude-opus-5-5
Link: https://github.com/openwrt/openwrt/pull/25674
Signed-off-by: Robert Marko <robimarko@gmail.com>
Starting with version 2026.10, the folder name inside the tarball
has changed: an additional `v` is prepended to the directory name.
Signed-off-by: Shiji Yang <yangshiji66@outlook.com>
Link: https://github.com/openwrt/openwrt/pull/25778
Signed-off-by: Robert Marko <robimarko@gmail.com>
The compressed LoongArch kernel (vmlinuz.efi) differs between builds in
different directories. The code and data of vmlinux are identical, but
its GNU build ID is not: the debug information of vmlinux holds a
single absolute path, the include directory of the compiler, e.g.
/builder/shared-workdir/build/staging_dir/toolchain-loongarch64_generic_gcc-14.4.0_musl/lib/gcc/loongarch64-openwrt-linux-musl/14.4.0/include
asm/loongarch.h includes the compiler's larchintrin.h, so core files
like arch/loongarch/kernel/cpu-probe.c refer to it. The prefix maps of
the kernel cover the build and the staging directory, but not the
toolchain directory. The build ID hashes all of vmlinux, so it depends
on the build directory, and with it the compressed kernel.
Map the toolchain directory as well, like the staging directory.
On loongarch64/generic, vmlinux built without the map holds this one
absolute path, like the published one; built with it, no absolute path
is left, while .text, .rodata and .data stay byte-identical.
Link: https://github.com/openwrt/openwrt/pull/25648
Signed-off-by: Paul Spooren <mail@aparcar.org>
With CONFIG_TARGET_IMAGES_GZIP enabled, which is the default, ext4
images are gzipped when they are copied to $(BIN_DIR), unless their
name ends with .gz or .dtb. This causes several problems:
- The JSON info contains the uncompressed name and hash, so
profiles.json lists images which don't exist on the download server:
profiles.json: openwrt-at91-sam9x-atmel_at91sam9263ek-ext4-root.ubi
sha256sums: openwrt-at91-sam9x-atmel_at91sam9263ek-ext4-root.ubi.gz
- The image is gzipped as a whole, including the metadata and signature
added by append-metadata. fwtool looks for them at the end of the
file, so sysupgrade rejects these images. Explicitly gzipped images
run append-metadata after gzip instead.
- A rule for the gzipped name is defined for every image, which clashes
with images that are built both plain and gzipped, as it was the case
for malta.
The previous commits removed the ext4 images which can't be used and
gzip the remaining ones explicitly, so drop the implicit gzip. The file
name in $(BIN_DIR) is now always the one from IMAGES. Targets wanting
gzipped images define them explicitly, e.g. sdcard.img.gz, or select
them via CONFIG_TARGET_IMAGES_GZIP like x86, armsr, loongarch64 and
malta. Limit the option to these targets.
The ext4 kernel.bin and factory.bin of ixp4xx, which only contain the
kernel, and the optional ext4 VM images of x86 and armsr are now stored
uncompressed, like their squashfs counterparts.
Link: https://github.com/openwrt/openwrt/pull/25658
Signed-off-by: Paul Spooren <mail@aparcar.org>
Image/Gzip lost its last user with commit df9ffedb75 ("archs38: drop
target") and Image/gzip-ext4-padded-squashfs with commit 8dfa38b82c
("malta: convert to Device macro image building"). The latter also
calls Image/Build/gzip, which isn't defined anywhere.
Remove both, along with Image/pad-root-squashfs, which was only used by
Image/gzip-ext4-padded-squashfs.
Link: https://github.com/openwrt/openwrt/pull/25658
Signed-off-by: Paul Spooren <mail@aparcar.org>
Every image is built for all filesystems in FILESYSTEMS. A device that
needs an image only for some of them, e.g. a flash image for squashfs
and a disk image for ext4, can only restrict FILESYSTEMS for the whole
device, which drops its other images as well.
Add FILESYSTEMS/<image> to build an image only for some filesystems of
the device, similar to IMAGE/<image>/<fs> for per filesystem recipes.
Like the recipes, it is not reset between devices, so it should be set
next to the recipe of the image.
Link: https://github.com/openwrt/openwrt/pull/25658
Signed-off-by: Paul Spooren <mail@aparcar.org>
The aarch64 toolchain is built without --with-cpu, so the
CPU_CFLAGS_<type> entry in target.mk is the only place a -mcpu option
comes from. The cortex-a72 (bcm27xx/bcm2711, mvebu/cortexa72) and
cortex-a76 (bcm27xx/bcm2712) CPU types have no entry, so userspace for
these subtargets is built with plain "-Os -pipe", which produces
generic armv8-a code. The "doesn't correspond to a known type" warning
is not visible because the target scan discards stderr.
Add the missing entries. For cortex-a76 this also enables the ARMv8.2
features such as LSE atomics, which replace the out of line atomic
helper calls.
Fixes: 2534141322 ("mvebu: add support for MACCHIATObin (cortex-a72)")
Fixes: 5e928acf22 ("brcm2708: add bcm2711 subtarget (RPi 4B aarch64)")
Fixes: 8b63d9ac4a ("bcm27xx: add new bcm2712 subtarget")
Assisted-by: Claude:claude-opus-5-5
Signed-off-by: Hauke Mehrtens <hauke@hauke-m.de>
Link: https://github.com/openwrt/openwrt/pull/25627
Signed-off-by: Robert Marko <robimarko@gmail.com>
Build/elx-header and Build/znet-header convert a hex string into \xHH
escape sequences using `fold -s2`. This relies on the obsolete -WIDTH
option syntax of GNU fold, which parses "-s2" as "-s -2". uutils
coreutils does not support this syntax and fails with:
error: unexpected argument '-2' found
tip: to pass '-2' as a value, use '-- -2'
Since fold writes nothing to stdout in this case, the command
substitution expands to an empty string. The following
`dd bs=N count=1 conv=sync` then writes zero bytes, because conv=sync
only pads partial input blocks and never pads empty input. The header
field is dropped entirely, and all subsequent fields move to lower
offsets.
The build still succeeds, because the failure happens inside the
subshell which is then handed to echo.
The malformed images are only rejected when a device tries to
boot them.
For example, on the Zyxel WSM20, the missing payload size field in the
znet-header moves the 0x12345678 magic and the payload to unexpected
offsets. The bootloader then rejects the image, even though sysupgrade
accepted it.
This affects all builds on Ubuntu 25.10 and later, because these
releases use uutils coreutils by default.
Replace `fold -s2` with `fold -w2`, which is the documented way to set
the line width and produces identical output with GNU fold.
Affected devices:
- Build/znet-header is used in all images of the Zyxel WSM20 and the
ELECOM WRC-X1800GS
- Build/elx-header is used in the factory images of the ELECOM
WAB-I1750-PS, WAB-S1167-PS, WAB-S600-PS, WRC-2533GHBK-I and
WRC-2533GHBK2-T, the I-O DATA WN-AC1167GR, WN-AC733GR3 and WNPR2600G,
the Edimax RA21S and RG21S, and the D-Link DAP-X1860 A1
Signed-off-by: Grische <github@grische.xyz>
Assisted-by: Claude:claude-opus-5
Link: https://github.com/openwrt/openwrt/pull/25410
Signed-off-by: Hauke Mehrtens <hauke@hauke-m.de>
UBIFS root file systems differ between builds, e.g. the one of the
Microchip SAMA7G5-EK: the superblock gets a random UUID, the inodes get
the ctime of the build host, which can't be reset with touch, and the
entries of each directory are added in readdir() order, which depends on
the host file system and decides inode numbers and layout.
Add a patch that, when SOURCE_DATE_EPOCH is set, uses a time-based
(version 1) UUID with that time and clamps all inode times to it. The
node ID of the UUID is taken from the new --uuid-node option, which the
image build sets from IMG_PART_DISKGUID, itself derived from
SOURCE_DATE_EPOCH and the kernel vermagic.
Directory entries are added sorted by name. Two trees with the same
contents but different ctimes and readdir() order now give identical
images; without SOURCE_DATE_EPOCH the UUID stays random.
Link: https://github.com/openwrt/openwrt/pull/25430
Signed-off-by: Paul Spooren <mail@aparcar.org>
The temporary directory used to create a kernel UBIFS image and its
kernel file receive wall-clock mtimes. mkfs.ubifs preserves these
times, making images differ between builds.
Set their timestamps to SOURCE_DATE_EPOCH before invoking mkfs.ubifs.
Signed-off-by: Robert Marko <robimarko@gmail.com>
Kernel/SetInitramfs writes the initramfs configuration into the kernel
tree and deletes .config.prev, so that the next configure stage puts the
regular configuration back. The check that decides if the configure stage
must run ignored this signal. The next build then compiled the regular
kernel with the initramfs configuration, and the squashfs images got a
kernel with an embedded root filesystem.
Run the configure stage if .config.prev is missing.
Fixes: e97e3f36bd ("kernel: configure the kernel only when an input changed")
Signed-off-by: John Crispin <john@phrozen.org>
Nothing tests either symbol any more. AUDIO_SUPPORT was also selected by
PCI_SUPPORT and USB_SUPPORT, which stood for "a sound card could be
plugged in here": audio hardware sits behind MMC, SPI, a Bluetooth UART
or an SoC PCM block just as often, so the inference never held. The
display feature was derived from a literal CONFIG_VGA_CONSOLE or CONFIG_FB
in a kernel fragment, which misses every target whose fbdev came from DRM
emulation.
Delete both symbols, the two selects, the feature translations and the
derivation.
Signed-off-by: Daniel Golle <daniel@makrotopia.org>
The U-Boot script embedded in the GL.iNet QSDK factory image hardcodes
the GL-B3000 flash layout. Replace the UBI offset and size with
placeholders filled in by the gl-qsdk-factory recipe, so other GL.iNet
boards with a different layout can use the same script.
The script is copied under a per-device name, which keeps the FIT
description unchanged and avoids devices overwriting each other's
substituted copy in KDIR_TMP.
Signed-off-by: Gilberto Conde <gilbertorconde@gmail.com>
CompileBPF calls clang with -g but without any prefix map. Stripping debug info
keeps .BTF and .BTF.ext, since libbpf needs them, and their line info carries
the absolute path of each source file. This makes BPF objects like qosify-bpf.o
depend on the build directory and breaks reproducibility:
-/builder/shared-workdir/build/build_dir/.../qosify-bpf.c
+/tmp/rebuilderd-openwrt.juCsuQ/src/build_dir/.../qosify-bpf.c
IREMAP_CFLAGS is not enough here: without CONFIG_REPRODUCIBLE_DEBUG_INFO it
only sets -fmacro-prefix-map, which leaves the line info alone, and it is empty
for external toolchains, which have no say over the host clang. Unlike regular
debug info, BTF ships with the package, so always pass -ffile-prefix-map for
the build and staging directories.
Add an fremap helper next to iremap for that, and let IREMAP_STAGING_DIR and
CompileBPF share the map of the staging directory, so they cannot drift apart.
Link: https://github.com/openwrt/openwrt/pull/25359
Signed-off-by: Paul Spooren <mail@aparcar.org>
Instead of carrying 100-no-testing.patch, pass -DBUILD_TESTING=OFF
directly to CMake's bootstrap script. This avoids refreshing the patch
on every CMake update.
The patch also changed the default in the installed CTest.cmake module.
Preserve that behavior by passing -DBUILD_TESTING=OFF by default in
cmake.mk for target and host builds. Packages can override it via
CMAKE_OPTIONS or CMAKE_HOST_OPTIONS.
OpenWrt never runs CMake tests, so building them only wastes build time.
This replaces the patch introduced in 8af80e79bd ("tools/cmake:
update to 3.17.0").
Signed-off-by: Josef Schlehofer <pepe.schlehofer@gmail.com>
Link: https://github.com/openwrt/openwrt/pull/24403
Signed-off-by: Robert Marko <robimarko@gmail.com>
Three cases kept a cached answer that no longer held:
A new configure script. A package with an autoreconf fixup regenerates
configure with the staged autoconf, automake and libtool, and a new
version of one of them can change what a cache variable means. The site
script now keeps a checksum of configure beside the cache and drops the
cache when the checksum changes. A new patch to configure is covered too.
A changed precious variable. configure aborts when LIBS, CPP or another
precious variable differs from the value in the cache, and because
nothing removes the cache, it aborts on every later run as well:
configure: error: 'LIBS' was not set in the previous run
configure: error: changes in the environment can compromise the build
The site script now makes the same comparison before configure does and
drops the cache on a difference.
Paths under other names. The libtool answers lt_cv_path_*, acl_cv_path_*
and lt_cv_sys_global_symbol_*, and gmp_cv_prog_m4, hold the path of a
system tool or of the staged sed and m4, and survived a wipe of
staging_dir. The site script now drops them with the ac_cv_path_ answers.
m4 and gmp still configure the same way with a warm cache as they do
without one, and still read it: the m4 configure stage takes 1.5 s warm
against 23 s cold.
Signed-off-by: John Crispin <john@phrozen.org>
The initramfs step copied /init into the root filesystem directory and
never removed it again. With CONFIG_TARGET_PER_DEVICE_ROOTFS the
initramfs kernel and the rootfs image of a device come from that same
directory. Nothing orders the two steps, so /init entered the squashfs
image or not, depending on which step ran first. The reproducible build
service reports this as a difference between the published image and a
rebuild.
Add /init to the initramfs from outside the root filesystem instead. A
built in initramfs gets it through a gen_init_cpio list file. A separate
cpio gets it appended to the finished archive. The mode is now fixed at
0755 and no longer follows the umask of the checkout.
The initramfs step also normalised the mtime of the root filesystem
directory. Do that in prepare_rootfs, so no step writes to the directory
after the image build can start.
Signed-off-by: John Crispin <john@phrozen.org>
iremap is called with BUILD_DIR and with PKG_BUILD_DIR, never with
STAGING_DIR. A header taken from the staging directory therefore reaches
__FILE__ and the debug information with its absolute path, and the result
depends on the directory the tree was built in.
Measured in the published snapshot of lantiq/xway: mt76-connac-lib.ko,
mt76x02-lib.ko and mt7925-common.ko hold
/builder/shared-workdir/build/staging_dir/target-mips_24kc_musl/usr/include/mac80211/net/cfg80211.h
That path is __FILE__ in .rodata, so the strip before the package is built
cannot reach it and the module ships it.
Map STAGING_DIR as well. Keep the staging_dir element in the replacement:
the last element of STAGING_DIR and of BUILD_DIR carry the same name, so
the bare name would make the two roots indistinguishable. The map of
BUILD_DIR does not change, so only a path that leaks today changes.
Verified with the target compiler on a header below a mapped directory:
-ffile-prefix-map removes the absolute path from both __FILE__ and the
debug information, -fmacro-prefix-map removes it from __FILE__ alone. The
shipped half is __FILE__, so both variants cover it.
A local build of mt76 also holds nine staging paths, but those are debug
information: the build directory copy of mt76-connac-lib.ko is 653456
bytes with 13 debug sections and nine such paths, the shipped copy is
84688 bytes with none. They are named here so that the next reader does
not take them for the same defect.
Signed-off-by: John Crispin <john@phrozen.org>
TF-A embeds __TIME__ and __DATE__ in its build message, so bl2 and bl31
differ on every build, and so do all images that contain them.
Pass BUILD_MESSAGE_TIMESTAMP, derived from the source date of the
package, in the same format.
Signed-off-by: John Crispin <john@phrozen.org>
A configure run spends its time on the compiler invocations behind its
checks, not on the shell. tools/m4 runs the compiler 633 times and takes
24 seconds. ccache does not help, because it refuses to cache an
autoconf conftest compile at all; it counts these separately as
"Autoconf compile/link" under its uncacheable calls.
A package that sets PKG_CONFIGURE_CACHE or HOST_CONFIGURE_CACHE keeps a
config.cache under tmp/, so a package that is cleaned and built again
reads the answers instead of running the checks. The configure stage of
tools/m4 drops from 24.5 s to 1.5 s. dirclean removes tmp/, so a full
reset of the tree starts from cold answers.
A package has to opt in, because a cache is only correct for a configure
script that keeps its side effects outside the AC_CACHE_VAL body.
ncurses appends -D_XOPEN_SOURCE and -D_DEFAULT_SOURCE to CPPFLAGS inside
that body, so a cached run restores the value, skips the append, and
then fails to compile wcwidth. ncurses_cfg.h is identical in both cases;
the difference is in the generated Makefile.
The path covers everything else that changes an answer. A target package
is keyed on the toolchain directory, which carries the gcc version. A
host package is keyed on the version string of the host compiler,
because autoconf cannot detect a new compiler behind an unchanged name;
gcc 14 turned an implicit function declaration into an error and changed
many answers that way. Both are keyed on a hash of the configure
arguments and the build flags, and autoconf refuses a cache written with
different flags on top of that.
A cached program path outlives a wipe of staging_dir, so ac_cv_path_ and
ac_cv_prog_ answers are dropped when the cache loads and configure
searches PATH again. Without that, a cache written before staging_dir
held sed pins /usr/bin/sed for good. A search runs no compiler, so this
costs nothing that matters.
The cache arrives through a site script rather than --cache-file, so a
hand written configure script never sees a new argument. The site script
removes the file of the previous arguments, so a different configuration
gets a cold cache instead of an autoconf error. It also unsets the
variable, and make exports it for the configure stage alone, so a nested
configure run keeps its own cache.
Set CONFIGURE_CACHE_DIR to keep the cache elsewhere, or to the empty
string to run every check again.
Host builds now read a site script on Linux as well. Before this change
only Darwin had one.
CONFIG_SITE holds one path name: autoconf before 2.70 reads the whole
value as that one name, so a second path would silence both scripts. It
names the cache script alone, and the cache script loads the site script
of the architecture from CONFIG_SITE_BASE.
Signed-off-by: John Crispin <john@phrozen.org>
libdeflate-gzip -12 of a kernel takes about a second, and its input
usually did not change. Run it through cache-run.sh, as Build/lzma.
libdeflate-gzip stores no mtime or file name, and -k -S .new gives the
same output as -c.
Signed-off-by: John Crispin <john@phrozen.org>
On a rebuild, the initrd and the initramfs kernel image usually have
the same content as before, but xz and lzma compress them again on
every build. For the OpenWrt One this takes about 6 s of the build.
Add scripts/cache-run.sh. It runs a command that creates one file from
another, and stores the output together with a hash of the input, the
command line and the tool. If they are the same in the next run, it
copies the stored output instead. The commands are deterministic, so
the result is identical.
On a hit the script compares the stored output with the file that is
already there and copies only when the two differ. The output then keeps
its timestamp, so a later step that compares timestamps does not treat it
as new work.
Use it for the initrd compression and for Build/lzma. bzip2 and lzop
stay as they are: bzip2 writes to stdout, and lzop stores the mtime of
its input. xz and gzip now keep the cpio, as the other tools do, so a
cache hit leaves the same files behind as a normal run.
The slot name follows the path of the output below $(KDIR), because two
targets in different directories can share a basename. cache-run.sh holds
a lock over the slot and swaps the stored output in with a rename, so two
image recipes on one slot never read a half written pair.
Signed-off-by: John Crispin <john@phrozen.org>
check_download_integrity hashes the downloaded file on every parse of
every Makefile that uses it. With large archives this costs seconds per
build, for example about 1.2 s for linux-firmware, which is on the
critical path of the package build.
Remember the identity (inode, size, mtime, ctime) of a file that
matched its expected hash, and hash it again only after it changed.
The identity comes from the stat of tools/coreutils, which a Linux host
does not build. Fall back to the stat of the host, and drop the cache
when neither one answers, so that a missing stat costs time and never
trust.
Signed-off-by: John Crispin <john@phrozen.org>
prepare-tmpinfo runs the metadata scan for the packages and for the targets
on every build. Each run starts two more make processes and walks the whole
directory, which takes 0.15 s per scan even when it finds nothing to do.
Keep a stamp per scan and skip the scan while no Makefile, no .mk file and
no directory below the scanned tree is newer than the stamp, and while
include, rules.mk and .config are unchanged as well.
package/kernel/linux reads target/linux/*/modules.mk through SCAN_DEPS, so
the package scan watches those files although they sit in the target tree.
This takes the time of the prereq step from 0.45 s to 0.21 s, and the time
of a full no-op build from 3.40 s to 3.25 s for the OpenWrt One.
Signed-off-by: John Crispin <john@phrozen.org>
The configure stage rewrites the kernel configuration on every build. It
runs kconfig.pl twice and package-metadata.pl once, which takes an eighth of
a second. target/compile and target/install both pay it.
The rule carries FORCE because the kernel configuration also depends on the
package metadata, which is not a prerequisite. Add a check file that the
rule creates only when the configure stage must run, and make the stamp
depend on it. The check covers the kernel configuration files, .config, the
package metadata, the scripts that read them, the two build system files
that hold the recipe, the target directory and the user headers.
This takes the time of target/linux install on an unchanged tree from 1.33 s
to 1.15 s, and the time of a full no-op build from 3.68 s to 3.40 s for the
OpenWrt One.
Signed-off-by: John Crispin <john@phrozen.org>
target/linux install runs the whole image build on every build. It creates
the root file system image, the kernel images, the FIT containers and the
factory images again, although the inputs are the same. This takes 0.7 s for
the OpenWrt One and much more for a target with many devices.
Record a state after a successful image build:
- a hash of the target directory, because the build normalises the
timestamps of the root file system and they cannot report a change,
- the name and the size of every image in the output directory, so that a
deleted image brings the build back,
- the name and the size of every file below json_info_files, because
image_prepare drops that directory and profiles.json comes from it.
Skip the image build while the state is the same and no file of the kernel
build directory, the kernel tree, the image directory of the staging
directory, the host tools, the target directory of the tree, include,
scripts or .config is newer than the state file.
This takes the time of target/linux install on an unchanged tree from 1.77 s
to 1.33 s, and the time of a full no-op build from 4.11 s to 3.72 s for the
OpenWrt One.
Signed-off-by: John Crispin <john@phrozen.org>
Josef Schlehofer
Daniel Golle
Julius Bairaktaris
Robert Marko
Sander van Deijck
John Audia
Paul Spooren
Grische
Shiji Yang
Ryan Leung
Hauke Mehrtens
John Crispin
Gilberto Conde