Fix the build on minimal tcc/BusyBox systems
The schema generator escaped quotes through awk's gsub(), but BusyBox awk drops the backslash in \", so it emitted an unescaped double quote inside a string literal and tcc failed on the generated schema_sql.c. The generator is now printf + POSIX sed only, whose escaping is defined and byte-identical between GNU sed and BusyBox sed. Leading "--" comments and blank lines are stripped before escaping, so the embedded DDL has no quotes left to escape at all (323 bytes instead of 785) while schema/used_networks.sql keeps its comments and stays the source of truth the README block matches. Output is staged in a temp file, asserted to contain the CREATE TABLE, and compiled before it is moved into place; the target also depends on the Makefile now, because a changed generator must re-run even when a stale, already-broken schema_sql.c looks up to date -- which is what kept this failure alive after the first fix. The link also asked for libraries a minimal system has no way to satisfy: -ldl, -lpthread and -lm, needed only because the vendored amalgamation was compiled with SQLite's default configuration. glibc 2.34 and later resolved dl and pthread into libc, and Buildroot-style installs ship no dev library for either or for libm, where tcc aborts the whole link on the first unsatisfiable -l flag. The amalgamation is now built SQLITE_THREADSAFE=0 (single-threaded CLI), SQLITE_OMIT_LOAD_EXTENSION (extensions are never loaded) and SQLITE_OMIT_FLOATING_POINT (fabs is SQLite's only libm reference, and the registry stores INTEGER and TEXT, never REAL), which removes every dl, pthread and math symbol: the binary links against libc alone. The defines reach every translation unit via CFLAGS, since sqlite3.h changes its float typedefs under that last define, and `make print-flags` publishes them so the strict-C89 build and the ASan runner cannot drift out of configuration. Verified: CURRENT_TIMESTAMP unchanged and second-accurate despite the float omission, 105/105 tests with and without sanitizers, strict C89 build clean, generator output identical under GNU sed and BusyBox sed.
This commit is contained in:
parent
b21f946db3
commit
db51d85f50
4 changed files with 104 additions and 25 deletions
1
.gitignore
vendored
1
.gitignore
vendored
|
|
@ -1,5 +1,6 @@
|
|||
# generated from schema/used_networks.sql by the Makefile
|
||||
schema_sql.c
|
||||
schema_sql.c.tmp
|
||||
|
||||
# build artifacts
|
||||
/ipvarch
|
||||
|
|
|
|||
81
Makefile
81
Makefile
|
|
@ -1,51 +1,96 @@
|
|||
# IPVarch - Internet Protocol Version Architect
|
||||
# Primary compiler is tcc (Tiny C Compiler); falls back to gcc automatically.
|
||||
# Primary compiler is tcc (Tiny C Compiler); the commented-out $(shell ...)
|
||||
# line below restores an automatic tcc-then-gcc fallback if you want it.
|
||||
# The vendored SQLite amalgamation (vendor/sqlite3.[ch], downloaded from
|
||||
# https://www.sqlite.org/2026/sqlite-amalgamation-3540100.zip) is compiled
|
||||
# separately with gcc - tcc links the resulting object fine.
|
||||
# separately with $(VCC) (tcc today; gcc works too). 'make check' and the
|
||||
# sanitizer runner build their own vendor objects, since they need gcc/clang.
|
||||
|
||||
#CC := $(shell command -v tcc 2>/dev/null || command -v gcc 2>/dev/null || echo cc)
|
||||
CC := tcc
|
||||
#VCC := gcc
|
||||
VCC := tcc
|
||||
CFLAGS := -Wall
|
||||
|
||||
# Embedder configuration for the vendored amalgamation, chosen so the link
|
||||
# needs no libraries at all: SQLITE_OMIT_LOAD_EXTENSION drops dlopen/dlclose,
|
||||
# SQLITE_THREADSAFE=0 drops the pthread mutex layer (the tool is
|
||||
# single-threaded and never loads extensions) and SQLITE_OMIT_FLOATING_POINT
|
||||
# drops the one libm reference left, fabs. glibc >= 2.34 resolved dl and
|
||||
# pthread into libc, and minimal systems (Buildroot and friends) ship no
|
||||
# libdl.so/libpthread.so/libm.so dev library, while tcc aborts the entire
|
||||
# link when a -l flag cannot be satisfied. Nothing is lost here: the
|
||||
# registry stores INTEGER and TEXT only -- no REAL column, no double in our
|
||||
# sources, and every sqlite3_snprintf format is %s or %d. If a future
|
||||
# schema needs REAL, drop the last define and link -lm.
|
||||
# The defines reach every translation unit, ours included: with
|
||||
# SQLITE_OMIT_FLOATING_POINT sqlite3.h changes its float typedefs, so our
|
||||
# declarations must match the compiled object.
|
||||
SQL_DEFS := -DSQLITE_THREADSAFE=0 -DSQLITE_OMIT_LOAD_EXTENSION -DSQLITE_OMIT_FLOATING_POINT
|
||||
CFLAGS := -Wall $(SQL_DEFS)
|
||||
TARGET := ipvarch
|
||||
|
||||
.PHONY: all check test test-sanitize clean
|
||||
.PHONY: all check test test-sanitize print-flags clean
|
||||
|
||||
all: $(TARGET)
|
||||
|
||||
$(TARGET): ipvarch.c ipvarch_db.c ipvtbl.c ipvarch_db.h ipvtbl.h schema_sql.c vendor/sqlite3.o
|
||||
$(TARGET): ipvarch.c ipvarch_db.c ipvtbl.c ipvarch_db.h ipvtbl.h schema_sql.c vendor/sqlite3.o Makefile
|
||||
$(CC) $(CFLAGS) -o $@ ipvarch.c ipvarch_db.c ipvtbl.c schema_sql.c \
|
||||
vendor/sqlite3.o -lpthread -ldl -lm
|
||||
vendor/sqlite3.o
|
||||
|
||||
# Schema DDL compiled into the binary (single source of truth is
|
||||
# schema/used_networks.sql; regenerate after editing it, e.g. with make).
|
||||
# Emits one C string literal per line into a NULL-terminated array, so
|
||||
# no single literal approaches ISO C90's 509-byte limit and the output
|
||||
# is C89-pedantic-clean.
|
||||
schema_sql.c: schema/used_networks.sql
|
||||
awk 'BEGIN{print "/* generated from schema/used_networks.sql - do not edit */"; print "const char *ipvarch_schema_lines[] = {"} {gsub(/\\/,"\\\\"); gsub(/"/,"\\\""); printf " \"%s\\n\",\n", $$0} END{print " 0\n};"}' $< > $@
|
||||
# Emits one C string literal per line into a NULL-terminated array, so no
|
||||
# single literal approaches ISO C90's 509-byte limit. Leading "--"
|
||||
# comments and blank lines stay in the SQL file for readers and are not
|
||||
# embedded here: the array then holds only live DDL, which carries no
|
||||
# double quotes to escape. Escaping uses printf + POSIX sed only, since
|
||||
# awk's gsub() replacement handling is implementation-defined and BusyBox
|
||||
# awk drops the backslash in \" -- that produced invalid C on a BusyBox
|
||||
# system. This target also depends on the Makefile: a changed generator
|
||||
# must re-run even when the .sql timestamp is older than a stale,
|
||||
# previously broken schema_sql.c.
|
||||
# The output is staged in a temp file, asserted non-empty, and compiled
|
||||
# (-x c -c -o /dev/null; tcc has no real -fsyntax-only and would try to
|
||||
# link) before being moved into place, so a broken generator fails here
|
||||
# instead of surfacing later as a confusing compile error.
|
||||
# Note the doubled $$ in the sed script: make reads $/ as an (empty)
|
||||
# variable reference, which would silently emit an empty array.
|
||||
schema_sql.c: schema/used_networks.sql Makefile
|
||||
{ printf '/* generated from schema/used_networks.sql - do not edit */\nconst char *ipvarch_schema_lines[] = {\n'; \
|
||||
sed -e '/^--/d' -e '/^$$/d' -e 's/\\/\\\\/g' -e 's/"/\\"/g' \
|
||||
-e 's/^/ "/' -e 's/$$/\\n",/' $<; \
|
||||
printf ' 0\n};\n'; } > $@.tmp && grep -q 'CREATE TABLE' $@.tmp \
|
||||
&& $(CC) -x c -c $@.tmp -o /dev/null && mv -f $@.tmp $@
|
||||
|
||||
vendor/sqlite3.o: vendor/sqlite3.c vendor/sqlite3.h
|
||||
$(VCC) -O1 -c vendor/sqlite3.c -o $@
|
||||
# Depends on the Makefile as well: a changed SQL_DEFS/$(VCC) must rebuild the
|
||||
# object, otherwise a stale one compiled with the old flags survives (the same
|
||||
# trap schema_sql.c documents above).
|
||||
vendor/sqlite3.o: vendor/sqlite3.c vendor/sqlite3.h Makefile
|
||||
$(VCC) -O1 $(SQL_DEFS) -c vendor/sqlite3.c -o $@
|
||||
|
||||
# Strict C89 conformance build of our sources (needs gcc; tcc has no such mode).
|
||||
# The vendor object is rebuilt with gcc here: the default tcc-built
|
||||
# vendor/sqlite3.o relies on tcc's libtcc1 runtime helpers, which gcc
|
||||
# cannot link, so check compiles its own copy (this is why check always
|
||||
# rebuilds it and clean removes the cached object).
|
||||
# rebuilds it and clean removes the cached object). No -l flags: see
|
||||
# SQL_DEFS above.
|
||||
check: $(TARGET) schema_sql.c
|
||||
@if command -v gcc >/dev/null 2>&1; then \
|
||||
mkdir -p /tmp/ipvarchtests && \
|
||||
gcc -O1 -c vendor/sqlite3.c -o /tmp/ipvarchtests/sqlite3-check.o && \
|
||||
gcc -std=c89 -pedantic -Wall -Wextra -Wno-long-long -D_FORTIFY_SOURCE=2 -O2 \
|
||||
gcc -O1 $(SQL_DEFS) -c vendor/sqlite3.c -o /tmp/ipvarchtests/sqlite3-check.o && \
|
||||
gcc -std=c89 -pedantic -Wall -Wextra -Wno-long-long -D_FORTIFY_SOURCE=2 -O2 $(SQL_DEFS) \
|
||||
-o /tmp/ipvarchtests/ipvarch-check ipvarch.c ipvarch_db.c ipvtbl.c schema_sql.c \
|
||||
/tmp/ipvarchtests/sqlite3-check.o -lpthread -ldl -lm && \
|
||||
/tmp/ipvarchtests/sqlite3-check.o && \
|
||||
echo "check: strict C89 build OK (/tmp/ipvarchtests/ipvarch-check)"; \
|
||||
else \
|
||||
echo "check: gcc not available, skipping (tcc has no strict mode)"; fi
|
||||
|
||||
# Build-time facts for the sanitizer runner: one source of truth for the
|
||||
# SQLite embedder defines, so an ASan build can never be configured
|
||||
# differently from the real one (same principle as the schema generator).
|
||||
print-flags:
|
||||
@printf '%s\n' $(SQL_DEFS)
|
||||
|
||||
# Validity test suite (python3, stdlib only); each run uses a fresh temp DB.
|
||||
test: $(TARGET)
|
||||
python3 tests/runner.py --bin ./$(TARGET)
|
||||
|
|
@ -56,5 +101,5 @@ test-sanitize: $(TARGET)
|
|||
|
||||
clean:
|
||||
rm -f $(TARGET) /tmp/ipvarchtests/ipvarch-asan /tmp/ipvarchtests/ipvarch-check
|
||||
rm -f schema_sql.c /tmp/ipvarchtests/sqlite3-check.o
|
||||
rm -f schema_sql.c schema_sql.c.tmp /tmp/ipvarchtests/sqlite3-check.o
|
||||
rm -f vendor/sqlite3.o
|
||||
|
|
|
|||
26
README.md
26
README.md
|
|
@ -115,7 +115,13 @@ The table layout lives in `schema/used_networks.sql`, the single source
|
|||
of truth. The Makefile compiles it into the binary at build time (as
|
||||
the generated `schema_sql.c`) — a purely development convenience: the
|
||||
shipped binary stays self-contained, and changing the schema means
|
||||
editing the SQL file and rebuilding. It is created with `IF NOT
|
||||
editing the SQL file and rebuilding. Only the live statements are
|
||||
embedded: leading `--` comments and blank lines stay in the file for
|
||||
readers and are dropped from the generated array, which also keeps the
|
||||
embedded SQL free of double quotes so C escaping can never be the weak
|
||||
point of a build. The generated file is refreshed whenever the SQL file
|
||||
*or* the Makefile changes, so an edited generator can't be undone by a
|
||||
stale artifact. It is created with `IF NOT
|
||||
EXISTS`, so re-opening an initialised database is a no-op:
|
||||
|
||||
```sql
|
||||
|
|
@ -193,10 +199,26 @@ Notes:
|
|||
- `ipvarch_db.c/.h` — used-networks SQLite backend.
|
||||
- `vendor/sqlite3.c`, `vendor/sqlite3.h` — vendored SQLite
|
||||
amalgamation 3.54.1 from https://www.sqlite.org/2026/sqlite-amalgamation-3540100.zip
|
||||
(public domain — free for any use, no attribution required); compiled separately with gcc and linked in, so the
|
||||
(public domain — free for any use, no attribution required); compiled
|
||||
separately with `$(VCC)` (tcc today, gcc works too) and linked in, so the
|
||||
only third-party dependency is vendored and there is nothing to
|
||||
install.
|
||||
|
||||
Nothing links against `libdl`, `libpthread` or `libm`. The amalgamation is
|
||||
built with `SQLITE_THREADSAFE=0`, `SQLITE_OMIT_LOAD_EXTENSION` and
|
||||
`SQLITE_OMIT_FLOATING_POINT` (the `SQL_DEFS` variable in the Makefile),
|
||||
which removes every reference those libraries would otherwise serve —
|
||||
`fabs` is the only math symbol SQLite needs at all. So the binary depends
|
||||
on `libc` alone, which is what lets a tcc-only Buildroot box build it:
|
||||
glibc 2.34 and later folded `dl` and `pthread` into `libc` anyway, minimal
|
||||
distributions often ship no dev libraries at all, and tcc aborts the entire
|
||||
link when a `-l` flag cannot be satisfied. Nothing IPVarch uses is given
|
||||
up: the registry stores INTEGER and TEXT and never REAL, and the tool is
|
||||
single-threaded. If a future schema needs `REAL` columns or `%f` in
|
||||
`sqlite3_mprintf`, drop the last define and add `-lm` back. `make check`,
|
||||
the schema generator and the sanitizer runner all read the same flags from
|
||||
the Makefile, so no build path can drift out of configuration.
|
||||
|
||||
### Code style
|
||||
|
||||
I write IPVarch in what I'll call a Stroustrup hybrid: an opening brace
|
||||
|
|
|
|||
|
|
@ -306,21 +306,32 @@ def build_sanitize(repo):
|
|||
if proc.returncode != 0 or not os.path.exists(schema_gen):
|
||||
return None, "schema generation failed:\n%s" % proc.stderr.strip()
|
||||
|
||||
# The Makefile owns the SQLite embedder defines (SQL_DEFS); ask it for
|
||||
# them instead of duplicating the list, so the sanitized build can never
|
||||
# end up configured differently from the real one. With those defines
|
||||
# the amalgamation references no dlopen/pthread/fabs symbol, so no -l
|
||||
# flags are needed here -- and minimal systems may not even have libdl,
|
||||
# libpthread or libm available to link.
|
||||
proc = subprocess.run(["make", "-s", "print-flags"], cwd=repo,
|
||||
capture_output=True, text=True)
|
||||
sql_defs = [f for f in proc.stdout.split() if f.startswith("-D")]
|
||||
if proc.returncode != 0 or not sql_defs:
|
||||
return None, "make print-flags failed:\n%s" % proc.stderr.strip()
|
||||
|
||||
cmd = [cc, "-O1", "-g", "-fsanitize=address,undefined",
|
||||
"-fno-omit-frame-pointer", "-c",
|
||||
os.path.join(repo, "vendor", "sqlite3.c"), "-o", vendor_obj]
|
||||
"-fno-omit-frame-pointer"] + sql_defs + [
|
||||
"-c", os.path.join(repo, "vendor", "sqlite3.c"), "-o", vendor_obj]
|
||||
proc = subprocess.run(cmd, capture_output=True, text=True)
|
||||
if proc.returncode != 0:
|
||||
return None, "sanitized sqlite build failed:\n%s" % proc.stderr.strip()
|
||||
|
||||
cmd = [cc, "-std=c89", "-O1", "-g",
|
||||
"-fsanitize=address,undefined", "-fno-omit-frame-pointer",
|
||||
"-fsanitize=address,undefined", "-fno-omit-frame-pointer"] + sql_defs + [
|
||||
os.path.join(repo, "ipvarch.c"),
|
||||
os.path.join(repo, "ipvarch_db.c"),
|
||||
os.path.join(repo, "ipvtbl.c"),
|
||||
schema_gen,
|
||||
vendor_obj, "-o", SAN_BIN,
|
||||
"-lpthread", "-ldl", "-lm"]
|
||||
vendor_obj, "-o", SAN_BIN]
|
||||
proc = subprocess.run(cmd, capture_output=True, text=True, cwd=repo)
|
||||
if proc.returncode != 0:
|
||||
return None, "sanitized build failed:\n%s" % proc.stderr.strip()
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue