From 584624a6246d8a8ed86193f265739b1bb8aca6d6 Mon Sep 17 00:00:00 2001 From: iceman1001 Date: Mon, 14 Sep 2026 18:12:16 +0200 Subject: [PATCH] fpga_compress: stop rejecting a bitstream set that exactly fills the buffer PLATFORM=PM3ULTIMATE has not built since c0ecb1c62 (2026-04-14), which fixed a real heap overflow by changing one character: - if (total_size > num_infiles * FPGA_CONFIG_SIZE) { + if (total_size >= num_infiles * FPGA_CONFIG_SIZE) { The old '>' checked after the fact, so a round could start with the buffer already full and write num_infiles * FPGA_INTERLEAVE_SIZE bytes past the end. But '>=' answers the wrong question: a buffer that is exactly full is not an error, it is the expected end state for a platform whose bitstreams are sized to FPGA_CONFIG_SIZE. Two things had to change. all_feof() tested the EOF flag, and C only sets that once a read has already run off the end -- consuming the last byte of a file leaves it clear. A file whose length is an exact multiple of FPGA_INTERLEAVE_SIZE therefore still looked unfinished after its final whole chunk, and the interleave loop ran one more round of pure zero padding. It now peeks a byte with fgetc/ungetc instead, so a file read to its last byte counts as finished right away. PM3ULTIMATE is the only platform this reaches: fpga_pm3_ult_felica.bit 69984 = 243 * 288 exactly fpga_pm3_ult_hf_15.bit 69983 fpga_pm3_ult_hf.bit 69980 fpga_pm3_ult_lf.bit 69980 243 rounds * 4 files * 288 = 279936, which is exactly 4 * FPGA_CONFIG_SIZE for 2s50vq144. Round 244 was padding only, and '>=' killed it there. Stock PM3's largest bitstream is 42172, not a chunk multiple, so its last round trips feof naturally and it never gets that far -- which is why this went unnoticed, nothing in CI builds ULTIMATE. The guard now asks whether the next round fits, which is the condition it was always meant to express and is strictly stronger than the original '>': if (total_size + (num_infiles * FPGA_INTERLEAVE_SIZE) > num_infiles * FPGA_CONFIG_SIZE) Verified: PM3ULTIMATE compresses 279936 bytes to 40195 and all four bitstreams decompress back byte-exact. Output is byte-identical to before for stock 2s30vq100 (169344 -> 106628), for icopyx XC3 (72864 -> 27292) and for the '-s' .data section path. fullimage builds for PM3ULTIMATE, PM3RDV4, PM3GENERIC and PM5. Note FPGA_CONFIG_SIZE is now exactly the size of the largest ULTIMATE bitstream. Regenerate that one byte larger and the guard fires again, correctly; bump the constant by one interleave step rather than touching the guard. Co-Authored-By: Claude Opus 5 (1M context) --- CHANGELOG.md | 1 + tools/fpga_compress/fpga_compress.c | 29 +++++++++++++++++++++++++---- 2 files changed, 26 insertions(+), 4 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 678c390c3..9ff998f3f 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -3,6 +3,7 @@ All notable changes to this project will be documented in this file. This project uses the changelog in accordance with [keepchangelog](http://keepachangelog.com/). Please use this to write notable changes, which is not the same as git commit log... ## [unreleased][unreleased] +- Fixed `fpga_compress` - PM3ULTIMATE builds again, its bitstreams are an exact multiple of the interleave size (@iceman1001) - Fixed `hf legic eload` - an upload after a command that left a different FPGA bitstream loaded now writes the whole image (@iceman1001) - Fixed `hf legic sim` - no longer wipes the emulator memory that `hf legic eload` just filled (@iceman1001) - Changed emulator memory - raised from 4096 to 8192 bytes (@iceman1001) diff --git a/tools/fpga_compress/fpga_compress.c b/tools/fpga_compress/fpga_compress.c index 29e5481ec..740d85346 100644 --- a/tools/fpga_compress/fpga_compress.c +++ b/tools/fpga_compress/fpga_compress.c @@ -40,11 +40,27 @@ static void usage(void) { fprintf(stdout, " Decompress . Write result to \n\n"); } +// feof() only reports true once a read has already run off the end, so a file +// whose length is an exact multiple of FPGA_INTERLEAVE_SIZE still looks unfinished +// after its last whole chunk. That earned one more interleave round of pure +// padding, which pushed total_size past the buffer and got the whole set rejected +// as "too big" -- PM3ULTIMATE's bitstreams are exactly 243 * 288 bytes. +// Peek one byte instead, so a file that has been read to its last byte counts as +// finished right away. static bool all_feof(FILE *infile[], uint8_t num_infiles) { for (uint16_t i = 0; i < num_infiles; i++) { - if (!feof(infile[i])) { - return false; + + if (feof(infile[i])) { + continue; } + + int c = fgetc(infile[i]); + if (c == EOF) { + continue; + } + + ungetc(c, infile[i]); + return false; } return true; } @@ -61,9 +77,14 @@ static int zlib_compress(FILE *infile[], uint8_t num_infiles, FILE *outfile, boo uint32_t total_size = 0; do { - if (total_size >= num_infiles * FPGA_CONFIG_SIZE) { + // Each round writes num_infiles * FPGA_INTERLEAVE_SIZE bytes, so ask whether + // that round fits rather than whether we have already filled the buffer. + // `>` alone let a round start with the buffer full and overrun it; `>=` + // rejected a set that exactly filled it, which is what the bitstreams of a + // platform sized to FPGA_CONFIG_SIZE do. + if (total_size + (num_infiles * FPGA_INTERLEAVE_SIZE) > num_infiles * FPGA_CONFIG_SIZE) { fprintf(stderr, - "Input files too big (total >= %li bytes). These are probably not PM3 FPGA config files.\n" + "Input files too big (total > %li bytes). These are probably not PM3 FPGA config files.\n" , num_infiles * FPGA_CONFIG_SIZE );