Three things that all cost memory or correctness once BigBuf is larger
than 64KB, which it already is on PM5.
BigBuf_max_traceLen() returned a uint16_t while s_bigbuf_hi is a
uint32_t. At 33272 on AT91 that is harmless; on AT32, where BigBuf runs
to several hundred kbyte, it truncates and LF sampling asks for a
fraction of the buffer that is there. Widened, along with the four
callers that assigned it straight back into a uint16_t.
BigBuf_malloc() and BigBuf_calloc() took a uint16_t chunksize, so a
request of exactly 65536 arrived as 0 and anything above wrapped. It
returned NULL for the 65536 case by accident, not by design. Both take a
uint32_t now and the guard tests the upper bound explicitly, so an
oversized request fails like any other allocation that does not fit.
The 14a tag simulation allocated its dynamic modulation buffer as a flat
512, or 4096 for ST25TA. prepare_tag_modulation() memcpy's the encoded
answer out of the ToSend buffer and tosend_stuffbit() hard caps that at
TOSEND_BUFFER_SIZE, so 1788 of ST25TA's 4096 could never be reached --
it was the largest single allocation of any tag simulation for nothing.
Meanwhile 512 is 68 bytes short of what a full 64 byte response encodes
to, at 9 bytes per byte plus 4, so those answers were refused at
modulation time.
Both are now derived from the response size and capped at what ToSend
can hold: 580 for the default tag types, 2308 for ST25TA. That also
fixes the reason the 4096 never helped -- all six call sites passed the
512 macro as max_buffer_size rather than the size actually allocated, so
ST25TA allocated 4096 and then bounds checked against 512.
Worst case for a simulation that also holds emulator memory drops from
12247 bytes of BigBuf to 10459.
Built for client, RDV4 and PM3GENERIC. Not yet run on hardware.
Co-Authored-By: Claude Opus 5 (1M context)
SPIFFS has no directory. SPIFFS_open() by name walks the object lookup page of
every block and reads the header of each index page it meets to strcmp the name,
so reopening per packet makes an upload cost packets x filesystem size. On a 2MB
part one open reads about 120KB of flash - storing a 512KB file spent 16MB of
reads finding the same file 131 times over, and 249MB back when a packet was one
256 byte flash page.
append_to_spiffs() now keeps the descriptor when the next packet names the same
file. Everything else calls spiffs_close_cached() on entry, so a held descriptor
is closed, and its writes flushed, before any other operation can look at the
file - all 15 direct SPIFFS_* call sites in this file are covered. No protocol
change: the CMD_SPIFFS_UNMOUNT the client already sends at the end of an upload
closes it, as does any other SPIFFS command. rdv40_spiffs_unmount() closes
before SPIFFS_unmount() since descriptors do not survive a mount, and the close
checks the mount state so a dead handle is dropped rather than passed on.
Flash reads to store a file, measured on a host build of this filesystem:
2MB fs, 512KB file, 4029 byte packets 16302 KB -> 371 KB
2MB fs, 512KB file, 256 byte packets 249415 KB -> 914 KB
2MB fs, 256KB file, 4029 byte packets 7834 KB -> 313 KB
256KB fs, 80KB file, 589 byte packets 1483 KB -> 82 KB
Written bytes are unchanged; only the searching went away.
Verified on the host against an order sensitive pattern - content byte identical,
size right, SPIFFS_check clean - both for a straight upload and for one with a
stat and an fsinfo landing mid transfer, which is the case that catches a missed
invalidation. Then on hardware: 80KB uploaded, 'mem spiffs tree' in between to
force a flush, downloaded again, byte identical.
Co-Authored-By: Claude Opus 5 (1M context)
CMD_SPIFFS_DOWNLOAD read the whole file into BigBuf first. BigBuf_calloc()
takes a uint16_t while the size is a uint32_t, so a 64K file allocated zero
bytes and anything larger wrapped to a short buffer the file was then read
straight past the end of. rdv40_spiffs_read_stream() opens the file once and
hands out one frame at a time, so the size never reaches the allocator, and the
SPIFFS path now honours start_index. Verified with an 80K round trip: upload,
download, byte identical.
The same BigBuf_calloc(filesize) wrap was in copy_in_spiffs(); it now copies in
SPIFFS_WRITE_CHUNK_SIZE chunks.
Separately, every flash failure was invisible. SPIFFS_CHECK_RES() only treats a
negative result as an error and the HAL hooks returned 128/129/130, so SPIFFS
saw success. The erase path was worst: Flash_Erase4k() only reports that the
command was sent, both Flash_CheckBusy() results were discarded, and
'return (SPIFFS_OK == erased)' yields 1 on failure. A flash dump from a device
that hit this showed one block written without being erased - its object index
header held '0x14000 & 0x10000', and the filename ANDed away to empty. Erases
are now read back and retried, and the HAL returns real SPIFFS error codes.
Flash_Write() returned len unconditionally even after a rejected page, which
also defeated the 'res == payload->len' check in CMD_FLASHMEM_WRITE.
Feedback, so none of this is silent again: CMD_SPIFFS_WRITE answers with the
SPIFFS result, the client aborts on the first refusal and names the byte it
stopped at, dl_it() checks the terminator status, and both transfer directions
print inline progress.
Co-Authored-By: Claude Opus 5 (1M context)
* introduced a new communication command CMD_FLASHMEM_PAGES64K to get the number of 64k pages
* "the last page" is a special, holding the dicts and some other stuff, relocated to there
* raised timeout when wiping a mem page (W25Q16 looks a bit slower than W25X20BV)
* loop all pages in Flash_WipeMemory()
- added spiffs check after flase wipe to force it to update its status
- added spiffs write and append to write in 8192 byte chunks to allow spiffs space to be freed in time.
- fixed spiffs dump to correctly handle issues if it could not allocate bugbuff space.
* .h include only the strict minimum for their own parsing
* this forces all files to include explicitment their needs and not count on far streched dependencies
* this helps Makefile to rebuild only the minimum
* according to this rule, most standalone .h are now gone
* big app.h is gone
* remove seldom __cplusplus, if c++ happens, everything will have to be done properly anyway
* all unrequired include were removed
* split common/ into common/ (client+arm) and common_arm/ (os+bootloader)
* bring zlib to common/
* bring stuff not really/not yet used in common back to armsrc/ or client/
* bring liblua into client/
* bring uart into client/
* move some portions of code around (dbprint, protocols,...)
* rename unused files into *_disabled.[ch] to make it explicit
* rename soft Uarts between 14a, 14b and iclass, so a standalone could use several without clash
* remove PrintAndLogDevice
* move deprecated-hid-flasher from client to tools
* Makefiles
* treat deps in armsrc/ as in client/
* client: stop on warning (-Werror), same as for armsrc/
Tested on:
* all standalone modes
* Linux