54 Commits
Author SHA1 Message Date
iceman1001 a8fcc081b5 add NULL checks 2026-09-15 08:17:05 +02:00
iceman1001andClaude Opus 5 (1M context) 5b909402b5 BigBuf: stop narrowing sizes, and size the 14a modulation buffer to fit
Three things that all cost memory or correctness once BigBuf is larger
than 64KB, which it already is on PM5.

BigBuf_max_traceLen() returned a uint16_t while s_bigbuf_hi is a
uint32_t. At 33272 on AT91 that is harmless; on AT32, where BigBuf runs
to several hundred kbyte, it truncates and LF sampling asks for a
fraction of the buffer that is there. Widened, along with the four
callers that assigned it straight back into a uint16_t.

BigBuf_malloc() and BigBuf_calloc() took a uint16_t chunksize, so a
request of exactly 65536 arrived as 0 and anything above wrapped. It
returned NULL for the 65536 case by accident, not by design. Both take a
uint32_t now and the guard tests the upper bound explicitly, so an
oversized request fails like any other allocation that does not fit.

The 14a tag simulation allocated its dynamic modulation buffer as a flat
512, or 4096 for ST25TA. prepare_tag_modulation() memcpy's the encoded
answer out of the ToSend buffer and tosend_stuffbit() hard caps that at
TOSEND_BUFFER_SIZE, so 1788 of ST25TA's 4096 could never be reached --
it was the largest single allocation of any tag simulation for nothing.
Meanwhile 512 is 68 bytes short of what a full 64 byte response encodes
to, at 9 bytes per byte plus 4, so those answers were refused at
modulation time.

Both are now derived from the response size and capped at what ToSend
can hold: 580 for the default tag types, 2308 for ST25TA. That also
fixes the reason the 4096 never helped -- all six call sites passed the
512 macro as max_buffer_size rather than the size actually allocated, so
ST25TA allocated 4096 and then bounds checked against 512.

Worst case for a simulation that also holds emulator memory drops from
12247 bytes of BigBuf to 10459.

Built for client, RDV4 and PM3GENERIC. Not yet run on hardware.

Co-Authored-By: Claude Opus 5 (1M context)
2026-09-14 13:06:14 +02:00
iceman1001andClaude Opus 5 (1M context) 05137e825a hold the spiffs file open across an upload
SPIFFS has no directory.  SPIFFS_open() by name walks the object lookup page of
every block and reads the header of each index page it meets to strcmp the name,
so reopening per packet makes an upload cost packets x filesystem size.  On a 2MB
part one open reads about 120KB of flash - storing a 512KB file spent 16MB of
reads finding the same file 131 times over, and 249MB back when a packet was one
256 byte flash page.

append_to_spiffs() now keeps the descriptor when the next packet names the same
file.  Everything else calls spiffs_close_cached() on entry, so a held descriptor
is closed, and its writes flushed, before any other operation can look at the
file - all 15 direct SPIFFS_* call sites in this file are covered.  No protocol
change: the CMD_SPIFFS_UNMOUNT the client already sends at the end of an upload
closes it, as does any other SPIFFS command.  rdv40_spiffs_unmount() closes
before SPIFFS_unmount() since descriptors do not survive a mount, and the close
checks the mount state so a dead handle is dropped rather than passed on.

Flash reads to store a file, measured on a host build of this filesystem:

  2MB fs, 512KB file, 4029 byte packets   16302 KB -> 371 KB
  2MB fs, 512KB file,  256 byte packets  249415 KB -> 914 KB
  2MB fs, 256KB file, 4029 byte packets    7834 KB -> 313 KB
  256KB fs, 80KB file, 589 byte packets    1483 KB ->  82 KB

Written bytes are unchanged; only the searching went away.

Verified on the host against an order sensitive pattern - content byte identical,
size right, SPIFFS_check clean - both for a straight upload and for one with a
stat and an fsinfo landing mid transfer, which is the case that catches a missed
invalidation.  Then on hardware: 80KB uploaded, 'mem spiffs tree' in between to
force a flush, downloaded again, byte identical.

Co-Authored-By: Claude Opus 5 (1M context)
2026-09-12 15:54:40 +02:00
iceman1001andClaude Opus 5 (1M context) 9a8e179266 fix spiffs downloads >= 64K, and stop losing flash errors
CMD_SPIFFS_DOWNLOAD read the whole file into BigBuf first.  BigBuf_calloc()
takes a uint16_t while the size is a uint32_t, so a 64K file allocated zero
bytes and anything larger wrapped to a short buffer the file was then read
straight past the end of.  rdv40_spiffs_read_stream() opens the file once and
hands out one frame at a time, so the size never reaches the allocator, and the
SPIFFS path now honours start_index.  Verified with an 80K round trip: upload,
download, byte identical.

The same BigBuf_calloc(filesize) wrap was in copy_in_spiffs(); it now copies in
SPIFFS_WRITE_CHUNK_SIZE chunks.

Separately, every flash failure was invisible.  SPIFFS_CHECK_RES() only treats a
negative result as an error and the HAL hooks returned 128/129/130, so SPIFFS
saw success.  The erase path was worst: Flash_Erase4k() only reports that the
command was sent, both Flash_CheckBusy() results were discarded, and
'return (SPIFFS_OK == erased)' yields 1 on failure.  A flash dump from a device
that hit this showed one block written without being erased - its object index
header held '0x14000 & 0x10000', and the filename ANDed away to empty.  Erases
are now read back and retried, and the HAL returns real SPIFFS error codes.

Flash_Write() returned len unconditionally even after a rejected page, which
also defeated the 'res == payload->len' check in CMD_FLASHMEM_WRITE.

Feedback, so none of this is silent again: CMD_SPIFFS_WRITE answers with the
SPIFFS result, the client aborts on the first refusal and names the byte it
stopped at, dl_it() checks the terminator status, and both transfer directions
print inline progress.

Co-Authored-By: Claude Opus 5 (1M context)
2026-09-12 14:58:08 +02:00
iceman1001 74f1936132 convert to our calloc instead since we prefer to know allocated shared memory is empty. Also removed a malloc(1) which is just a waste of bytes just like @NVX said a year ago at DefCon 2025-05-25 09:55:32 +02:00
iceman1001 4c97151ecc style 2025-03-22 15:46:55 +01:00
iceman1001 c0af6cd7d2 fiddled with pm3 client informative text 2025-03-22 13:09:55 +01:00
iceman1001 14d50f7ac5 change mem spiffs tree to show id number in decimal. Previously shown in hex. 2025-02-23 21:05:09 +01:00
Iceman f66d5970be Merge branch 'master' into fix_spiffs_tree_linkname
Signed-off-by: Iceman <iceman@iuse.se>
2024-11-21 18:48:58 +01:00
ANTodorov 99d5a72879 fixed symlink name in mem spiffs tree
the used strtok(...,".")` is cutting the name to the first dot
linkname.txt.lnk -> linkname

with the fix it is
linkname.txt.lnk -> linkname.txt
2024-11-21 14:28:39 +02:00
ANTodorov 7b99bfa3d6 fix reported file/link names when mem spiffs wipe 2024-11-21 12:31:48 +02:00
ANTodorov 48ec109a1e use all spi flash v0.1
* introduced a new communication command CMD_FLASHMEM_PAGES64K to get the number of 64k pages
* "the last page" is a special, holding the dicts and some other stuff, relocated to there
* raised timeout when wiping a mem page (W25Q16 looks a bit slower than W25X20BV)
* loop all pages in Flash_WipeMemory()
2024-11-17 19:03:15 +02:00
iceman1001 39639c803c fix a wrong size when clearning allocated memory 2024-06-15 20:36:11 +02:00
iceman1001 b9a583cdb5 swapped out to use bigbuff memory allocation and also show an empty message 2024-05-21 18:31:51 +02:00
nvx 8dd963d305 make style 2023-08-24 16:34:33 +10:00
Henry Gabryjelski e973aebae6 Workaround SPIFFS lack of const 2023-08-08 22:56:51 -07:00
Henry Gabryjelski b279df2d46 more const - spiffs 2023-08-08 22:50:34 -07:00
iceman1001 3f92d5ee67 style 2022-10-29 01:26:12 +02:00
iceman1001 ba225905d3 changed "mf eload" / "iclass eload" to be able to transfer a file from SPIFFS to EMULATOR memory. Original idea from @natesales 2022-10-26 06:57:21 +02:00
iceman1001 71f96ba1e7 make style 2022-10-16 17:56:12 +02:00
mwalker33 9406ef9fd3 spiffs prep
added define for chuck size
update changelog
2022-09-01 08:15:30 +10:00
mwalker33 96876c0111 Update spiffs.c
patch type size comparison
2022-08-31 19:10:15 +10:00
mwalker33 d8c18e6db8 Prep spiffs for bigger data files
- added spiffs check after flase wipe to force it to update its status
- added spiffs write and append to write in 8192 byte chunks to allow spiffs space to be freed in time.
- fixed spiffs dump to correctly handle issues if it could not allocate bugbuff space.
2022-08-31 18:51:39 +10:00
iceman1001 79cfa1d8fa adapt SPIFFS from 128kb -> 192kb. Increased GC to fit sector size 4kb. (thanks @mwalker33) 2022-08-28 12:21:21 +02:00
Philippe Teuwen b703bb746b Adapting license headers, WIP 2022-01-06 02:20:38 +01:00
Philippe Teuwen 88308ea727 typos 2021-10-10 01:35:45 +02:00
Philippe Teuwen 5b54385347 rename globals 2021-08-21 23:08:26 +02:00
iceman1001 27184d7f5b style 2021-04-08 10:44:31 +02:00
iceman1001 1b59b38058 fix coverity 314850, and textual changes 2021-04-05 08:47:03 +02:00
iceman1001 1b2091932e fix 'mem spiffs view' and bad file stats retrieval 2021-03-10 10:43:49 +01:00
iceman1001 0be35a8e7e style 2020-09-07 10:35:09 +02:00
iceman1001 b356a2e18c debug textual 2020-08-31 01:41:11 +02:00
Philippe Teuwen 4ed57c7c4d make style 2020-08-13 12:25:04 +02:00
iceman1001 ccbfdf9e88 add: mem spiffs wipe 2020-08-08 12:33:12 +02:00
iceman1001 9dca01d1bc mem spiffs tree - textual 2020-08-06 16:16:24 +02:00
iceman1001 4dabccf9db style 2020-07-14 15:44:23 +02:00
iceman1001 3832cf26fd color 2020-06-11 19:20:59 +02:00
Philippe Teuwen 8160131119 fix standalone warnings 2020-05-11 13:48:57 +02:00
Philippe Teuwen 026707b960 arm: fix prototypes 2020-05-11 13:48:57 +02:00
Philippe Teuwen 8bdda55115 Fix prompt colors in log -> spurious space in color macros -> adjust lots of files... 2020-04-22 02:22:55 +02:00
iceman1001 4be59e295f whitespace 2020-02-03 21:37:51 +01:00
Philippe Teuwen 945fb82b48 make style 2019-10-17 13:58:22 +02:00
iceman1001 51022d4329 textual 2019-10-16 14:24:23 -04:00
Ludovic Rousseau a1852eaa4b Fix typos
Thanks to Debian lintian for the reports:
I: proxmark3: spelling-error-in-binary usr/bin/proxmark3 formated formatted
I: proxmark3: spelling-error-in-binary usr/bin/proxmark3 succesfully successfully
I: proxmark3: spelling-error-in-binary usr/bin/proxmark3 Skiped Skipped
I: proxmark3: spelling-error-in-binary usr/bin/proxmark3 standart standard
I: proxmark3: spelling-error-in-binary usr/bin/proxmark3 supress suppress
I: proxmark3: spelling-error-in-binary usr/bin/proxmark3 successfull successful
I: proxmark3: spelling-error-in-binary usr/bin/proxmark3 Succeded Succeeded
I: proxmark3: spelling-error-in-binary usr/bin/proxmark3 Overriden Overridden
I: proxmark3: spelling-error-in-binary usr/bin/proxmark3 aquire acquire
I: proxmark3: spelling-error-in-binary usr/bin/proxmark3 occured occurred
I: proxmark3: spelling-error-in-binary usr/bin/proxmark3 delimeter delimiter
2019-09-14 17:44:58 +02:00
iceman1001 6f9ec6dd24 FIX: 'spiffs : fixes the device going rouge with bad memory reading' Thanks @megabug 2019-08-19 15:18:17 +02:00
Philippe Teuwen 3771c150f0 make style 2019-08-13 17:42:03 +02:00
Philippe Teuwen 0ace6bffb8 missing includes for spiffs 2019-08-13 17:39:48 +02:00
cjbrigato 961f98c125 SPIFFS : FIXES FS corruption against high numbero f current losses, high number of files manipulation. FIXES LOG_BLOCK erasing handlers. FIXES Garbage Collector. Overall, SPIFFS should now be WAY MORE resilient to any form of misbehavior. In case of corrupted filesystem (should still be readable !), spamming 'mem spiffs check' from the client will ensure at least 2 new free 4k blocks each tile (if number of LOG_PAGE available is enough) and in the same pass will move/reorganize the LOG_PAGES of contextual blocks. Yiha 2019-08-13 17:39:48 +02:00
iceman1001 5acc77bc95 chg: mem spiffs info - text output 2019-08-12 16:33:27 -04:00
Philippe Teuwen d19754567d summer restructuring:
* .h include only the strict minimum for their own parsing
  * this forces all files to include explicitment their needs and not count on far streched dependencies
  * this helps Makefile to rebuild only the minimum
  * according to this rule, most standalone .h are now gone
  * big app.h is gone
  * remove seldom __cplusplus, if c++ happens, everything will have to be done properly anyway
* all unrequired include were removed
* split common/ into common/ (client+arm) and common_arm/ (os+bootloader)
  * bring zlib to common/
  * bring stuff not really/not yet used in common back to armsrc/ or client/
  * bring liblua into client/
  * bring uart into client/
  * move some portions of code around (dbprint, protocols,...)
* rename unused files into *_disabled.[ch] to make it explicit
* rename soft Uarts between 14a, 14b and iclass, so a standalone could use several without clash
* remove PrintAndLogDevice
* move deprecated-hid-flasher from client to tools
* Makefiles
  * treat deps in armsrc/ as in client/
  * client: stop on warning (-Werror), same as for armsrc/

Tested on:

* all standalone modes
* Linux
2019-08-11 21:42:01 +02:00