* feat: add snprintf/sprintf hook
* test: add unit tests for SnprintfHook
* fix: resolve isort lint errors in tests
* fix: resolve black formatting and mypy type errors
The OSS database rebuild changes build-specific fields such as
file_path (e.g. CMakeFiles/zlibstatic.dir/inffast.c.obj). Drop the
hardcoded file_path and function_name asserts and keep only stable
metadata.
Previously only non-static values were rejected, silently allowing
--summary --string-type static. --summary is its own static-only view, so
any --string-type/--no-string-type is now an argument error.
Undo the opt-in default from 0abe0c2 (default now runs all string types,
deobfuscation included) per PR #1376 discussion — the final behavior is
still being debated. The interactive deobfuscation prompt stays removed.
The --summary static-only shortcut and --analyze-functions are unchanged.
The interactive deobfuscation prompt (and the confusing -y/--yes /
--no-prompt flag that skipped it) is removed entirely. String deobfuscation
is now opt-in: a plain run defaults to static (and language) strings only,
and stack/tight/decoded must be requested explicitly via --string-type.
--analyze-functions requests function-level deobfuscation, so it is not
disabled by the default. This resolves the silent redirect behavior from
#1176: there is no implicit 'n' anymore, just an explicit default.
--yes implied it enables deobfuscation, but the flag skips the prompt and
defaults to not running it, which confused reviewers. --no-prompt states
the intent. The dest and Options field become no_prompt (store_true).
Only gp.StringHashDatabase checked for a Git LFS pointer; the oss, winapi,
and expert loaders failed with a confusing BadGzipFile/msgspec error on a
fresh clone without git lfs pull. A shared ensure_not_lfs_pointer() helper
now raises the actionable message, and gp's loader is refactored onto it.
--summary auto-disables stack/tight/decoded and --analyze-functions
disables static, so the combination silently left every string type
disabled and rendered an empty summary. Since the summary only covers
static strings, which --analyze-functions cannot show, the two are
incompatible and now raise a clear argument error.
PELayout/ELFLayout mark_structures overrides only recursed into children,
never applying structures_by_address to their own strings, so strings in
the ELF header/program-header gap (attached to the root node) and PE
header-gap strings rendered with an empty .structure. Both now annotate
their own strings before recursing, matching MachOLayout.
The key was sha256+version only, so interpreting the same bytes under a
different --format (sc32 vs sc64) could serve the other variant's
stack/tight/decoded results. The CLI already disables caching for
non-auto formats, but API callers passing cache_dir were unprotected.
Format is now part of the key, so variants never collide.
The help text says -y defaults to not running deobfuscation, but the
prompt branch -y bypasses was the only place stack/tight/decoded got
disabled, so a non-interactive Go/Rust run with -y ran full deobfuscation
anyway (only .NET was force-disabled). -y now skips the prompt and
disables deobfuscation, matching the documented default.
XOR-obfuscated headers append ' (XOR decoded with key: 0x...)' to the
layout name, so is_structured_layout()'s exact 'pe'/'elf' match failed
and compute_layout() silently fell back to classic static strings,
losing section names, code/reloc/XOR tags, and structure annotations.
Strip any parenthetical suffix before matching.
This PR fundamentally revamps the terminal summary display mode for clearer navigation and density.
Architectural Improvements:
- Integrates `interesting` strings inline directly beneath their exact binary sections in the structural map.
- Strictly aligns summary rendering boundaries to precisely mirror the main FLOSS display spacing and aesthetic (e.g. gray offsets).
- Enforces a 5-string max sampling quota per section to maintain strict summary conciseness without overwhelming the console layout on large binaries.
- Establishes a clean visual section separation utilizing explicitly bracketed headers and empty newlines, entirely stripping out extraneous table headers and placeholders.
- Unified the concept of "interesting strings" across both the LayoutFilter (--interesting) parser and the summary output. We now correctly drop strings mapped purely to noisy tags (#code, #common), but successfully rescue any strings invoking highlight rules (e.g. valid #capa rules hitting code intersections).
TAG=agy
load() probed the cache entry with path.is_file() unguarded, but on
Python 3.8+ PermissionError from stat() through a locked cache directory
is not swallowed (only ENOENT-like errnos are), so it crashed the
analysis instead of treating the entry as a miss.
FLOSS_CACHE_ENABLE=False or FLOSS_CACHE_REFRESH=True were evaluated
verbatim, so the uppercase variants silently did nothing. Normalize the
value with .lower() so 0/false/no/n and 1/true/yes/y work regardless of
capitalization.
materialize() filtered the document to the requested -n but left
metadata.min_length at the cached extraction threshold, so --json on a
hit (e.g. -n 6 against a -n 4 entry) advertised min_length: 4. Set it to
the requested value, matching what load() does for user-supplied
documents and what a fresh run reports.
cache_dir.mkdir() in store() and tempfile.mkstemp() in _write_atomic()
ran outside the OSError guard: an unwritable cache directory or a full
disk raised straight through and aborted the analysis. Both now log a
warning and skip caching.
Deleting a cache entry manually is the only forced-refresh path today, and
FLOSS_CACHE_ENABLE=0 disables reads and writes entirely. FLOSS_CACHE_REFRESH=1
bypasses the cache on the current run and overwrites the entry with the
fresh document, documented in --help alongside the other envars.
the new materialize tag tests asserted layout.strings without proving the
layout is present, failing the CI mypy check. assert layout is not None
first so the union narrows.
covers() treated surplus tag data as a miss: if not wanted.enable_tags and
cached.analysis.enable_tags was a miss, forcing a full re-analysis just
because the cache holds more than requested. Tag false-positive cleanup
(remove_false_positive_lib_strings) only redacts tags, never strings, so
the cached data is a faithful superset.
covers() now accepts it, and materialize() redacts the tags across the
static, language, and layout-tree strings when tags are not wanted.
load() removed stale entries with path.unlink() and store() replaced the
entry with os.replace(), neither guarded. On Windows a cache file held
open by another process or an antivirus scanner raises PermissionError,
which propagated and aborted the analysis.
Writes and removals are now best-effort: _write_atomic() logs and skips
on any OSError during mkstemp/write/replace and cleans up the temporary
file, and _drop_cache_entry() logs and continues when an entry cannot be
removed.
A cache hit for a document that holds more string types than the user
requested previously left those arrays populated: covers() treats a
superset as a hit, but materialize() only flipped the analysis flags.
With --json the output then contained data a fresh run would omit.
materialize() now mirrors a fresh run: disabled types are cleared from
the document (static also drops the layout that holds them) and the
enable_layout/enable_tags flags are synced to the requested analysis.
covers() no longer treats a cached layout as a miss for a no-layout
request: materialize() now drops the layout when it is not wanted. Tags
remain a miss because a tags-enabled document has already had
false-positive strings removed. Move the --analyze-functions comment
inline with the caching guard it documents.
msvcrt.locking cannot lock a byte range past EOF, so the lock on an
empty lock file always failed on Windows and store() never wrote a cache
entry. Ensure the lock file has at least one byte before locking.
test_get_cache_dir_default asserted the default path ends with 'floss',
but the Windows default is %LOCALAPPDATA%\floss\Cache. Assert it
equals platformdirs' user_cache_dir('floss') instead.
Add an autouse fixture that points FLOSS_CACHE_DIR at a throwaway
directory per test so the suite never touches the real platform cache.
Add integration tests for a cache hit (identical output, hit surfaced
via -d), FLOSS_CACHE_ENABLE=0, no caching for user-supplied JSON result
documents, and non-default analysis variants (explicit --format).
Loading a saved results document with a --minimum-length below the value used
at extraction silently drops strings that were already gone at extraction time
and cannot be recovered. Instead of just warning, abort with InvalidLoadConfig
so the user is not misled into thinking the smaller threshold applies.
Remove two redundant tests: test_filter_tag_family_winapi duplicated
test_filter_by_tag (the winapi family is a single-element set, so the family
and direct-tag paths are indistinguishable), and test_main_columns_accumulate
duplicated test_main_columns_flag and would pass even if accumulation broke.
Add coverage for previously untested branches:
- test_filter_tag_normalization: #/case-insensitive tag matching
- test_summary_section_counts_thread_fat_macho: analyze_layout Mach-O
fat-arch section threading
- test_layout_none_warns_filters_ignored: the warning emitted when an
active layout filter has no layout tree to apply to
--summary only covers static strings, so a --string-type/--no-string-type
selection for stack/tight/decoded/language is a shadow arg that does nothing.
Reject it at parse time instead of silently ignoring it; --summary alone (or
with an explicit static selection) still works and skips the slow
deobfuscation for the recovered string types.
Now that -y suppresses the deobfuscation prompt, the manual --language override
tests can exercise the real floss.main entry point via JSON output instead of
calling the pipeline directly.
Drives floss.main with --string-type language and the new -y flag instead of
calling the pipeline directly to dodge the deobfuscation prompt, so the test
exercises the real entry point and the prompt-suppression flag it relies on.
--language auto is the default and there's no value in a spot-check test for
one of many possible choices; keeping it invites testing every type and arg.
Loading a saved results document with a --minimum-length below the value used
at extraction silently drops any shorter strings, but those strings were
already gone at extraction time and cannot be recovered. Warn so users aren't
misled into thinking the smaller threshold applies.
Loading a saved results document and disabling static strings (--no-string-type
static) previously dropped the entire metadata table, because the layout branch
matched on layout presence alone and the inner static-enabled check swallowed
the output. When a layout exists but static strings are disabled, fall back to
the classic metadata view (file path, hashes, language) instead of rendering
nothing. Also warn when layout-aware filters are active but no layout tree
exists, so --section/--tag/--query/--max-strings/--interesting don't silently
no-op on unsupported formats.
- --language choices are now auto/go/rust/none per spec 3.4.7; dotnet is
rejected because .NET extraction is unimplemented
- --interesting now drops any string carrying a noisy tag, even alongside a
non-noisy tag (#winapi #common is dropped), matching the spec's 'excludes
the noisy tags' wording
The earlier guard only honored a manual --language go/rust when auto-detection
returned unknown. When auto-detection wrongly identified a different language
(e.g. rust), the manual override was ignored and the auto-detected language was
used. A concrete manual selection now always wins; the language version is kept
only when it matches the selection. Also make the missing-file DB test robust
by using tmp_path instead of a hardcoded /nonexistent path (which raises
PermissionError on some Linux CI environments).
The summary's layout-derived sections (section counts, tag histogram,
high-value strings) cover static strings only, but --summary still spun up the
slow stack/tight/decoded deobfuscation. When --summary is requested and no
string types were explicitly selected, disable the recovered string types;
explicit --string-type/--no-string-type selection overrides the default.
render() only emitted the known TOP_LEVEL_KEYS, so a future document with an
additional top-level field would silently lose it. Append any extra keys after
the fixed ordering (metadata first), preserving detect_file_type behavior.
Add tests for the extra-key guard and the manual --language override.
--columns used store (last wins) while the other filter flags extend
(accumulate). Switch to extend with an empty default and fall back to
DEFAULT_COLUMNS in render when nothing was passed, so repeated --columns
flags accumulate and the default behavior is unchanged.
