SentinelLABS has conducted an evaluation of OpenAI's context compaction feature, a technique designed to manage and compress the history of interactions for long-running agent tasks. This pattern aims to reduce the volume of input tokens, thereby lowering costs and minimizing noise in the data processed by AI models, without sacrificing the quality of the output. The study focused on applying this compaction method to automated malware analysis, a domain that presents unique challenges for agent memory and state management.
Automated malware analysis is inherently complex for AI agents. The process involves tasks such as identifying key functions, interpreting code paths, analyzing strings and API calls, and renaming components based on observed behavior. To achieve high scores, agents must maintain a coherent theory about the malware, track collected evidence, and manage open questions. This iterative process often involves multiple rounds of investigation, where connections between different parts of the code may be unclear or require significant effort to uncover.
During their evaluations, SentinelLABS observed that agents tended to accumulate a large volume of tokens over time, carrying the full history of the analysis. This is analogous to a human analyst who, while working, compresses their understanding and externalizes detailed notes rather than holding every raw observation in active memory. Compaction addresses this by distinguishing between working memory, which holds the current state and active hypotheses, and durable memory, which stores specific findings and exact artifacts.
SentinelLABS implemented compaction by using it to carry forward the agent's working state, including its current goal, past actions, learned information, active hypotheses, and open questions. Crucial evidence, such as tool outputs and decompiled functions, was stored externally in durable storage. This separation allowed the agent to retrieve exact evidence when needed without relying on the compacted context to preserve it verbatim, thus avoiding the potential loss of critical details during summarization.
The evaluation compared runs with and without compaction enabled. The results showed a significant reduction in input tokens by approximately 86%, with corresponding decreases in output and reasoning tokens, and a reduction in model calls. Crucially, the aggregate evaluation score for the malware analysis task remained effectively unchanged. This indicates that compaction successfully maintained the necessary state for the workflow to continue correctly while dramatically reducing the computational overhead.
However, the analysis did identify a minor drawback: a decrease in the model's ability to recover higher-level domain objects and structures. This suggests that compaction occasionally compressed structural reasoning that could be valuable for later analysis. This finding reinforces the importance of storing exact artifacts in durable storage, as relying solely on compacted context might lead to the flattening of crucial analytical details.
SentinelLABS also explored different implementation methods for compaction. OpenAI offers server-side compaction, which automatically compresses context when a threshold is met, and a standalone compaction endpoint for more explicit control. Other providers like Anthropic and Google, as well as frameworks like LangChain, offer similar approaches under different names. The choice between server-side and standalone compaction depends on the specific use case, with standalone being useful for distinct phase boundaries in multi-stage workflows, and server-side being simpler for long-running coding agents or chat assistants.
