Java Memory Leak Audit

Systematically analyze Java code for memory-leak and resource-leak patterns, report findings with severity and evidence, and apply minimal, safe fixes.

Operating modes

Determine the mode from the user's request before starting:

• SCAN — analyze and report only. Default when the user says "scan", "audit", "review", "check".
• FIX — analyze, report, then implement fixes. Use when the user says "fix", "remediate", "make the changes". Always show the findings report before editing code unless the user explicitly says to just fix everything.

If unclear, ask once. Never silently rewrite code the user only asked you to review.

Workflow

Step 0 — Evidence triage (only when runtime evidence exists)

If the user provides — or can provide — application logs, GC logs, heap dumps, hprof files, or class histograms, analyze those before the static scan so the scan is targeted rather than blanket:

• Logs / GC logs / runtime symptoms → read references/log-analysis.md. Map each symptom to catalog categories and produce the Symptom Triage table; scan the implicated categories first.
• Heap dumps / histograms → read references/heap-dump-analysis.md. Run programmatic analysis if a shell is available (histogram diff, MAT headless leak-suspects), otherwise guide the user through MAT/VisualVM and interpret what they paste back. The dump's GC-root path names the accumulation point — audit that class directly.
• WebSphere / J9 environments (verbosegc XML logs, javacore*.txt, .phd dumps) → also read references/websphere.md; the formats and enablement steps differ from HotSpot.
• No runtime evidence but an active production incident → recommend the capture commands from heap-dump-analysis.md §1 (histogram first — it's safe and instant) and proceed with the static scan meanwhile.

Findings supported by both runtime evidence and code analysis are marked Runtime-confirmed — the highest confidence tier. Purely static findings keep the Confirmed/Likely/Potential scale.

Step 1 — Context discovery (always do this first, before reading any target class)

Leaks are judged relative to the runtime environment, so establish it up front:

1. Java version. Read pom.xml (maven.compiler.release / source/target) or build.gradle (sourceCompatibility / toolchain). Record it — it changes both detection and fixes:
   • Java 7+: try-with-resources is the required fix idiom for all AutoCloseable leaks.
   • Java 9+: finalize() is deprecated — flag any finalizer and fix with java.lang.ref.Cleaner or explicit close().
   • Java 11+: HttpClient misuse patterns apply; prefer sharing one client, not one per request.
   • Java 21+: if virtual threads are in use (Executors.newVirtualThreadPerTaskExecutor, spring.threads.virtual.enabled), ThreadLocal retention multiplies across millions of threads — raise severity of ThreadLocal findings and mention ScopedValue as the modern alternative.
2. Frameworks and versions. Spring Boot version (lifecycle hooks, cache defaults, OSIV default on), Hibernate/JPA, Netty (ByteBuf reference-counting), Caffeine/Guava on the classpath (determines which bounded-cache fix is available), HikariCP config.
3. Runtime shape. Servlet container (Tomcat/Jetty thread pools → ThreadLocal and classloader leaks become high-probability), scheduled jobs, async executors, batch jobs, hot-redeploy environments. If WebSphere (traditional WAS or Liberty) or an IBM J9/OpenJ9/Semeru JVM is detected (was.install.root, Liberty server.xml, ibm-web-bnd.xml, .phd/javacore files, Semeru toolchain), read references/websphere.md — GC log format, dump artifacts, connection-leak tracing, and several leak patterns differ from the HotSpot/Tomcat defaults.
4. Scope. Identify the target classes/packages. If given a package, module, or repo, enumerate the .java files. Individual files named by the user are entry points for tracing, not the boundary of the audit.

Step 2 — Build a lifetime map

Before pattern-matching, map object lifetimes across the codebase, because a leak is a reachability problem and cannot be judged one file at a time:

• Which classes are JVM-lifetime: Spring singletons (@Service, @Component, @RestController, @Configuration beans), static fields, pooled threads, long-lived executors, caches.
• Which are request/task-scoped: controllers' method locals, per-message handlers, batch items.
• Who references whom: any path where a request-scoped object becomes reachable from a JVM-lifetime object (added to a static map, registered as a listener on a singleton, stored in a ThreadLocal on a pooled thread, cached without bounds) is a candidate leak.

Step 3 — Scan against the pattern catalog

Read references/leak-patterns.md and check the target code against every category. The catalog covers:

1. Static collections & unbounded caches
2. ThreadLocal misuse (especially with thread pools)
3. Unclosed resources (streams, files, HTTP clients, Stream<Path>)
4. Database & persistence leaks — unclosed JDBC resources, connection-pool exhaustion, streaming query results, unbounded queries, persistence-context (Hibernate session) bloat, open transactions, OSIV
5. Listener / callback / observer registration without deregistration
6. Non-static inner classes & lambdas capturing this into long-lived contexts
7. Executor / scheduler / thread lifecycle leaks
8. Spring-specific leaks (singletons holding request state, unbounded @Cacheable, prototype-in-singleton, event listeners, per-request WebClient/RestTemplate construction)
9. Collection key/equality bugs (mutable keys, broken equals/hashCode)
10. ClassLoader / redeployment leaks (JDBC driver registration, shutdown hooks, static references)
11. Buffer/substring retention and oversized graphs held longer than needed
12. Finalizers and reference-type misuse

Trace across files. For every candidate, follow the retention path wherever it leads: a DAO returning a Stream is only a leak if no consumer closes it — check the consumers. A ThreadLocal.set() in a filter is only safe if a remove() exists in a finally on the same request path — check the whole path. A callback registered in class A against singleton B leaks only if nothing ever deregisters it — search for the deregistration.

Step 4 — Verify each finding before reporting

False positives destroy trust. Before reporting:

• Confirm the retaining object is actually long-lived (static, singleton bean, pooled thread, JVM-lifetime executor).
• Search for an existing cleanup path anywhere in the codebase: @PreDestroy, close(), removeListener, eviction policy, finally block, @Scheduled cleanup job.
• Distinguish a true leak (unbounded retention) from a deliberate bounded cache. Caffeine with maximumSize/expireAfterWrite is not a leak; a raw static HashMap used as a cache is.
• Assign confidence: Confirmed (clear unbounded retention path), Likely (path exists; bound depends on usage volume), Potential (depends on runtime context — state exactly what to check).

Step 5 — Report

ALWAYS use this exact structure:

# Memory Leak Audit — <scope>

## Environment
Java <version> · <frameworks + versions> · <runtime shape notes>

## Summary
<1–3 sentences: N findings across M classes, most severe first>

## Findings

### [CRITICAL|HIGH|MEDIUM|LOW] F1: <short title>
- **Location:** `com.example.Foo#method` (line N) — plus every file on the retention path
- **Pattern:** <catalog category>
- **Confidence:** Runtime-confirmed | Confirmed | Likely | Potential
- **Runtime evidence:** <log line / dominator entry / histogram delta supporting it, if any>
- **Retention path:** <the full chain: who holds the reference, across which classes, and why it is never released>
- **Impact:** <runtime symptom: heap growth driver, OOM trigger, pool exhaustion/deadlock, thread starvation>
- **Fix:** <specific change with a short code sketch; note if behavior-affecting>

## Clean classes
<classes scanned with no findings — list them so coverage is explicit>

## Runtime verification
<how to confirm: heap dump dominator tree targets, JFR OldObjectSample, hikari leak-detection-threshold, metrics to watch>

Severity guide: CRITICAL = unbounded growth on a hot path, or guaranteed pool/resource exhaustion; HIGH = unbounded growth on a realistic path; MEDIUM = large bounded retention, or leak only on error/redeploy paths; LOW = hygiene issue unlikely to matter at current scale.

Step 6 — Fix (FIX mode only)

Read references/fix-recipes.md and apply the canonical fix for each confirmed/likely finding. Rules:

• Minimal diffs. Fix the leak; do not refactor unrelated code, rename identifiers, or "improve" style.
• Preserve behavior. A cache stays a cache (add bounds/TTL, don't delete it). A listener keeps working (add deregistration). A query returns the same data.
• Match the environment from Step 1. Use idioms valid for the detected Java version; use Caffeine only if it's already on the classpath, otherwise offer the no-dependency alternative and ask before adding dependencies.
• Flag behavior-affecting fixes (pagination on an unbounded query, closing a shared client, changing fetch strategy) as proposals requiring approval rather than applying silently.
• After editing, re-read every modified file: imports added, braces balanced, resources still reachable where used, and the fix didn't introduce a new lifetime bug (e.g., closing a connection other code still uses, or evicting cache entries something depends on).
• End with a per-fix changelog: file, lines, what was fixed, behavior caveats.

Things that are NOT leaks — do not report these

• Bounded caches with eviction (Caffeine/Guava with limits, bounded Spring CacheManager)
• static final constants and immutable lookup tables
• Singleton beans holding stateless collaborators (other beans, config objects)
• Large but short-lived request-scoped objects
• ThreadLocal on provably non-pooled short-lived threads (rare — verify before dismissing)
• Connections managed entirely by JdbcTemplate / Spring @Transactional (Spring closes them)

Closing the loop after fixes

For every applied fix, prescribe the runtime verification that proves it (from log-analysis.md / heap-dump-analysis.md): the histogram class that should stop growing, the GC floor that should flatten, the Hikari timeouts that should stop. A fix isn't done until the metric that found the leak confirms it's gone. Include these in the changelog.

Fix Recipes

Canonical fixes per pattern. Global rules: minimal diff, preserve behavior, match the Java/framework versions detected in Step 1, and flag behavior-affecting changes for approval instead of applying silently. Numbering matches leak-patterns.md.

1. Static collections & unbounded caches

Keep the cache; bound it.

Caffeine on classpath (preferred):

private static final Cache<String, UserSession> SESSIONS = Caffeine.newBuilder()
    .maximumSize(10_000)
    .expireAfterAccess(Duration.ofMinutes(30))
    .build();

No new dependency (LRU via LinkedHashMap):

private static final Map<String, UserSession> SESSIONS =
    Collections.synchronizedMap(new LinkedHashMap<>(16, 0.75f, true) {
        @Override protected boolean removeEldestEntry(Map.Entry<String, UserSession> e) {
            return size() > 10_000;
        }
    });

If entries have a natural end-of-life event (logout, task completion), add explicit remove() at that event instead of — or in addition to — bounding. Choose sensible bounds from context and state them in the changelog; ask the user if the right bound is unclear.

2. ThreadLocal

remove() in finally on the same path as set():

CONTEXT.set(userContext);
try {
    chain.doFilter(req, res);
} finally {
    CONTEXT.remove();
}

If set/remove live in different classes (filter/interceptor pair), put the remove() at the outermost layer that did the set. Java 21+ with virtual threads: mention ScopedValue as the structural fix, but don't migrate unless asked.

3. Unclosed resources (general)

Try-with-resources, preserving existing error handling:

try (Stream<String> lines = Files.lines(p)) {
    return lines.filter(s -> !s.isBlank()).toList();
}

If a method returns an open resource, either (a) refactor to accept a consumer/callback so the method controls closure, or (b) keep the signature and fix every call site with try-with-resources — prefer (b) for minimal diff, and document the ownership transfer in the method's Javadoc.

Per-request HttpClient/OkHttpClient/WebClient: hoist to a shared singleton (Spring bean or static final). Closing a shared client used elsewhere is a behavior change — never do it without checking all users.

4. Database & persistence

4a. Raw JDBC → try-with-resources for all three levels:

try (Connection c = ds.getConnection();
     PreparedStatement ps = c.prepareStatement(SQL)) {
    ps.setLong(1, id);
    try (ResultSet rs = ps.executeQuery()) {
        ...
    }
}

Where the class already uses Spring, migrating the method to JdbcTemplate is the cleaner fix — offer it, apply only with approval since it's a larger diff.

4b. Pool exhaustion → fix the specific non-returned path (usually an early return/throw between acquire and close → restructure with try-with-resources). Additionally recommend:

spring:
  datasource:
    hikari:
      leak-detection-threshold: 30000

as a config suggestion (logs stack traces of held connections — safe, observability-only).

4c. Streaming queries → close at the consumer, ensure a transaction:

@Transactional(readOnly = true)
public void processOpen() {
    try (Stream<Order> s = repo.streamByStatus(OPEN)) {
        s.forEach(this::process);
    }
}

4d. Unbounded queries → behavior-affecting; propose, don't silently apply. Options: Pageable pagination, Stream + recipe 4c, or a LIMITed query. State the trade-off (callers must handle pages).

4e. Persistence-context bloat (batch) → periodic flush/clear preserving batch logic:

int i = 0;
for (Row r : rows) {
    em.persist(map(r));
    if (++i % 500 == 0) { em.flush(); em.clear(); }
}

OSIV: recommend spring.jpa.open-in-view: false as a proposal only — it changes lazy-loading behavior in controllers/views and can surface LazyInitializationException; requires the user's sign-off and possibly fetch adjustments.

4f. Open transactions → commit/rollback in finally, or replace manual demarcation with @Transactional/TransactionTemplate (which guarantee cleanup).

5. Listeners/callbacks

Add the symmetric deregistration at the end of the registrant's lifecycle:

• Spring bean → @PreDestroy calling removeListener(...)
• Request/task-scoped registrant → try/finally around the operation
• If the listener source supports it, prefer weak-listener registration APIs

Never delete the registration itself — the feature must keep working.

6. Inner classes & lambdas

Convert to a static nested class, or capture only the needed values as locals before the lambda:

String jobId = this.jobId;                    // copy the needed field
scheduler.scheduleAtFixedRate(() -> poll(jobId), 0, 1, MINUTES);

Only when the enclosing instance is genuinely needed and long retention is unacceptable, hold it via WeakReference inside the task and no-op if cleared — document this behavior change.

7. Executors/schedulers/threads

• Bean-owned executor → shut it down in lifecycle:

@PreDestroy
void shutdown() {
    executor.shutdown();
    try {
        if (!executor.awaitTermination(10, SECONDS)) executor.shutdownNow();
    } catch (InterruptedException e) {
        executor.shutdownNow();
        Thread.currentThread().interrupt();
    }
}

• Per-request executor creation → hoist to a shared bean/static final.
• Unbounded queue growth → bounded ThreadPoolExecutor with an explicit RejectedExecutionHandler; behavior-affecting (submissions can now be rejected/blocked), so propose with the trade-off stated.
• Timer → prefer ScheduledExecutorService (Timer dies silently on task exceptions); propose if touching that code anyway.

8. Spring-specific

• Singleton holding request state → move state to method locals/parameters; if genuinely per-request, use a request-scoped bean with proxyMode = TARGET_CLASS. Also note the thread-safety bug in the finding.
• Unbounded @Cacheable → configure a bounded CacheManager (Caffeine):

@Bean
CacheManager cacheManager() {
    CaffeineCacheManager mgr = new CaffeineCacheManager();
    mgr.setCaffeine(Caffeine.newBuilder().maximumSize(10_000).expireAfterWrite(Duration.ofMinutes(10)));
    return mgr;
}

• Prototype-in-singleton → ObjectProvider<T> injection, calling getObject() per use.
• Per-request client construction → single shared WebClient/RestTemplate bean.

9. Collection key bugs

Implement equals/hashCode on the key type (or key by an immutable ID field instead). If the key must stay mutable, key the map by an immutable identifier and store the mutable object as the value. Removing entries under the old hash is impossible after the fact — note that existing stuck entries clear only on restart or map replacement.

10. ClassLoader/redeploy

In a ServletContextListener.contextDestroyed (or @PreDestroy on a lifecycle bean):

• DriverManager.deregisterDriver(driver) for drivers this webapp loaded
• Remove shutdown hooks the app added
• Deregister MBeans, JUL handlers, security providers the app registered
• Ensure static ThreadLocals are removed (recipe 2) — they pin the classloader via container threads

11. Buffers/oversized graphs

• Copy the needed slice instead of pinning the whole buffer: Arrays.copyOfRange(...) / new String(chars, from, len)
• Stream large files/payloads instead of full in-memory reads (behavior/perf trade-off — propose)
• Netty: every retain() needs a matching release(); extend SimpleChannelInboundHandler where auto-release semantics fit; enable -Dio.netty.leakDetection.level=paranoid in tests as a verification suggestion
• Store projections/IDs in caches and sessions instead of full entity graphs

12. Finalizers & references

• finalize() → implement AutoCloseable with explicit close() at call sites (recipe 3); where closure can't be guaranteed by callers, add a java.lang.ref.Cleaner as a safety net (Java 9+). Note: on Java 8, keep the finalizer only if nothing better exists, and say so.
• Useless WeakReference/misused SoftReference → replace with the semantically correct structure: bounded cache (recipe 1) for capacity concerns, WeakHashMap only for canonical-mapping semantics.
• Unpolled ReferenceQueue → poll it where cleanup should happen, or remove it if unused.

After every fix

1. Re-read the full modified file: imports present, braces balanced, resource still in scope everywhere it's used.
2. Confirm no new lifetime bug: nothing shared got closed, nothing needed got evicted, no remove() runs before a later read on the same path.
3. Changelog entry: file, lines, pattern fixed, behavior caveats.

Heap Dump Analysis & Remediation

Use this when the user has (or can capture) a heap dump, or when static findings need runtime confirmation. Two modes: programmatic (the agent runs tools itself — preferred when a shell is available, e.g. Windsurf/Devin) and guided (walk the user through MAT/VisualVM and interpret what they paste back).

Every analysis ends the same way: retained class → retaining field → GC-root path → catalog pattern (leak-patterns.md) → fix recipe (fix-recipes.md). The dump tells you what leaks; the code tells you why; the recipe fixes it.

Table of contents

1. Capturing evidence
2. Cheapest first: class-histogram diff
3. Programmatic dump analysis (MAT headless)
4. Guided analysis (user drives MAT/VisualVM)
5. Interpreting results → remediation
6. Common dominator signatures cheat sheet

---

1. Capturing evidence

Recommend in this order (least to most invasive):

# 1. Class histogram — seconds, tiny output, safe in prod
jcmd <pid> GC.class_histogram > histo-$(date +%s).txt

# 2. Heap dump of LIVE objects only (forces a GC first — brief pause)
jcmd <pid> GC.heap_dump -all=false /tmp/heap-live.hprof
# (or: jmap -dump:live,format=b,file=/tmp/heap-live.hprof <pid>)

# 3. Automatic dump at the moment of failure — set proactively
-XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=/dumps

Notes to pass on:

• Always prefer live-only dumps for leak hunting; full dumps include garbage that muddies retained sizes.
• Dump size ≈ heap used size; ensure disk space, and warn that dumping a large heap pauses the JVM for seconds — do it on a node pulled from the load balancer if possible.
• Dumps contain real user data (strings, entities). Treat as sensitive; don't ask the user to upload one anywhere they wouldn't put production data.
• For a growing leak, two dumps ~30–60 min apart beat one dump: the delta is the leak.

2. Class-histogram diff

The highest signal-to-effort technique, and it needs no tooling beyond jcmd and a diff. Take two histograms at steady load, minutes to hours apart, then compute per-class instance/byte deltas.

When given two histogram files, write a small script to parse and diff them (columns: instances, bytes, class name), sort by byte growth, and report the top ~15 growers. Interpret:

• Growing domain classes (com.yourco.OrderDto) → whatever collection/cache holds them; scan §1/§4e/§8 for that type.
• Growing char[]/String/byte[] alone → strings/buffers retained; find which top-growing container class correlates (the arrays are leaves, not causes).
• Growing java.util.HashMap$Node / ConcurrentHashMap$Node → a map is growing; correlate with the domain class growing at a similar count.
• Growing java.lang.ref.Finalizer → finalizer backlog (§12).
• Growing java.lang.Thread → thread leak (§7); confirm with thread dump.
• Growing framework entity/proxy classes (...$HibernateProxy...) → persistence-context bloat (§4e).

Report growth rates (instances/hour, MB/hour) and estimated time-to-OOM given -Xmx.

3. Programmatic dump analysis

When a shell is available and the dump file is accessible, run Eclipse MAT headless — it does the dominator/leak-suspects math without a GUI:

# Get MAT (once): download & unzip Eclipse MAT, or use the ce-provided mat if present
# Then run the Leak Suspects report headless:
<mat-dir>/ParseHeapDump.sh /path/heap-live.hprof \
  org.eclipse.mat.api:suspects \
  org.eclipse.mat.api:top_components

This produces zipped HTML reports (*_Leak_Suspects.zip, *_Top_Components.zip) next to the dump. Unzip and read the HTML: MAT names suspect #1/#2 with retained sizes, the accumulation point (the exact field, e.g. com.example.SessionRegistry.sessions), and the GC-root path. Extract those and jump to §5.

Heap for MAT itself: parsing an N-GB dump needs roughly N GB for MAT — set -Xmx in MemoryAnalyzer.ini/ParseHeapDump.sh accordingly. If the environment can't fit the dump, fall back to histogram diff (§2) or OQL-free jhat-style summaries, and say which fidelity was lost.

Alternative when MAT isn't installable: jhat is removed in modern JDKs; prefer VisualVM headless is not practical — so histogram diff + targeted jcmd GC.heap_dump inspection via MAT remains the recommended path; if truly toolless, guide the user instead (§4).

4. Guided analysis

When the user runs the tool and pastes results back:

In Eclipse MAT (preferred):

1. Open dump → run Leak Suspects report first; paste the suspect summary back.
2. Open Dominator Tree, sort by retained heap. Ask for the top 5 rows (class, retained size, percentage).
3. For the top suspect: right-click → Path to GC Roots → exclude weak/soft references. Paste the path. This path names the retaining field and class — the audit target.
4. If suspects are collections: right-click → Show Retained Set or use OQL to sample entries, e.g. SELECT s FROM com.example.UserSession s — sampling entry contents often reveals what kind of entries accumulate (all same user? all completed tasks that should've been removed?).

In VisualVM (simpler, less capable): Load dump → Summary → biggest objects; Classes view sorted by size; instance view → nearest GC root. Good enough for obvious leaks; escalate to MAT for tangled ones.

Key concept to apply (and explain if the user asks): shallow size is the object itself; retained size is everything that would become garbage if it were collected. Leak hunting sorts by retained size. One HashMap with 10 GB retained is the answer; a million small strings with big shallow totals are usually its leaves.

5. Interpreting results → remediation

Given a dominator/suspect result, produce a Runtime-confirmed finding in the standard report format, with the retention path taken from the GC-root path rather than inferred:

1. Identify the accumulation point — the field named in the GC-root path (e.g. TaskScheduler.completedTasks).
2. Open that class in the code and match it to a catalog pattern. The dump has already done Step-3 tracing for you — verify in code that no cleanup path exists (Step 4 discipline still applies; MAT can't see a cleanup job that hasn't run yet).
3. Apply the matching fix recipe, following all FIX-mode rules (minimal diff, preserve behavior, approval for behavior changes).
4. Prescribe verification: after the fix, the same measurement that found the leak proves the fix — histogram diff shows the class flat; GC floor stops rising; Hikari timeouts stop. State the concrete metric and expected result in the changelog.

If the dump shows a leak in framework or third-party code: check the retaining path for the app-owned frame (there almost always is one — the app configured, registered, or invoked something). If genuinely a library bug, identify the library version, search for the known issue, and recommend the upgrade/workaround rather than patching app code around it.

6. Common dominator signatures

Dominator tree shows	Almost always means	Catalog / recipe
One HashMap/ConcurrentHashMap retaining most of heap, GC root = static field or singleton bean	Unbounded cache/registry	§1
Thread objects each retaining large ThreadLocalMap entries	ThreadLocal not removed on pooled threads	§2
org.hibernate.internal.SessionImpl / StatefulPersistenceContext with huge retained set	Long-lived session, batch without flush/clear, OSIV	§4e
Many com.zaxxer.hikari.pool.PoolEntry strongly held outside the pool	Borrowed connections never returned	§4b
Listener list (CopyOnWriteArrayList on a singleton/event bus) retaining many dead-looking components	Missing deregistration	§5
ScheduledThreadPoolExecutor queue retaining tasks that each retain a large outer instance	Lambda/inner-class capture into a long-lived scheduler	§6 + §7
java.lang.ref.Finalizer chain with big retained set	Finalizer backlog	§12
Multiple copies of the same app class / WebappClassLoader instances after redeploys	Classloader leak	§10
io.netty.buffer.PoolChunk / direct ByteBuffer growth	ByteBuf release / direct buffer leak	§11

Java Memory Leak Pattern Catalog

Check target code against every category. For each candidate, trace the full retention path across files before concluding anything. The core question is always: who holds a reference to this object, and when — if ever — is it released?

Table of contents

1. Static collections & unbounded caches
2. ThreadLocal misuse
3. Unclosed resources (general)
4. Database & persistence leaks
5. Listener/callback registration without deregistration
6. Inner classes & lambdas capturing this
7. Executor/scheduler/thread lifecycle
8. Spring-specific leaks
9. Collection key & equality bugs
10. ClassLoader & redeployment leaks
11. Buffer/substring retention & oversized graphs
12. Finalizers & reference-type misuse

---

1. Static collections & unbounded caches

Look for: static (or singleton-bean) Map/List/Set fields that are added to but never removed from, cleared, or bounded. Homemade caches, registries, "recently seen" trackers, dedupe sets, metrics accumulators.

// LEAK: grows forever, one entry per user, never evicted
private static final Map<String, UserSession> SESSIONS = new HashMap<>();
public void onLogin(User u) { SESSIONS.put(u.getId(), buildSession(u)); }

Verify: search the whole codebase for remove/clear/eviction on that field before reporting. A @Scheduled cleanup job elsewhere may exist.

Also check: memoization maps keyed by unbounded input (request params, user input strings); interning user-controlled strings; accumulating List in a singleton used as an audit trail.

2. ThreadLocal misuse

Look for: ThreadLocal.set() without a guaranteed remove() in a finally on the same execution path, in any code that runs on pooled threads (servlet containers, @Async, executors, message listeners).

// LEAK on pooled threads: last request's context retained per thread forever
CONTEXT.set(userContext);
chain.doFilter(req, res);   // remove() missing, or not in finally

Trace: the set and the remove are often in different classes (filter sets, interceptor removes). Follow the request path end to end. An exception between set and a non-finally remove also leaks.

Severity modifiers: raise severity if the value graph is large (entities, buffers) or if Java 21+ virtual threads are enabled (retention multiplies per virtual thread). Static ThreadLocals also pin classloaders (see §10).

3. Unclosed resources (general)

Look for: any AutoCloseable not managed by try-with-resources or a guaranteed finally: InputStream/OutputStream, Reader/Writer, Files.lines()/Files.walk() (returns a Stream holding a file handle), Scanner, ZipFile, channels/sockets, per-request HttpClient/OkHttpClient construction.

// LEAK: Files.lines holds an open file handle; stream never closed
List<String> read(Path p) throws IOException {
    return Files.lines(p).filter(s -> !s.isBlank()).toList();
}

Verify: closure may happen in the caller. Trace who consumes returned resources; a method that returns an open resource transfers the obligation — check every call site.

4. Database & persistence leaks

The highest-value category for Spring backends. Symptoms are often pool exhaustion (requests hang) rather than OOM.

4a. Unclosed JDBC resources

Raw Connection/Statement/PreparedStatement/ResultSet outside try-with-resources, or closed outside finally so exception paths leak. Manual dataSource.getConnection() inside @Transactional code bypasses Spring's managed connection and is never released by Spring.

4b. Connection pool exhaustion

Connections borrowed and not returned on every path. Watch for early return / throw between acquire and close. Symptom: Hikari timeout exceptions, requests queueing. Recommend spring.datasource.hikari.leak-detection-threshold as runtime confirmation.

4c. Streaming query results

Spring Data Stream<T> return types and Hibernate ScrollableResults hold the connection open until closed. Requires try-with-resources at the consumer and an open transaction (@Transactional(readOnly = true)) around consumption. Check every caller of any repository method returning Stream.

4d. Unbounded queries

findAll() on large/growing tables, missing pagination or LIMIT, IN clauses built from unbounded lists, N+1 patterns hydrating huge object graphs per request. Not a classic leak, but a heap-spike/OOM driver — report as MEDIUM–HIGH with pagination/streaming as the fix.

4e. Persistence-context bloat

Long-lived EntityManager/Hibernate Session: the first-level cache retains every entity touched. Classic case: batch job iterating millions of rows in one transaction without periodic flush()/clear(). Also: extended persistence contexts, entities cached in singleton beans, open-session-in-view (OSIV, on by default in Spring Boot) holding the session and its entities for the full request including view rendering.

4f. Transactions left open

Manual beginTransaction() without commit/rollback in finally; programmatic TransactionTemplate misuse. Pins the connection and locks.

5. Listener/callback registration without deregistration

Look for: addListener/register/subscribe/observe against a long-lived object (singleton, static event bus, scheduler) by a shorter-lived object, with no matching remove/unsubscribe. The long-lived side retains the short-lived side and everything it references.

Trace: registration and deregistration are usually in different classes/lifecycle methods. Absence of any remove*/unsubscribe call for that listener anywhere in the codebase = Confirmed.

6. Inner classes & lambdas capturing this

Look for: non-static inner classes, anonymous classes, and lambdas that capture the enclosing instance, then get handed to something long-lived (executor queue, cache, listener list, static field). The captured outer instance — and its whole graph — is retained.

// LEAK: anonymous Runnable captures the enclosing request handler
scheduler.scheduleAtFixedRate(new Runnable() {
    public void run() { poll(); }   // implicit reference to outer instance
}, 0, 1, MINUTES);

Fix direction: static nested class, or lambda referencing only the specific fields needed (copied to locals).

7. Executor/scheduler/thread lifecycle

Look for: ExecutorService/ScheduledExecutorService/Timer created without a corresponding shutdown() (in @PreDestroy for beans); executors created per-request/per-call; unbounded task queues (newFixedThreadPool uses an unbounded queue — tasks accumulate if producers outpace consumers); threads started and never joined/interrupted; CompletableFuture chains retaining large inputs.

8. Spring-specific leaks

• Singleton beans holding per-request state in instance fields (mutable field written per request in a @Service) — both a leak and a concurrency bug.
• Unbounded @Cacheable: default ConcurrentMapCacheManager has no eviction. Cacheable methods keyed by user input grow forever. Check the configured CacheManager before reporting.
• Prototype bean injected into singleton: the singleton holds one prototype instance forever; with method-injection misuse, prototypes accumulate.
• ApplicationListener registered programmatically without removal (annotation-driven @EventListener on singleton beans is fine).
• Per-request RestTemplate/WebClient/HttpClient construction: each carries pools/threads. Should be a shared singleton bean.
• Custom scopes / request-scoped beans referenced from singletons without scoped proxies.

9. Collection key & equality bugs

Look for: HashMap/HashSet keyed by objects lacking equals/hashCode, or keyed by mutable objects whose hash-relevant fields change after insertion. Entries become unreachable-by-lookup but still retained — remove() silently fails and the map only grows.

10. ClassLoader & redeployment leaks

Relevant for WAR deployments and hot-redeploy environments:

• JDBC drivers registered by the webapp and never deregistered on shutdown
• Shutdown hooks referencing app classes, never removed
• Static ThreadLocals on container threads pinning the webapp classloader
• App classes registered with JVM-global registries (JMX MBeans, java.util.logging, security providers) without deregistration in a ServletContextListener/@PreDestroy

Each redeploy leaks the entire previous classloader and every class it loaded → Metaspace OOM.

11. Buffer/substring retention & oversized graphs

• Retaining a small slice of a huge buffer/array in a way that pins the whole thing (e.g., wrapping large byte[] regions, ByteBuffer.slice() semantics)
• Reading entire large files/payloads into memory when streaming would do
• Netty ByteBuf reference-count leaks: retain() without matching release(); check SimpleChannelInboundHandler vs manual release discipline
• Large object graphs stored where only a small projection is needed (whole entity in a session/cache when an ID would do)

12. Finalizers & reference-type misuse

• Any finalize() override (deprecated since Java 9): delays collection by at least one GC cycle, resurrection risks. Flag for replacement with Cleaner/explicit close.
• SoftReference used as a correctness mechanism (entries vanish under pressure) or WeakReference wrapping objects that something else strongly holds anyway (useless) — or not used where a canonical map genuinely needs WeakHashMap semantics.
• ReferenceQueue polled nowhere → the reference objects themselves accumulate.

Log Analysis for Memory & Resource Leaks

Use this when the user provides application logs, GC logs, or container/server logs — or when they describe runtime symptoms. The goal is to map symptoms to catalog categories (leak-patterns.md) so the code scan is targeted, and to confirm static findings with runtime evidence.

Table of contents

1. OutOfMemoryError variants → root-cause direction
2. Connection pool symptoms
3. GC log analysis
4. Framework & container warnings that name the leak
5. Thread & native symptoms
6. Symptom triage output format

---

1. OutOfMemoryError variants

Each OOM message points to a different catalog section. Extract the exact message and the stack trace's top frames (the allocation site is a victim, not necessarily the culprit — say so in the report).

Log message	Meaning	Scan first
java.lang.OutOfMemoryError: Java heap space	Heap exhausted — classic leak or oversized allocation	§1 static collections, §4 DB/persistence, §2 ThreadLocal, §5 listeners
OutOfMemoryError: GC overhead limit exceeded	Heap nearly full, GC churning >98% time — late-stage version of the above	same as heap space
OutOfMemoryError: Metaspace (or PermGen on Java 7-)	Class metadata exhausted — classloader leak, usually redeploy-driven, or runaway proxy/bytecode generation	§10 classloader leaks; check redeploy history; check dynamic proxy/CGLIB generation in loops
OutOfMemoryError: unable to create new native thread	Thread exhaustion, not heap	§7 executor/thread lifecycle (executors created per-request, threads never terminated)
OutOfMemoryError: Direct buffer memory	Off-heap NIO/direct buffers exhausted	§11 buffers; Netty ByteBuf release discipline; unclosed channels
OutOfMemoryError: Requested array size exceeds VM limit	Single huge allocation attempt	§4d unbounded queries, oversized payload reads — usually a spike bug, not a slow leak
Container OOMKilled (exit 137) with no Java OOM in logs	Native/RSS growth beyond heap: direct buffers, Metaspace, thread stacks, or heap sized larger than container limit	Check -Xmx vs container limit first; then direct memory & thread counts

Allocation-site caveat: the stack trace on an OOM shows whoever happened to allocate last. Report it as "the straw", and direct root-cause at whatever the dominator analysis or targeted scan finds retaining the heap.

2. Connection pool symptoms

HikariCP (Spring Boot default):

• HikariPool-1 - Connection is not available, request timed out after 30000ms → pool exhaustion (§4b). If it appears under normal load and recovers only on restart: connections are being leaked, not merely undersized.
• Apparent connection leak detected + stack trace (when leak-detection-threshold is set) → this is gold: the stack trace shows the exact method that borrowed and never returned. Audit that method and its error paths directly.
• HikariPool-1 - Thread starvation or clock leap detected → often GC pauses from a near-full heap; correlate with GC logs.

If leak detection is not enabled, recommend enabling it as the first remediation step:

spring.datasource.hikari.leak-detection-threshold: 30000

Symptom pattern to ask about / look for in logs: latency climbing gradually, requests hanging on DB calls, thread dumps showing many threads parked in HikariPool.getConnection — all pool exhaustion signatures even without explicit errors.

3. GC log analysis

The definitive leak signature: the post-full-GC heap floor rises over time. Normal apps show a sawtooth returning to a stable baseline; leaking apps show a rising staircase.

What to compute from the logs:

1. Extract heap-after values from full GC (or G1 mixed/Full) events over the covered time span.
2. Trend the minimums. Rising floor across hours/days at steady load = leak confirmed. Flat floor = the OOM (if any) was a spike, not a leak — redirect to §4d oversized-allocation patterns.
3. Full-GC frequency increasing over time = heap pressure growing.
4. Long pauses (>1s) correlate with the "thread starvation" and timeout symptoms elsewhere in the logs.

Log formats (match to Java version from Step 1):

• Java 9+ unified logging: lines like [gc] GC(123) Pause Full (G1 Compaction Pause) 3072M->2900M(4096M) 450ms — the -> value after Full GC events is the floor to trend. Enabled via -Xlog:gc*:file=gc.log:time,uptime.
• Java 8 legacy: [Full GC (Ergonomics) ... 3072M->2900M(4096M), 0.45 secs] with -XX:+PrintGCDetails -XX:+PrintGCDateStamps.

If the user has no GC logs, recommend enabling them (near-zero overhead) and re-checking after a representative period.

Parsing at scale: when given a large GC log file, write a small script to extract and trend the post-full-GC values rather than eyeballing; include the trend summary (start floor, end floor, growth rate per hour) in the report. Growth rate + heap headroom gives estimated time-to-OOM — include it, it drives urgency.

4. Framework & container warnings

These warnings often name the leaking class directly — search logs for them:

• Tomcat shutdown/redeploy warnings (§10 confirmed, with the culprit named):
  • The web application [app] appears to have started a thread named [X] but has failed to stop it
  • The web application [app] created a ThreadLocal with key of type [X] ... but failed to remove it when the web application was stopped
  • The web application [app] registered the JDBC driver [X] but failed to unregister it
• Hibernate:
  • HHH000104: firstResult/maxResults specified with collection fetch; applying in memory! → whole result set loaded to paginate in memory (§4d)
  • HHH000444 (batch fetch) and session-size warnings → persistence-context bloat (§4e)
• Spring: Bean 'X' of type ... is not eligible for getting processed by all BeanPostProcessors is not a leak — don't report it. But repeated context refreshes in logs suggest redeploy cycles feeding §10.
• Netty: LEAK: ByteBuf.release() was not called before it's garbage-collected → §11, and the records show the access points. Recommend -Dio.netty.leakDetection.level=advanced to get full traces.

5. Thread & native symptoms

• Thread count climbing in metrics/logs, or thread dumps growing with pool-named threads (pool-47-thread-1) → executors created repeatedly and never shut down (§7). The pool number incrementing is the tell: pool-1, pool-2, ... pool-900 means an executor is created per operation.
• Too many open files (java.io.IOException: Too many open files) → file/socket handle leak (§3); lsof -p <pid> | wc -l trending up confirms. The lsof type breakdown (files vs sockets vs pipes) narrows which resource.
• Timer/scheduler silence: java.util.Timer threads die permanently on unhandled task exceptions — a task exception in logs followed by that job never running again indicates §7's Timer hazard.

6. Symptom triage output format

When starting from logs, produce this before (or alongside) the code scan:

## Runtime Symptom Triage

| # | Symptom (log evidence, timestamp) | Diagnosis | Catalog category | Confidence |
|---|---|---|---|---|
| S1 | `OutOfMemoryError: Metaspace` after 3rd redeploy | Classloader leak | §10 | Confirmed |
| S2 | Hikari timeouts starting ~2h after deploy | Connection leak | §4b | Likely |

**Scan priority:** <categories ordered by symptom evidence>
**Missing evidence:** <what to enable — leak-detection-threshold, GC logs, heap dump — to confirm the rest>

Findings later confirmed by both static scan and log evidence are marked Runtime-confirmed — the highest confidence tier, above static "Confirmed".

WebSphere (WAS) & IBM J9/Semeru Specifics

Use this when Step 1 context discovery reveals WebSphere (traditional WAS or Liberty) or an IBM J9/OpenJ9/Semeru JVM. The generic guidance in log-analysis.md and heap-dump-analysis.md assumes HotSpot; this file overrides the deltas. Detection signals: was.install.root, server.xml (Liberty), ibm-web-bnd.xml, wsadmin scripts, websphere dependencies in the build, mention of Semeru/OpenJ9, .phd files, javacore*.txt files.

Table of contents

1. Enabling the right logs on WAS
2. J9 verbose GC analysis
3. OOM artifacts: javacore, PHD, snap
4. WAS connection leak detection
5. PHD heap dump analysis
6. WAS-specific leak patterns to add to the scan

---

1. Enabling the right logs on WAS

Priority order for a leak hunt (least invasive first):

Verbose GC (near-zero overhead, always recommend first):

• Traditional WAS: Admin Console → Servers → Server Types → WebSphere application servers → server → Java and Process Management → Process definition → Java Virtual Machine → Generic JVM arguments, add:

  -Xverbosegclog:${SERVER_LOG_ROOT}/verbosegc.%Y%m%d.%H%M%S.%pid.txt,5,10000
  

  (5 rotating files × 10,000 GC cycles). The "Verbose garbage collection" checkbox works too but writes into native_stderr.log unrotated — prefer the argument. Restart required.
• Liberty: same argument in ${server.config.dir}/jvm.options, restart the server.

Connection leak detection (WAS-managed datasources): see §4. Runtime-enableable, no restart.

Already-on sources: SystemOut.log/SystemErr.log (traditional), messages.log/console.log (Liberty), and FFDC incident files under logs/ffdc/ — FFDC captures first-failure context for OOMs automatically. Search these for OutOfMemoryError, HHH000104, J2CA0045E (connection wait timeout), CWOBJ/TRAS heap messages.

Dump agents (set proactively so the artifacts exist when the OOM happens): J9 produces javacore + PHD + snap on OOM by default. To also capture on demand or on other events, Generic JVM arguments accept -Xdump clauses, e.g. -Xdump:heap:events=user makes kill -QUIT produce a heap dump.

2. J9 verbose GC analysis

J9 GC logs are XML, not HotSpot text. The leak signature is the same — a rising floor — but extract it differently:

• Trend <mem-info ... free="..." total="..."> inside <gc-end type="global"> (or <gc-op type="global mark/sweep"> events). Falling free-after-global-GC over time at steady load = leak.
• Gencon collector (default): ignore the frequent scavenge (nursery) events for floor-trending; only global collections tell you about tenured retention.
• <exclusive-end durationms="..."> gives pause times for correlating with timeout symptoms.
• Balanced/metronome collectors have different event names but the same principle: trend free heap after the most-complete collection type present.

When given a J9 GC log, write a small XML-aware parsing script (the files are large; don't eyeball) and report: start floor, end floor, growth per hour, estimated time to -Xmx exhaustion.

GUI alternative to recommend to the user: IBM Garbage Collection and Memory Visualizer (GCMV) reads these logs directly and plots the trend.

3. OOM artifacts

On OOM, J9 automatically writes to the profile/server working directory:

• javacore.<date>.<time>.<pid>.<seq>.txt — thread dump plus much more. Read programmatically; the high-value sections:
  • 0SECTION TITLE + 1TISIGINFO — what triggered the dump (Dump Event "systhrow" ... java/lang/OutOfMemoryError and which variant)
  • MEMINFO section — heap segments, free/used at failure
  • LOCKS and THREADS sections — for the pool-exhaustion case: many threads waiting in getConnection = §4b confirmed
  • Native memory (NATIVEMEMINFO) — distinguishes heap OOM from native/DirectByteBuffer exhaustion
• heapdump.<...>.phd — the heap dump (see §5)
• Snap.<...>.trc — trace buffer snapshot (rarely needed for leaks)

On-demand equivalents: kill -3 <pid> → javacore only. Full set: wsadmin → $AdminControl invoke $jvm generateHeapDump / dumpThreads, or jcmd-equivalent ${JAVA_HOME}/bin/jcmd <pid> Dump.heap on recent Semeru.

4. WAS connection leak detection

WAS's equivalent of Hikari's leak-detection-threshold, for WAS-managed datasources:

• Enable at runtime: Admin Console → Troubleshooting → Logs and Trace → server → Diagnostic Trace → Change log detail levels → add ConnLeakLogic=all (traditional WAS; on Liberty use logging traceSpecification in server.xml).
• Effect: connections held in-use beyond ~10s get their borrowing stack trace written to trace.log. That stack names the exact application method that acquired and never returned — audit that method and its exception paths directly (catalog §4b).
• Related symptom messages to search for even without the trace: J2CA0045E: Connection not available (pool exhausted, callers timing out), J2CA0086W (connection usage anomalies), and hung-thread warnings WSVR0605W: Thread ... has been active for ... which often accompany threads stuck waiting on the pool.
• Also check pool config vs symptom: Admin Console → Resources → JDBC → Data sources → ds → Connection pool properties. Maximum connections too low mimics a leak; a leak exhausts any maximum given time — the distinguishing test is whether restart-to-timeout interval scales with load.

If the app instead bundles Hikari (Spring Boot WAR on WAS), the generic Hikari guidance in log-analysis.md applies unchanged.

5. PHD heap dump analysis

.phd (Portable Heap Dump) is not hprof:

• Eclipse MAT needs the IBM DTFJ adapter plugin to open PHD files (install from IBM's update site into MAT), or use IBM's Memory Analyzer packaging (bundled DTFJ). Without DTFJ, MAT silently can't parse it — tell the user this upfront to save an hour.
• Once loaded, everything in heap-dump-analysis.md applies unchanged: Leak Suspects report, dominator tree by retained size, path to GC roots → accumulation point → catalog pattern → fix recipe. MAT headless (ParseHeapDump.sh) also works on PHD once DTFJ is installed.
• PHD limitation: it contains object graphs and sizes but no field values / primitive content (no string contents). Sampling map entries to see what kind of data accumulates isn't possible from a PHD — if content inspection is needed, capture a system core instead (-Xdump:system:events=user) and load it via DTFJ, or infer entry nature from the key/value classes alone.
• Histogram-diff alternative when MAT/DTFJ isn't available: two javacores' MEMINFO/class sections are coarse; better is jcmd <pid> GC.class_histogram on recent Semeru, or GCMV's object-allocation view. State the fidelity loss.

6. WAS-specific leak patterns

Add these to the standard catalog scan when WAS is detected:

• Classloader leaks amplified: WAS traditional keeps apps hot-restartable; every app restart with a §10-pattern leak (static ThreadLocals, unremoved shutdown hooks, JDBC drivers, JMX MBeans) leaks a full CompoundClassLoader. Symptom: OOM in native/class storage after N restarts without a JVM bounce. WAS logs WSVR0606W/leak warnings less helpfully than Tomcat — don't rely on the container to name the culprit; scan §10 patterns directly.
• com.ibm.websphere.* API misuse: WorkManager / async beans (WorkManager.startWork) with unbounded submissions and no join; scheduled work never cancelled — same lifecycle rules as §7.
• HTTP session bloat: WAS replicates/persists sessions; large object graphs in HttpSession (entities, results) both leak memory per active session and hammer replication. Scan for session.setAttribute with large/entity types; fix = store IDs/projections. Session timeout config (Admin Console → Session management) is the bound — check it's not set to unlimited.
• Shared library scoping: app-provided JDBC drivers or pooling libs in the app classpath (instead of WAS shared libraries) worsen classloader retention and can double-pool connections — flag as configuration finding, not a code fix.
• javax vs jakarta: traditional WAS 8.5/9 apps are Java EE (javax.*); don't suggest jakarta.*-based fixes (or Spring Boot 3+ idioms requiring it) unless the target runtime is Liberty with Jakarta EE 9+ features enabled. Check server.xml features / WAS version first — this is part of Step 1 context discovery.