Editing a Web Content Structure Results in a thread deadlock 2026.q2

[Mark_Clarke]

Hi all,

I have a web content structure and a template. I have about 200 articles already using this structure. I need to edit this structure to turn off a fields required settings. When I save the changes to the structure liferay becomes unresponsive. I have asked AI to summarise the findings.

Liferay DXP 2026.Q2 — Saving DDM Structure causes portal-wide freeze when structure has 100+ dependent articles

Environment:

• Liferay DXP 2026.q2.0, Docker deployment
• PostgreSQL (native, not containerized)
• Elasticsearch 8.19.18 (native)
• Java 21 (Zulu), G1GC, 12GB heap
• Upgraded from 2026.Q1.5-LTS

Symptom: Editing a Web Content Structure and clicking Save causes the entire portal to become unresponsive — not just for the editing session, but for all users. No errors are thrown in docker logs, no exceptions in the browser console. This structure has 100+ Web Content articles associated with it.

Diagnosis so far:

1. Thread dump during the freeze shows 6+ worker threads (default-N) BLOCKED (on object monitor), all waiting on the same lock — a ConcurrentHashMap inside CounterFinderImpl.getCounterRegister. All are in this call chain:

   DDMStructureModelListener.onAfterUpdate
     → DDMFieldLocalServiceImpl.updateDDMFormValues
       → CounterLocalServiceImpl.increment
         → CounterFinderImpl.getCounterRegister [BLOCKED]
   
   

2. The lock-holding thread is itself inside CounterFinderImpl._obtainIncrement, executing a Hibernate query — while holding the global counter lock.

3. pg_stat_activity during the freeze shows all related sessions as idle in transaction / wait_event ClientRead — meaning Postgres already returned results and is waiting on the application, not the other way around. No entries in pg_locks show unblocked/waiting locks — ruling out a genuine DB-level deadlock.

4. Counter batching itself appears correctly configured (counter.increment=2000), and counters like DDMField/DDMFieldAttribute show healthy incrementing values in the Counter table — so this doesn’t look like a batching misconfiguration.

5. Connection pool (jdbc/LiferayPool, Tomcat JNDI resource) is generous (maxActive=100), so pool exhaustion seems unlikely to be the root trigger, though it may compound the impact.

Working theory: CounterFinderImpl’s counter-increment lock is global (not scoped per counter name), so when onAfterUpdate cascades through 100+ articles — each needing sequential ID allocations across multiple counters (DDMFormValues, DDMFormFieldValue, etc.) — all this work serializes through one lock. Combined with whatever else on the portal also needs new IDs at the same moment, this could produce a much longer real-world stall than expected, though 5+ minutes for ~100 articles still seems disproportionate.

Question for the community:

• Is this a known scaling characteristic of DDMStructureModelListener.onAfterUpdate / CounterFinderImpl locking when a structure has a large number of dependent articles?
• Has anyone seen this specific freeze pattern after upgrading to 2026.Q2, or is this likely unrelated to the version and just a data-volume issue we hadn’t hit before?
• Is there a supported way to defer/background this cascade (e.g., process structure-dependent reindexing asynchronously) rather than having it run synchronously in the save request thread?

[Allison_McGill]

Hi @Mark_Clarke

You are not the first to report this behavior unfortunately : (

Updating a DDM Structure is a synchronous operation. When a structure is modified, the platform must immediately propagate that change to all dependent Web Content instances, including database updates, cache invalidation, and search reindexing.

I know there are plans to implement some fixes to try to minimize these issues, but at the moment large numbers of content per structure may still cause impact to performance.

What some other people have done to reduce the impact is:

1. Delete old Web Content versions and keep only the latest version per article and them limit the maximum article versions to prevent versions from accumulating.
2. Disable journal.articles.index.all.versions=false to reduce reindexing work triggered by the structure updates.
3. You could try increasing the counter.increment up to 5000-10000 to reduces how frequently the CounterFinderImpl lock is contended

[Mark_Clarke]

Ok, so I cleaned out all versions, leaving just 1 for all the affected content, and there are only 188 articles. I increased the counter.increment = 10000. The problem still persisted. The thread lock issue persists. I then looked at the database structure and this is what I found with the assistance of AI.

---

Liferay DXP 2026.Q2 — DDM Structure save freezes portal; also found duplicate DDMField/DDMFieldAttribute rows accumulating from repeated saves

Environment:

• Liferay DXP 2026.q2.0, Docker deployment
• PostgreSQL 16+ (native), Elasticsearch 8.19.18 (native)
• Java 21 (Zulu), G1GC
• Upgraded from 2026.Q1.5-LTS

Original symptom: Editing a Web Content Structure and clicking Save (even a trivial change like toggling a field from required to optional — no schema impact) causes the entire portal to become unresponsive for all users, sometimes for 5+ minutes, with no errors logged anywhere.

Root cause #1 — confirmed via thread dump: DDMStructureModelListener.onAfterUpdate cascades through every JournalArticle/DDMFormValues version tied to the structure. Each one calls CounterLocalServiceImpl.increment(), which goes through CounterFinderImpl.getCounterRegister() — a single global lock (ConcurrentHashMap), not scoped per counter name. With enough dependent content, every counter allocation across the entire portal serializes through this one lock, one at a time. We had one structure with 1,981 total article versions attached; the cascade took 5+ minutes and blocked unrelated portal activity system-wide.

Mitigations that helped (all recommended by staff/community here, thank you):

• Trimmed excess JournalArticle versions per article (1,981 → 188 for this structure)
• journal.article.version.max=5 going forward
• journal.articles.index.all.versions=false
• counter.increment raised from 2000 to 10000 (note: we hit a real gotcha here — a duplicate counter.increment line across merged portal-ext.properties layers caused PropsValues.COUNTER_INCREMENT to silently resolve to 0, disabling batching entirely and making things worse. Worth checking PropsValues.COUNTER_INCREMENT directly via Script console, not just PropsUtil.get(), since the latter showed a concatenated "10000,2000" string that masked the bug.)

None of these eliminate the underlying architecture — they reduce frequency/severity. This matches what a community member told us: “not the first to report this… plans to implement some fixes… at the moment large numbers of content per structure may still cause impact to performance.”

Root cause #2 — new finding, possibly more concerning: While investigating a specific checkbox field (Checkbox58879974) that appeared in article content and the structure’s DDMStructureLayout, but was missing from DDMStructureVersion.definition JSON, we found the field is genuinely live (confirmed via DDMStructureLocalServiceUtil.getStructure().getDDMForm().getDDMFormFields() — the true source of truth). But querying DDMField directly for this structure’s structureVersionId revealed 13 duplicate rows for one radio field and 2 for the checkbox, all with distinct fieldIds under the same structureVersionId, same fieldName:

1759110 | Radio88009340 | radio
1759118 | Radio88009340 | radio
1759826 | Radio88009340 | radio
... (13 total)
1766306 | Checkbox58879974 | checkbox
1790312 | Checkbox58879974 | checkbox

We suspect this duplication comes from repeated/interrupted structure-save attempts (several of ours were killed mid-save while debugging the freeze above, via container restarts) leaving orphaned DDMField/DDMFieldAttribute rows behind rather than the save being transactional/idempotent. This may also help explain why DDMField/DDMFieldAttribute counters were already at ~1.7-1.8 million on a fairly modest single-tenant install — repeated interrupted saves burning through IDs for the same logical fields over and over.

Questions for the community/staff:

1. Is the CounterFinderImpl global-lock bottleneck for structure-save cascades scheduled for a specific upcoming release, or is there a tracking ticket we could follow?
2. Is duplicate DDMField row accumulation from interrupted/killed structure saves a known issue? Is there a supported cleanup/dedup path, or does this need manual DB intervention?
3. Is there a safer way to interrupt a hung structure-save cascade (e.g., a supported “cancel” mechanism) rather than restarting the container, given that appears to be what caused the duplication?