Issue with MEMORY_CLUSTERED jobs and server restarts in Liferay DXP 2024.Q1.12 (Cloud, 2 nodes)

General
1

Hello everyone,

We are currently working with Liferay Digital Experience Platform 2024.Q1.12 deployed on Liferay Cloud with 2 nodes.

What we are doing

  • We have implemented schedulers to import products from SAP into Liferay.

  • The scheduler registers background tasks.

  • Background tasks are configured to run 10 tasks in parallel.

  • Each background task is responsible for fetching product data from SAP and inserting/updating it in Liferay.

The issue

  • After some time, we see the following message in the logs:

    57 MEMORY_CLUSTERED jobs started running on this node

  • When nodes switch a couple of times, the server restarts unexpectedly.

  • This behavior seems related to how jobs are being executed in MEMORY_CLUSTERED mode instead of being persisted.

Questions

  1. Why are these jobs running in MEMORY_CLUSTERED mode even though we are registering background tasks?

  2. How can we configure these background tasks (or schedulers) to run in persistent mode so they survive node switches and restarts?

  3. Is there a recommended approach for running multiple background tasks in a clustered Liferay Cloud environment without hitting this issue?

Any guidance or best practices for configuring persistent schedulers/background tasks in Liferay Cloud would be greatly appreciated.

Thanks in advance!

1 Answer

1
2

Hi @pballal ,

While implementation of persistent schedules and the recommended approach for Liferay Cloud is outside of my wheelhouse, I can provide resources the help answer question 1 and 2.

  1. Why are these jobs running in MEMORY_CLUSTERED mode even though we are registering background tasks?

MEMORY_CLUSTERED jobs are the default… for Liferay schedulers.

Possibly meaning unless you explicitly request PERSISTED, the Quartz library and Liferay will choose MEMORY_CLUSTERED regardless of how it’s registered. For more info see: How to diagnose and recover Liferay cluster.

  1. How can we configure these background tasks (or schedulers) to run in persistent mode so they survive node switches and restarts?

Liferay Source Code outlines the three types of storage, this is how you can configure PERSISTED.

Coupling this with the instructions noted in the portal.properties, this should give you insight on how to configure the scheduler to run in PERSISTED mode.