WPS — asynchronous workflows for SuiteCRM 7 and 8, with plugins for both versions

Show and Tell
,
1

I open-sourced Workflow Processing Service (WPS) under the MIT license. It is a standalone service that runs business processes over CRM records. There are two plugins for it — one for SuiteCRM 7.14 and one for SuiteCRM 8.10.

- Service: < GitHub - madarch-labs/Workflow-Processing-Service · GitHub >

- SuiteCRM 7 plugin: < GitHub - madarch-labs/WPS_SuiteCRM7_plugin · GitHub >

- SuiteCRM 8 plugin: < GitHub - madarch-labs/WPS_SuiteCRM8_plugin · GitHub >

I built it because the built-in AOW workflow module stopped being enough for me. Below is what did not work for me, how WPS is put together, and what is still bad about it.

The problems we wanted to solve

AOW is fine for simple rules. These are the points where it held us back (checked against the AOW_WorkFlow code):

AOW Workflow

  1. Workflows run inside the save. An “On Save” workflow runs all its actions in the user’s request (after_save). The user waits for record creation and emails, and a failing action fails the save. “In the Scheduler” avoids this but reacts only on the next cron run.
  2. One record in scope. A workflow sees the saved record and records linked by relationships. You cannot look up an arbitrary record by a field (a name, an email, an external code), follow a chain such as case → account → account manager, or process a list of records (say, all open tasks of a deal).
  3. No composition and no flow control. A workflow cannot call another workflow and pass data to it. There are no if / else branches, no loops and no “three days after…” delays. Chains only happen implicitly, when a modified record triggers another workflow — with no link in the audit and no protection against loops.
  4. No integrations. There is no built-in way to call an external REST API, fire a webhook or post to a messenger.
  5. Thin audit, no retries. Process Audit shows Complete / Failed, but no error text, no request and no retry policy for temporary failures.
  6. No way to test a workflow without real records.

How WPS works

The plugin adds before_save, after_save and after_delete logic hooks that do a single thing: XADD the record into a Redis Stream. Since plugin 1.2.0 (event contract 2.0) the event also carries the event type (create / update / delete), the old values of the changed fields, the list of changed fields and the user who saved the record. The Redis client is pure PHP, so phpredis is not needed. If Redis is unreachable, the save still succeeds (1 s timeout) and the error goes to suitecrm.log.

  • WPS (PHP 8.3 + Swoole) consumes the stream asynchronously, picks the processes for that module and event type, and runs their actions. It talks back to the CRM through a REST API that the same plugin provides: /api/v9/{module}[/{id}], filters with operators, sorting, cursor pagination, counting, create, update, delete and relationships. The API uses JSON:API with an X-Auth-Key header that changes every hour. WPS reads GET /capabilities to see what the CRM supports, so older plugins keep working.

  • Processes, executions, a step-level audit, sleeping (waiting) executions, a side-effect journal and encrypted secrets live in PostgreSQL. A Vue 3 UI (English / Russian) and a REST API (50+ endpoints) are available for configuring processes.

    What you can do in a process

    image
    image1440×1614 237 KB

    Triggers. A process reacts to create, update and/or delete events of a module.

  • Variables and preload. Besides object (the saved record), load any CRM record into a named variable, by id or by any field. Each lookup value can come from the object, from another variable (chains: case → account → manager) or be typed in. No relationship is required.

  • Conditions. There are 25 operators:

    • equality and ordering (greater_than, less_or_equal…);
    • ranges (between);
    • lists (in, not_in);
    • substrings (starts_with, ends_with, contains);
    • multi-select values (contains_any, contains_all, contains_none);
    • emptiness (is_empty, is_null);
    • field changes (changed, changed_to, changed_from, not_changed).

    Values are type-cast (int, decimal, date, list…), with case-insensitive and trimmed comparison available. The right-hand side can be a constant, a field of another variable (object.assigned_user_id not equals manager.id), an expression (account.credit_limit * 0.8) or a relative date (today+30d, start_of_quarter). Conditions nest into AND / OR groups. Preload is lazy: if the conditions on the record itself already fail, the CRM is not called.

  • Expressions and templates. There are 121 functions with exact decimal arithmetic:

    • strings, including regex, translit and slug;
    • numbers;
    • dates and time zones;
    • lists (to_list, unique, sort, pluck, where, sum);
    • if / switch / coalesce;
    • JSONPath.

    Templates such as "Deal {{ object.name }} for {{ object.amount | number:2 }}" work too. Expressions and templates can use any field of any variable, plus system variables sys.* (execution id, process name and version, event type, changed fields, user, current time, last error) and per-process constants config.*.

  • Modify and save anything. Update the record, a related record, or a new variable — which becomes a new record. Only changed fields are sent. Change types also include date shift, increment / decrement, append to a multi-select, a lookup table, and an aggregate over CRM records (count, sum, avg, min, max).

  • Child processes. “Start next process” runs child processes asynchronously and in parallel:

    • each child gets a copy of the event and all the parent’s variables;
    • each child has its own conditions and run-once setting;
    • a failing child affects neither the parent nor its siblings;
    • the audit links parent and child executions;
    • depth is limited and cycles are rejected when a process is saved;
    • one child process can be reused by several parents.

image
image1440×1797 233 KB

Actions beyond “update a field”

Group Actions
CRM data preload, modify, save, find records (find_records), update every record of a list (update_records), link / unlink records
Flow if / else-if / else branches (condition), loops (foreach), delays (delay), early exit (stop_process), process variables and accumulators (set_variable)
People and external systems email (send_email, SMTP), notifications (notify: the CRM bell, Telegram, Slack, Microsoft Teams), HTTP requests (http_request), assignment (assign: round robin, least busy, manager, specific user)
Processes child processes (start_process)

Feedback and contributions welcome

We would really like to hear from the SuiteCRM community:

  • :star: Star the repositories if this looks useful — it helps others find the project: service, SuiteCRM 7 plugin, SuiteCRM 8 plugin.
  • :bug: Open issues for bugs, other SuiteCRM versions, or workflows you can’t express yet.
  • :hammer_and_wrench: Pull requests are welcome:
    • editor support for the new actions;
    • more notification channels;
    • nginx setups;
    • other CRMs;
    • UI translations.

Questions about installation or the comparison with AOW are welcome in this thread. Thanks!

3 Likes