Commit graph

396 commits

Author SHA1 Message Date
Sipke Schoorstra f99f37407c
feat(bpmn): host port and command applier (#7942)
* feat(bpmn): host-side applier for the Bpmn.Semantics port

Translates the interpreter's three host commands onto ActivityExecutionContext
and feeds its four entry points, plus the minimum BpmnProcess container needed
to exercise them end to end through IWorkflowRunner.

StartWork schedules the bound activity, CancelWorkSubtree calls the public
CancelActivityAsync extension (already recursive), and SignalEnclosingScope
sends a BpmnScopeSignal up the ancestor chain. OnWorkFaulted rides the
FaultSignal seam: it asks the interpreter what BPMN made of the fault and calls
StopPropagation only on a Caught disposition, leaving a Propagated one strictly
alone so an enclosing scope or the incident strategy takes it.

A unit of work is keyed on the child ActivityExecutionContext.Id, recorded in
the scope's own persisted ledger, never on Tag: the completion-callback dispatch
rewrites the receiving context's Tag, so a nested scope wears a different tag
than its parent remembers it by. Interpreter correlation travels on the child's
context rather than on the shared activity instance.

Evaluations go through one queue per workflow instance, so a scope signalled
mid-apply is drained after the command list rather than re-entering the
interpreter. Commands are applied in the order returned.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* test(bpmn): cover the teardown refusal path

Adds a focused unit test that drives BpmnWorkTeardown.CancelSubtreeAsync
into the NotSupportedException branch by constructing a real context tree
with a scheduled-but-not-invoked descendant, so a regression that silently
drops the detection is caught. Also records why BpmnWorkLedger's
append-only, handle-keyed Records list cannot strand a context on a
duplicate StartWork for a live (BindingRef, IterationId) slot, a case the
port's own guarantee makes unreachable from this applier.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(bpmn): keep a refused teardown from stranding ledger state

A subtree cancellation refused with NotSupportedException is absorbed
into an incident under ContinueWithIncidentsStrategy rather than
crashing, so the end-of-command ledger save was being skipped and the
persisted ledger kept claiming work BPMN had just torn down. Save the
ledger removal before the possible throw instead of after, so a later
completion callback for the stranded activity finds no live record and
is discarded instead of being fed to the interpreter.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
2026-08-14 01:42:31 +02:00
Sipke Schoorstra 22886809e0
test(bpmn): guard that Elsa.Bpmn never duplicates the library (#7939)
* test(bpmn): guard against Elsa.Bpmn* reimplementing Bpmn.* library types

Adds a reflection-based architecture test asserting no type under Elsa.Bpmn
or Elsa.Bpmn.Interchange shares a type name with Bpmn.Model/Bpmn.Semantics
(and Bpmn.Interchange for the interchange assembly), plus a positive check
that both assemblies still depend on their respective library packages -
closing the gap that let elsa-foundation grow a parallel BpmnElement/BpmnGraph
semantics core undetected.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* test(bpmn): move the library-duplication guard to the interchange test project

Elsa.Bpmn.UnitTests referencing Elsa.Bpmn.Interchange (plus three
redundant PackageReferences already flowing transitively) inverted the
layering the guard exists to protect. Elsa.Bpmn.Interchange.UnitTests
already sees both assemblies transitively with no new references, so
the guard moves there unchanged apart from namespace and doc. Also
drops the silent null-forgive on Assembly.GetName().Name in favor of
an explained fallback.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* test(bpmn): drop the dependency-direction assertions the compiler already enforces

Mutation testing showed AssertDependsOnPackage and the two positive facts using
it can never go red: any state that would trip them fails the test project's
build first (CS0234), because the guard's own typeof bindings already require
the Bpmn.* packages. Delete the decorative facts and the now-unused helper,
and document that the typeof bindings are load-bearing on purpose.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* test(bpmn): prove the duplication detector actually detects

Split AssertNoTypeNameCollisions into a thin assertion wrapper around a
new FindTypeNameCollisions helper, and add a fact that points the
detector at the test assembly (which carries a deliberately colliding
BpmnGraph fixture) so CI sees the guard fail as well as pass.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
2026-08-13 04:09:53 +02:00
Sipke Schoorstra 735e953dc5
feat(bpmn): scaffold Elsa.Bpmn and Elsa.Bpmn.Interchange modules (#7938)
Adds the valence-works/bpmn package feed and pins Bpmn.Model,
Bpmn.Semantics and Bpmn.Interchange at 0.1.1-preview.19, then wires
two new module projects consuming those libraries without
reimplementing anything they provide (D12): Elsa.Bpmn for BPMN
execution and Elsa.Bpmn.Interchange for XML import/export, kept as a
separate package so hosts that only execute BPMN don't take the XML
reader. Both are marked IsPackable=false until the Bpmn.* packages
are published to nuget.org. Test projects are added for both, unit
and integration, and all four are wired into Elsa.sln so PR CI
discovers and runs them. This unblocks #7925 and the rest of the
BPMN runtime work in #7909.

Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
2026-08-13 01:17:46 +02:00
Sipke Schoorstra 5faba75906
fix(core): a fault a container claimed is not an incident (#7923)
* fix(core): a fault a container claimed is not an incident (#7911)

RecoverFromFault reset the counts and the status but left behind the two other
things Fault recorded: the ActivityIncident and the exception. So a container
that successfully handled a child's fault still left the workflow carrying an
incident.

That is not cosmetic. Code reads a non-empty WorkflowExecutionContext.Incidents
as "this workflow failed" without looking further; HttpWorkflowsMiddleware is
one, and it hands the caller a fault response. A workflow whose container caught
the error and finished normally was reported to its caller as failed.

RecoverFromFault is now the inverse of Fault: it removes the incident Fault
appended, matched on this activity's node id and most recent first so an
activity that faults, recovers and faults again keeps the incident that was
never recovered, and it clears the recorded exception so the activity does not
sit in Running carrying one.

The execution log still records the failure, so nothing is hidden from anyone
reading the journal. Two integration assertions that encoded the old behaviour
are updated; they were written from the reasoning this change corrects.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(core): tie an incident to the execution that raised it, not its node

Recovery matched the incident to remove on ActivityNodeId, which identifies the
static workflow node rather than an execution of it. A node inside a loop,
retried, or run concurrently raises one incident per execution, all under the
same node id, so recovering one execution could remove another's incident and
leave its own behind.

ActivityIncident now carries the ActivityInstanceId of the execution that raised
it, and recovery matches on that. Within a single execution the most recent is
still taken, so fault, recover, fault again keeps the incident that was never
recovered. The property is optional: an incident recorded against the workflow
itself has no execution, and so do incidents persisted before this existed.

Caught by review on #7923.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(api-client): mirror ActivityInstanceId on the client incident model

The server model gained the property in the previous commit and the API client
carries a hand-maintained copy of it. Left alone, a client deserializing an
incident would silently drop the only field that says which execution raised it.

Also records two consequences of recovery that were implicit: it relies on the
incident collection preserving insertion order to pick an execution's newest
incident, which holds only because the collection is list-backed; and clearing
the exception also clears it from the activity's execution record, which is
intended for the same reason the incident goes, with the journal keeping the
evidence either way.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
2026-08-12 05:20:22 +02:00
Sipke Schoorstra 13eb48e002
feat(core): let a container handle a child's fault via FaultSignal
A container activity had no way to learn that one of its children faulted.
ExceptionHandlingMiddleware caught the exception, called context.Fault(e) and
handed off to the workflow-global IIncidentStrategy; the container's completion
callback never fired, because the child never completed.

Add a seam on the ancestor-bubbling signal channel that already exists:

- FaultSignal(Exception, ActivityExecutionContext), beside CancelSignal. Its XML
  doc carries the contract, including why a handler must not call
  RecoverFromFault and why the CompleteActivityAsync sweep is a backstop rather
  than the mechanism.
- An internal bool-returning TrySendSignalAsync, since SignalContext
  .StopPropagationRequested is internal and SendSignalAsync reported nothing.
  SendSignalAsync keeps its public signature and delegates to it.
- ExceptionHandlingMiddleware sends the signal after faulting and, when an
  ancestor stops propagation, calls RecoverFromFault once and returns instead of
  raising an incident.

RecoverFromFault now transitions to Running only when the activity is still
Faulted. It is called after the handler runs, so the unconditional transition
would otherwise undo a handler that cancelled or completed the faulted child.
The counts are still reset unconditionally, and the one pre-existing caller is
unaffected.

Behavior is unchanged when nobody handles the signal: verified by running the
new unhandled-fault theory against the pre-change middleware, and by
IncidentStrategyTests and Primitives/FaultTests passing unmodified.

Refs #7911

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-11 01:11:25 +02:00
Sipke Schoorstra 5d7db9d057
Merge remote-tracking branch 'origin/main' into 012-output-converters 2026-07-31 13:29:06 +02:00
Sipke Schoorstra 0737def83d
Harden output converter contracts 2026-07-31 13:29:01 +02:00
Sipke Schoorstra d698e6b005
Add output converter support at binding boundaries 2026-07-31 04:10:48 +02:00
Sipke Schoorstra ec9acd4f3f
Fix tenant service mutation race (#7898)
Serialize tenant lifecycle mutations and keep synchronization available through shutdown.

Closes #7771.
2026-07-30 02:47:44 +02:00
Sipke Schoorstra 04e58690ff
chore: apply safe dependency upgrades (#7896) 2026-07-26 21:42:27 +02:00
Sipke Schoorstra 74b4778552
Merge pull request #7756 from MohitGuptaC/main
fix: correct Oracle identifier quoting and NVARCHAR2 type cast in GenerateOracleUpsert
2026-07-11 15:37:45 +02:00
Sipke Schoorstra 33181b2c9d
test: cover Oracle bulk upsert SQL generation
Exercise quoted Oracle identifiers, NVARCHAR2 casts, multi-row SQL, update/insert clauses, and parameter ordering. Reuse the provider SQL generation helper so identifier delimiting stays aligned with Oracle EF Core.
2026-07-11 14:35:36 +02:00
Sipke Schoorstra 5336f9ea06
Increase AI host test coverage 2026-06-27 04:04:39 +02:00
Sipke Schoorstra 571c1c7b2c Shorten configured application instance names
Configured stable application instance names that exceed the Azure Service Bus transport entity limit are now shortened deterministically instead of failing startup. The same configured value resolves to the same shortened name across restarts, preserving stable per-instance transport identity while supporting normal Kubernetes pod names.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-06-20 12:50:02 +02:00
Sipke Schoorstra b08b10132c
address greptile review feedback (greploop iteration 1)
- Make validation constants internal and expose via InternalsVisibleTo so tests reference single source of truth
- Add empty-string guard to IsValidConfiguredInstanceName to prevent IndexOutOfRangeException
- Add ClusteringFeature_UsesConfiguredInstanceNameProvider test for Features.ClusteringFeature to match existing ShellFeatures coverage

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-06-20 00:13:33 +02:00
Sipke Schoorstra 55284aa1c1
Add stable application instance name configuration
Forward-port the opt-in stable application instance name support from PR #7734 so clustered deployments can reuse per-instance transport entities across restarts while preserving random names by default.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-06-20 00:04:15 +02:00
Sipke Schoorstra 8e20386a28
Merge pull request #7705 from elsa-workflows/codex/7579-tenant-agnostic-api-key
fix: use tenant-agnostic application lookup for API keys
2026-06-17 20:16:22 +02:00
Sipke Schoorstra 46cf0739e6
Merge remote-tracking branch 'origin/main' into codex/7708-notfoundactivity-guard
# Conflicts:
#	src/modules/Elsa.Workflows.Core/Serialization/Converters/ActivityJsonConverter.cs
2026-06-14 22:26:41 +02:00
Sipke Schoorstra 8f721e1ea2
Guard HTTP fault handling when workflow reload returns null (#7714)
* Guard HTTP fault handling when workflow reload returns null

* Add HTTP fault handler reload guard tests

* Address Greptile review feedback
2026-06-14 22:23:38 +02:00
Sipke Schoorstra 0d7c810688
address greptile review feedback 2026-06-14 19:07:15 +02:00
Sipke Schoorstra 48a087e71e
Restore RequestAborted after timed HTTP workflow failures (#7712)
* Fix RequestAborted restoration for timed HTTP workflows

* Add timeout restoration tests for HttpWorkflowsMiddleware
2026-06-14 19:02:23 +02:00
Sipke Schoorstra 9c24f5efe5
Dispose parsed activity JsonDocuments (#7713)
* Dispose parsed activity JsonDocuments

* Add ActivityJsonConverter disposal regression test

* Fix ActivityJsonConverter disposal regression test file

* address greptile test coverage feedback
2026-06-11 20:33:10 +02:00
Sipke Schoorstra 42ef838323 Add regression test for missing NotFoundActivity descriptor 2026-06-09 21:28:29 +02:00
Sipke Schoorstra 1d1c68282c
fix: use tenant-agnostic application lookup for api keys 2026-06-09 06:09:47 +02:00
Sipke Schoorstra 50ef957031
[codex] Add runtime entity validator coverage (#7697)
* Add runtime entity validator coverage

* Potential fix for pull request finding

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

---------

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
2026-06-06 04:14:12 +02:00
Sipke Schoorstra 3936258146
Implement Weaver AI Copilot core (#7523)
* Implement Weaver AI Copilot core

* Address Greptile review feedback

* Address Greptile persistence feedback

* Address Greptile orchestration feedback

* Address Greptile tool isolation feedback

* Wire chat audit events

* Stream chat events over SSE

* Use server identity for AI endpoints

* Validate AI proposal persistence

* Isolate AI audit failures

* Enforce AI tool lookup scope

* Support AI tool result continuations

* Handle AI chat reconnects safely

* Tighten AI context and reconnect behavior

* Guard AI conversation and persistence setup

* Persist AI tool-loop progress

* Tighten AI tool registry and reconnect cleanup

* Handle AI preparation failures cleanly

* Order AI tool messages after assistant turns

* Initialize AI provider sessions

* Align AI context capabilities

* Prevent completed AI reconnect replay

* Enforce AI conversation ownership

* Default AI proposal creation time

* Persist AI session and retention defaults

* Allow AI context provider overrides

* Scope AI tool results per turn

* Apply AI provider configuration

* Scope AI proposal reads

* Avoid duplicate AI tool continuations

* Resolve AI tool registry scopes

* Tighten AI reconnect cleanup

* Honor default AI proposal tools

* Pass AI provider session to turns

* Close AI observability gaps

* Fix AI capabilities options alias

* Harden AI orchestration lifetimes

* Track actual AI reconnect conversation

* Address AI audit and context review findings

* Fix AI reconnect and persistence capabilities

* Handle AI session startup failures

* Tighten AI orchestration review gaps

* Warn on placeholder AI context

* Filter disabled AI provider tools

* Add durable AI conversation persistence

* Fix AI orchestrator persistence lifetime

* Handle failed AI reconnect edge cases

* Harden AI reconnect failure handling

* Address AI reconnect and cleanup review gaps

* Tighten AI audit and cleanup persistence

* Keep expired AI cleanup best effort

* Tighten AI tool lookup and cleanup fallback

* Handle AI provider and tenant edge cases

* Tighten AI proposal and agent authorization

* Address AI tool scope cleanup review

* Close remaining AI greptile findings

* Harden AI stores and tool defaults

* Harden AI conversation persistence edge cases

* Cover AI proposal and tool visibility guards

* Fix AI capabilities and audit batch resilience

* Fix AI conversation truncation for unicode

* Resolve remaining AI persistence review nits

* Wire AI conversation persistence option

* Address AI audit and proposal style review

* Fix AI stream truncation surrogate handling

* Address AI context and cleanup review

* Preserve AI titles and tenant tool defaults

* Guard AI conversation user ownership

* Align in-memory AI conversation ownership

* Fix expired AI conversation cleanup tracking

* Harden AI proposal persistence retry

* Tighten AI proposal reads and cleanup SQL

* Harden AI reconnect and provider defaults

* Optimize AI tool listing and message trimming

* Preserve AI conversation timestamps

* Address final AI persistence review nits

* Normalize AI acronym casing

* Address Copilot AI review comments

* Normalize default tenant handling for AI stores

* Harden AI registry and message truncation

* Make AI tool filtering explicit

* Align AI contracts with implementation

* Align remaining AI review contracts

* address greptile ai persistence feedback

* Address Copilot AI persistence feedback

* Address Copilot AI host feedback

* Order persisted AI conversation messages

* Address Copilot chat and cleanup feedback

* Release unused AI reconnect reservations

* Address Copilot AI review feedback

* Address Copilot tool and conversation feedback

* Address Copilot governance feedback

* Address Copilot tool test feedback

* Address AI review follow-ups

* Address Copilot AI follow-ups

* Clean up AI persistence tests

* Address IAITool disposal review

* Address AI integration review follow-ups

* Address AI chat persistence review

* Address AI registry and truncation review

* Enable read-only AI tools by default

* Potential fix for pull request finding

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

* Potential fix for pull request finding

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

---------

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
2026-06-04 11:35:44 +02:00
Sipke Schoorstra c16995014c
Add Persistence vNext provider-neutral POC
Adds the Persistence vNext proof of concept, including provider-neutral schema/document abstractions, relational and document provider POCs, Elsa integration, runtime-defined entities, physicalization planning, workflow runtime evaluation, and focused tests.
2026-06-02 21:05:21 +02:00
Sipke Schoorstra 39e22bea31
Add operational dashboard API
* Add dashboard API foundation

Merge dashboard API foundation into dashboard-v2

* Harden dashboard API provider

Documents the dashboard API contract, adds provider behavior tests, and degrades diagnostics capability states independently.
2026-06-02 02:33:11 +02:00
Sipke Schoorstra 87ee19f5b3
Implement secret update functionality, add related tests, and update project references. 2026-06-01 23:47:05 +02:00
Sipke Schoorstra aa55274bcf
Auto stash before checking out "origin/main" 2026-06-01 22:42:16 +02:00
Sipke Schoorstra 69d9661fc0
Use shared secret name normalization 2026-06-01 22:28:38 +02:00
Sipke Schoorstra 863a43d318
Validate secrets SQLite migrations in repository tests 2026-06-01 21:56:36 +02:00
Sipke Schoorstra 7a5569819b
Merge remote-tracking branch 'origin/codex/secrets-efcore-persistence' into codex/secrets-efcore-persistence 2026-06-01 21:48:49 +02:00
Sipke Schoorstra b001a28ace
Merge remote-tracking branch 'origin/main' into codex/secrets-efcore-persistence 2026-06-01 21:48:42 +02:00
Sipke Schoorstra 9fac0c83c0
Harden EF Core secret name uniqueness 2026-06-01 20:57:11 +02:00
Sipke Schoorstra e446b1090c
Merge remote-tracking branch 'origin/codex/workflow-secret-references-s2' into codex/workflow-secret-references-s5 2026-06-01 09:37:54 +02:00
Sipke Schoorstra b438551c78
Add Secret expression runtime 2026-06-01 09:20:55 +02:00
Sipke Schoorstra c8ffcb2a8a
Protect sensitive workflow inputs 2026-06-01 09:15:34 +02:00
Sipke Schoorstra e083d3b307
Add EF Core secrets persistence 2026-05-31 23:23:43 +02:00
Sipke Schoorstra d84d83f828
Remove SecretProviderAdapter and ISecretProvider to streamline secret management; update solution and project references accordingly. 2026-05-31 15:41:12 +02:00
Sipke Schoorstra 48c4259844
Pin test environment to Production in IngressRateLimitingTests to stabilize assertions. 2026-05-31 14:44:34 +02:00
Sipke Schoorstra 209f93bfb5
Preserve activity descriptors on empty refresh 2026-05-31 13:34:30 +02:00
Sipke Schoorstra 66af304364
Optimize activity registry descriptor lookup (#7538)
* Optimize activity registry lookup

Add a latest-version descriptor index per tenant registry so ActivityRegistry.Find(type) can avoid scanning every registered descriptor. Keep the index in sync when descriptors are added, removed, refreshed, or cleared.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Handle provider descriptor removal during refresh

Remove descriptors from registries that a refreshed provider no longer contributes to keep descriptor indexes consistent when a provider drops a tenant group entirely.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Address activity registry review feedback

* Clear manual activity descriptors

* Integrate ConsoleLogStreaming.Core and ConsoleLogStreaming.SignalR packages; refine activity registry lookup logic.

* Reduce activity registry refresh allocations

---------

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-05-31 13:15:42 +02:00
Sipke Schoorstra c2fb027c41
Refactor: Overhauls workflow JSON type serialization (#7549)
* Avoid null endpoint DTO metadata in tests

* Enforce console logs hub read permission

* Remove unused console logs hub import

* Support mapped endpoint metadata in auth tests

* Reduce console log capture throughput impact

* Address Copilot console logs review

* Refactor task scheduling to support tenant-level background work and enhance logging functionality.

* Introduce ConsoleStreamHook for stdout/stderr tee and enhance logging validation. Adjust test cases and startup warnings for distributed lock provider usage.

* Refactor console logging pipeline with capture optimization and new ConsoleLogsHost; update tests accordingly.

* Add Ansi SGR parser for console logs and associated unit tests

* Remove ANSI color renderings and parsers; integrate ConsoleLogScopeAccessor for improved logging context with workflow instance ID support.

* Address console logs code quality feedback

* Address PR review feedback

* Preserve console logs extension points

* Stabilize console logs host lifecycle

* Address final automated review comments

* Tighten console log capture shutdown

* Address console log review feedback

* Address follow-up review feedback

* Cover final review feedback

* Avoid recursive console provider initialization

* Guard console host lease shutdown

* Preserve console log scope and provider lifetime

* Correlate console log scope fallback

* Tighten console scope correlation

* Expose host services during provider construction

* Redact ANSI-normalized console lines

* Remove `ConsoleCaptureTee` and related services and tests

* Use pipeline contributors for console log context

* Update CShells package versions to 0.0.24-preview.132

* Filter live console logs by workflow instance

* Enhance console logging with activity execution metadata and extend test coverage.

* Address console logs stream consumption comment

* Add diagnostics OpenTelemetry backend

* Introduce dedicated workflow JSON type registry and hardening

This change addresses GitHub issue #7541 by establishing a separate type registry (`IWorkflowJsonTypeRegistry`) for workflow JSON serialization. This decouples workflow type resolution from expression type aliases, enforcing a strict trust boundary.

Key aspects:
- New workflow JSON emits preferred aliases for registered types.
- Existing persisted workflows can be loaded via registered legacy names.
- Unknown, abstract, interface, open generic, or inappropriate collection types are rejected during deserialization, enhancing security.
- Public APIs (e.g., incident strategies) now expose consistent workflow JSON type identifiers.

This ensures secure, predictable, and backward-compatible handling of types within workflow definitions and payloads.

* Remove unused project references and streamline console log endpoint

* Move serialization type aliases to Elsa.Common

* Update serialization integration fixtures for aliases

* Stabilize missing rate limiter policy test
2026-05-31 11:09:39 +02:00
Sipke Schoorstra b280a711b2
[codex] Fix diagnostics live feed regressions (#7548)
* Avoid null endpoint DTO metadata in tests

* Enforce console logs hub read permission

* Remove unused console logs hub import

* Support mapped endpoint metadata in auth tests

* Reduce console log capture throughput impact

* Address Copilot console logs review

* Refactor task scheduling to support tenant-level background work and enhance logging functionality.

* Introduce ConsoleStreamHook for stdout/stderr tee and enhance logging validation. Adjust test cases and startup warnings for distributed lock provider usage.

* Refactor console logging pipeline with capture optimization and new ConsoleLogsHost; update tests accordingly.

* Add Ansi SGR parser for console logs and associated unit tests

* Remove ANSI color renderings and parsers; integrate ConsoleLogScopeAccessor for improved logging context with workflow instance ID support.

* Address console logs code quality feedback

* Address PR review feedback

* Preserve console logs extension points

* Stabilize console logs host lifecycle

* Address final automated review comments

* Tighten console log capture shutdown

* Address console log review feedback

* Address follow-up review feedback

* Cover final review feedback

* Avoid recursive console provider initialization

* Guard console host lease shutdown

* Preserve console log scope and provider lifetime

* Correlate console log scope fallback

* Tighten console scope correlation

* Expose host services during provider construction

* Redact ANSI-normalized console lines

* Add OpenTelemetry diagnostics backend foundation

* Add OTLP HTTP ingestion parsing

* Document OpenTelemetry diagnostics setup

* Enforce OpenTelemetry hub permissions

* Remove `ConsoleCaptureTee` and related services and tests

* Add OpenTelemetry HTTP ingestion integration test

* Use pipeline contributors for console log context

* Update CShells package versions to 0.0.24-preview.132

* Add OpenTelemetry ingestion security tests

* Add OpenTelemetry API authorization tests

* Filter live console logs by workflow instance

* Add OpenTelemetry hub tests

* Add OpenTelemetry gRPC metadata hook

* Assert OpenTelemetry workflow tags survive ingestion

* Mark OpenTelemetry core build verified

* Enhance console logging with activity execution metadata and extend test coverage.

* Address console logs stream consumption comment

* Wire OpenTelemetry diagnostics into core sample

* Address Core diagnostics review feedback

* Address Core Copilot follow-up feedback

* Add OpenTelemetry metric instrument names

* Address Core Copilot provider feedback

* Address Core Copilot diagnostics follow-up

* Address Core Copilot live feed feedback

* Address Core Copilot store feedback

* Integrate OpenTelemetry for logging, tracing, and metrics in ModularServer and update launch settings and docker-compose configuration.

* Refactor to replace `ConsoleLogStream.Core` with `ConsoleLogStreaming.Core` across codebase and update `ConsoleStreamHook` installation.

* Add diagnostics OpenTelemetry backend

* Fix OpenTelemetry live hub subscription

* Fix modular OpenTelemetry exporter endpoints

* Add CShells logging configuration in appsettings.json

* Remove obsolete unit tests and helper classes

* Restore default activity exception handling

* Simplify type serialization and alias management

This commit refactors the internal type serialization and alias management system to reduce boilerplate, improve robustness, and simplify the developer experience:

-   Removed numerous explicit `ExpressionOptions` type alias registrations across various modules.
-   Updated `TypeJsonConverter` and polymorphic serialization to reliably handle types using assembly-qualified names when a short alias is not explicitly registered.
-   Streamlined `ExcludeFromHashConverter` to strictly adhere to `ExcludeFromHashAttribute` for hash calculations, removing complex `JsonIgnoreCondition` logic.
-   Eliminated several helper classes (`WorkflowJsonTypeResolver`, `WorkflowTypeValidator`, `IWorkflowTypeRegistry`, `WorkflowFactoryDictionary`, `JavaScriptExceptionTypeAliasRegistrar`, `WorkflowRuntimeTypeAliasRegistrar`) and their associated unit tests, simplifying the codebase.

Additionally, this commit introduces a comprehensive markdown document (`product-website-feature-source.md`) outlining Elsa's core features, Studio capabilities, extension ecosystem, and architectural selling points, intended as source material for the product website.

* Refine type serialization for improved robustness and alias handling

This commit further enhances the type serialization and deserialization mechanisms:

*   Centralizes type resolution and alias management through `IWellKnownTypeRegistry` and `WorkflowJsonTypeResolver`.
*   Prioritizes registered type aliases when serializing type metadata in `PolymorphicObjectConverter`, resulting in more concise JSON output.
*   Enhances deserialization in `PolymorphicObjectConverter` and `VariableMapper` to gracefully handle unknown or non-instantiable types, providing fallbacks and logging warnings.
*   Simplifies `TypeJsonConverter` by delegating complex type resolution logic to the `WorkflowJsonTypeResolver`.
*   Adds `JsonArray` to the well-known type aliases for direct recognition.

* Fix console logs packaging and workflow type resolution

* Fix console log metadata and type resolution

* Address Copilot review feedback

* Enhance type resolution, improve console log handling, and update tests

- Streamlined `WorkflowDictionaryExtensions` for better workflow registration validation.
- Refined `ConsoleLogsAuthorizationTests` with the new `SetJsonRequest` helper to improve test requests handling.
- Updated `OrderDefinition` to ignore JSON serialization for `KeySelector`.
- Enhanced `WorkflowRuntimeFeature` for improved workflow registration and type alias configuration.
- Added tests to ensure `ConsoleLogProvider` metadata filtration in various scenarios.
- Improved type serialization logic in `WorkflowJsonTypeResolver`.
- Updated README to fix references related to diagnostics.
- Optimized `ExcludeFromHashConverter` for property serialization conditions.
- Modified `TriggerIndexer` for streamlined trigger management.
- Tested payload checks in `PublishEventTests`.
- Adjusted `Endpoint` in `ConsoleLogs` for automatic JSON request handling.
- Ensured registration of workflow type aliases in `WorkflowsFeature`.

* Restore CLR workflow registration compatibility

* Align JSON island serialization fixtures

* Add Console Logs Services and Enhance Endpoint Handling

- Introduced `ActivityExecutionsEndpointTests` to validate route exposure.
- Added `ConsoleLogCaptureHostedService` for console log streaming.
- Implemented `ConsoleStreamJsonConverter` for JSON conversion of console streams.
- Developed `ElsaConsoleLogRecentBuffer` to handle recent log buffering.
- Updated `ConsoleLogsAuthorizationTests` with new test cases for stream filter mapping.
- Consolidated console log provider dependencies and registration, including recent buffering.
- Enhanced `ElsaConsoleLogProvider` to use recent buffer for filtering.
- Adjusted `Program.cs` for streamlined logging service setup.

* Enhance type resolution and test coverage; streamline console log integration

- Added `ConsoleStreamHook` for streamlined log streaming.
- Updated `WorkflowJsonTypeResolverTests` to improve type resolution and test new scenarios.
- Simplified type resolution by removing trusted assembly checks.

* Fix CI smoke and package restore failures

* Fix Docker smoke image project paths

* Fix Docker Python runtime packages

* Fix Docker CA smoke teardown

* Refresh Elsa roadmap

* Implement background processors and mediation coordination

- Added `BackgroundCommandProcessor`, `BackgroundJobProcessor`, and `BackgroundNotificationProcessor` classes for handling commands, jobs, and notifications, respectively.
- Introduced `MediatorBackgroundProcessingCoordinator` to coordinate the execution of all background processors.
- Implemented `MediatorBackgroundTask` for wrapping `MediatorBackgroundProcessingCoordinator` in `BackgroundTask`.
- Added unit tests for `MediatorBackgroundTask` to ensure proper start and stop behavior.
- Refactored `BackgroundCommandSenderHostedService` to utilize `BackgroundCommandProcessor`.
- Introduced 'elsa-roadmap-refresh' skill configuration for roadmap updates.

* Address workflow type resolution review feedback

* Address follow-up review feedback

* Restore recent console logs execute path

* Address Copilot follow-up review

* Decouple workflow JSON aliases from expressions

* Fix workflow management unit test setup

* Fix console logs recent endpoint handler shape

* Respect workflow JSON strict type aliases

* Remove unused console log contracts reference

* Address Copilot review feedback

* Address Copilot follow-up comments

* Synchronize ring buffer dropped count

* Address background processor strategy replay

* Fix diagnostics live feed regressions
2026-05-31 09:40:02 +02:00
Sipke Schoorstra 842cf7c162
[codex] Fix console log metadata and type resolution (#7542)
* Avoid null endpoint DTO metadata in tests

* Enforce console logs hub read permission

* Remove unused console logs hub import

* Support mapped endpoint metadata in auth tests

* Reduce console log capture throughput impact

* Address Copilot console logs review

* Refactor task scheduling to support tenant-level background work and enhance logging functionality.

* Introduce ConsoleStreamHook for stdout/stderr tee and enhance logging validation. Adjust test cases and startup warnings for distributed lock provider usage.

* Refactor console logging pipeline with capture optimization and new ConsoleLogsHost; update tests accordingly.

* Add Ansi SGR parser for console logs and associated unit tests

* Remove ANSI color renderings and parsers; integrate ConsoleLogScopeAccessor for improved logging context with workflow instance ID support.

* Address console logs code quality feedback

* Address PR review feedback

* Preserve console logs extension points

* Stabilize console logs host lifecycle

* Address final automated review comments

* Tighten console log capture shutdown

* Address console log review feedback

* Address follow-up review feedback

* Cover final review feedback

* Avoid recursive console provider initialization

* Guard console host lease shutdown

* Preserve console log scope and provider lifetime

* Correlate console log scope fallback

* Tighten console scope correlation

* Expose host services during provider construction

* Redact ANSI-normalized console lines

* Add OpenTelemetry diagnostics backend foundation

* Add OTLP HTTP ingestion parsing

* Document OpenTelemetry diagnostics setup

* Enforce OpenTelemetry hub permissions

* Remove `ConsoleCaptureTee` and related services and tests

* Add OpenTelemetry HTTP ingestion integration test

* Use pipeline contributors for console log context

* Update CShells package versions to 0.0.24-preview.132

* Add OpenTelemetry ingestion security tests

* Add OpenTelemetry API authorization tests

* Filter live console logs by workflow instance

* Add OpenTelemetry hub tests

* Add OpenTelemetry gRPC metadata hook

* Assert OpenTelemetry workflow tags survive ingestion

* Mark OpenTelemetry core build verified

* Enhance console logging with activity execution metadata and extend test coverage.

* Address console logs stream consumption comment

* Wire OpenTelemetry diagnostics into core sample

* Address Core diagnostics review feedback

* Address Core Copilot follow-up feedback

* Add OpenTelemetry metric instrument names

* Address Core Copilot provider feedback

* Address Core Copilot diagnostics follow-up

* Address Core Copilot live feed feedback

* Address Core Copilot store feedback

* Integrate OpenTelemetry for logging, tracing, and metrics in ModularServer and update launch settings and docker-compose configuration.

* Refactor to replace `ConsoleLogStream.Core` with `ConsoleLogStreaming.Core` across codebase and update `ConsoleStreamHook` installation.

* Add diagnostics OpenTelemetry backend

* Fix OpenTelemetry live hub subscription

* Fix modular OpenTelemetry exporter endpoints

* Add CShells logging configuration in appsettings.json

* Remove obsolete unit tests and helper classes

* Restore default activity exception handling

* Simplify type serialization and alias management

This commit refactors the internal type serialization and alias management system to reduce boilerplate, improve robustness, and simplify the developer experience:

-   Removed numerous explicit `ExpressionOptions` type alias registrations across various modules.
-   Updated `TypeJsonConverter` and polymorphic serialization to reliably handle types using assembly-qualified names when a short alias is not explicitly registered.
-   Streamlined `ExcludeFromHashConverter` to strictly adhere to `ExcludeFromHashAttribute` for hash calculations, removing complex `JsonIgnoreCondition` logic.
-   Eliminated several helper classes (`WorkflowJsonTypeResolver`, `WorkflowTypeValidator`, `IWorkflowTypeRegistry`, `WorkflowFactoryDictionary`, `JavaScriptExceptionTypeAliasRegistrar`, `WorkflowRuntimeTypeAliasRegistrar`) and their associated unit tests, simplifying the codebase.

Additionally, this commit introduces a comprehensive markdown document (`product-website-feature-source.md`) outlining Elsa's core features, Studio capabilities, extension ecosystem, and architectural selling points, intended as source material for the product website.

* Refine type serialization for improved robustness and alias handling

This commit further enhances the type serialization and deserialization mechanisms:

*   Centralizes type resolution and alias management through `IWellKnownTypeRegistry` and `WorkflowJsonTypeResolver`.
*   Prioritizes registered type aliases when serializing type metadata in `PolymorphicObjectConverter`, resulting in more concise JSON output.
*   Enhances deserialization in `PolymorphicObjectConverter` and `VariableMapper` to gracefully handle unknown or non-instantiable types, providing fallbacks and logging warnings.
*   Simplifies `TypeJsonConverter` by delegating complex type resolution logic to the `WorkflowJsonTypeResolver`.
*   Adds `JsonArray` to the well-known type aliases for direct recognition.

* Fix console logs packaging and workflow type resolution

* Fix console log metadata and type resolution

* Address Copilot review feedback

* Enhance type resolution, improve console log handling, and update tests

- Streamlined `WorkflowDictionaryExtensions` for better workflow registration validation.
- Refined `ConsoleLogsAuthorizationTests` with the new `SetJsonRequest` helper to improve test requests handling.
- Updated `OrderDefinition` to ignore JSON serialization for `KeySelector`.
- Enhanced `WorkflowRuntimeFeature` for improved workflow registration and type alias configuration.
- Added tests to ensure `ConsoleLogProvider` metadata filtration in various scenarios.
- Improved type serialization logic in `WorkflowJsonTypeResolver`.
- Updated README to fix references related to diagnostics.
- Optimized `ExcludeFromHashConverter` for property serialization conditions.
- Modified `TriggerIndexer` for streamlined trigger management.
- Tested payload checks in `PublishEventTests`.
- Adjusted `Endpoint` in `ConsoleLogs` for automatic JSON request handling.
- Ensured registration of workflow type aliases in `WorkflowsFeature`.

* Restore CLR workflow registration compatibility

* Align JSON island serialization fixtures

* Add Console Logs Services and Enhance Endpoint Handling

- Introduced `ActivityExecutionsEndpointTests` to validate route exposure.
- Added `ConsoleLogCaptureHostedService` for console log streaming.
- Implemented `ConsoleStreamJsonConverter` for JSON conversion of console streams.
- Developed `ElsaConsoleLogRecentBuffer` to handle recent log buffering.
- Updated `ConsoleLogsAuthorizationTests` with new test cases for stream filter mapping.
- Consolidated console log provider dependencies and registration, including recent buffering.
- Enhanced `ElsaConsoleLogProvider` to use recent buffer for filtering.
- Adjusted `Program.cs` for streamlined logging service setup.

* Enhance type resolution and test coverage; streamline console log integration

- Added `ConsoleStreamHook` for streamlined log streaming.
- Updated `WorkflowJsonTypeResolverTests` to improve type resolution and test new scenarios.
- Simplified type resolution by removing trusted assembly checks.

* Fix CI smoke and package restore failures

* Fix Docker smoke image project paths

* Fix Docker Python runtime packages

* Fix Docker CA smoke teardown

* Refresh Elsa roadmap

* Implement background processors and mediation coordination

- Added `BackgroundCommandProcessor`, `BackgroundJobProcessor`, and `BackgroundNotificationProcessor` classes for handling commands, jobs, and notifications, respectively.
- Introduced `MediatorBackgroundProcessingCoordinator` to coordinate the execution of all background processors.
- Implemented `MediatorBackgroundTask` for wrapping `MediatorBackgroundProcessingCoordinator` in `BackgroundTask`.
- Added unit tests for `MediatorBackgroundTask` to ensure proper start and stop behavior.
- Refactored `BackgroundCommandSenderHostedService` to utilize `BackgroundCommandProcessor`.
- Introduced 'elsa-roadmap-refresh' skill configuration for roadmap updates.

* Address workflow type resolution review feedback

* Address follow-up review feedback

* Restore recent console logs execute path

* Address Copilot follow-up review

* Decouple workflow JSON aliases from expressions

* Fix workflow management unit test setup

* Fix console logs recent endpoint handler shape

* Respect workflow JSON strict type aliases

* Remove unused console log contracts reference

* Address Copilot review feedback

* Address Copilot follow-up comments

* Synchronize ring buffer dropped count

* Address background processor strategy replay
2026-05-30 22:52:01 +02:00
Sipke Schoorstra 67ded31cba
Increase secrets unit coverage (#7545) 2026-05-30 08:12:12 +02:00
Sipke Schoorstra 5245599131
Enhance console logging with improved context and lifecycle (#7536)
* Avoid null endpoint DTO metadata in tests

* Enforce console logs hub read permission

* Remove unused console logs hub import

* Support mapped endpoint metadata in auth tests

* Reduce console log capture throughput impact

* Address Copilot console logs review

* Refactor task scheduling to support tenant-level background work and enhance logging functionality.

* Introduce ConsoleStreamHook for stdout/stderr tee and enhance logging validation. Adjust test cases and startup warnings for distributed lock provider usage.

* Refactor console logging pipeline with capture optimization and new ConsoleLogsHost; update tests accordingly.

* Add Ansi SGR parser for console logs and associated unit tests

* Remove ANSI color renderings and parsers; integrate ConsoleLogScopeAccessor for improved logging context with workflow instance ID support.

* Address console logs code quality feedback

* Address PR review feedback

* Preserve console logs extension points

* Stabilize console logs host lifecycle

* Address final automated review comments

* Tighten console log capture shutdown

* Address console log review feedback

* Address follow-up review feedback

* Cover final review feedback

* Avoid recursive console provider initialization

* Guard console host lease shutdown

* Preserve console log scope and provider lifetime

* Correlate console log scope fallback

* Tighten console scope correlation

* Expose host services during provider construction

* Redact ANSI-normalized console lines

* Remove `ConsoleCaptureTee` and related services and tests

* Use pipeline contributors for console log context

* Update CShells package versions to 0.0.24-preview.132

* Filter live console logs by workflow instance

* Enhance console logging with activity execution metadata and extend test coverage.

* Address console logs stream consumption comment
2026-05-27 00:14:28 +02:00
Sipke Schoorstra 8e301d4e1e
[codex] Scope console logs to workflow instances (#7535)
* Avoid null endpoint DTO metadata in tests

* Enforce console logs hub read permission

* Remove unused console logs hub import

* Support mapped endpoint metadata in auth tests

* Reduce console log capture throughput impact

* Address Copilot console logs review

* Refactor task scheduling to support tenant-level background work and enhance logging functionality.

* Introduce ConsoleStreamHook for stdout/stderr tee and enhance logging validation. Adjust test cases and startup warnings for distributed lock provider usage.

* Refactor console logging pipeline with capture optimization and new ConsoleLogsHost; update tests accordingly.

* Add Ansi SGR parser for console logs and associated unit tests

* Remove ANSI color renderings and parsers; integrate ConsoleLogScopeAccessor for improved logging context with workflow instance ID support.

* Address console logs code quality feedback

* Address PR review feedback

* Preserve console logs extension points

* Stabilize console logs host lifecycle

* Address final automated review comments

* Tighten console log capture shutdown

* Address console log review feedback

* Address follow-up review feedback

* Cover final review feedback

* Avoid recursive console provider initialization

* Guard console host lease shutdown

* Preserve console log scope and provider lifetime

* Correlate console log scope fallback

* Tighten console scope correlation

* Expose host services during provider construction

* Redact ANSI-normalized console lines
2026-05-25 11:49:51 +02:00
Sipke Schoorstra 0d305d276e
[codex] Enforce console logs hub read permission (#7533)
* Avoid null endpoint DTO metadata in tests

* Enforce console logs hub read permission

* Remove unused console logs hub import

* Support mapped endpoint metadata in auth tests

* Reduce console log capture throughput impact

* Address Copilot console logs review
2026-05-23 15:57:27 +02:00