[codex] Fix console log metadata and type resolution (#7542)
* Avoid null endpoint DTO metadata in tests
* Enforce console logs hub read permission
* Remove unused console logs hub import
* Support mapped endpoint metadata in auth tests
* Reduce console log capture throughput impact
* Address Copilot console logs review
* Refactor task scheduling to support tenant-level background work and enhance logging functionality.
* Introduce ConsoleStreamHook for stdout/stderr tee and enhance logging validation. Adjust test cases and startup warnings for distributed lock provider usage.
* Refactor console logging pipeline with capture optimization and new ConsoleLogsHost; update tests accordingly.
* Add Ansi SGR parser for console logs and associated unit tests
* Remove ANSI color renderings and parsers; integrate ConsoleLogScopeAccessor for improved logging context with workflow instance ID support.
* Address console logs code quality feedback
* Address PR review feedback
* Preserve console logs extension points
* Stabilize console logs host lifecycle
* Address final automated review comments
* Tighten console log capture shutdown
* Address console log review feedback
* Address follow-up review feedback
* Cover final review feedback
* Avoid recursive console provider initialization
* Guard console host lease shutdown
* Preserve console log scope and provider lifetime
* Correlate console log scope fallback
* Tighten console scope correlation
* Expose host services during provider construction
* Redact ANSI-normalized console lines
* Add OpenTelemetry diagnostics backend foundation
* Add OTLP HTTP ingestion parsing
* Document OpenTelemetry diagnostics setup
* Enforce OpenTelemetry hub permissions
* Remove `ConsoleCaptureTee` and related services and tests
* Add OpenTelemetry HTTP ingestion integration test
* Use pipeline contributors for console log context
* Update CShells package versions to 0.0.24-preview.132
* Add OpenTelemetry ingestion security tests
* Add OpenTelemetry API authorization tests
* Filter live console logs by workflow instance
* Add OpenTelemetry hub tests
* Add OpenTelemetry gRPC metadata hook
* Assert OpenTelemetry workflow tags survive ingestion
* Mark OpenTelemetry core build verified
* Enhance console logging with activity execution metadata and extend test coverage.
* Address console logs stream consumption comment
* Wire OpenTelemetry diagnostics into core sample
* Address Core diagnostics review feedback
* Address Core Copilot follow-up feedback
* Add OpenTelemetry metric instrument names
* Address Core Copilot provider feedback
* Address Core Copilot diagnostics follow-up
* Address Core Copilot live feed feedback
* Address Core Copilot store feedback
* Integrate OpenTelemetry for logging, tracing, and metrics in ModularServer and update launch settings and docker-compose configuration.
* Refactor to replace `ConsoleLogStream.Core` with `ConsoleLogStreaming.Core` across codebase and update `ConsoleStreamHook` installation.
* Add diagnostics OpenTelemetry backend
* Fix OpenTelemetry live hub subscription
* Fix modular OpenTelemetry exporter endpoints
* Add CShells logging configuration in appsettings.json
* Remove obsolete unit tests and helper classes
* Restore default activity exception handling
* Simplify type serialization and alias management
This commit refactors the internal type serialization and alias management system to reduce boilerplate, improve robustness, and simplify the developer experience:
- Removed numerous explicit `ExpressionOptions` type alias registrations across various modules.
- Updated `TypeJsonConverter` and polymorphic serialization to reliably handle types using assembly-qualified names when a short alias is not explicitly registered.
- Streamlined `ExcludeFromHashConverter` to strictly adhere to `ExcludeFromHashAttribute` for hash calculations, removing complex `JsonIgnoreCondition` logic.
- Eliminated several helper classes (`WorkflowJsonTypeResolver`, `WorkflowTypeValidator`, `IWorkflowTypeRegistry`, `WorkflowFactoryDictionary`, `JavaScriptExceptionTypeAliasRegistrar`, `WorkflowRuntimeTypeAliasRegistrar`) and their associated unit tests, simplifying the codebase.
Additionally, this commit introduces a comprehensive markdown document (`product-website-feature-source.md`) outlining Elsa's core features, Studio capabilities, extension ecosystem, and architectural selling points, intended as source material for the product website.
* Refine type serialization for improved robustness and alias handling
This commit further enhances the type serialization and deserialization mechanisms:
* Centralizes type resolution and alias management through `IWellKnownTypeRegistry` and `WorkflowJsonTypeResolver`.
* Prioritizes registered type aliases when serializing type metadata in `PolymorphicObjectConverter`, resulting in more concise JSON output.
* Enhances deserialization in `PolymorphicObjectConverter` and `VariableMapper` to gracefully handle unknown or non-instantiable types, providing fallbacks and logging warnings.
* Simplifies `TypeJsonConverter` by delegating complex type resolution logic to the `WorkflowJsonTypeResolver`.
* Adds `JsonArray` to the well-known type aliases for direct recognition.
* Fix console logs packaging and workflow type resolution
* Fix console log metadata and type resolution
* Address Copilot review feedback
* Enhance type resolution, improve console log handling, and update tests
- Streamlined `WorkflowDictionaryExtensions` for better workflow registration validation.
- Refined `ConsoleLogsAuthorizationTests` with the new `SetJsonRequest` helper to improve test requests handling.
- Updated `OrderDefinition` to ignore JSON serialization for `KeySelector`.
- Enhanced `WorkflowRuntimeFeature` for improved workflow registration and type alias configuration.
- Added tests to ensure `ConsoleLogProvider` metadata filtration in various scenarios.
- Improved type serialization logic in `WorkflowJsonTypeResolver`.
- Updated README to fix references related to diagnostics.
- Optimized `ExcludeFromHashConverter` for property serialization conditions.
- Modified `TriggerIndexer` for streamlined trigger management.
- Tested payload checks in `PublishEventTests`.
- Adjusted `Endpoint` in `ConsoleLogs` for automatic JSON request handling.
- Ensured registration of workflow type aliases in `WorkflowsFeature`.
* Restore CLR workflow registration compatibility
* Align JSON island serialization fixtures
* Add Console Logs Services and Enhance Endpoint Handling
- Introduced `ActivityExecutionsEndpointTests` to validate route exposure.
- Added `ConsoleLogCaptureHostedService` for console log streaming.
- Implemented `ConsoleStreamJsonConverter` for JSON conversion of console streams.
- Developed `ElsaConsoleLogRecentBuffer` to handle recent log buffering.
- Updated `ConsoleLogsAuthorizationTests` with new test cases for stream filter mapping.
- Consolidated console log provider dependencies and registration, including recent buffering.
- Enhanced `ElsaConsoleLogProvider` to use recent buffer for filtering.
- Adjusted `Program.cs` for streamlined logging service setup.
* Enhance type resolution and test coverage; streamline console log integration
- Added `ConsoleStreamHook` for streamlined log streaming.
- Updated `WorkflowJsonTypeResolverTests` to improve type resolution and test new scenarios.
- Simplified type resolution by removing trusted assembly checks.
* Fix CI smoke and package restore failures
* Fix Docker smoke image project paths
* Fix Docker Python runtime packages
* Fix Docker CA smoke teardown
* Refresh Elsa roadmap
* Implement background processors and mediation coordination
- Added `BackgroundCommandProcessor`, `BackgroundJobProcessor`, and `BackgroundNotificationProcessor` classes for handling commands, jobs, and notifications, respectively.
- Introduced `MediatorBackgroundProcessingCoordinator` to coordinate the execution of all background processors.
- Implemented `MediatorBackgroundTask` for wrapping `MediatorBackgroundProcessingCoordinator` in `BackgroundTask`.
- Added unit tests for `MediatorBackgroundTask` to ensure proper start and stop behavior.
- Refactored `BackgroundCommandSenderHostedService` to utilize `BackgroundCommandProcessor`.
- Introduced 'elsa-roadmap-refresh' skill configuration for roadmap updates.
* Address workflow type resolution review feedback
* Address follow-up review feedback
* Restore recent console logs execute path
* Address Copilot follow-up review
* Decouple workflow JSON aliases from expressions
* Fix workflow management unit test setup
* Fix console logs recent endpoint handler shape
* Respect workflow JSON strict type aliases
* Remove unused console log contracts reference
* Address Copilot review feedback
* Address Copilot follow-up comments
* Synchronize ring buffer dropped count
* Address background processor strategy replay
2026-05-30 20:52:01 +00:00
|
|
|
# Elsa Diagnostics OpenTelemetry
|
|
|
|
|
|
|
|
|
|
Provides the OpenTelemetry diagnostics collector surface for Elsa hosts.
|
|
|
|
|
|
|
|
|
|
This module owns the Core-side OTEL contracts, HTTP/protobuf ingestion, bounded in-memory diagnostics store, feature registration, permissions, read APIs, live update hub, and collector metadata that Studio consumes.
|
|
|
|
|
|
|
|
|
|
## Scope
|
|
|
|
|
|
|
|
|
|
`Elsa.Diagnostics.OpenTelemetry` is collector/read-side diagnostics infrastructure. It does not create workflow spans, mutate `Activity.Current`, export telemetry to vendors, or provide durable OpenTelemetry persistence. Workflow telemetry production remains owned by existing `Elsa.Workflows` `ActivitySource` and `Meter` instrumentation.
|
|
|
|
|
|
|
|
|
|
The historical `Elsa.OpenTelemetry` module from `elsa-extensions` is intentionally not ported into this feature. It is producer-side workflow tracing middleware, duplicates current `Elsa.Workflows` instrumentation, and mutates `Activity.Current`; this module is collector/read-side diagnostics infrastructure.
|
|
|
|
|
|
|
|
|
|
## Routes
|
|
|
|
|
|
|
|
|
|
OTLP HTTP/protobuf ingestion is exposed under `OpenTelemetryDiagnosticsOptions.HttpEndpointPath`, which defaults to `/elsa/otlp/v1`:
|
|
|
|
|
|
|
|
|
|
- `POST /elsa/otlp/v1/traces`
|
|
|
|
|
- `POST /elsa/otlp/v1/metrics`
|
|
|
|
|
- `POST /elsa/otlp/v1/logs`
|
|
|
|
|
|
|
|
|
|
Diagnostics read APIs are exposed through Elsa API endpoints:
|
|
|
|
|
|
|
|
|
|
- `POST /diagnostics/opentelemetry/resources/search`
|
|
|
|
|
- `POST /diagnostics/opentelemetry/traces/search`
|
|
|
|
|
- `GET /diagnostics/opentelemetry/traces/{traceId}`
|
|
|
|
|
- `POST /diagnostics/opentelemetry/metrics/search`
|
|
|
|
|
- `POST /diagnostics/opentelemetry/logs/search`
|
|
|
|
|
- `GET /diagnostics/opentelemetry/storage`
|
|
|
|
|
- `GET /diagnostics/opentelemetry/collector-configuration`
|
|
|
|
|
|
|
|
|
|
Live updates are exposed through SignalR at `OpenTelemetryDiagnosticsOptions.HubRoute`, which defaults to `/elsa/hubs/diagnostics/opentelemetry`.
|
|
|
|
|
|
|
|
|
|
## Security
|
|
|
|
|
|
|
|
|
|
All diagnostics read APIs require the OpenTelemetry diagnostics read permission. OTLP ingestion allows unauthenticated loopback traffic by default for local development only when no API key is configured. When `OpenTelemetryDiagnosticsOptions.ApiKey` is set, every sender must provide the configured API key header, including loopback senders.
|
|
|
|
|
|
|
|
|
|
Collector configuration returns endpoint and required-header names only. It never returns secret header values.
|
|
|
|
|
|
|
|
|
|
## Storage
|
|
|
|
|
|
|
|
|
|
The default provider is bounded in-memory storage. Capacity options cover traces, spans, metric points, OTLP log records, and live subscriber queues. When a buffer exceeds capacity, the oldest item for that signal is dropped and diagnostics counters are incremented.
|