Decree.Event.Handler behaviour (decree v0.1.0)

Copy Markdown View Source

Defines the behaviour an event handler must implement and provides a convenience macro that implements the behaviour, allowing you to handle only the events you are interested in processing.

You should start your event handlers using a Supervisor to ensure they are restarted on error.

Example

defmodule ExampleHandler do
  use Decree.Event.Handler,
    application: ExampleApp,
    name: "ExampleHandler"

  def handle(%AnEvent{..}, _metadata) do
    # ... process the event
    :ok
  end
end

Start your event handler process (or use a Supervisor):

{:ok, _handler} = ExampleHandler.start_link()

Event handler name

The name you specify is used when subscribing to the event store. You must use a unique name for each event handler you start. Also, you should not change the name once the handler has been deployed. A new subscription will be created if you change the name and the event handler will receive already handled events.

You can use the module name of your event handler using the __MODULE__ special form:

defmodule ExampleHandler do
  use Decree.Event.Handler,
    application: ExampleApp,
    name: __MODULE__
end

Subscription options

You can choose to start the event handler's event store subscription from :origin, :current position, or an exact event number using the start_from option. The default is to use the origin so your handler will receive all events.

Use the :current position when you don't want newly created event handlers to go through all previous events. An example would be adding an event handler to send transactional emails to an already deployed system containing many historical events.

The start_from option only applies when the subscription is initially created, the first time the handler starts. Whenever the handler restarts the subscription will resume from the next event after the last successfully processed event. Restarting an event handler does not restart its subscription.

Example

Set the start_from option (:origin, :current, or an explicit event number) when using Decree.Event.Handler:

defmodule ExampleHandler do
  use Decree.Event.Handler,
    application: ExampleApp,
    name: "ExampleHandler",
    start_from: :origin
end

You can optionally override :start_from by passing it as option when starting your handler:

{:ok, _handler} = ExampleHandler.start_link(start_from: :current)

Subscribing to an individual stream

By default event handlers will subscribe to all events appended to any stream. Provide a subscribe_to option to subscribe to a single stream.

defmodule ExampleHandler do
  use Decree.Event.Handler,
    application: ExampleApp,
    name: __MODULE__,
    subscribe_to: "stream1234"
end

This will ensure the handler only receives events appended to that stream.

If the stream is hard deleted and an append to its uuid creates it again, from version 1, the handler handles the recreated events: its subscription follows the stream and tells it ({:subscription_stream_recreated, subscription}), and it forgets the last event number it saw, so its already-seen check does not skip them. Events of the deleted stream still waiting in its batch buffer are dropped unhandled, as on a lost lock (sparkle-space/sparklespace#8386).

Runtime event handler configuration

Runtime options can be provided to the event handler's start_link/1 function or its child spec. The init/1 callback function can also be used to define runtime configuration.

Example

Provide runtime configuration to start_link/1:

{:ok, _pid} =
  ExampleHandler.start_link(
    application: ExampleApp,
    name: "ExampleHandler"
  )

Or when supervised:

Supervisor.start_link([
  {ExampleHandler, application: ExampleApp, name: "ExampleHandler"}
], strategy: :one_for_one)

Event handler state

An event handler can define and update state which is held in the GenServer process memory. It is passed to the handle/2 (or handle_batch/1) function as part of the metadata using the :state key. The state is transient and will be lost whenever the process restarts.

Initial state can be set in the init/1 callback function by adding a :state key to the config. It can also be provided when starting the handler process:

ExampleHandler.start_link(state: initial_state)

Or when supervised:

Supervisor.start_link([
  {ExampleHandler, state: initial_state}
], strategy: :one_for_one)

State can be updated by returning {:ok, new_state} from any handle/2 (or handle_batch/2) function. Returning an :ok reply will keep the state unchanged.

Handler state is also included in the Decree.Event.FailureContext struct passed to the error/3 callback function.

Example

defmodule StatefulEventHandler do
  use Decree.Event.Handler,
    application: ExampleApp,
    name: __MODULE__

  def init(config) do
    config = Keyword.put_new(config, :state, %{})

    {:ok, config}
  end

  def handle(event, metadata) do
    %{state: state} = metadata

    new_state = mutate_state(state)

    {:ok, new_state}
  end
end

Concurrency

An event handler may be configured to start multiple processes to handle the events concurrently. By default one process will be started, processing events one at a time in order. The :concurrency option determines how many event handler processes are started. It must be a positive integer.

Note with concurrent processing events will likely be processed out of order. If you need to enforce an order, such as per stream or by using a field from an event, you can define a partition_by/2 callback function in the event handler module. The function will receive each event and its metadata and must return a consistent term indicating the event's partition. Events which return the same term are guaranteed to be processed in order by the same event handler instance. While events with different partitions may be processed concurrently by another instance. An attempt will be made to distribute events as evenly as possible to all running event handler instances.

Example

defmodule ConcurrentProcessingEventHandler do
  alias Decree.EventStore.RecordedEvent

  use Decree.Event.Handler,
    application: ExampleApp,
    name: __MODULE__,
    concurrency: 10

  def init(config) do
    # Fetch the index of this event handler instance (0..9 in this example)
    index = Keyword.fetch!(config, :index)

    {:ok, config}
  end

  def handle(event, metadata) do
    :ok
  end

  # Partition events by their stream
  def partition_by(event, metadata) do
    %{stream_id: stream_id} = metadata

    stream_id
  end
end

Batching

Event handlers can process events in batches rather than one at a time. This allows writing to a target system in a single transaction and acknowledging all events at once, reducing overhead.

To enable batching, set the batch_size option and implement handle_batch/1 instead of handle/2. The callback receives a list of {event, metadata} tuples. On returning :ok, all events in the batch are acknowledged. There is no partial acknowledgement.

Batching and concurrency are not supported together; setting both will raise a compilation error.

batch_size

The batch_size option controls the Decree.Store subscription's in-flight buffer — how many events the subscription can deliver without waiting for acknowledgement. It is not an accumulation mechanism at the handler level. The handler processes each delivery from the subscription immediately.

During live operation, when events arrive one at a time, handle_batch/1 is called with a single event — regardless of the configured batch_size. Larger batches form naturally only when the subscription has multiple events queued, such as during catch-up replay, back-pressure recovery, or bulk appends.

When batch_timeout is configured, batch_size also serves as the maximum size of the handler's internal buffer — once the buffer reaches batch_size events, it is flushed immediately regardless of the timeout.

batch_timeout

The batch_timeout option adds time-based buffering at the handler level. When set, events are accumulated in the handler process and flushed when either condition is met first:

  • batch_size events have accumulated, OR
  • batch_timeout milliseconds have elapsed since the first buffered event

This enables use cases such as accumulating changes before writing them in one go to a projection, batching bulk database inserts, or reducing per-event overhead for side effects — even when events arrive individually during steady-state operation.

batch_timeout accepts a positive integer (milliseconds) or :infinity (the default — no buffering, backwards compatible with previous behaviour). Requires batch_size to be configured.

Example

defmodule OrderProjector do
  use Decree.Event.Handler,
    application: ExampleApp,
    name: "OrderProjector",
    batch_size: 50,
    batch_timeout: 100

  def handle_batch(events) do
    # Receives up to 50 events, or whatever accumulated within 100ms
    :ok
  end
end

Telemetry

Batch telemetry events include a flush_reason field:

  • :size — batch reached batch_size
  • :timeout — batch_timeout elapsed before the batch filled
  • :immediate — no batch_timeout configured; events processed as delivered

Dynamic application

An event handler's application can be provided as an option to start_link/1. This can be used to start the same handler multiple times, each using a separate Decree application and event store.

Example

Start an event handler process for each tenant in a multi-tenanted app, guaranteeing that the data and processing remains isolated between tenants.

for tenant <- [:tenant1, :tenant2, :tenant3] do
  {:ok, _app} = MyApp.Application.start_link(name: tenant)
  {:ok, _handler} = ExampleHandler.start_link(application: tenant)
end

Typically you would start the event handlers using a supervisor:

children =
  for tenant <- [:tenant1, :tenant2, :tenant3] do
    {ExampleHandler, application: tenant}
  end

Supervisor.start_link(children, strategy: :one_for_one)

The above example requires three named Decree applications to have already been started.

Personal fields

An event can declare personal fields (Decree.PersonalFields), which are stored encrypted. A handler receives them in plaintext for each event it names in a head of handle/2, and Decree.Projections.Ecto's project names one. The set is derived from the heads at compile time and there is no list to keep: __personal_fields__/0 answers it. Every other event reaches the handler's catch-all as it is stored, so one lost key does not stall a handler over an event it ignores.

A head is read when its first argument is a struct, descending through =. A trailing clause ignoring both arguments, with _-prefixed variables its body does not read and no guard, is the one catch-all: an underscored argument the body reads does not count as ignored. Any other head can't be read: a bound first argument, %module{}, a map, an ignored event beside a bound or matched metadata argument, a guarded or non-trailing catch-all. So can't handle_batch/1, which gets no decrypt.

A module with such a head names nothing and sees stored values. If it reads no personal field it says so, and is then never decrypted for:

use Decree.Event.Handler,
  application: ExampleApp,
  name: __MODULE__,
  personal_fields: :opaque

:opaque is the only value :personal_fields takes, and anything else is a CompileError.

Every handler, :opaque or not, also answers __handle_heads__/0: each handle/2 clause it defines, in order and a use-injected one included, as {line, event_pattern, guards}, the expanded first argument and its list of guards ([] without one). Decree's own catch-all is not in it, and a handle_batch/1 handler answers []. It is for a tool that checks what a handler's heads match, so it reads the compiled heads rather than the source.

Every module that uses the handler, and so every Decree.Projections.Ecto projector, also persists the attribute decree_event_handler: [true], read back as module.module_info(:attributes)[:decree_event_handler]. It lets such a tool tell decree's expansion from a module that only declares @behaviour Decree.Event.Handler and writes its own __handle_heads__/0. It marks the use, and proves nothing more: a module that registers and sets the persisted attribute by hand forges it, which is deliberate tampering, as below.

The export is checked after compile: the module's final handle/2 clauses must be the heads __handle_heads__/0 answers, followed by decree's catch-all. A CompileError names the module when they are not, as when a module defines __handle_heads__/0 itself, or when another use defines or overrides handle/2 in a @before_compile registered after decree's. Put such a use before use Decree.Event.Handler. The check covers what macros and compile hooks do, not deliberate tampering such as removing decree's @after_compile.

A head that can't be read and isn't :opaque is a CompileError naming the module and the head's line.

When a key can't be read

{:error, :key_unknown}, from the decrypt or from handle/2, never reaches the handler's own error/3, whose answer could skip the event. It goes to the application's on_event_handler_error. nil, :stop and :backoff stop the handler, and a stop never acks. A module's error/3 decides, and a :skip from it is turned into a stop, so nothing can skip an event at this seam. The event stays unacked until the key can be read.

When a command meets an unknown event type

During a rolling deploy a handler on an older release can be part-way through an event it knows when a newer release appends an event of a type the older one does not know to the stream its follow-up command targets. The aggregate refuses that command with {:error, :unknown_event_type} rather than fold past the event, and handle/2 or handle_batch/1 returns it.

{:error, :unknown_event_type} takes the same route as {:error, :key_unknown}, from either callback: it never reaches the handler's own error/3 and goes to the application's on_event_handler_error. nil, :stop and :backoff stop the handler, and a stop never acks. A module's error/3 decides, and a :skip from it is turned into a stop, so nothing can skip an event at this seam. The event, or the batch, stays unacked until a pod that knows the type serves the command.

From handle/2 it logs one warning naming the event's number, without the event. From handle_batch/1 it logs through the generic batch error line, which shows the batch's first and last events as stored, as {:error, :key_unknown} does.

What stays out of logs

A failure on an event decrypted for this handler is treated as carrying the value. A raise from handle/2 becomes a Decree.PersonalFields.RedactedError with a scrubbed stacktrace before anything sees it, error/3 included. Decree's own log lines, telemetry and stop reasons show an {:error, reason} return reduced by Decree.PersonalFields.RedactedError.reduce/1. An exit leaves reduced the same way, and a throw leaves as the exit {:nocatch, reduced}, so the handler stops: thrown on, a reduced {:noreply, _} would be read as the callback's return and keep the handler running on a state of :redacted. A handler must not put a declared value in an error reason, because decree reduces it in its own log and telemetry but hands it to error/3 as it stands. A handler must not keep a declared value in its state either, because decree hides it in its own output and hands it to the handler as it stands: once a decrypted event's handle/2 answers {:ok, state}, telemetry and format_status/1 show :redacted, and every later failure of that handler is treated as carrying the value.

The RecordedEvent is never rewritten, so a failed event logged, passed to error/3 or carried by telemetry holds ciphertext, as does partition_by/2's.

Telemetry

  • [:decree, :event, :handle, :start]

    • Description: Emitted when an event handler starts handling an event
    • Measurements: %{system_time: integer()}
    • Metadata: %{application: Decree.Application.t(), context: map(), handler_name: String.t(), handler_module: atom(), handler_state: map(), recorded_event: RecordedEvent.t()}
  • [:decree, :event, :handle, :stop]

    • Description: Emitted when an event handler stops handling an event
    • Measurements: %{duration: non_neg_integer()}
    • Metadata: %{:application => Decree.Application.t(), :context => map(), :handler_name => String.t(), :handler_module => atom(), :handler_state => map(), :recorded_event => RecordedEvent.t(), optional(:error) => any()}
  • [:decree, :event, :handle, :exception]

    • Description: Emitted when an event handler raises an exception
    • Measurements: %{duration: non_neg_integer()}
    • Metadata: %{application: Decree.Application.t(), context: map(), handler_name: String.t(), handler_module: atom(), handler_state: map(), recorded_event: RecordedEvent.t(), kind: :throw | :error | :exit, reason: any(), stacktrace: list()}
  • [:decree, :event, :batch, :start]

    • Description: Emitted when an event handler starts handling a batch of events
    • Measurements: %{system_time: integer()}
    • Metadata: %{application: Decree.Application.t(), context: map(), handler_name: String.t(), handler_module: atom(), handler_state: map(), first_event_id: binary(), last_event_id: binary(), event_count: integer(), flush_reason: :size | :timeout | :immediate}
  • [:decree, :event, :batch, :stop]

    • Description: Emitted when an event handler stops handling a batch of events
    • Measurements: %{duration: non_neg_integer()}
    • Metadata: %{application: Decree.Application.t(), context: map(), handler_name: String.t(), handler_module: atom(), handler_state: map(), first_event_id: binary(), last_event_id: binary(), event_count: integer(), recorded_event: RecordedEvent.t() | nil, flush_reason: :size | :timeout | :immediate, optional(:error) => any()}
  • [:decree, :event, :batch, :exception]

    • Description: Emitted when an event batch handler raises an exception
    • Measurements: %{duration: non_neg_integer()}
    • Metadata: %{application: Decree.Application.t(), context: map(), handler_name: String.t(), handler_module: atom(), handler_state: map(), recorded_event: => RecordedEvent.t() | nil, first_event_id: binary(), last_event_id: binary(), event_count: integer(), flush_reason: :size | :timeout | :immediate, kind: :throw | :error | :exit, reason: any(), optional(:stacktrace) => list()}
  • [:decree, :event, :retry, :scheduled]

    • Description: Emitted when an event handler schedules the retry of a failed event or batch for after a delay, once per scheduled retry
    • Measurements: %{delay: non_neg_integer(), system_time: integer()}
    • Metadata: %{application: Decree.Application.t(), context: map(), handler_name: String.t(), handler_module: atom(), handler_state: map() | :redacted, recorded_event: RecordedEvent.t() | nil, event_count: non_neg_integer()}

Summary

Callbacks

Optional initialisation callback function called when the handler starts.

Called before an event handler gets reset

Called when an event handle/2 callback returns an error.

Handle a domain event and its metadata.

Handle a batch of domain events and their metadata.

init() deprecated

Optional callback function called to configure the handler before it starts.

Determine which partition an event belongs to.

Functions

Returns a specification to start this module under a supervisor.

Types

domain_event()

@type domain_event() :: struct()

metadata()

@type metadata() :: map()

subscribe_from()

@type subscribe_from() :: :origin | :current | non_neg_integer()

Callbacks

after_start(handler_state)

@callback after_start(handler_state :: term()) ::
  :ok | {:ok, state :: map()} | {:stop, reason :: any()}

Optional initialisation callback function called when the handler starts.

Can be used to start any related processes when the event handler is started.

This callback function must return :ok, {:ok, state} to return new state, or {:stop, reason} to stop the handler process. Any other return value will terminate the event handler with an error.

Example

defmodule ExampleHandler do
  use Decree.Event.Handler,
    application: ExampleApp,
    name: "ExampleHandler"

  # Optional initialisation
  def after_start(handler_state) do
    new_handler_state = Map.put(handler_state, :foo, "bar")
    {:ok, new_handler_state}
  end

  def handle(%AnEvent{..}, _metadata) do
    # Process the event ...
    :ok
  end
end

before_reset()

(optional)
@callback before_reset() :: :ok

Called before an event handler gets reset

error(error, failed_event, failure_context)

(optional)
@callback error(
  error :: term(),
  failed_event :: domain_event() | [domain_event()] | nil,
  failure_context :: Decree.Event.FailureContext.t()
) ::
  {:retry, context :: map() | Decree.Event.FailureContext.t()}
  | {:retry, delay :: non_neg_integer(),
     context :: map() | Decree.Event.FailureContext.t()}
  | :skip
  | {:stop, reason :: term()}

Called when an event handle/2 callback returns an error.

The error/3 function allows you to control how event handling failures are handled. The function is passed the error returned by the event handler (e.g. {:error, :failure}), the event causing the error, and a context map containing state passed between retries.

Use pattern matching on the error and/or failed event to explicitly handle certain errors or events. Use the context map to track any transient state you need to access between retried failures.

{:error, :key_unknown} and {:error, :unknown_event_type} never reach it: the application's on_event_handler_error decides them, and nothing can skip them. See "When a key can't be read" and "When a command meets an unknown event type" in the module documentation.

You can return one of the following responses depending upon the error severity:

  • {:retry, context} - retry the failed event, provide a context map, or updated Decree.Event.FailureContext struct, containing any state to be passed to subsequent failures. This could be used to count the number of failures, stopping after too many.

  • {:retry, delay, context} - retry the failed event after the requested delay (in milliseconds). Context is a map or Decree.Event.FailureContext struct as described in {:retry, context} above.

    The handler does not sleep through the delay: it sets a timer and goes back to its mailbox, a delay of 0 included. While it waits it answers system messages (such as :sys.get_state/1), and it stops on a DOWN from its subscription or a non-normal exit signal, a supervisor's shutdown included, without waiting for the timer. Events delivered meanwhile are held. When the timer fires it retries the failed event (or batch), then handles the rest of that delivery, then the held deliveries, in the order they arrived. A lock loss or a reset during the wait drops the pending retry and every held event unhandled: none was acknowledged, so the subscription delivers them again from the last acknowledged event, and the failed event's next failure starts from a fresh context.

  • :skip - skip the failed event by acknowledging receipt. In batching mode, this will acknowledge and skip all events up to and including the failed event

  • {:stop, reason} - stop the event handler with the given reason.

The default behaviour if you don't provide an error/3 callback is to stop the event handler using the exact error reason returned from the handle/2 function. If the event handler is supervised using restart permanent or transient stopping on error will cause the handler to be restarted. It will likely crash again as it will reprocess the problematic event. This can lead to cascading failures going up the supervision tree.

Example error handling

defmodule ExampleHandler do
  use Decree.Event.Handler,
    application: ExampleApp,
    name: __MODULE__

  require Logger

  alias Decree.Event.FailureContext

  def handle(%AnEvent{}, _metadata) do
    # simulate event handling failure
    {:error, :failed}
  end

  def error({:error, :failed}, %AnEvent{} = event, %FailureContext{context: context}) do
    context = record_failure(context)

    case Map.get(context, :failures) do
      too_many when too_many >= 3 ->
        # skip bad event after third failure
        Logger.warning("Skipping bad event, too many failures: " <> inspect(event.__struct__))

        :skip

      _ ->
        # retry event, failure count is included in context map
        {:retry, context}
    end
  end

  defp record_failure(context) do
    Map.update(context, :failures, 1, fn failures -> failures + 1 end)
  end
end

handle(domain_event, metadata)

(optional)
@callback handle(domain_event(), metadata()) ::
  :ok
  | {:ok, new_state :: any()}
  | {:error, :already_seen_event}
  | {:error, reason :: any()}

Handle a domain event and its metadata.

Return :ok on success, {:error, :already_seen_event} to ack and skip the event, or {:error, reason} on failure.

handle_batch(list)

(optional)
@callback handle_batch([{domain_event(), metadata()}]) ::
  :ok | {:ok, new_state :: any()} | {:error, reason :: any()}

Handle a batch of domain events and their metadata.

Return :ok on success. All events in the batch will be acknowledged.

On error, you can either return {:error, reason} to indicate something with the whole batch went wrong

Note that this interface may change as more experience with use cases for batching is gained.

init()

(optional)
This callback is deprecated. Use the after_start/1 callback instead..
@callback init() :: :ok | {:stop, reason :: any()}

init(config)

(optional)
@callback init(config :: Keyword.t()) :: {:ok, Keyword.t()}

Optional callback function called to configure the handler before it starts.

It is passed the merged compile-time and runtime config, and must return the updated config as {:ok, config}.

Note this function is called before the event handler process is started and is not run from the handler's process. You cannot use self() to access the handler's PID.

Example

The init/1 function is used to define the handler's application and name based upon a value provided at runtime:

defmodule ExampleHandler do
  use Decree.Event.Handler

  def init(config) do
    {tenant, config} = Keyword.pop!(config, :tenant)

    config =
      config
      |> Keyword.put(:application, Module.concat([ExampleApp, tenant]))
      |> Keyword.put(:name, Module.concat([__MODULE__, tenant]))

    {:ok, config}
  end
end

Usage:

{:ok, _pid} = ExampleHandler.start_link(tenant: :tenant1)

partition_by(domain_event, metadata)

(optional)
@callback partition_by(domain_event(), metadata()) :: any()

Determine which partition an event belongs to.

Only applicable when an event handler has been configured with more than one instance via the :concurrency option.

Functions

child_spec(init_arg)

Returns a specification to start this module under a supervisor.

See Supervisor.