There is an edge case, if you use `forward/4` and use Plug.ErrorHandler,
then when an exception reaches the outer router, then Plug.send_resp
will be called, triggering `[:phoenix, :endpoint, :stop]`, and the span
will be gone by the time the outer router gets the exception. This
causes this telemetry handler to crash and be detached.
Sequence of events:
- [:phoenix, :endpoint, :start]
- [:phoenix, :router_dispatch, :exception] (inner router)
- [:phoenix, :endpoint, :stop]
- [:phoenix, :router_dispatch, :exception] (outer router) ** here there is no span, crashes
Initial approach follows Ecto instrumentation, recording spans for all
Redix `[:redix, :pipeline, :stop]` events.
The command sanitization is inspired-by and adapted from [Java
instrumentation][1], from where I've also copied the actual commands and
what configuration should they follow.
Network attributes are tracked via a "sidecar" process, which keeps
track of connection attributes also via `telemetry`. This extra bit of
bookkeeping is needed as command events doesn't include that piece of
information, unfortunately.
[1]: b2bc41453b/instrumentation-api/src/main/java/io/opentelemetry/instrumentation/api/db/RedisCommandSanitizer.java
By default a new trace is automatically started when a job is processed
by monitoring these events:
* `[:oban, :job, :start]` — at the point a job is fetched from the database and will execute
* `[:oban, :job, :stop]` — after a job succeeds and the success is recorded in the database
* `[:oban, :job, :exception]` — after a job fails and the failure is recorded in the database
To also record a span when a job is created and to link traces together
`Oban.insert/2` has to be replaced by `OpentelemetryOban.insert/2`.
Before:
```elixir
%{id: 1, in_the: "business", of_doing: "business"}
|> MyApp.Business.new()
|> Oban.insert()
```
After:
```elixir
%{id: 1, in_the: "business", of_doing: "business"}
|> MyApp.Business.new()
|> OpentelemetryOban.insert()
```
Co-authored-by: Tristan Sloughter <t@crashfast.com>
* Set error status on error instead of just adding attribute
* Use Exception.message if error is an exception
Co-authored-by: Bryan Naegele <bryannaegele@users.noreply.github.com>
As it stands, when source is nil (which, for example, can happen if there is a
call to `Repo.transaction`), the name of the span ends in an odd ':'. This
removes that ':'.
Co-authored-by: Bryan Naegele <bryannaegele@users.noreply.github.com>
* Update otel-phoenix deps
* CI not triggering
* Remove deprecated plug conn property
* Retry shared matrix with 1.11 include fix
* Try reading the file again
* Still can't get file reading right
* Update source links
Derek Kraan
Ho-Yon Mak
Bryan Naegele
Andrew Rosa
Aaron Renner
matthijsqd
Indrek Juhkam
Timmo Verlaan
Seth Archer Brown
Norberto Lopes
Manu
Marco "Efesto" Polita