Log formats & field mapping

HeliosLogs is permissive: it accepts the JSON shapes that common loggers already emit, and it parses several non-JSON text formats. This page documents exactly how incoming events are mapped onto HeliosLogs's schema-on-read model.

Universal-core fields and their aliases

Only three keys get special treatment; HeliosLogs recognizes common aliases for each (the first present wins). Everything else keeps its name and becomes a queryable dynamic field.

Core field	Recognized keys
timestamp	timestamp, ts, time, @timestamp, Timestamp, eventTime, datetime, @t, asctime
message	message, msg, Body, body, event, Msg, log.body, @m, @mt
source	source, log.source, _source

These cover zap, pino, logrus, bunyan, Docker, log4j2/logback (ECS / logstash encoder), Serilog (CLEF), python-json-logger, OpenTelemetry, and more. If a source isn't in the event, the ?source= query parameter is used as a fallback.

Timestamp parsing

timestamp accepts strings or numbers:

• Strings — ISO 8601 (2026-06-14T18:00:00Z, with offsets), space-separated ISO (2026-06-14 18:00:00, assumed UTC), Apache/CLF (10/Oct/2024:13:55:36 +0000), and Python comma-millis (2024-01-15 10:30:45,123).
• Numbers — the unit is inferred from magnitude: seconds (including fractional like 1779126000.123), milliseconds, microseconds, or nanoseconds.

If the timestamp is missing or unparseable, the event is routed to the current day rather than rejected.

Nested objects → dotted paths

Nested JSON is flattened into dotted paths at ingest, so you query nested data the same way as flat data:

{ "message": "upstream failed",
  "error": { "type": "Timeout", "code": 504, "detail": { "ms": 5000 } } }

becomes the fields error.type, error.code, and error.detail.ms, queryable as:

error.type:Timeout  error.code:504  error.detail.ms:5000

Objects are flattened up to 16 levels deep; anything deeper is stored as one searchable string. Field names are matched case-insensitively.

Arrays → multi-valued fields

Array elements are shredded under the same path, making the field multi-valued:

{ "items": [ { "sku": "A1", "qty": 1 }, { "sku": "B2", "qty": 2 } ] }

produces items.sku and items.qty with two values each. An equality query like items.qty:1 matches if any element qualifies. (Range and aggregation use the first element.) Up to 1000 array elements are shredded per array; the rest remain in the full raw event.

OpenTelemetry containers

OTel events nest attributes under Attributes / Resource containers. HeliosLogs flattens these one level into the top-level namespace before mapping (the lowercase attributes / resource are handled too):

{ "Timestamp": "2026-06-14T19:01:00Z", "Body": "upstream failed",
  "Attributes": { "service.name": "payment", "http.status_code": 502 },
  "Resource": { "host.name": "payment-99" } }

→ service.name:payment, http.status_code:502, host.name:payment-99, with Body mapped to message. (A top-level key always wins over a same-named container key.)

Index templating

The ingest index parameter may contain placeholders that resolve per event against the (pre-flatten) JSON, so one stream can fan out to many indexes:

# each event routes by its service field: app-checkout, app-payment, …
curl -X POST 'http://localhost:7300/api/ingest?index=app-{{service}}' --data-binary @events.ndjson

# nested path
curl -X POST 'http://localhost:7300/api/ingest?index=logs-{{kubernetes.namespace}}' --data-binary @events.ndjson

Dotted paths address nested fields; a missing value becomes unknown. Resolved index names are lowercased and sanitized to [a-z0-9_-] (other characters fold to -) and truncated to 64 characters.

Non-JSON text formats

The /api/ingest/raw endpoint and pull sources can parse text formats directly, selected with format= (or auto-detected):

Format	What it parses
ndjson	One JSON object per line.
json	A single JSON value: an array of objects, or one object.
text	Plain lines — each becomes { "message": line }.
syslog	RFC 3164 / RFC 5424 (see Syslog).
logfmt	key=value key="quoted value" pairs (e.g. logrus text formatter).
csv / tsv	Delimited columns with a header row (delimiter sniffed).
grok	Field extraction via a grok preset, %{...} pattern, or named-capture regex.
cef	ArcSight CEF security-appliance events.
w3c	W3C extended logs (#Fields: header) — CloudFront / IIS.
auto	Sniffs the content and picks one of the above (falls back to text).

Unparseable input always degrades to a message field rather than being dropped, and parse failures are counted in the response's errors.

Everything else

Any other key lands in the dynamic columns under its original name and is immediately queryable as key:value — no declaration, no migration. See Fields & discovery to explore what's present in your data.