CrispyWaffle.ElasticSearch

CrispyWaffle.ElasticSearch connects CrispyWaffle to Elasticsearch through the official Elastic.Clients.Elasticsearch client. It provides:

• A log provider that indexes every log message as a document and deletes old log documents once, shortly after startup.
• A small wrapper for reading and writing documents.
• Index helpers that derive index names from your types.

Use it when you want CrispyWaffle logs searchable in Elasticsearch or Kibana, or when you need simple document storage without writing client code yourself.

How it works

Connecting

ElasticConnector builds an ElasticsearchClient from an IConnection:

• The scheme is https when Port is 443, and http otherwise.
• Basic authentication is added only when Credentials isn't null and Credentials.Username isn't blank.
• The client's default index is the defaultIndexName constructor argument. When you don't pass one, it's logs-{EnvironmentHelper.ApplicationName}-{EnvironmentHelper.Version}.

Register the connector as a singleton. ElasticWrapper and ElasticSearchLogProvider take it in their constructors. The static Extensions class resolves it from ServiceLocator.

Log provider

ElasticSearchLogProvider is an ILogProvider. Each logging call that passes the level check builds a LogMessage and indexes it in the connector's default index. The LogMessage carries:

• The application name, date, host name and process ID.
• The local and remote IP addresses.
• The operation, user agent, thread ID and thread name.

Indexing calls are started but not awaited. Logging never blocks, and indexing errors aren't reported back to the caller.

The provider's level starts with no flags set, so call SetLevel after adding it.

Ten seconds after it's constructed, the provider runs a single delete-by-query against the default index. The query removes LogMessage documents whose Date is older than logRetentionDays days. This cleanup runs once per provider instance, not on a schedule.

AbortGarbageCollector() and Dispose() cancel the cleanup if it hasn't run yet.

Index names

Extensions.GetIndexName<T>() decides which index a type belongs to:

• If the type has an [IndexName] attribute, that name is used.
• Otherwise, the type name in lowercase is used (with spaces replaced by -).

ElasticWrapper's *WithResolver methods and the Extensions methods use this name. Types used with them must implement the marker interface IIndexable.

Getting started

Start Elasticsearch locally. The repository's docker-compose.yml runs elasticsearch:8.8.0 as a single node on port 9200 with security disabled.

using CrispyWaffle.Composition;
using CrispyWaffle.Configuration;
using CrispyWaffle.ElasticSearch.Log;
using CrispyWaffle.ElasticSearch.Utils.Communications;
using CrispyWaffle.Log;

var connection = new Connection { Host = "localhost", Port = 9200 };

var connector = new ElasticConnector(connection, "logs-my-app");
ServiceLocator.Register(connector);

// Index log messages and delete log documents older than 30 days.
LogConsumer
    .AddProvider(new ElasticSearchLogProvider(connector, 30))
    .SetLevel(LogLevel.Production);

LogConsumer.Info("Application started");

Reference

ElasticConnector

public sealed class ElasticConnector creates and holds the ElasticsearchClient. It has the [ConnectionName("ElasticSearch")] attribute.

ElasticSearchLogProvider

public class ElasticSearchLogProvider : ILogProvider, IDisposable indexes log messages in Elasticsearch.

ElasticWrapper

public sealed class ElasticWrapper has convenience methods for single-document reads and writes. The synchronous methods block on the asynchronous client calls with .Result or .Wait(). Prefer the *Async versions.

The id parameter is typed as object but is cast to Elastic.Clients.Elasticsearch.Id, so pass an Id instance such as new Id("42"). A plain string or number throws InvalidCastException.

Extensions

public static class Extensions (namespace CrispyWaffle.ElasticSearch.Helpers) has index-management helpers.

The extension methods use the ElasticConnector resolved from ServiceLocator the first time the class is used. Register the connector before that.

IIndexable

public interface IIndexable is a marker interface with no members. Implement it on document types that you use with GetIndexName<T>(), the *WithResolver methods and the Extensions methods.

IndexNameAttribute

public sealed class IndexNameAttribute : Attribute sets the index name for a class. Apply it to classes only.

Examples

Read and write a typed document

using CrispyWaffle.ElasticSearch.Helpers;
using CrispyWaffle.ElasticSearch.Utils.Communications;
using Elastic.Clients.Elasticsearch;

[IndexName("products")]
public class Product : IIndexable
{
    public string Id { get; set; }

    public string Name { get; set; }
}

var wrapper = new ElasticWrapper(connector);

await wrapper.SetDocumentWithResolverAsync(new Product { Id = "42", Name = "Waffle iron" });

var product = await wrapper.GetDocumentWithResolverAsync<Product>(new Id("42"));
var exists = await wrapper.IndexExistsAsync<Product>();

Remove old documents and add an alias

using CrispyWaffle.ElasticSearch.Helpers;

public class AuditEntry : IIndexable
{
    public DateTime Date { get; set; }

    public string Action { get; set; }
}

var deleted = await new AuditEntry().DeleteByQueryAsync(e => e.Date, "auditentry*", 90);

await new AuditEntry().AliasAsync("audit-current");

See also

• Logging
• CrispyWaffle.Configuration
• Service locator