> ## Documentation Index
> Fetch the complete documentation index at: https://docs.onefirewall.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Elasticsearch index settings for PoV

## Elasticsearch index configuration

To setup the PoV elasticsearch index we apply an Index Lifecycle Management (ILM) policy with rollover and automated deletion, gaining several benefit:

1. Automatic data growth management
   * With rollover (max\_age: 1d or max\_size: 50gb), you don’t need to manually monitor index size or age.
   * As soon as an index reaches the threshold, Elasticsearch creates a new one (poc\_traffic-000002, etc.) and automatically updates the alias poc\_traffic.

2. Better query and update performance
   * Oversized indices slow down searches and updates.
   * By splitting them regularly, shards remain smaller, keeping queries, aggregations, and writes efficient.

3. Automatic cleanup of old data
   * The delete phase (min\_age: 34d) removes indices older than 34 days.
   * No need for external jobs (cron, scripts) to enforce data retention → lower risk of wasting disk space.

4. Resource usage optimization
   * number\_of\_shards: 1 and number\_of\_replicas: 0 reduce overhead when high availability is not required.
   * index.translog.flush\_threshold\_size: 512mb and refresh\_interval: 30s optimize ingestion performance compared to immediate search.
   * Prevents the cluster from being overloaded with either too many small shards or oversized ones.

5. Easier management with index templates
   * With an index template (poc\_traffic\_template), each new rollover index automatically inherits the same settings.
   * No need to reapply configurations like refresh\_interval or max\_result\_window manually.

6. Elasticity and scalability
   * Ideal for time-series data (like logs or traffic data) that continuously grows.
   * The combination of alias + rollover + ILM is the recommended Elastic pattern for scalable data management.

```bash theme={null}
curl -XPUT "http://localhost:9200/_ilm/policy/poc_traffic_policy" -H "kbn-xsrf: reporting" -H "Content-Type: application/json" -d'
{
  "policy": {
    "phases": {
      "hot": {
        "actions": {
          "rollover": {
            "max_age": "1d",
            "max_size": "50gb"
          }
        }
      },
      "delete": {
        "min_age": "34d",
        "actions": {
          "delete": {}
        }
      }
    }
  }
}'
```

```bash theme={null}
curl -XPUT "http://localhost:9200/poc_traffic-000001" -H "kbn-xsrf: reporting" -H "Content-Type: application/json" -d'
{
  "aliases": {
    "poc_traffic": {
      "is_write_index": true
    }
  },
  "settings": {
    "number_of_shards": 1,
    "number_of_replicas": 0,
    "refresh_interval": "30s",
    "index.lifecycle.name": "poc_traffic_policy",
    "index.lifecycle.rollover_alias": "poc_traffic",
    "index.translog.flush_threshold_size": "512mb",
    "max_result_window": 100000
  }
}'
```

```bash theme={null}
curl -XPUT "http://localhost:9200/_index_template/poc_traffic_template" -H "kbn-xsrf: reporting" -H "Content-Type: application/json" -d'
{
  "index_patterns": ["poc_traffic-*"],
  "template": {
    "settings": {
      "number_of_shards": 1,
      "number_of_replicas": 0,
      "refresh_interval": "30s",
      "index.lifecycle.name": "poc_traffic_policy",
      "index.lifecycle.rollover_alias": "poc_traffic",
      "index.translog.flush_threshold_size": "512mb",
      "max_result_window": 100000
    }
  },
  "priority": 500
}'
```