> ## Documentation Index
> Fetch the complete documentation index at: https://docs.onefirewall.com/llms.txt
> Use this file to discover all available pages before exploring further.

# S3 Log Processing Architecture

> Authentication methods and ingestion patterns for processing Amazon S3 logs

## Overview

This document outlines the architectural patterns and security requirements for ingesting and processing log files stored in Amazon S3 via the **onefirewall** Virtual Machine.

***

## Prerequisites & Core Requirements

<CardGroup cols={2}>
  <Card title="Security First" icon="shield-check">
    Avoid static long-term credentials (IAM Access Keys) wherever possible in favor of short-lived tokens.
  </Card>

  <Card title="Least Privilege" icon="key">
    Restrict permissions strictly to `s3:GetObject` and `s3:ListBucket` on the designated log prefixes.
  </Card>
</CardGroup>

```json theme={null}
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "AllowS3LogReadAccess",
      "Effect": "Allow",
      "Action": [
        "s3:GetObject",
        "s3:ListBucket"
      ],
      "Resource": [
        "arn:aws:s3:::your-log-bucket-name",
        "arn:aws:s3:::your-log-bucket-name/*"
      ]
    }
  ]
}
```

***

## Authentication Patterns

The authentication model depends on whether the **onefirewall** VM is hosted natively inside AWS or in an external environment.

<Tabs>
  <Tab title="AWS Native (EC2)">
    ### IAM Instance Profiles

    If the VM is running inside AWS as an EC2 instance, **do not** use access keys. Attach an **IAM Role** directly to the VM instance profile.

    * **Mechanism:** AWS Instance Metadata Service (IMDSv2) automatically issues short-lived security credentials.
    * **Rotation:** Managed automatically by AWS without application downtime.
    * **Code Integration:** AWS SDKs pick up the role credentials transparently.

    ```bash theme={null}
    # Test access directly from the VM using the instance role
    aws s3 ls s3://your-log-bucket-name/
    ```
  </Tab>

  <Tab title="Non-AWS / On-Premises">
    ### 1. IAM Roles Anywhere (Recommended)

    Establishes trust between external servers and AWS IAM using PKI and X.509 digital certificates.

    * **Pros:** Issues short-lived credentials without storing AWS static keys on the VM.
    * **Requires:** An existing internal Certificate Authority (CA) or AWS Private CA.

    ### 2. Static IAM Access Keys (Fallback)

    If IAM Roles Anywhere cannot be implemented, use dedicated IAM User keys with strict guardrails:

    <Warning>
      Static keys must never be hardcoded in application source code. Store them in secure secret stores like HashiCorp Vault or environment variables.
    </Warning>

    * Mandate key rotation **every 90 days**.
    * Enforce IP-based explicit deny conditions in the IAM Policy.
  </Tab>
</Tabs>

***

## Ingestion Models

**scheduled polling** based on latency requirements.

### 1. AWS OFA Log Adapter

Scanning the bucket continuously to new incoming log files and processing with AWS OFA Log Adapter

```mermaid theme={null}
graph LR
    A[S3 Log Creation] -->|s3:GetObject| B(AWS OFA Log Adapter)
    B -->|ofa traffic_api| C[OneFirewall Alliance OnPrem/Whitelabel]
```