S3 Log Processing Architecture

Security First

Avoid static long-term credentials (IAM Access Keys) wherever possible in favor of short-lived tokens.

Restrict permissions strictly to s3:GetObject and s3:ListBucket on the designated log prefixes.

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "AllowS3LogReadAccess",
      "Effect": "Allow",
      "Action": [
        "s3:GetObject",
        "s3:ListBucket"
      ],
      "Resource": [
        "arn:aws:s3:::your-log-bucket-name",
        "arn:aws:s3:::your-log-bucket-name/*"
      ]
    }
  ]
}

Non-AWS / On-Premises

IAM Instance Profiles

If the VM is running inside AWS as an EC2 instance, do not use access keys. Attach an IAM Role directly to the VM instance profile.

Mechanism: AWS Instance Metadata Service (IMDSv2) automatically issues short-lived security credentials.
Rotation: Managed automatically by AWS without application downtime.
Code Integration: AWS SDKs pick up the role credentials transparently.

# Test access directly from the VM using the instance role
aws s3 ls s3://your-log-bucket-name/

1. IAM Roles Anywhere (Recommended)

Establishes trust between external servers and AWS IAM using PKI and X.509 digital certificates.

Pros: Issues short-lived credentials without storing AWS static keys on the VM.
Requires: An existing internal Certificate Authority (CA) or AWS Private CA.

2. Static IAM Access Keys (Fallback)

If IAM Roles Anywhere cannot be implemented, use dedicated IAM User keys with strict guardrails:

Static keys must never be hardcoded in application source code. Store them in secure secret stores like HashiCorp Vault or environment variables.

Mandate key rotation every 90 days.
Enforce IP-based explicit deny conditions in the IAM Policy.

Get Started

Essentials

Releases

Case Studies

WCF Agent

Products

Updates

Overview

Prerequisites & Core Requirements

Security First

Least Privilege

Authentication Patterns

IAM Instance Profiles

1. IAM Roles Anywhere (Recommended)

2. Static IAM Access Keys (Fallback)

Ingestion Models

1. AWS OFA Log Adapter

​Overview

​Prerequisites & Core Requirements

Security First

Least Privilege

​Authentication Patterns

​IAM Instance Profiles

​1. IAM Roles Anywhere (Recommended)

​2. Static IAM Access Keys (Fallback)

​Ingestion Models

​1. AWS OFA Log Adapter

Overview

Prerequisites & Core Requirements

Authentication Patterns

IAM Instance Profiles

1. IAM Roles Anywhere (Recommended)

2. Static IAM Access Keys (Fallback)

Ingestion Models

1. AWS OFA Log Adapter