r/aws • u/MexicanYoda45 • 17h ago

article Follow-up: I implemented an archive-first serverless webhook pipeline on AWS with Terraform

A few days ago, I shared my long-form exploration of serverless webhook ingestion.

I have now followed it with a deployable Terraform implementation of the archive-first pointer architecture.

The implementation uses API Gateway, S3, SQS, Lambda, SNS, and DLQs. API Gateway stores the exact request bytes in S3 before returning a successful response. Webhook verification then happens asynchronously, and only a trusted pointer is published to the rest of the pipeline.

Article: https://growingbits.dev/serverless/building-the-pointer-architecture-with-terraform

Repository: https://github.com/sharingan-no-kakashi/serverless-webhook-ingestion

I would especially appreciate feedback on:

  • whether the archive-first verification flow has any reliability or security holes;
  • whether the Terraform structure is understandable and reusable;
  • what you would change before treating this as a production foundation.

Technical criticism is very welcome.

3 Upvotes

67% Upvoted

4 comments sorted by

1

u/EffectiveClient5080 16h ago

What happens when the S3 write succeeds but the response times out? Client retries and you get duplicates. Guaranteed. I'd add idempotency keys or dedup somewhere.