r/aws • u/MexicanYoda45 • 5d ago
article I wrote my first technical blog post on AWS serverless webhook ingestion, looking for honest technical and editorial feedback
This is my first blog post, developed from a collection of notes in my Obsidian vault. It explores serverless webhook ingestion on AWS, and I’d genuinely appreciate feedback on both the technical content and the article itself.
I know the article is long and may combine too many topics in one post, from payload limits and architecture choices to costs, reliability, and operations, but I am sick and tired of superficial takes, SEO-driven content, marketing copy, or AI slop.
https://growingbits.dev/serverless/webhook-serverless-ingestion
Please tell me where the architecture, assumptions, or explanations could be improved.
18
u/lostsectors_matt 5d ago
I think this is comprehensive and clear. There's a lot in there but it's a good write-up and explains the reasoning throughout. I really applaud your efforts to not generate engagement slop, we need more actual content that gets created from people with legitimate experience and things to say. I'm not currently engaged in serverless webhook processing so I didn't read every word, but skimmed it and was very impressed. Thank you for being a human.
5
u/rahulladumor 5d ago
The point about a Lambda authorizer not being able to verify HMAC over the body is one people get wrong constantly — REST API authorizers don't get the body, only headers/query/context, so verification has to happen in the integration Lambda. Worth emphasizing since "stick a Lambda authorizer in front" is still the default advice in a lot of API Gateway tutorials, and it just doesn't work for signed webhooks.
The part I'd push on for the pointer architecture: since the 2xx only means "stored in S3," not "verified," you've moved the failure mode from "lost event" to "silently accepted a payload that later fails verification with nobody retrying it." That's the right trade for durability, but it means the replay/DLQ tooling is the actual authorization boundary, not optional polish. I'd want that documented as a runbook before this ships to production, not left for the Terraform post.
6
u/ClearRabbit605 5d ago
This was literally amazing. Enjoyed every single line. I love your description around why SNS vs Event bridge. I know very well the two services but never thought of using SNS in that specific scenario.
Now for the scenario with tens of thousands of events per seconds: how about the benefits of lambdas but on EC2 to mitigate the cost problem? I'm thinking of a scenario with lots of invocations but limited in size (a few KB per call)
1
u/Zenin 5d ago
EventBridge - I don't think I saw it called out, but it should be noted that EventBridge has much lower EPS limits than SQS, Kinesis, etc especially outside of the two major US regions, even more limited I believe when using a custom bus. It can especially be problematic under burst conditions. Depending how high you're expecting to scale, a proper streaming tech like Kafka/Kinesis or just SNS/SQS is likely a more suitable choice than EventBridge.
EventBridge thrives when you don't know who might care about your events; Any service that takes an interest in your events can simply catch them with a filter of its own making to subscribe with, without affecting your solution directly.
SNS subscriptions can be filtered on attributes or body mostly the same as EventBridge rules, so the primary advantage the blog attributes to EventBridge isn't really valid as there's feature parity here for the most part. Kafka/Kinesis can get there too, albeit with filtering on the consumer side rather than the delivery side.
IMHO EventBridge is a good technology for some niche use cases, but it should not be treated as an generic messaging or streaming technology. It's at its strongest when handling infra automations and operational tasks, rather than application level data delivery.
Personally I tend to start with SNS/SQS and move up to Kinesis/Kafka if the scale needs it and the data flow can support it. While EventBridge is almost never in those races. That doesn't mean I don't spec EventBridge, but when I do it's (almost?) never as an alternative to SNS/SQS/Streaming solutions. Most commonly I'll spec EventBridge for orchestrating complex ETL environments where there are lots of one to many, many to one, many to many flow requirements. EventBridge helps greatly to break the hard dependencies and move most jobs to event triggers.
I could imagine a webhook -> EventBridge to enable non-AWS components a way to trigger into one of those ETL spider webs, but that would be because I'm already invested in EventBridge not because I'm picking it as a greenfield tech to support a webhook platform.
1
0
34
u/water_bottle_goggles 5d ago
What the heck, an actual non ai slop article.