Project goal
This sample case study explores how a document-search system could accept uploads, process them asynchronously, and make their contents searchable.
Architecture
A static frontend calls an API for a presigned upload URL. The browser uploads to S3. An event-driven queue separates upload traffic from document workers, which extract text and index it in OpenSearch.
Why these choices?
- A queue lets workers process documents at their own pace.
- A dead-letter queue keeps repeatedly failing messages available for investigation.
- Separate IAM roles limit what each component can access.
Verification to document
Upload a document, track its processing, search for a known phrase, and test how a worker handles a repeated event. Replace this example with your actual results and architecture diagram.
Processing flow (Mermaid sample)
flowchart LR
A[Browser] --> B[API Gateway]
B --> C[Presigned S3 upload]
C --> D[S3 event]
D --> E[SQS queue]
E --> F[Python worker]
F --> G[OpenSearch]
E -. Failed after retries .-> H[DLQ]