Flow Diagrams

Simplified flowcharts showing decision flows and process steps for each endpoint in the Web Search module. These diagrams illustrate the logical flow, decision points, and error handling paths for each operation.

Diagrams

1. flow-get-web-search-results.md - Web search results flow

   • Complete flow: Authentication → Search → Process by result-type → Create chunks → Return

   • Shows Firecrawl SaaS/Self-Hosted fallback logic

   • Demonstrates different processing paths based on result type

2. flow-get-web-search-urls.md - Web search URLs flow

   • Streamlined flow: Authentication → Search → Extract URLs → Return

   • Shows Firecrawl hybrid engine selection

   • Efficient URL extraction without content processing

3. flow-get-web-search-map.md - Website mapping flow

   • Complete flow: Authentication → Validate URL → Map website → Extract links → Return

   • Shows Firecrawl hybrid engine selection

   • Handles empty results and link conversion to MapLink objects

4. flow-get-web-page.md - Web page scraping flow

   • Complete flow: Authentication → Validate PDF → Scrape → Extract metadata → Return

   • Shows PDF URL validation and error handling

   • Demonstrates metadata extraction process

5. flow-get-web-page-snippets.md - Snippet extraction flow

   • Complete flow: Authentication → Scrape → Extract snippets via LLM → Create chunks → Return

   • Shows LLM-based snippet extraction process

   • Demonstrates chunk creation from extracted snippets

6. flow-get-web-page-keypoints.md - Keypoint extraction flow

   • Complete flow: Authentication → Scrape → Extract keypoints via LLM → Create chunks → Return

   • Shows LLM-based keypoint extraction process

   • Demonstrates structured keypoint chunk creation

Usage

These diagrams can be viewed:

• In any Markdown viewer that supports Mermaid (GitHub, GitLab, etc.)

• In VS Code with the Mermaid extension

• Online at https://mermaid.live/

• In documentation tools like MkDocs, Docusaurus, etc.

Updating Diagrams

When updating the codebase, please update the relevant diagrams to reflect architectural changes. The diagrams should be kept in sync with the implementation.