Data Setup Guide

This section provides a comprehensive guide for developers and clients on the significance and purpose of configuration data within the Smart Search environment.

Overview of Requirements

Before utilizing the Deep Search and Browse APIs, the following data must be configured in the Smart Search database:

• Prompts: Predefined prompts used specifically for the browse endpoint to guide user queries.

• Browse Profile: A specific profile for the browse endpoint that maps to the Swirl search provider.

• Deep Search Profile: A specific profile for the deep-search endpoint that maps to the Swirl search provider.

• Configuration Data: This includes keys, default codenames, and various other settings essential for API operation.

If you are setting up the Smart Search environment using the provided guide, the process will automatically pre-seed the database with the basic data needed to allow the project to run immediately. All data can be viewed in the admin page of Smart Search at https://search-api.gdplabs.id/admin/ using email credentials. The following sections explain the meaning and significance of each field in the configuration.

Prompt

Field Name	Description	Example
id	Auto-incremented ID (Primary Key)	2
codename	The codename for the prompt, usually used as the main identifier (Unique, String 100)	default_system_prompt
title	Title or description of the prompt (String 255)	System Prompt for /browse
value	Actual prompt content (Text)	You are a helpful assistant...
variables	Comma-separated list of variable names used in the prompt (String 1000)	query,context,result

Browse Profile

Field Name	Description	Example
id	Auto-incremented ID (Primary Key)	2
order	Order value for database sort (Integer)	2
codename	String-valued name to be used as identifier for browse profile (Unique, String 100)	general
title	Title of profile (String 255)	General (Firecrawl)
description	Short description of profile (String 255)	All URLs using Firecrawl Search Provider
provider	Provider identifier to be mapped to the profile (String 100)	firecrawl
hosts	Additional hosts to filter URLs, limiting which URLs can be browsed (Array of String 100)	["catapa.com", "gdplabs.id"]
system_prompt	Foreign key reference to prompt.id for system prompt. Must reference an existing prompt record	1
user_prompt	Foreign key reference to prompt.id for user prompt. Must reference an existing prompt record	2

Deep Search Profile

Field Name	Description	Example
id	Auto-incremented ID (Primary Key)	2
codename	String-valued name to be used as identifier for deep search profile (Unique, String 100)	hr
title	Title of profile (String 255)	HR Web Search
description	Short description of profile (String 255)	HR Profile for deep search
providers	Provider identifiers to be mapped to the profile (String 100, default: empty string)	firecrawl,google
additional_query	Hardcoded query to be appended to user query (String 1000)	site:kemnaker.go.id OR site:bkn.go.id
filters	Additional filters to apply to search queries (String 1000)	language:en
updated_time	Timestamp of last update (DateTime, auto-set to UTC)	2024-01-15 10:30:00
expand_query	Whether to expand the query using LLM (Boolean, default: False)	true
query_system_prompt	Foreign key reference to prompt.id for query expansion system prompt (Integer, Foreign Key)	3
query_user_prompt	Foreign key reference to prompt.id for query expansion user prompt (Integer, Foreign Key)	4

Configurations

Field Name	Description	Example
id	Auto-incremented ID (Primary Key)	2
name	String-valued name to be used as identifier for configuration (String 100)	LEARNINGS_MODEL
title	Short description of the configuration (String 255)	Learning Model Configuration
value	Actual value of the configuration (String 1000)	gpt4o-mini

Specifically for configurations, you need to create configuration value for the following:

• OPENAI_API_KEY: API key for OpenAI

• DEEP_SEARCH_PROFILE_DEFAULT: Default profile used for deep search if not provided in request

• LEARNINGS_MODEL: Learning model used for LLM

• GITHUB_AGENT_ID: GitHub Agent ID in AIP

• GOOGLE_MAIL_AGENT_ID: Google Mail Agent ID in AIP

• GOOGLE_CALENDAR_AGENT_ID: Google Calendar Agent ID in AIP

• GOOGLE_DRIVE_AGENT_ID: Google Drive Agent ID in AIP

• FAQ_DEFAULT_SYSTEM_PROMPT: FAQ Default System Prompt

• FAQ_DEFAULT_USER_PROMPT: FAQ Default User Prompt

• DS_PROFILE_SELECTOR_SYSTEM_PROMPT: Deep Search Profile Selector System Prompt

• DS_PROFILE_SELECTOR_USER_PROMPT: Deep Search Profile Selector User Prompt