 {"id":519641,"date":"2025-08-16T23:13:00","date_gmt":"2025-08-17T06:13:00","guid":{"rendered":"https:\/\/jorgep.com\/blog\/?p=519641"},"modified":"2025-12-29T11:01:17","modified_gmt":"2025-12-29T18:01:17","slug":"build-log-creating-a-custom-ai-research-agent","status":"publish","type":"post","link":"https:\/\/jorgep.com\/blog\/build-log-creating-a-custom-ai-research-agent\/","title":{"rendered":"Build Log: Creating a Custom AI Research Agent"},"content":{"rendered":"\n<h1 class=\"wp-block-heading\"><\/h1>\n\n\n\n<p><strong>Author:<\/strong> Jorge Pereira<\/p>\n\n\n\n<p><strong>Date:<\/strong> August 2025<\/p>\n\n\n\n<p><strong>Tech Stack:<\/strong> n8n (Docker), Ollama, Open WebUI, JavaScript\/HTML, Windows 11<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">The Vision<\/h2>\n\n\n\n<p>The goal was to build an autonomous &#8220;Deep Research&#8221; agent capable of performing real-time web searches and returning professional, high-quality Markdown reports. I wanted a tool that behaved like Perplexity but lived in my own home lab environment.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">1. The n8n Architecture: The Agent&#8217;s &#8220;Brain&#8221;<\/h2>\n\n\n\n<p>Before tackling the deployment, I designed the internal logic in n8n to ensure the data was accurate and highly structured.<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li><strong>The Entry Point (Webhook):<\/strong> The workflow starts with a <strong>Webhook Node<\/strong> listening for a <code>POST<\/code> request. This makes the agent accessible to any external interface.<\/li>\n\n\n\n<li><strong>The Search Phase (Tavily\/Serper):<\/strong> Instead of relying on stagnant AI training data, I integrated a <strong>Search Tool<\/strong>. The agent executes a live web search to retrieve the most current data from the internet.<\/li>\n\n\n\n<li><strong>The Intelligence Layer (AI Agent):<\/strong> The raw search results are passed to an AI Model (via <strong>Ollama<\/strong> or <strong>OpenRouter<\/strong>). I used a strict system prompt to force the AI into &#8220;Analyst Mode,&#8221; requiring it to synthesize data into Markdown tables and headers.<\/li>\n\n\n\n<li><strong>The Output Phase (Respond to Webhook):<\/strong> Using the <strong>Respond to Webhook<\/strong> node, the workflow sends the finished Markdown report back to the caller immediately upon completion.<\/li>\n<\/ul>\n\n\n\n<h2 class=\"wp-block-heading\">2. Technical Hurdles &amp; Solutions<\/h2>\n\n\n\n<p>Building this in a Windows-based Docker environment presented several unique challenges that required specific workarounds.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">The Permission &amp; File System Struggle<\/h3>\n\n\n\n<p>The first major roadblock was getting n8n to save files to my local Windows drive (<code>C:\\localdata\\dockerapps\\n8n-ollama\\data<\/code>).<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li><strong>The Challenge:<\/strong> Docker containers run as a Linux user (<code>node<\/code>), which Windows NTFS doesn&#8217;t recognize. This caused constant &#8220;Not Writable&#8221; errors.<\/li>\n\n\n\n<li><strong>The Solution:<\/strong> I had to grant <strong>&#8220;Everyone: Full Control&#8221;<\/strong> to the folder in Windows. In the <code>docker-compose.yml<\/code>, I unlocked the filesystem using <code>N8N_BLOCKS_ENABLE_ALL=true<\/code> and eventually forced the container to run as <code>user: root<\/code> to bypass all permission checks.<\/li>\n<\/ul>\n\n\n\n<h3 class=\"wp-block-heading\">The AI &#8220;Summarization&#8221; Trap<\/h3>\n\n\n\n<p>When I first connected n8n to Open WebUI, the interface worked, but the experience was poor.<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li><strong>The Challenge:<\/strong> The AI model (Qwen\/Llama) would receive the perfect Markdown table from n8n and then try to &#8220;helpfully&#8221; summarize it. This turned my beautiful tables into long, boring paragraphs.<\/li>\n\n\n\n<li><strong>The Solution:<\/strong> I updated the <strong>Tool<\/strong> Python code in Open WebUI with a strict docstring: <em>&#8220;CRITICAL: Return output exactly as received. DO NOT summarize.&#8221;<\/em> This forced the model to act as a pass-through for the n8n data.<\/li>\n<\/ul>\n\n\n\n<h2 class=\"wp-block-heading\">3. The Final Evolution: The Research Dashboard<\/h2>\n\n\n\n<p>To achieve the best user experience, I built a standalone <strong>Research Lab<\/strong> page. This eliminated the &#8220;middleman&#8221; AI and allowed for perfect rendering of complex data.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">Key Features of the Dashboard:<\/h3>\n\n\n\n<ol start=\"1\" class=\"wp-block-list\">\n<li><strong>Marked.js Integration:<\/strong> Instantly converts n8n\u2019s Markdown output into professional HTML tables.<\/li>\n\n\n\n<li><strong>GitHub-Style CSS:<\/strong> Ensures the reports look like clean, technical documentation.<\/li>\n\n\n\n<li><strong>Research History:<\/strong> A sidebar that tracks all searches in a session, allowing me to toggle between reports without re-running the workflow.<\/li>\n\n\n\n<li><strong>One-Click Export:<\/strong> A &#8220;Download as .md&#8221; button that saves the research locally for my records.<\/li>\n<\/ol>\n\n\n\n<h2 class=\"wp-block-heading\">Final Thoughts<\/h2>\n\n\n\n<p>This project proved that while local AI tools are powerful, the real value comes from the &#8220;plumbing&#8221;\u2014the webhooks, the permission fixes, and the custom UIs that make the data usable. I now have a private, high-speed research agent that provides better-formatted data than most commercial tools.<\/p>\n\n\n\n<p><\/p>\n","protected":false},"excerpt":{"rendered":"<p>Author: Jorge Pereira Date: August 2025 Tech Stack: n8n (Docker), Ollama, Open WebUI, JavaScript\/HTML, Windows 11 The Vision The goal was to build an autonomous &#8220;Deep Research&#8221; agent capable of performing real-time web searches and returning professional, high-quality Markdown reports. I wanted a tool that behaved like Perplexity but lived in my own home lab&#8230;<\/p>\n","protected":false},"author":2,"featured_media":519644,"comment_status":"closed","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_kad_blocks_custom_css":"","_kad_blocks_head_custom_js":"","_kad_blocks_body_custom_js":"","_kad_blocks_footer_custom_js":"","ngg_post_thumbnail":0,"episode_type":"","audio_file":"","podmotor_file_id":"","podmotor_episode_id":"","cover_image":"","cover_image_id":"","duration":"","filesize":"","filesize_raw":"","date_recorded":"","explicit":"","block":"","itunes_episode_number":"","itunes_title":"","itunes_season_number":"","itunes_episode_type":"","_kad_post_transparent":"","_kad_post_title":"","_kad_post_layout":"","_kad_post_sidebar_id":"","_kad_post_content_style":"","_kad_post_vertical_padding":"","_kad_post_feature":"","_kad_post_feature_position":"","_kad_post_header":false,"_kad_post_footer":false,"_kad_post_classname":"","footnotes":""},"categories":[17],"tags":[],"class_list":["post-519641","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-journey"],"taxonomy_info":{"category":[{"value":17,"label":"Journey"}]},"featured_image_src_large":["https:\/\/jorgep.com\/blog\/wp-content\/uploads\/FeaturedBuildLog-ResearchIAgent.png",1024,512,false],"author_info":{"display_name":"Jorge Pereira","author_link":"https:\/\/jorgep.com\/blog\/author\/jorge\/"},"comment_info":0,"category_info":[{"term_id":17,"name":"Journey","slug":"journey","term_group":0,"term_taxonomy_id":18,"taxonomy":"category","description":"","parent":0,"count":318,"filter":"raw","cat_ID":17,"category_count":318,"category_description":"","cat_name":"Journey","category_nicename":"journey","category_parent":0}],"tag_info":false,"_links":{"self":[{"href":"https:\/\/jorgep.com\/blog\/wp-json\/wp\/v2\/posts\/519641","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/jorgep.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/jorgep.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/jorgep.com\/blog\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/jorgep.com\/blog\/wp-json\/wp\/v2\/comments?post=519641"}],"version-history":[{"count":1,"href":"https:\/\/jorgep.com\/blog\/wp-json\/wp\/v2\/posts\/519641\/revisions"}],"predecessor-version":[{"id":519642,"href":"https:\/\/jorgep.com\/blog\/wp-json\/wp\/v2\/posts\/519641\/revisions\/519642"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/jorgep.com\/blog\/wp-json\/wp\/v2\/media\/519644"}],"wp:attachment":[{"href":"https:\/\/jorgep.com\/blog\/wp-json\/wp\/v2\/media?parent=519641"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/jorgep.com\/blog\/wp-json\/wp\/v2\/categories?post=519641"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/jorgep.com\/blog\/wp-json\/wp\/v2\/tags?post=519641"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}