From Experiment to Infrastructure: Lessons from Building Our AI Portal

Introduction

When AI tools first became widely available, our team jumped in with curiosity and creativity. We explored ChatGPT, Claude, Gemini, and a handful of open-source interfaces to see how AI could help us work faster and smarter. One tool, OpenWebUI, quickly became a favorite: open, flexible, and able to connect to almost any model through the OpenAI protocol. It was our playground, and our proving ground.

As our experiments grew, we discovered the challenges that come with scaling AI beyond a single user: security, compliance, cost management, and consistency. What began as experimentation naturally evolved into infrastructure, and each step along the way taught us something new about building AI responsibly at scale.

The Experiment: Trying OpenWebUI in the Wild

OpenWebUI initially seemed perfect. Anyone could log in, select a model, and start experimenting. It supported both open-source and commercial APIs alike. As more teams began sharing the same AI workspace, the environment naturally shifted from a playground to infrastructure.

That’s when we began to see the cracks:

• Security risks: sensitive API keys, model tokens, and credentials were too exposed.
• API incompatibility: Bedrock, Vertex AI, and Azure each speak their own dialect of “AI API.”
• Cost opacity: pay-per-token billing meant we could rack up a surprisingly large bill before the month was over.
• Admin complexity: model names, versions, and permissions quickly became inconsistent.

We started with OpenWebUI, an outstanding open-source foundation for speed and flexibility, and extended it with the governance and security controls enterprises need. From there, we enhanced it with the controls, permissions, and security needed for enterprise use.

How We Evolved: Building the Tag1 AI Portal

We built on what worked. The goal was to keep OpenWebUI’s usability, but wrap it in an architecture that met enterprise-level standards for security and scalability.

The result is what we now call our Internal AI portal.

At a high level, the system works like this:

• OpenWebUI provides the familiar chat interface.
• Our proxy layer, one each for AWS Bedrock, Google Vertex AI, and soon Microsoft Azure, does the heavy lifting.

These proxies serve two critical functions:

1. Security isolation. Each proxy holds the sensitive keys; OpenWebUI never sees them.
2. Protocol translation. Bedrock, Vertex, and Azure all have proprietary APIs, but our proxies translate everything into a common, OpenAI-compatible language.

The result:

OpenWebUI thinks it’s talking to OpenAI, but in reality, it’s speaking through secure internal services that handle routing, authentication, and logging.

That architectural shift gave us both safety and flexibility, allowing us to connect any model we want, from Claude on Bedrock to Gemini on Vertex AI, and even open-source models like GPT-OSS.

Security by Design

Security was the primary driver for this change.

Because our portal routes requests through private instances of non-training models running on our AWS Bedrock infrastructure, all processing stays within Tag1’s environment, no internal or client data is exposed externally, or used for training. That’s the foundation of our secure AI design.

With the new setup, no API keys or credentials are ever exposed to end users. Each proxy sits inside a private network and holds one key, its own. If someone’s personal API key leaks, it can only access their user space, not the organization’s infrastructure.

We also enforce workspace permissions so teams can collaborate confidently without risk of cross-project data exposure.

Each workspace is isolated, meaning only team members assigned to a specific project can access its prompts, data, and AI outputs. This ensures that sensitive information stays contained within the right team and reduces the risk of accidental leaks.

The Billing Reality: Every Token Counts

If you’re used to paying $20 a month for ChatGPT Plus, you might not realize how different enterprise billing is.

Every token, every word in, every word out has a cost. Bedrock, Vertex AI, and Azure all charge per million tokens, and each has its own rates and quotas.

Early on, we realized this could get expensive fast. We designed a cost-tracking system directly into the portal:

• Every user and every API key has its own usage limits.
• At $50, $75, and $100 thresholds, the system automatically alerts both the user and admin.
• When a user hits their limit, their access pauses until the budget is approved.

This ensures that one over-enthusiastic coding session doesn’t accidentally rack up a four-digit cloud bill.

We’re extending this with usage dashboards and an admin interface powered by PostgreSQL, giving us a real-time view of how AI is being used across the organization.

How It Changed Our Work

The biggest impact wasn’t technical; it was cultural.

Before the portal, developers were hesitant to use AI with any code that touched client projects. Now, they can safely ask questions, share prompts, and even prototype with real data.

That shift unlocked collaboration we hadn’t seen before, the portal turned AI from a curiosity into a daily companion for developers. Teams share chats, fork conversations, and build on each other’s work, just like they would with code.

We’ve even used AI to refactor our own systems: running complexity scores on production code, simplifying functions, and generating cleaner, faster implementations.

Lessons from the Journey

Looking back, a few lessons stand out:

• Start messy then standardize. Our first setup was a Wild West of models and keys. Structure came later, and that’s okay.
• Keep models behind proxies. It’s the only way to combine flexibility with security.
• Track your tokens. Visibility into usage changes how you design AI systems.
• Name things well. We now follow a consistent pattern that we enforce in code: provider.model-version-variant (for example, openai.gpt-5, google.gemini-2.5-flash, or anthropic.claude-sonnet-4.5). It sounds small, but it saves hours of debugging, and ideally, we’d love to see this naming scheme adopted across the entire enterprise. Standardization would make life much easier for everyone.
• Show, don’t tell. The best way to get buy-in is to demonstrate value, AI simplifying a real piece of code speaks louder than any slide deck.

Each of these lessons helped transform AI from a set of tools into part of how we think and build.

What’s Next

The next major milestone is rolling out full cost tracking and analytics for all users. Once complete, the portal will provide real-time insights into AI usage and ROI across teams.

We’re also building the Azure proxy to complete the tri-cloud setup, ensuring every major provider can run securely under our architecture.

And we’re thinking bigger.

Our long-term goal is to make this framework reproducible, something we can help our clients implement internally if they want their own secure, private AI environment.

What started in open source continues to grow there, our blueprint for enterprise AI is built on the same shared foundations that make open innovation possible.

Closing Thought

OpenWebUI gave us a head start, and building on it taught us what really matters when AI becomes part of your infrastructure: security, cost awareness, and collaboration.

At Tag1, that’s what responsible AI means: engineering systems that are powerful, transparent, and built to last.

Bring practical, proven AI adoption strategies to your organization, let's start a conversation! We'd love to hear from you.