Mux Engineers Pioneer Custom Video Intelligence Agent Using Vercel Eve Framework to Solve Internal Slack Search Challenges

The modern software engineering workplace is inundated with asynchronous communication tools, yet finding critical institutional knowledge buried within corporate chat platforms remains a persistent operational friction point. At Mux, a leading video infrastructure platform, engineering teams frequently share rapid iteration updates, technical demonstrations, and product walk-throughs via video recordings directly inside Slack channels. While this culture of visual communication fosters high collaboration and swift alignment, it introduces a severe information retrieval bottleneck. Video assets posted to fast-moving communication streams easily vanish into digital oblivion, rendering historical context difficult or impossible to locate when subsequent feature development or debugging requires it.

To address this systemic enterprise communication challenge, developers at Mux initiated an internal experiment to determine whether a custom agent harness could successfully bridge the gap between raw video processing APIs and day-to-day workflow execution. Rather than relying entirely on generic, off-the-shelf development assistants or terminal-bound large language models, the team sought to build a proprietary, highly integrated solution capable of acting as an omniscient, private video library historian for the company. The resulting system leverages Vercel’s newly introduced Eve framework—frequently characterized by developers as Next.js for autonomous agents—alongside Mux’s native video robots workflows to bring advanced video intelligence directly into active workplace communication threads.

The Genesis of the Experiment: Bridging APIs and Autonomous Workflows

The fundamental question driving the Mux experiment centered on the distinction between making application programming interfaces available and genuinely owning the user experience workflow. Off-the-shelf tools, such as Anthropic’s Claude or GitHub Copilot, provide exceptional capabilities when developers operate locally within terminal environments or dedicated software development kits. Engineers can readily provision a local command-line interface, inject API keys, and prompt models to execute localized coding tasks. However, these generic interfaces frequently fall short when organizations require persistent, cloud-deployed integrations that monitor enterprise messaging channels, manage complex permission boundaries, and execute asynchronous tasks reliably without human babysitting.

Mux recognized that raw technical functionality—such as transcription generation, semantic embedding creation, and video chunking—is insufficient on its own. While Model Context Protocol (MCP) servers and standard command-line utilities allow developers to expose capabilities to third-party harnesses, doing so relinquishes control over the surrounding product experience. By constructing a custom harness, engineering teams maintain total oversight of request routing, asynchronous state tracking, tenant boundaries, and thread-level context management. This orchestration layer transforms basic utility plumbing into a seamless, reliable software experience.

Adopting Vercel Eve for Multi-Channel Agent Architecture

To accelerate development without writing a custom agent runtime from scratch, Mux utilized Vercel’s eve framework. The framework provided a structured, modular environment that separated the core agent definition from the specific communication channels delivering messages to and from the system. Within the project architecture, the agent’s core logic resides in inspectable, cleanly separated files: instructions.md defines the persistent identity, safety protocols, tenant boundaries, and routing rules; agent.ts establishes the primary decision-making loop; and dedicated directories house modular tools, skills, and background libraries.

This architectural separation proved critical to the experiment’s success. Slack functions as one interface channel wrapped around the core agent, but the underlying intelligence is not tightly coupled to enterprise chat applications. The exact same agent logic, tools, and behavioral definitions can theoretically interface with web applications, integrated development environment terminal extensions, or customer-facing dashboards without requiring a complete rewrite of the agent’s core capabilities. By decoupling the agent’s execution engine from its presentation layers, developers achieved high reusability and maintainability across diverse deployment targets.

Transforming Passive Video Sharing into Actionable Enterprise Context

The operational workflow implemented by the Mux team fundamentally alters how internal video assets are consumed and indexed. Under the new paradigm, when an engineer shares a product demonstration or technical deep-dive within a Slack thread, team members can explicitly mention the Mux Agent. Upon invocation, the agent automatically routes the video file into Mux’s established ingestion and processing pipeline, providing real-time progress updates to the chat thread as background jobs execute.

Once the underlying APIs complete the heavy lifting—transcribing audio, generating dense semantic embeddings, and indexing visual frames—the video transforms from a static, passive recording into a fully searchable, interactive knowledge base. Team members can query the thread about specific topics discussed in the recording, request automated executive summaries, or ask the agent to locate precise timestamps where specific user interface elements, such as a pricing screen or architecture diagram, were displayed. Furthermore, the agent supports multimodal search inputs; if an engineer remembers a visual concept but cannot recall keywords, they can upload a screenshot directly into the Slack thread, prompting the agent to scan visual frames and return the exact moment the subject appeared.

Separation of Concerns: Deterministic APIs Versus Probabilistic Reasoning

A core architectural principle established during the Mux experiment involves maintaining a strict division of labor between deterministic domain logic and probabilistic agent reasoning. Industry best practices suggest that developers should never rely on large language models to execute precise computational tasks or manage state transitions. Instead, Mux kept deterministic operations—such as video transcoding, audio transcription, vector embedding generation, and asynchronous job orchestration—strictly within traditional backend APIs.

The agent itself is strictly scoped to handle high-level user intent, tool selection, context coordination, and natural language communication. When a user issues a prompt in Slack, the agent analyzes the request, selects the appropriate Mux API tool, monitors the asynchronous execution of the task, and formats the retrieved data into a clean, human-readable response. This clean separation of concerns significantly simplifies debugging and iteration, allowing developers to update underlying video processing capabilities without destabilizing the agent’s natural language interface.

Furthermore, owning the agent harness grants engineering teams unprecedented visibility into compound workflows and user behavior patterns. As team members begin chaining capabilities together—such as using the output of a transcript summary as context for a subsequent video search query—developers can instrument these interaction sequences. This observability reveals valuable, unanticipated compound workflows that can be formally productized and optimized over time, insights that remain entirely obscured when tools operate within third-party proprietary harnesses.

Broader Industry Implications and Future Horizons

While the initial deployment of the Mux video intelligence agent was tailored specifically for internal Slack usage, industry analysts note that the underlying architectural pattern carries significant implications for enterprise software development at large. As organizations increasingly adopt agentic workflows to automate knowledge management, the ability to build custom harnesses using frameworks like Vercel Eve represents a vital competitive advantage. Companies are moving past the initial phase of LLM experimentation and entering an era of deep workflow integration, where the ownership of routing, context persistence, and user experience determines whether an AI implementation becomes mission-critical infrastructure or remains a passing novelty.

Looking forward, Mux’s experiment opens the door to broader commercial applications. While the current iteration serves primarily as a private video historian for internal engineering teams, the underlying architecture provides a blueprint for external customer-facing solutions. Over time, similar agentic workflows could empower Mux customers to deploy automated video intelligence directly into their own operational environments, enabling end-users to search, comprehend, and manipulate massive enterprise video libraries directly from the productivity tools where they work every day.

The success of the Mux experiment underscores a broader lesson for software architects and product developers: while off-the-shelf tools and generic harnesses are invaluable for exploring the outer limits of what artificial intelligence can achieve, building a proprietary harness is essential for shaping a cohesive, reliable product experience around specialized domain tools. By starting small with a singular, high-friction workflow and maintaining strict boundaries between API execution and agent reasoning, engineering organizations can unlock profound efficiency gains and uncover powerful new ways to interact with enterprise data.