5 Steps from Chatbots to Secure Enterprise AI Agents in 2025

Though LLMs are great at text generation, their

• Capability to interact with enterprise systems is limited.
• Hallucinations prevent them from many production scenarios. ¹

Enterprise AI agents are being built by leading SaaS vendors to address these issues. These bots need to be able to:

While these are table stakes, how they are implemented is important. Based on my ~2 decades of enterprise automation experience, I have reviewed the offerings of major SaaS vendors and identified these products:

Top enterprise AI agents 

Read more: AI agent builders, agentic AI.

Enterprise AI agents deep-dive

1. Salesforce Einstein Copilot

Salesforce Einstein Copilot is a natively embedded AI agent across the Salesforce platform. It can answer queries, generate content, and carry out actions.

Video 1: What is Einstein Copilot

Source: Salesforce²

Einstein Copilot offers a set of low-code functions integrated with Data Cloud, which lets users customize and embed AI prompts and actions:

• Salesforce Einstein Copilot BYOL (bring your own lake) enables businesses to link their data sources with Data Cloud and access their data from data lakes such as AWS Redshift, Google BigQuery, Databricks, and Snowflake within Salesforce. This eliminates the need for the ETL (extract-transform-load) to move data between platforms.
• Model builder with bring your own LLM allows different LLMs, including those from Anthropic, Hugging Face, Google, and OpenAI to be used.
• Copilot builder: Users can use it to configure their AI assistant to execute context-specific skills such as updating an account record or receiving a shipment update. For example, Mulesoft and copilot integration allows developers to extend the copilot’s functionality (i.e. skills) with external APIs.³

Example use cases:

• Service organizations can execute tasks such as launching fee reversal requests for disputed transactions, approving provisional credits, and retrieving lists of recent customer transactions.
• Sales employees can query historical customer transcripts and send automated follow-up emails.
• Marketers can design more intelligent campaigns automatically.
• Retailers can create e-commerce websites and develop automated promotions

Read more: Large action models (LAMs).

2. Workato Agentic

Workato Agentic allows business users to access AI agents called Genies. There are pre-built Genies (e.g. for sales and HR) and businesses can also build their own Genies. Genies have granular governance controls and have access to business context enabling them to

• Respond to user queries taking into account that user’s permissions
• Limit their activities to their focus area, limiting hallucinations.

Orchestrating agents

There is also an overall Genie that routes user requests to specific Genies. Genies can also talk among one another for task orchestration.

This level of anthropomorphization of bots could confuse novice users but it can also help businesses better understand process flows, user requests and manage governance in a granular manner.

Skills

Since Workato is typically used as an iPaaS, it has a broad range of integrations. Its users build thousands of integration or automation workflows. These can be leveraged as skills by the Genies. Example use cases:

• IT: App provisioning, Q&A
• Sales: CRM updates, preparing presentations, drafting emails
• HR: Onboarding, signing up for available training sessions
• Finance: PO preparation, invoice and expense approvals
• Cybersecurity: Automated remediation based on alerts

3. IBM Watsonx Orchestrate

IBM Watsonx Orchestrate calls itself a platform for digital labor. Users can integrate Watsonx Orchestrate with common business applications, such as Salesforce, SAP and Workday.

When a user submits a request to IBM Watsonx Orchestrate, it may utilize basic skills (adding a row to a table) or complex abilities (finding contacts from the database, creating a table of those contacts, and then emailing them).

IBM Watsonx Orchestrate offers:

• 1,000+ prebuilt skills
• Prebuilt AI assistants to search for an answer in a knowledge base, and refer users to human agents for more support.
• AI assistant builder to create and deploy purpose-built AI assistants that help end users complete tasks.
• Low-code skills studio to create customized skills and processes without the need for coding experience.

IBM Watsonx Orchestrate’s use cases include:

HR: Talent acquisition (TA), HR procurement, onboarding

Onboarding and assisting hires:

Video 2: Human resources process automation

Source: IBM⁴

Example integrations for HR actions:

• Human resources information system: BambooHR
• Candidate sourcing: Oracle Human Capital Management (HCM), SAP SuccessFactors, ThisWay Global, Workday

Sales: Automating sales cycle

IBM Watsonx Orchestrate can help sellers automate lead generation, and improve interactions with prospects and existing customers.

Example integrations for sales actions:

• Application integration services: Amazon SNS, Amazon SQS
• Business spend management: Coupa
• Business planning and performance management: Anaplan

Procurement: Vendor, contract, and order management 

IBM Watsonx Orchestrate can help companies with:

• Contact management: Select contact templates and chat with your stakeholders to enhance operational efficiency
• Procure to pay & order management: Automate invoice processing and payment execution.

Example integrations for procurement actions:

• Salesforce: Enables account, campaign and contact creation, tracking of leads and opportunities; initiating orders etc.
• SAP ARIBA: Ensures accurate data to create a purchase request
• Coupa: Enables management and analysis of business spending across procurement, supply chain & finance.

Other key integrations:

• Communicating and collaborating: Calendly, Gmail, Microsoft Outlook, Microsoft Teams, Slack
• Data visualization: Microsoft Power BI
• Directory and identity management: Microsoft Entra ID
• File sharing and hosting: Amazon S3, Box, Dropbox, Microsoft OneDrive for Business, Microsoft OneNote

Specialist AI agents

4. Beam AI

Beam AI agents automate routine back-office operations thanks to integrations with current internal systems (CRM, ERP, etc.) and databases.

Beam AI’s agent automation is divided into two categories:

1. communication use cases (such as customer assistance or supplier/vendor contacts) 
2. data extraction/entry business workflows (such as invoice processing and order management)

Video 3: Order processing with Beam AI

Source: Beam AI⁵

Selected use cases:

• Customer service:
  • Order placement
  • Inventory check and allocation
  • Payment processing
  • Order fulfillment
  • Shipping and tracking
• Healthcare:
  • Patient’s preliminary data collection
  • Real-time transcription documentation
  • Integration into electronic health record (EHR)
  • Follow-up messaging and patient engagement
• Insurance:
  • Data extraction 
  • Customer verification 
  • Fraud detection and risk assessment
  • Claim resolution and payment processing

Key integrations:

• Finacial operations: Vantaca
• Insurance operations: Duck Creek
• Medical operations: AdvancedMD
• Document management: Box
• Productivity management: Clockify
• Business management: SAP

5. Regie.ai

~50% of Chief Sales Officers surveyed by Gartner ranked outbound prospecting as a top-three priority for scaling their company.⁶ Regie.ai is a sales auto-pilot (like virtual sales reps) that enables teams to understand which audiences in their CRM are most likely to book appointments with and identify similar leads to prioritize outreach to.  

Regie.ai can:

• source contacts from your database
• write content
• call leads
• send custom email messages

The outbound agent is best suited for companies that have a large number of companies in their addressable market and that haven’t yet established contact with a significant share of these companies. It can help reps focus high-priority messages via phone and social media.

Figure 1: Auto discovery, content generation, and task execution workflow for Regie.ai

Source: Regie.ai⁷

Some key features include:

• List creation creates a lead list with a similar audience.
• Personalized emails and LinkedIn messages based on predetermined data, such as ideal customer profile, ICP’s pain areas etc.
• Outbound prioritization based on buyer intent signals to improve the frequency of touchpoints with prospects.

Recommendations to buyers

Investigate your enterprise orchestration platform‘s (e.g. your iPaaS or automation vendor of choice) agentic capabilities. It is likely to be the provider of your enterprise AI agents. You already have your automation flows there, these flows can be made accessible across the enterprise with a text/voice interface thanks to enterprise AI agents.

Invest in flexible and extensible platforms. This is an emerging technology. For example, you don’t want to get stuck with limited options to choose LLMs.

Governance and reliability are key. Enterprise automation without governance mechanisms or high rates of reliability is a recipe for disaster. Your PoC needs to investigate the governance mechanisms. Important questions are:

• How much effort will it take to input our governance model into the platform?
• What are the challenges encountered by initial users?

Enterprise AI agents: 5 distinct capabilities

1. Have a ubiquitous text or voice interface to interact with employees

AI agents need to be accessible to be useful. They need to be present in the company’s messaging system (e.g. Slack, MS Teams) and be ready to respond to voice and text interfaces.

Figure 5: Conversation preview on Salesforce Einstein Copilot

Source: Salesforce⁸

2. Access the company’s knowledge base to be informed by the business context

A generic agent is of limited value in an enterprise context. Agents need to be able to access a company’s knowledge base to be informed about the company’s policies. It can carry out:

• Context-aware search: The AI agent can perform searches within the knowledge base, understanding the nuances of business-specific terminology and the relationships between different context-specific information. 
• Hierarchical understanding: The agent can navigate complex, hierarchical knowledge structures, understanding the organization’s data architecture, including categories, subcategories, and metadata.

For example, an AI customer support agent can access a company’s product and return policy databases. Similarly, an AI marketer agent can access customer analytics databases to be informed by marketing context.

3. Carry out actions on enterprise systems

An agent without access to systems is just a talking head.

Agents deployed on systems need to be able to participate in complex workflows and processes across various enterprise functions like finance, HR, supply chain, and customer service. They can:

• Query data.
• Determine specific actions to perform (e.g., data entry, report generation, customer support) across systems.
• Execute API integrations based on the defined objectives to communicate with enterprise systems.

4. Provide context-specific guardrails to minimize mistakes

Multi agent systems need contextual guardrails and governance. This helps reduce misuse and ground agents.

For example, consider a customer service agent. Customers may initiate a discussion with the agent, asking for the return of a specific product. Input guardrails can assist in assessing if the person seeking information has the authority to activate the model and obtain the information.

Without a guardrail:

• Prompt: “How many customers you’ve served today”
• Response: “I have served 45 customers”

With guardrails:

• Prompt: “How many customers you’ve served today”
• Response:  “Sorry, but I can’t assist with that.”

5. Log all actions in a detailed audit log for process analysis

Without an audit log, businesses would lose the opportunity to have a granular view of their processes. Businesses are investing millions in buying process mining solutions to access such log data, they shouldn’t miss the chance to generate accessible log files detailing user actions.

When an end-user interacts with an AI agent each action the AI agent takes should be logged as an entry with:

• A timestamp: The exact time when the action was performed.
• Action description: A detailed description of the action taken.
• User and agent ID: Identifier for the AI agent or user who initiated the action.
• System and module affected: The specific enterprise system or module where the action was performed.
• Input data: Any input data or parameters used by the AI agent for the action.
• Outcome: The result of the action (e.g., success, failure, error code).

The AI agent then can use a centralized logging service (e.g., Elasticsearch, Splunk, or a custom database) to store the audit logs.

For more context on why enterprise AI agents are now being offered:

LLM-based chatbots to enterprise AI Agents: From tokenization to reasoning

LLMs started working on text without any planning. Enterprise AI agents can work with enterprise systems and plan their actions.

Actions vs text

LLMs are trained for causal language modeling which takes a sequence of text tokens as input and returns the probability distribution for the next token. For instance, typing in “John bought…” to an LLM-based chatbot may result in a suggestion like “a laptop”.

These LLM token-prediction capabilities are trained on large volumes of internet text.
Figure 2: General working flow of an LLM predicting the next word

Source: NVIDIA⁹

Thus, LLMs can manipulate text (e.g. carry out engaging conversations, answer questions, write code). However, LLMs alone cannot browse the internet, run code, or retrieve data from a knowledge base. For such tasks, they need access to enterprise systems. With AI agents, we can add external capabilities to LLM.

Planning

In agentic AI, LLMs are used to break down tasks into smaller sub-components, assess the results of potential actions, take actions and evaluate their consequences. This enables them to complete more complex processes.

Table 2: Differences between enterprise AI agents and traditional chatbots

Figure 3: “plan-and-execute” diagram for AI agents

Source: GitHub¹⁰

The advantages of these “plan-and-execute” agents include:

• Explicit long-term planning
• Capability to utilize smaller/weaker models for the execution phase and just larger/better models for the planning step.

This shows why an agent can accomplish tasks effortlessly whereas a chatbot will underperform. The AI agent benefits from several LLM calls and an externally required system for planning, thinking, assessing, and carrying out tasks.

The GPT model’s performance benchmark demonstrates this point. The GPT-3.5 model wrapped in a reflection loop (95%) outperforms GPT-4 (~65%) in zero-shot prompting (executing a task without prior examples or specific training).

Figure 4: Human Eval coding performance benchmark

Reflection: The LLM reviews its own work to determine how it may be improved.

Source: OpenAI (2023)¹¹

Grounding

Without grounding, hallucinations harm LLMs usability as agents. With each step in the process (e.g. planning, assessing), the probability of hallucinations increases. Enterprise AI agents use several approaches to ground themselves:

• Searching enterprise knowledge bases for facts
• Context: If a user from the sales department is calling the agent, this fact can be used to significantly reduce the solution space for actions. For example, a sales personnel wouldn’t be expected to pay an invoice or answer to an internal IT request
• Focus: If an enterprise AI agent is focused on finance, then it will not expect to take actions in the IT domain, making it easier to choose which actions to take.

Further reading

• Compare 20 LLM Security Tools & Open-Source Frameworks 
• What are the Top 10 LLM Security Risks

External Links

• 1. What Makes Chatbots ‘Hallucinate’ or Say the Wrong Thing? - The New York Times. The New York Times
• 2. Einstein Copilot In-Depth: What It Is, How It Works, and What It Can Do - Salesforce.
• 3. Optimize Integration with Copilot Actions, APIs and MuleSoft - YouTube.
• 4. HR Process Automation – IBM watsonx Orchestrate .
• 5. Leading Platform for Agentic Automation & AI Agents | Beam AI.
• 6. Gartner Identifies Three Priorities for Chief Sales Officers in Response to Economic Headwinds .
• 7. Top Use Cases for RegieOne, the World's First AI SEP | Regie.ai.
• 8. Agentforce: Create Powerful AI Agents | Salesforce US.
• 9. How to Get Better Outputs from Your Large Language Model | NVIDIA Technical Blog. NVIDIA Technical Blog
• 10. langgraph/examples/plan-and-execute/plan-and-execute.ipynb at main · langchain-ai/langgraph · GitHub.
• 11. ”GPT-4 Technical Report“. (PDF). March 27, 2023. Retrieved on August 17, 2024.