Tool Modality Selection for Large Language Model Agents: An Empirical Evaluation of API-Based Retrieval and Browser Automation for Latency, Cost, and Reliability Optimization

Abstract

Large Language Models (LLM) agents are gradually becoming dependent on external tools for knowledge grounding, and autonomous decision-making. The study offers empirical comparison of the structured API retrieval and browser automation as reliable LLM verbal to agents. Based on the verification agent of type document and registry developed by Amazon Bedrock AgentCore, API Gateway integration allows retrieving documents through an orderly structure as a tool of the first order. Based on latency, tail latency, cost of use, determinism in output, accuracy of verification, and runtime errors, identical verification tasks are measured in both API and browser modes. Another factor of API-based integration that can reduce operational cost includes less overhead for browsers, fewer requirements for browsers and less usage of computational resources. Experimental evidence shows that API-based retrieval has a better performance, pairing the mean latency to 220 ms versus 950 ms when using browser automation, and has an increased reliability, as well as a decrease in parsing and timeout failures. Machine learning models such as Logistic Regression, Random Forest and Gradient boosting are used to determine tool selection prediction whereby the best two are Gradient Boosting with 94.5% potential accuracy and 0.99 ROC-AUC. This paper suggests new API-first-with-browser-fallback architecture to direct effective, dependable and scalable use of enterprise LLM agents.