4.4 Action and Execution 4.4.1 Tool Calling



原 PPT 独立图片资源

4.4 Action and Execution 4.4.1 Tool Calling

原 PPT 独立图片资源

Agent need to take Action


原 PPT 独立图片资源
Agent need to take Action
Beyond “Tell” to “Do”
Closing the loop: from understanding to executing tasks (e.g., sending emails, scheduling)
Tool-Calling & External APIs
Interface with calculators, search engines, IoT devices, business systems
Enables multi-step workflows under LLM orchestration
Agentic Autonomy
Persistent agents can monitor, adapt, and intervene in real-world processes
Frees users from manual coordination of complex tasks

Tool Calling


原 PPT 独立图片资源
Tool Calling
Definition: Enabling an LLM to invoke external functions or services (APIs, calculators, search engines) during conversation.
Why it matters:
Keeps knowledge up-to-date beyond training cutoff
Adds deterministic computation (math, code execution)
Connects LLMs to real-world data and actions
Introduced by Open AI at 2023.

Define Tools


原 PPT 独立图片资源
Define Tools
原 PPT 独立图片资源
https://medium.com/@developer.yasir.pk/tool-calling-for-llms-a-detailed-tutorial-a2b4d78633e2

Prepare the Conversation


原 PPT 独立图片资源
Prepare the Conversation
https://medium.com/@developer.yasir.pk/tool-calling-for-llms-a-detailed-tutorial-a2b4d78633e2
原 PPT 独立图片资源

Pass Tools to the LLM


原 PPT 独立图片资源
Pass Tools to the LLM
https://medium.com/@developer.yasir.pk/tool-calling-for-llms-a-detailed-tutorial-a2b4d78633e2

原 PPT 独立图片资源

Interpret LLM Output


原 PPT 独立图片资源
Interpret LLM Output
https://medium.com/@developer.yasir.pk/tool-calling-for-llms-a-detailed-tutorial-a2b4d78633e2

原 PPT 独立图片资源

Execute Tool Calls


原 PPT 独立图片资源
Execute Tool Calls
https://medium.com/@developer.yasir.pk/tool-calling-for-llms-a-detailed-tutorial-a2b4d78633e2

原 PPT 独立图片资源

Incorporate Tool Results


原 PPT 独立图片资源
Incorporate Tool Results
https://medium.com/@developer.yasir.pk/tool-calling-for-llms-a-detailed-tutorial-a2b4d78633e2

原 PPT 独立图片资源

Continue Conversation


原 PPT 独立图片资源
Continue Conversation
https://medium.com/@developer.yasir.pk/tool-calling-for-llms-a-detailed-tutorial-a2b4d78633e2

原 PPT 独立图片资源

4.4 Action and Execution 4.4.2 Coding



原 PPT 独立图片资源

4.4 Action and Execution 4.4.2 Coding

原 PPT 独立图片资源

Agent should write Code


原 PPT 独立图片资源
Agent should write Code
Automating Development
Rapid prototyping: generate boilerplate, tests, and integration snippets
Lower barrier for non-programmers to implement data pipelines and analyses
Self-Improving Systems
Models can modify their own tooling or workflows in response to new requirements
Enables continuous adaptation without full manual redevelopment
Explainable & Reproducible
Generated code with comments provides a clear audit trail of the model’s logic

From Tool Calling to Coding


原 PPT 独立图片资源
From Tool Calling to Coding
Coding is the interface to control the natural world;
Tool Calling: calling human written code;
Coding: generating code to satisfy new requirements.

Coding is Long-Term Goal of LLM


原 PPT 独立图片资源
Coding is Long-Term Goal of LLM
原 PPT 独立图片资源

https://arxiv.org/pdf/2406.00515

Coding contributes a lot in Advances of LLM


原 PPT 独立图片资源
Coding contributes a lot in Advances of LLM
https://arxiv.org/pdf/2303.10420
原 PPT 独立图片资源

Training to Generate Code


原 PPT 独立图片资源
Training to Generate Code
https://arxiv.org/pdf/2406.00515
原 PPT 独立图片资源

Data is Foundation


原 PPT 独立图片资源
Data is Foundation
https://arxiv.org/pdf/2406.00515
原 PPT 独立图片资源
原 PPT 独立图片资源

Code Act


原 PPT 独立图片资源
Code Act
https://github.com/xingyaoww/code-act
Overview