Chain of Thoughts vs Tree of Thoughts
Last year I came across this publication and was fascinated by the concept. Fast-forward 6 months and I am even more amazed at the possibilities and potential use cases ToT can support. Stay tuned for my workbook exploring this!
What is Tree of Thoughts?
Tree of Thoughts (ToT) and Chain of Thoughts (CoT) are two frameworks for guiding how language models generate and organise text. CoT follows a straightforward, linear path, moving from one idea to the next, while ToT branches out into multiple possibilities (hence the name ‘Tree of Thoughts’), exploring different reasoning paths simultaneously. These techniques shape how models think—either sticking to a single thread or branching out to consider various scenarios—offering unique advantages depending on the complexity of the task.
Chain of Thoughts
The Chain of Thoughts framework is a method where reasoning follows a linear, step-by-step progression. Imagine you're solving a math problem by methodically moving from one calculation to the next, ensuring each step logically follows the previous one. In natural language processing, CoT operates similarly, with each generated word or phrase being directly influenced by the preceding ones.
This approach works well for tasks that require a clear and linear logical flow, such as basic arithmetic problems or straightforward text summarisation, where the goal is to move seamlessly from start to finish without deviating from the path.
Key Characteristics of CoT:
Linear Reasoning: Follows a single, sequential path from one thought to the next.
Step-by-Step Logic: Each token or decision is directly influenced by the preceding one.
Limited Exploration: Focuses on a straightforward, continuous flow without considering alternative scenarios.
Simpler Tasks: Best suited for tasks requiring clear, linear logical progression.
Tree of Thoughts
The Tree of Thoughts framework, is like navigating a complex maze where multiple paths are possible. Rather than sticking to a single, linear progression, ToT allows for branching out into different reasoning paths. For example, when assessing the potential impact of inflation on various economic sectors, ToT could potentially enable you to explore how inflation might affect consumer spending, housing markets, and investment strategies simultaneously.
Each branch can be evaluated, backtracked, or expanded upon, leading to a more comprehensive and nuanced understanding. This makes ToT particularly valuable for tasks involving complex decision-making and strategic planning, where exploring multiple possibilities is crucial.
Key Characteristics of ToT:
Branching Reasoning: Explores multiple reasoning paths simultaneously, like branches on a tree.
Deliberate Decision-Making: Evaluates and compares different thought paths before selecting the best option.
Complex Exploration: Allows for lookahead, backtracking, and reassessment of decisions.
Advanced Tasks: Ideal for tasks involving strategic planning, complex problem-solving, and deep analysis.
I’ll be updating an building a kickstarter repo to be able to get started with ToT in Azure ML studio over the coming weeks. Here is a great place to start if you’re a python-head and want to play around with ToT.
Just an FYI, you may need a decent spec PC to compile this! Check the repo out now!