MyGO Multiplex CoT: A Method for Self-Reflection in Large Language Models via Double Chain of Thought Thinking

MyGO 多重 CoT：通过双重思维链实现大语言模型自我反思的方法

Abstract

摘要

Recent advancements in large language models (LLMs) have demonstrated their impressive abilities in various reasoning and decision-making tasks. However, the quality and coherence of the reasoning process can still benefit from enhanced introspection and selfreflection. In this paper, we introduce Multiplex CoT (Chain of Thought), a method that enables LLMs to simulate a form of self-review while reasoning, by initiating double Chain of Thought (CoT) thinking. Multiplex CoT leverages the power of iterative reasoning, where the model generates an initial chain of thought and subsequently critiques and refines this reasoning with a second round of thought generation. This recursive approach allows for more coherent, logical, and robust answers, improving the overall decision-making process. We demonstrate how this method can be effectively implemented using simple prompt engineering in existing LLM architectures, achieving an effect similar to that of the Learning-Refinement Model (LRM) without the need for additional training. Additionally, we present a practical guide for implementing the method in Google Colab, enabling easy integration into real-world applications.

近年来，大语言模型 (LLM) 在各种推理和决策任务中展现了令人印象深刻的能力。然而，推理过程的质量和连贯性仍可以通过增强内省和自我反思来提升。本文提出了一种名为 Multiplex CoT (Chain of Thought) 的方法，通过启动双重 Chain of Thought (CoT) 思维，使大语言模型能够在推理过程中模拟一种自我审查的形式。Multiplex CoT 利用了迭代推理的力量，模型首先生成初始的思维链，随后通过第二轮思维生成来批判和优化这一推理过程。这种递归方法能够生成更加连贯、逻辑性强且稳健的答案，从而提升整体决策过程。我们展示了如何通过简单的提示工程在现有的大语言模型架构中有效实现这一方法，达到类似于学习-优化模型 (LRM) 的效果，而无需额外的训练。此外，我们还提供了在 Google Colab 中实现该方法的实用指南，便于将其集成到实际应用中。

1 Introduction

1 引言

Large language models (LLMs) have revolutionized natural language processing (NLP) by excelling in tasks ranging from translation to text generation. However, these models often struggle with producing coherent, logical reasoning when faced with complex decision-making scenarios. One of the key limitations of LLMs is their inability to critically reflect on their own thought process, which can lead to inconsistencies and errors in the final output. While recent research has explored methods for improving reasoning in LLMs, including Chain of Thought (CoT) reasoning and fine-tuning approaches, there is still room for improvement in terms of the model’s ability to refine and critique its own reasoning.

大语言模型 (LLMs) 通过在从翻译到文本生成等任务中的卓越表现，彻底改变了自然语言处理 (NLP) 领域。然而，这些模型在面对复杂的决策场景时，往往难以生成连贯、逻辑的推理。LLMs 的一个关键局限在于它们无法批判性地反思自己的思维过程，这可能导致最终输出的不一致性和错误。尽管最近的研究探索了改进 LLMs 推理的方法，包括思维链 (Chain of Thought, CoT) 推理和微调方法，但模型在改进和批判自身推理能力方面仍有提升空间。

In this paper, we propose Multiplex CoT, a novel method for enhancing LLM reasoning by prompting the model to perform a self-reflection process. The technique involves generating an initial CoT and then initiating a second round of reasoning, which critiques and refines the initial chain of thought. By employing this iterative process, the model can simulate a form of self-review, leading to more coherent and logical outputs. Importantly, this method does not require additional training but instead utilizes a simple prompt engineering approach, making it easy to implement in existing LLM architectures.

在本文中，我们提出了 Multiplex CoT，这是一种通过提示模型执行自我反思过程来增强大语言模型推理能力的新方法。该技术涉及生成初始的 CoT（Chain of Thought），然后启动第二轮推理，对初始的思维链进行批判和优化。通过采用这种迭代过程，模型可以模拟一种自我审查的形式，从而产生更加连贯和逻辑的输出。重要的是，这种方法不需要额外的训练，而是利用简单的提示工程方法，使其易于在现有的大语言模型架构中实现。

2 Background

2 背景

2.1 Chain of Thought (CoT) Reasoning

2.1 思维链 (Chain of Thought, CoT) 推理

Chain of Thought (CoT) reasoning has been proposed as a technique to improve the logical coherence of LLM outputs. The method involves prompting the model to produce a step-bystep sequence of thoughts, which guides the reasoning process and helps the model arrive at more

思维链 (Chain of Thought, CoT) 推理被提出作为一种提高大语言模型输出逻辑连贯性的技术。该方法通过提示模型生成逐步的思维序列，从而引导推理过程，帮助模型得出更合理的结论。

accurate conclusions. CoT has been shown to significantly improve performance in tasks that require complex reasoning, such as mathematical problem-solving and commonsense reasoning.

准确结论。CoT 已被证明在需要复杂推理的任务中显著提升性能，例如数学问题解决和常识推理。

2.2 Learning-Refinement Models (LRM)

2.2 学习-精炼模型 (Learning-Refinement Models, LRM)

Learning-Refinement Models (LRM) aim to improve model performance by iterative ly refining the outputs through multiple training steps. These models typically involve a feedback loop where the initial predictions are revised based on some form of error analysis or critique. While LRM-based approaches have proven effective in certain contexts, they often require additional training and fine-tuning, which can be computationally expensive and time-consuming.

学习-精炼模型 (Learning-Refinement Models, LRM) 旨在通过多次训练步骤迭代精炼输出，以提高模型性能。这些模型通常涉及一个反馈循环，其中初始预测基于某种形式的错误分析或批评进行修订。尽管基于 LRM 的方法在某些场景中已被证明有效，但它们通常需要额外的训练和微调，这可能会带来较高的计算成本和耗时。

2.3 Self-Reflection in AI

2.3 AI 中的自我反思

Self-reflection is a cognitive process in which an agent reviews its own reasoning to identify errors or inconsistencies. While traditional LLMs are not equipped for self-reflection, recent work in meta-learning and reinforcement learning has explored ways to enable models to reflect on their actions. This line of research has shown promise in improving decision-making processes, particularly in scenarios where error correction or refinement is crucial.

自我反思是一种认知过程，AI智能体通过这一过程回顾自身的推理，以识别错误或不一致之处。虽然传统的大语言模型不具备自我反思的能力，但最近的元学习和强化学习研究探索了让模型反思其行为的方法。这一研究方向在改进决策过程方面显示出潜力，尤其是在错误纠正或优化至关重要的场景中。

3 Multiplex CoT: A Double Chain of Thought Approach

3 多重 CoT：一种双链思维方法

Multiplex CoT combines the benefits of CoT reasoning with a self-reflection mechanism. The process is outlined as follows:

多路 CoT 结合了 CoT 推理的优势与自我反思机制。其过程概述如下：

This two-phase process mimics human-like self-reflection, where the first phase involves generating ideas, and the second phase focuses on evaluating and refining those ideas. The method is designed to work seamlessly with existing LLM architectures, without requiring any changes to the underlying model parameters.

这个两阶段过程模仿了人类的自我反思，其中第一阶段涉及生成想法，第二阶段则专注于评估和完善这些想法。该方法旨在与现有的大语言模型架构无缝协作，无需对底层模型参数进行任何更改。

3.1 Prompt Engineering

3.1 提示工程 (Prompt Engineering)

To implement Multiplex CoT, we utilize a simple prompt engineering technique. By structuring the input prompt to request both the initial reasoning and a follow-up critique, the model is able to generate and refine its reasoning within the same inference cycle. The prompt is designed to encourage the model to "think twice" about its initial response, leading to better overall performance.

为了实现多路复用思维链 (Multiplex CoT)，我们采用了一种简单的提示工程 (Prompt Engineering) 技术。通过构建输入提示以请求初始推理和后续的批判性分析，模型能够在同一推理周期内生成并优化其推理过程。该提示的设计旨在鼓励模型对其初始响应进行“二次思考”，从而提升整体表现。

3.2 Example

3.2 示例

Consider the following example: Prompt:

考虑以下示例：提示：

Please solve the following problem: What is the capital of France? First, generate a Chain of Thought for how you would arrive at the answer. Then, review your answer and critique it. If you find any inconsistencies or errors, correct them and provide the final answer.

请解决以下问题：法国的首都是哪里？首先，生成一个思维链，说明你将如何得出答案。然后，回顾你的答案并对其进行批评。如果发现任何不一致或错误，请纠正并提供最终答案。

The model might respond with: Step 1 (Initial CoT):

模型可能会回应：步骤 1（初始 CoT）：

Step 2 (Review and Refinement):

步骤 2 (审查与优化):

4 Mathematical Analysis of Multiplex CoT: Refining the Reasoning Process

4 多重 CoT 的数学分析：优化推理过程

To mathematically validate the effectiveness of Multiplex CoT in improving the reasoning quality of large language models (LLMs), we will introduce formal definitions for the concepts of logical consistency, coherence, and error correction rate. These metrics provide a quantitative way to assess the impact of self-reflection on reasoning quality.

为了从数学上验证多路复用思维链 (Multiplex CoT) 在提升大语言模型推理质量方面的有效性，我们将引入逻辑一致性、连贯性和纠错率等概念的形式化定义。这些指标为评估自我反思对推理质量的影响提供了量化方法。

4.1 Logical Consistency and Coherence

4.1 逻辑一致性与连贯性

The key advantage of Multiplex CoT lies in its ability to iterative ly improve the reasoning process by reviewing and refining the initial output. We define logical consistency as the number of valid logical connections between consecutive reasoning steps. If $s_{i}$ represents the $i$ -th step in the Chain of Thought, and $\mathbb{I}(s_{i},s_{i+1})$ is an indicator function that returns 1 if there is a logical connection between $s_{i}$ and $s_{i+1}$ , then the logical consistency $C$ for a single Chain of Thought is:

Multiplex CoT 的关键优势在于其能够通过审查和优化初始输出来迭代改进推理过程。我们将逻辑一致性定义为连续推理步骤之间有效逻辑连接的数量。如果 $s_{i}$ 表示思维链中的第 $i$ 步，且 $\mathbb{I}(s_{i},s_{i+1})$ 是一个指示函数，当 $s_{i}$ 和 $s_{i+1}$ 之间存在逻辑连接时返回 1，那么单个思维链的逻辑一致性 $C$ 为：

where $n$ is the total number of steps in the reasoning chain. A higher value of $\zeta_{\mathrm{CoT}}$ indicates that the reasoning steps are logically consistent and connected.

其中 $n$ 是推理链中的总步数。$\zeta_{\mathrm{CoT}}$ 的值越高，表示推理步骤在逻辑上越一致且连贯。

When applying Multiplex CoT, a second round of reasoning is conducted, which critiques and refines the initial reasoning. We define the coherence of the reasoning process as the degree of alignment between the initial and refined reasoning steps. The coherence $H$ can be quantified as:

在应用多路 CoT 时，会进行第二轮推理，对初始推理进行批判和优化。我们将推理过程的一致性定义为初始推理步骤与优化后推理步骤之间的对齐程度。一致性 $H$ 可以量化为：

where $s_{i}^{\mathrm{refined}}$ is the corresponding statement in the refined reasoning chain, and $\mathbb{I}(s_{i},s_{i}^{\mathrm{refined}})$ is 1 if the statement in the second round is consistent with the original reasoning. Coherence measures how well the second round of reasoning preserves the logic of the initial thought while refining it.

其中 $s_{i}^{\mathrm{refined}}$ 是精炼推理链中的对应陈述，$\mathbb{I}(s_{i},s_{i}^{\mathrm{refined}})$ 在第二轮推理中的陈述与原始推理一致时为 1。连贯性衡量第二轮推理在精炼初始思路时保持逻辑的能力。

The overall improvement in reasoning due to Multiplex CoT can be defin