从‘理性Agent’到‘学习Agent’用Python一步步构建你的第一个AI智能体在人工智能领域智能体(Agent)是最基础也最核心的概念之一。一个智能体可以简单理解为能够感知环境并采取行动以实现目标的实体。从最简单的自动调温器到复杂的自动驾驶系统智能体的设计理念贯穿了AI应用的各个层面。本文将带你从零开始用Python构建一个能够不断进化的智能体系统。1. 智能体基础与环境建模1.1 理性Agent的核心概念理性Agent是指在给定感知序列和先验知识的情况下能够采取使其性能度量最大化的行动的Agent。要构建一个理性Agent我们需要明确几个关键组件class RationalAgent: def __init__(self, performance_measure, environment, actuators, sensors): self.performance performance_measure self.env environment self.actuators actuators self.sensors sensorsPEAS模型是描述Agent任务环境的标准框架组件描述示例(扫地机器人)Performance衡量Agent成功程度的标准清洁面积百分比EnvironmentAgent运作的上下文房间布局、家具位置ActuatorsAgent执行动作的机制移动轮子、吸尘装置SensorsAgent感知环境的方式灰尘传感器、碰撞检测1.2 环境性质分析在设计Agent前我们需要分析环境的性质可观察性完全可观察 vs 部分可观察确定性确定性的 vs 随机的连续性离散的 vs 连续的动态性静态的 vs 动态的以下代码展示了如何用Python模拟不同环境性质class Environment: def __init__(self, observableTrue, deterministicTrue, discreteTrue, staticTrue): self.observable observable self.deterministic deterministic self.discrete discrete self.static static def get_state(self): if self.observable: return self.current_state else: return self._partial_observation()2. Agent架构的演进路径2.1 简单反射Agent最简单的Agent形式是基于当前感知直接选择行动不考虑历史信息class SimpleReflexAgent: def __init__(self, rules): self.rules rules # 条件-动作规则集 def act(self, perception): for condition, action in self.rules.items(): if condition(perception): return action return self.default_action这种Agent的局限性很明显无法处理部分可观察环境也无法从经验中学习。2.2 基于模型的反射Agent通过维护内部状态Agent可以追踪感知历史class ModelBasedReflexAgent: def __init__(self, rules, state_update_fn): self.rules rules self.state_update state_update_fn self.state None def act(self, perception): self.state self.state_update(self.state, perception) for condition, action in self.rules.items(): if condition(self.state): return action return self.default_action2.3 基于目标的Agent引入目标概念后Agent可以评估不同行动对实现目标的贡献class GoalBasedAgent: def __init__(self, goal_test, actions): self.goal_test goal_test self.actions actions def plan(self, state): # 简单的广度优先搜索规划 from collections import deque queue deque([(state, [])]) visited set() while queue: current_state, path queue.popleft() if self.goal_test(current_state): return path if current_state in visited: continue visited.add(current_state) for action in self.actions: new_state self.result(current_state, action) queue.append((new_state, path [action])) return None3. 从理性到学习构建学习Agent3.1 学习Agent的架构学习Agent由四个关键组件构成性能元件选择外部行动评判元件评估Agent表现学习元件根据反馈改进问题产生器提出新的探索行动class LearningAgent: def __init__(self, performance_element, critic, learning_element, problem_generator): self.performance performance_element self.critic critic self.learner learning_element self.explorer problem_generator def run_episode(self, env): state env.reset() total_reward 0 while True: action self.performance.select_action(state) next_state, reward, done env.step(action) feedback self.critic.evaluate(state, action, reward, next_state) self.learner.update(feedback) if self.explorer.should_explore(): action self.explorer.suggest_action() state next_state total_reward reward if done: return total_reward3.2 强化学习实现Q学习是一种经典的强化学习算法适合实现学习Agentimport numpy as np class QLearningAgent: def __init__(self, state_space, action_space, alpha0.1, gamma0.9, epsilon0.1): self.q_table np.zeros((state_space, action_space)) self.alpha alpha # 学习率 self.gamma gamma # 折扣因子 self.epsilon epsilon # 探索率 def learn(self, state, action, reward, next_state): best_next_action np.argmax(self.q_table[next_state]) td_target reward self.gamma * self.q_table[next_state][best_next_action] td_error td_target - self.q_table[state][action] self.q_table[state][action] self.alpha * td_error def select_action(self, state): if np.random.random() self.epsilon: return np.random.randint(0, self.q_table.shape[1]) return np.argmax(self.q_table[state])4. 实战构建游戏AI智能体4.1 环境设置网格世界我们创建一个简单的网格世界环境来测试我们的Agentclass GridWorld: def __init__(self, size5, obstaclesNone, goal(4,4)): self.size size self.obstacles obstacles or [] self.goal goal self.state (0, 0) def reset(self): self.state (0, 0) return self.state def step(self, action): x, y self.state if action 0: # 上 y max(0, y-1) elif action 1: # 右 x min(self.size-1, x1) elif action 2: # 下 y min(self.size-1, y1) elif action 3: # 左 x max(0, x-1) if (x, y) in self.obstacles: return self.state, -10, False self.state (x, y) done (x, y) self.goal reward 10 if done else -1 return self.state, reward, done4.2 Agent训练与评估训练我们的学习Agent并评估其性能def train_agent(episodes1000): env GridWorld(obstacles[(1,1), (2,2), (3,3)]) agent QLearningAgent(state_space25, action_space4) rewards [] for episode in range(episodes): state env.reset() total_reward 0 done False while not done: action agent.select_action(state[0]*5 state[1]) next_state, reward, done env.step(action) agent.learn(state[0]*5 state[1], action, reward, next_state[0]*5 next_state[1]) state next_state total_reward reward rewards.append(total_reward) return agent, rewards4.3 可视化训练过程使用matplotlib可视化Agent的学习曲线import matplotlib.pyplot as plt def plot_learning_curve(rewards, window100): moving_avg [np.mean(rewards[i-window:i]) for i in range(window, len(rewards))] plt.plot(moving_avg) plt.xlabel(Episode) plt.ylabel(Average Reward) plt.title(Learning Curve (Moving Average)) plt.show()5. 进阶从单一Agent到多Agent系统5.1 多Agent环境建模在多Agent环境中每个Agent的行为会影响其他Agentclass MultiAgentEnvironment: def __init__(self, agents, size5): self.agents agents self.size size self.positions {agent: (0,0) for agent in agents} def step(self, actions): new_positions {} rewards {} # 先计算所有Agent的意向移动 for agent, action in actions.items(): x, y self.positions[agent] # 移动逻辑... new_positions[agent] (new_x, new_y) # 处理冲突 position_count {} for pos in new_positions.values(): position_count[pos] position_count.get(pos, 0) 1 # 更新位置并计算奖励 for agent in self.agents: new_pos new_positions[agent] if position_count[new_pos] 1: # 冲突 rewards[agent] -5 new_pos self.positions[agent] # 保持原位置 else: rewards[agent] -1 self.positions[agent] new_pos return self.positions, rewards5.2 竞争与合作策略在多Agent系统中Agent可以采取不同策略竞争策略最大化自身利益合作策略追求集体利益混合策略根据情境调整class CooperativeAgent(QLearningAgent): def __init__(self, *args, **kwargs): super().__init__(*args, **kwargs) self.other_agents [] def update_policy(self, joint_state, joint_action, rewards): # 考虑其他Agent的奖励 total_reward sum(rewards.values()) for agent, action in joint_action.items(): state joint_state[agent] self.learn(state, action, total_reward, joint_state[agent])6. 实际应用与优化技巧6.1 处理高维状态空间当状态空间很大时Q表不再适用可以使用函数近似from keras.models import Sequential from keras.layers import Dense class DQNAgent: def __init__(self, state_size, action_size): self.state_size state_size self.action_size action_size self.model self._build_model() def _build_model(self): model Sequential() model.add(Dense(24, input_dimself.state_size, activationrelu)) model.add(Dense(24, activationrelu)) model.add(Dense(self.action_size, activationlinear)) model.compile(lossmse, optimizeradam) return model def remember(self, state, action, reward, next_state, done): self.memory.append((state, action, reward, next_state, done)) def replay(self, batch_size): minibatch random.sample(self.memory, batch_size) for state, action, reward, next_state, done in minibatch: target reward if not done: target reward self.gamma * np.amax(self.model.predict(next_state)[0]) target_f self.model.predict(state) target_f[0][action] target self.model.fit(state, target_f, epochs1, verbose0)6.2 超参数调优Agent性能很大程度上取决于超参数选择参数影响典型值范围调整策略学习率(α)更新步长0.01-0.5从大到小衰减折扣因子(γ)未来奖励重要性0.9-0.99长期任务取高值探索率(ε)探索-利用权衡0.1-0.3随时间衰减class AdaptiveParameter: def __init__(self, initial_value, decay_rate, min_value0.01): self.value initial_value self.decay_rate decay_rate self.min_value min_value def decay(self): self.value max(self.min_value, self.value * self.decay_rate) return self.value7. 从理论到实践完整项目示例7.1 项目结构设计一个完整的Agent系统通常包含以下模块/agent_project │── /agents │ ├── base_agent.py │ ├── reflex_agent.py │ ├── learning_agent.py │── /environments │ ├── grid_world.py │ ├── maze.py │── /utils │ ├── visualizer.py │ ├── logger.py │── train.py │── evaluate.py7.2 训练流程实现# train.py from agents.learning_agent import DQNAgent from environments.grid_world import GridWorld from utils.visualizer import plot_learning_curve def main(): env GridWorld(size10, obstacles[(2,2),(3,5),(7,8)]) agent DQNAgent(state_size100, action_size4) batch_size 32 episodes 1000 for e in range(episodes): state env.reset() state np.reshape(state, [1, 100]) total_reward 0 for time in range(500): action agent.act(state) next_state, reward, done env.step(action) next_state np.reshape(next_state, [1, 100]) agent.remember(state, action, reward, next_state, done) state next_state total_reward reward if done: print(fEpisode: {e}/{episodes}, Score: {time}, Reward: {total_reward}) break if len(agent.memory) batch_size: agent.replay(batch_size) plot_learning_curve(agent.rewards)7.3 性能评估与部署评估Agent性能时需要考虑多个指标成功率完成任务的比例效率完成任务的平均步数稳健性在不同初始条件下的表现一致性适应性环境变化后的快速调整能力# evaluate.py def evaluate_agent(agent, env, trials100): results { success_rate: 0, average_steps: 0, total_reward: 0 } successful_trials 0 for _ in range(trials): state env.reset() done False steps 0 total_reward 0 while not done and steps 1000: action agent.act(state) state, reward, done env.step(action) total_reward reward steps 1 if done: successful_trials 1 results[average_steps] steps results[total_reward] total_reward results[success_rate] successful_trials / trials if successful_trials 0: results[average_steps] / successful_trials return results8. 前沿发展与未来方向8.1 深度强化学习进展近年来深度强化学习在Agent设计中取得显著进展DQN结合深度学习的Q学习A3C异步优势Actor-Critic算法PPO近端策略优化SAC柔性Actor-Critic# 使用Stable Baselines3实现PPO算法 from stable_baselines3 import PPO from stable_baselines3.common.env_util import make_vec_env env make_vec_env(GridWorld-v0, n_envs4) model PPO(MlpPolicy, env, verbose1) model.learn(total_timesteps100000) model.save(ppo_gridworld)8.2 多模态感知集成现代Agent系统正整合多种感知模态视觉处理CNN处理图像输入自然语言Transformer理解指令传感器融合整合激光雷达、IMU等数据class MultiModalAgent: def __init__(self): self.vision_model load_vision_model() self.nlp_model load_language_model() self.sensor_fusion SensorFusionNetwork() def perceive(self, observations): visual_feats self.vision_model(observations[image]) text_feats self.nlp_model(observations[text]) sensor_feats self.sensor_fusion(observations[sensors]) return torch.cat([visual_feats, text_feats, sensor_feats], dim-1)8.3 元学习与通用Agent元学习(Meta-Learning)使Agent能够学习如何学习class MetaLearner: def __init__(self, inner_lr0.1, outer_lr0.001): self.inner_lr inner_lr self.outer_lr outer_lr self.meta_optimizer torch.optim.Adam(self.parameters(), lrouter_lr) def adapt(self, task, support_set): fast_weights OrderedDict(self.named_parameters()) for x, y in support_set: logits self.forward(x, fast_weights) loss F.cross_entropy(logits, y) grads torch.autograd.grad(loss, fast_weights.values(), create_graphTrue) fast_weights OrderedDict( (name, param - self.inner_lr * grad) for (name, param), grad in zip(fast_weights.items(), grads) ) return fast_weights def meta_update(self, tasks): meta_loss 0 for task in tasks: support_set, query_set task fast_weights self.adapt(task, support_set) for x, y in query_set: logits self.forward(x, fast_weights) meta_loss F.cross_entropy(logits, y) self.meta_optimizer.zero_grad() meta_loss.backward() self.meta_optimizer.step()
从‘理性Agent’到‘学习Agent’：用Python一步步构建你的第一个AI智能体
发布时间：2026/5/22 21:20:22
从‘理性Agent’到‘学习Agent’用Python一步步构建你的第一个AI智能体在人工智能领域智能体(Agent)是最基础也最核心的概念之一。一个智能体可以简单理解为能够感知环境并采取行动以实现目标的实体。从最简单的自动调温器到复杂的自动驾驶系统智能体的设计理念贯穿了AI应用的各个层面。本文将带你从零开始用Python构建一个能够不断进化的智能体系统。1. 智能体基础与环境建模1.1 理性Agent的核心概念理性Agent是指在给定感知序列和先验知识的情况下能够采取使其性能度量最大化的行动的Agent。要构建一个理性Agent我们需要明确几个关键组件class RationalAgent: def __init__(self, performance_measure, environment, actuators, sensors): self.performance performance_measure self.env environment self.actuators actuators self.sensors sensorsPEAS模型是描述Agent任务环境的标准框架组件描述示例(扫地机器人)Performance衡量Agent成功程度的标准清洁面积百分比EnvironmentAgent运作的上下文房间布局、家具位置ActuatorsAgent执行动作的机制移动轮子、吸尘装置SensorsAgent感知环境的方式灰尘传感器、碰撞检测1.2 环境性质分析在设计Agent前我们需要分析环境的性质可观察性完全可观察 vs 部分可观察确定性确定性的 vs 随机的连续性离散的 vs 连续的动态性静态的 vs 动态的以下代码展示了如何用Python模拟不同环境性质class Environment: def __init__(self, observableTrue, deterministicTrue, discreteTrue, staticTrue): self.observable observable self.deterministic deterministic self.discrete discrete self.static static def get_state(self): if self.observable: return self.current_state else: return self._partial_observation()2. Agent架构的演进路径2.1 简单反射Agent最简单的Agent形式是基于当前感知直接选择行动不考虑历史信息class SimpleReflexAgent: def __init__(self, rules): self.rules rules # 条件-动作规则集 def act(self, perception): for condition, action in self.rules.items(): if condition(perception): return action return self.default_action这种Agent的局限性很明显无法处理部分可观察环境也无法从经验中学习。2.2 基于模型的反射Agent通过维护内部状态Agent可以追踪感知历史class ModelBasedReflexAgent: def __init__(self, rules, state_update_fn): self.rules rules self.state_update state_update_fn self.state None def act(self, perception): self.state self.state_update(self.state, perception) for condition, action in self.rules.items(): if condition(self.state): return action return self.default_action2.3 基于目标的Agent引入目标概念后Agent可以评估不同行动对实现目标的贡献class GoalBasedAgent: def __init__(self, goal_test, actions): self.goal_test goal_test self.actions actions def plan(self, state): # 简单的广度优先搜索规划 from collections import deque queue deque([(state, [])]) visited set() while queue: current_state, path queue.popleft() if self.goal_test(current_state): return path if current_state in visited: continue visited.add(current_state) for action in self.actions: new_state self.result(current_state, action) queue.append((new_state, path [action])) return None3. 从理性到学习构建学习Agent3.1 学习Agent的架构学习Agent由四个关键组件构成性能元件选择外部行动评判元件评估Agent表现学习元件根据反馈改进问题产生器提出新的探索行动class LearningAgent: def __init__(self, performance_element, critic, learning_element, problem_generator): self.performance performance_element self.critic critic self.learner learning_element self.explorer problem_generator def run_episode(self, env): state env.reset() total_reward 0 while True: action self.performance.select_action(state) next_state, reward, done env.step(action) feedback self.critic.evaluate(state, action, reward, next_state) self.learner.update(feedback) if self.explorer.should_explore(): action self.explorer.suggest_action() state next_state total_reward reward if done: return total_reward3.2 强化学习实现Q学习是一种经典的强化学习算法适合实现学习Agentimport numpy as np class QLearningAgent: def __init__(self, state_space, action_space, alpha0.1, gamma0.9, epsilon0.1): self.q_table np.zeros((state_space, action_space)) self.alpha alpha # 学习率 self.gamma gamma # 折扣因子 self.epsilon epsilon # 探索率 def learn(self, state, action, reward, next_state): best_next_action np.argmax(self.q_table[next_state]) td_target reward self.gamma * self.q_table[next_state][best_next_action] td_error td_target - self.q_table[state][action] self.q_table[state][action] self.alpha * td_error def select_action(self, state): if np.random.random() self.epsilon: return np.random.randint(0, self.q_table.shape[1]) return np.argmax(self.q_table[state])4. 实战构建游戏AI智能体4.1 环境设置网格世界我们创建一个简单的网格世界环境来测试我们的Agentclass GridWorld: def __init__(self, size5, obstaclesNone, goal(4,4)): self.size size self.obstacles obstacles or [] self.goal goal self.state (0, 0) def reset(self): self.state (0, 0) return self.state def step(self, action): x, y self.state if action 0: # 上 y max(0, y-1) elif action 1: # 右 x min(self.size-1, x1) elif action 2: # 下 y min(self.size-1, y1) elif action 3: # 左 x max(0, x-1) if (x, y) in self.obstacles: return self.state, -10, False self.state (x, y) done (x, y) self.goal reward 10 if done else -1 return self.state, reward, done4.2 Agent训练与评估训练我们的学习Agent并评估其性能def train_agent(episodes1000): env GridWorld(obstacles[(1,1), (2,2), (3,3)]) agent QLearningAgent(state_space25, action_space4) rewards [] for episode in range(episodes): state env.reset() total_reward 0 done False while not done: action agent.select_action(state[0]*5 state[1]) next_state, reward, done env.step(action) agent.learn(state[0]*5 state[1], action, reward, next_state[0]*5 next_state[1]) state next_state total_reward reward rewards.append(total_reward) return agent, rewards4.3 可视化训练过程使用matplotlib可视化Agent的学习曲线import matplotlib.pyplot as plt def plot_learning_curve(rewards, window100): moving_avg [np.mean(rewards[i-window:i]) for i in range(window, len(rewards))] plt.plot(moving_avg) plt.xlabel(Episode) plt.ylabel(Average Reward) plt.title(Learning Curve (Moving Average)) plt.show()5. 进阶从单一Agent到多Agent系统5.1 多Agent环境建模在多Agent环境中每个Agent的行为会影响其他Agentclass MultiAgentEnvironment: def __init__(self, agents, size5): self.agents agents self.size size self.positions {agent: (0,0) for agent in agents} def step(self, actions): new_positions {} rewards {} # 先计算所有Agent的意向移动 for agent, action in actions.items(): x, y self.positions[agent] # 移动逻辑... new_positions[agent] (new_x, new_y) # 处理冲突 position_count {} for pos in new_positions.values(): position_count[pos] position_count.get(pos, 0) 1 # 更新位置并计算奖励 for agent in self.agents: new_pos new_positions[agent] if position_count[new_pos] 1: # 冲突 rewards[agent] -5 new_pos self.positions[agent] # 保持原位置 else: rewards[agent] -1 self.positions[agent] new_pos return self.positions, rewards5.2 竞争与合作策略在多Agent系统中Agent可以采取不同策略竞争策略最大化自身利益合作策略追求集体利益混合策略根据情境调整class CooperativeAgent(QLearningAgent): def __init__(self, *args, **kwargs): super().__init__(*args, **kwargs) self.other_agents [] def update_policy(self, joint_state, joint_action, rewards): # 考虑其他Agent的奖励 total_reward sum(rewards.values()) for agent, action in joint_action.items(): state joint_state[agent] self.learn(state, action, total_reward, joint_state[agent])6. 实际应用与优化技巧6.1 处理高维状态空间当状态空间很大时Q表不再适用可以使用函数近似from keras.models import Sequential from keras.layers import Dense class DQNAgent: def __init__(self, state_size, action_size): self.state_size state_size self.action_size action_size self.model self._build_model() def _build_model(self): model Sequential() model.add(Dense(24, input_dimself.state_size, activationrelu)) model.add(Dense(24, activationrelu)) model.add(Dense(self.action_size, activationlinear)) model.compile(lossmse, optimizeradam) return model def remember(self, state, action, reward, next_state, done): self.memory.append((state, action, reward, next_state, done)) def replay(self, batch_size): minibatch random.sample(self.memory, batch_size) for state, action, reward, next_state, done in minibatch: target reward if not done: target reward self.gamma * np.amax(self.model.predict(next_state)[0]) target_f self.model.predict(state) target_f[0][action] target self.model.fit(state, target_f, epochs1, verbose0)6.2 超参数调优Agent性能很大程度上取决于超参数选择参数影响典型值范围调整策略学习率(α)更新步长0.01-0.5从大到小衰减折扣因子(γ)未来奖励重要性0.9-0.99长期任务取高值探索率(ε)探索-利用权衡0.1-0.3随时间衰减class AdaptiveParameter: def __init__(self, initial_value, decay_rate, min_value0.01): self.value initial_value self.decay_rate decay_rate self.min_value min_value def decay(self): self.value max(self.min_value, self.value * self.decay_rate) return self.value7. 从理论到实践完整项目示例7.1 项目结构设计一个完整的Agent系统通常包含以下模块/agent_project │── /agents │ ├── base_agent.py │ ├── reflex_agent.py │ ├── learning_agent.py │── /environments │ ├── grid_world.py │ ├── maze.py │── /utils │ ├── visualizer.py │ ├── logger.py │── train.py │── evaluate.py7.2 训练流程实现# train.py from agents.learning_agent import DQNAgent from environments.grid_world import GridWorld from utils.visualizer import plot_learning_curve def main(): env GridWorld(size10, obstacles[(2,2),(3,5),(7,8)]) agent DQNAgent(state_size100, action_size4) batch_size 32 episodes 1000 for e in range(episodes): state env.reset() state np.reshape(state, [1, 100]) total_reward 0 for time in range(500): action agent.act(state) next_state, reward, done env.step(action) next_state np.reshape(next_state, [1, 100]) agent.remember(state, action, reward, next_state, done) state next_state total_reward reward if done: print(fEpisode: {e}/{episodes}, Score: {time}, Reward: {total_reward}) break if len(agent.memory) batch_size: agent.replay(batch_size) plot_learning_curve(agent.rewards)7.3 性能评估与部署评估Agent性能时需要考虑多个指标成功率完成任务的比例效率完成任务的平均步数稳健性在不同初始条件下的表现一致性适应性环境变化后的快速调整能力# evaluate.py def evaluate_agent(agent, env, trials100): results { success_rate: 0, average_steps: 0, total_reward: 0 } successful_trials 0 for _ in range(trials): state env.reset() done False steps 0 total_reward 0 while not done and steps 1000: action agent.act(state) state, reward, done env.step(action) total_reward reward steps 1 if done: successful_trials 1 results[average_steps] steps results[total_reward] total_reward results[success_rate] successful_trials / trials if successful_trials 0: results[average_steps] / successful_trials return results8. 前沿发展与未来方向8.1 深度强化学习进展近年来深度强化学习在Agent设计中取得显著进展DQN结合深度学习的Q学习A3C异步优势Actor-Critic算法PPO近端策略优化SAC柔性Actor-Critic# 使用Stable Baselines3实现PPO算法 from stable_baselines3 import PPO from stable_baselines3.common.env_util import make_vec_env env make_vec_env(GridWorld-v0, n_envs4) model PPO(MlpPolicy, env, verbose1) model.learn(total_timesteps100000) model.save(ppo_gridworld)8.2 多模态感知集成现代Agent系统正整合多种感知模态视觉处理CNN处理图像输入自然语言Transformer理解指令传感器融合整合激光雷达、IMU等数据class MultiModalAgent: def __init__(self): self.vision_model load_vision_model() self.nlp_model load_language_model() self.sensor_fusion SensorFusionNetwork() def perceive(self, observations): visual_feats self.vision_model(observations[image]) text_feats self.nlp_model(observations[text]) sensor_feats self.sensor_fusion(observations[sensors]) return torch.cat([visual_feats, text_feats, sensor_feats], dim-1)8.3 元学习与通用Agent元学习(Meta-Learning)使Agent能够学习如何学习class MetaLearner: def __init__(self, inner_lr0.1, outer_lr0.001): self.inner_lr inner_lr self.outer_lr outer_lr self.meta_optimizer torch.optim.Adam(self.parameters(), lrouter_lr) def adapt(self, task, support_set): fast_weights OrderedDict(self.named_parameters()) for x, y in support_set: logits self.forward(x, fast_weights) loss F.cross_entropy(logits, y) grads torch.autograd.grad(loss, fast_weights.values(), create_graphTrue) fast_weights OrderedDict( (name, param - self.inner_lr * grad) for (name, param), grad in zip(fast_weights.items(), grads) ) return fast_weights def meta_update(self, tasks): meta_loss 0 for task in tasks: support_set, query_set task fast_weights self.adapt(task, support_set) for x, y in query_set: logits self.forward(x, fast_weights) meta_loss F.cross_entropy(logits, y) self.meta_optimizer.zero_grad() meta_loss.backward() self.meta_optimizer.step()
