Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Value error when running DQN.fit #388

Open
GravermanDev opened this issue Apr 16, 2022 · 2 comments
Open

Value error when running DQN.fit #388

GravermanDev opened this issue Apr 16, 2022 · 2 comments

Comments

@GravermanDev
Copy link

GravermanDev commented Apr 16, 2022

I tried teaching AI how to play breakout but my code crashes when I try to teach DQN model.
``
import gym
import numpy as np
import tensorflow as tf
from rl.agents.dqn import DQNAgent
from rl.policy import LinearAnnealedPolicy, EpsGreedyQPolicy
from rl.memory import SequentialMemory
from keras.layers import Dense, Flatten, Convolution2D

env = gym.make('ALE/Breakout-v5', render_mode='rgb_array')
height, width, channels = env.observation_space.shape
actions = env.action_space.n

episodes = 10
for episode in range(1, episodes + 1):
env.reset()
done = False
score = 0

def buildModel(height, width, channels, actions):
model = tf.keras.Sequential()
model.add(Convolution2D(32, (8, 8), strides=(4, 4), activation='relu', input_shape=(3,height, width, channels)))
model.add(Convolution2D(64, (4, 4), strides=(2, 2), activation='relu'))
model.add(Convolution2D(64, (3, 3), activation='relu'))
model.add(Flatten())
model.add(Dense(512, activation='relu'))
model.add(Dense(256, activation='relu'))
model.add(Dense(actions, activation='linear'))
return model

def buildAgent(model, actions):
policy = LinearAnnealedPolicy(EpsGreedyQPolicy(), attr='eps', value_max=1., value_min=.1, value_test=.2, nb_steps=10000)
memory = SequentialMemory(limit=1000, window_length=3)
dqn = DQNAgent(model=model, memory=memory, policy=policy,
enable_dueling_network=True, dueling_type='avg',
nb_actions=actions, nb_steps_warmup=1000)
return dqn

model = buildModel(height, width, channels, actions)

DQN = buildAgent(model, actions)
DQN.compile(tf.keras.optimizers.Adam(learning_rate=1e-4), metrics=['mae'])
DQN.fit(env, nb_steps=1000000, visualize=True, verbose=1)

scores = DQN.test(env, nb_episodes=1000, visualize=True)
print(np.mean(scores.history['episode_reward']))
``

Error: ValueError: The truth value of an array with more than one element is ambiguous. Use a.any() or a.all()

@JasonXian
Copy link

JasonXian commented Jul 7, 2022

I had the same issue and traced the error to if not np.isreal(value): on rl/core.py.

This section checks each value in the info object, but seems to break when given arrays. I'm not sure if it was an issue with using arrays in general or the values that I used.

However, my use case didn't require me to pass arrays into the info object, so I simply removed them and the issue went away.

@R-Liebert
Copy link

I'm also getting error trying dqn.fit. ValueError: setting an array element with a sequence. It seems like it comes keras/backend.py. But this repo seems dead so I doubt there is any hopes for fixes.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants