Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

InvalidArgumentError: Graph execution error: #20006

Closed
iwqculrbud opened this issue Jul 17, 2024 · 10 comments
Closed

InvalidArgumentError: Graph execution error: #20006

iwqculrbud opened this issue Jul 17, 2024 · 10 comments
Assignees
@mehtamansi29
Labels
stale stat:awaiting response from contributor type:Bug

Comments

@iwqculrbud
Copy link

Can anyone help me

import keras
from keras import layers
import matplotlib.pyplot as plt
from keras.utils import image_dataset_from_directory
​
epochs = 30
image_size = (373, 454)
​
train_ds, val_ds = image_dataset_from_directory("/kaggle/input/fractured-images/images",
validation_split=0.4,
subset="both",
image_size=image_size,
seed=1337,
batch_size=128)
​
​
def model_function(shape):
inputs = keras.Input(shape)
w = layers.Rescaling(1.0 /255)(inputs)
for size in [256, 512, 728, 1024]:
w = layers.Conv2D(size, 3, activation="relu")(w)
w = layers.MaxPooling2D(pool_size=2)(w)
w = layers.Conv2D(1024, 3, activation="relu")(w)
w = layers.Flatten()(w)
outputs = layers.Dense(1, activation="sigmoid")(w)
return keras.Model(inputs=inputs, outputs=outputs)
​
model = model_function(shape=image_size + (3,))
model.summary()
keras.utils.plot_model(model,
f"fractures_test_{epochs}1.png",
show_shapes=True)
​
callbacks = [
keras.callbacks.ModelCheckpoint(
filepath=f"fractured_model_test{epochs}_1.keras",
save_best_only=True,
monitor="val_loss"
)
]
​
model.compile(
loss="binary_crossentropy",
optimizer="rmsprop",
metrics=["accuracy"]
)
​
history = model.fit(
train_ds,
epochs=epochs,
validation_data=val_ds,
callbacks=callbacks
)

2024-07-17 16:02:44.093403: E tensorflow/core/lib/jpeg/jpeg_mem.cc:327] Premature end of JPEG data. Stopped at line 382/454
2024-07-17 16:02:44.133631: E tensorflow/core/lib/jpeg/jpeg_mem.cc:327] Premature end of JPEG data. Stopped at line 430/454

InvalidArgumentError Traceback (most recent call last)
Cell In[16], line 48
34 callbacks = [
35 keras.callbacks.ModelCheckpoint(
36 filepath=f"fractured_model_test_{epochs}_1.keras",
(...)
39 )
40 ]
42 model.compile(
43 loss="binary_crossentropy",
44 optimizer="rmsprop",
45 metrics=["accuracy"]
46 )
---> 48 history = model.fit(
49 train_ds,
50 epochs=epochs,
51 validation_data=val_ds,
52 callbacks=callbacks
53 )
55 history_dict = history.history
56 loss_valus = history_dict["loss"]

File /opt/conda/lib/python3.10/site-packages/keras/src/utils/traceback_utils.py:122, in filter_traceback..error_handler(*args, **kwargs)
119 filtered_tb = _process_traceback_frames(e.traceback)
120 # To get the full stack trace, call:
121 # keras.config.disable_traceback_filtering()
--> 122 raise e.with_traceback(filtered_tb) from None
123 finally:
124 del filtered_tb

File /opt/conda/lib/python3.10/site-packages/tensorflow/python/eager/execute.py:53, in quick_execute(op_name, num_outputs, inputs, attrs, ctx, name)
51 try:
52 ctx.ensure_initialized()
---> 53 tensors = pywrap_tfe.TFE_Py_Execute(ctx._handle, device_name, op_name,
54 inputs, attrs, num_outputs)
55 except core._NotOkStatusException as e:
56 if name is not None:

InvalidArgumentError: Graph execution error:

Detected at node decode_image/DecodeImage defined at (most recent call last):

jpeg::Uncompress failed. Invalid JPEG data or crop window.
[[{{node decode_image/DecodeImage}}]]
[[IteratorGetNext]] [Op:__inference_one_step_on_iterator_11223]
2024-07-17 16:02:44.259599: E tensorflow/core/lib/jpeg/jpeg_mem.cc:327] Premature end of JPEG data. Stopped at line 446/454
2024-07-17 16:02:44.441775: E tensorflow/core/lib/jpeg/jpeg_mem.cc:327] Premature end of JPEG data. Stopped at line 350/454
@mehtamansi29
Copy link
Collaborator

Hi @iwqculrbud -

Thanks for reporting this issue. Currently I am to find the dataset directory to reproduce this issue. If possible Can you help me to with your dataset? Thanks..!!

@iwqculrbud
Copy link
Author

https://kaggle.com/datasets/e26fb9d642838aaf211ab8e41238df185bb60f6199ce07eafffae86085759d89
This is an alternate address,thank you!!!

@mehtamansi29
Copy link
Collaborator

Hi @iwqculrbud -

Thanks for the dataset. Actually "InvalidArgumentError: Graph execution error" error cause by there were corrupt images are there in your dataset. To resolve this error remove corrupt images and apply some preprocessing will resolve this error.

Here is the solution you can try to create train_ds and val_ds and then train your model:

image_paths=[]
labels=[]
for root, dirs, filenames in os.walk(files_dir):
    for filename in filenames:
        if filename.lower().endswith((".jpg", ".jpeg")):
            if not filename.startswith('.'):
                image_path = os.path.join(root, filename)
                img = cv2.imread(image_path)
                if img is not None:
                    image_paths.append(image_path) 
                    labels.append(os.path.basename(root))

def decode_image(image_path, label):
    img = keras.utils.load_img(image_path, target_size=(224, 224))
    img = keras.utils.img_to_array(img)
    img = tf.cast(img, tf.float32) / 255.0
    return img, label

def decode_image_wrapper(image_path, label):
    img, label = tf.numpy_function(decode_image, [image_path, label], [tf.float32, tf.string])
    img.set_shape((224, 224, 3))
    label.set_shape(())
    return img, label

image_paths_ds = tf.data.Dataset.from_tensor_slices(image_paths)
labels_ds = tf.data.Dataset.from_tensor_slices(labels)
dataset = tf.data.Dataset.zip((image_paths_ds, labels_ds))
dataset = dataset.map(decode_image_wrapper, num_parallel_calls=tf.data.AUTOTUNE)
train_size = int(0.8 * len(image_paths))
val_size = len(image_paths) - train_size
train_ds = dataset.take(train_size)
val_ds = dataset.skip(train_size)

@iwqculrbud
Copy link
Author

你好 -

感谢您的数据集。实际上，“InvalidArgumentError： Graph execution error”错误是由数据集中存在损坏的图像引起的。要解决此错误，请删除损坏的图像并应用一些预处理将解决此错误。

下面是您可以尝试创建train_ds和val_ds然后训练模型的解决方案：

image_paths=[]
labels=[]
for root, dirs, filenames in os.walk(files_dir):
    for filename in filenames:
        if filename.lower().endswith((".jpg", ".jpeg")):
            if not filename.startswith('.'):
                image_path = os.path.join(root, filename)
                img = cv2.imread(image_path)
                if img is not None:
                    image_paths.append(image_path) 
                    labels.append(os.path.basename(root))

def decode_image(image_path, label):
    img = keras.utils.load_img(image_path, target_size=(224, 224))
    img = keras.utils.img_to_array(img)
    img = tf.cast(img, tf.float32) / 255.0
    return img, label

def decode_image_wrapper(image_path, label):
    img, label = tf.numpy_function(decode_image, [image_path, label], [tf.float32, tf.string])
    img.set_shape((224, 224, 3))
    label.set_shape(())
    return img, label

image_paths_ds = tf.data.Dataset.from_tensor_slices(image_paths)
labels_ds = tf.data.Dataset.from_tensor_slices(labels)
dataset = tf.data.Dataset.zip((image_paths_ds, labels_ds))
dataset = dataset.map(decode_image_wrapper, num_parallel_calls=tf.data.AUTOTUNE)
train_size = int(0.8 * len(image_paths))
val_size = len(image_paths) - train_size
train_ds = dataset.take(train_size)
val_ds = dataset.skip(train_size)

hello

This doesn't work for me, thank you

@mehtamansi29
Copy link
Collaborator

Hi @iwqculrbud -

Your dataset images are corrupted hence, we need to do preprocessing for corrupted images. You can find attached gist with detailed solution.

@github-actions GitHub Actions
Copy link

github-actions bot commented Aug 6, 2024

This issue is stale because it has been open for 14 days with no activity. It will be closed if no further activity occurs. Thank you.

@github-actions github-actions bot added the stale label Aug 6, 2024
@mehtamansi29
Copy link
Collaborator

Hi @iwqculrbud -

Does the issue is still reproduce to you ?

@github-actions GitHub Actions
Copy link

This issue is stale because it has been open for 14 days with no activity. It will be closed if no further activity occurs. Thank you.

@github-actions github-actions bot added the stale label Dec 11, 2024
@github-actions GitHub Actions
Copy link

This issue was closed because it has been inactive for 28 days. Please reopen if you'd like to work on this further.

@google-ml-butler Google-ML-Butler
Copy link

Are you satisfied with the resolution of your issue?
Yes
No

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@mehtamansi29 mehtamansi29

Labels
stale stat:awaiting response from contributor type:Bug
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@iwqculrbud @mehtamansi29 @sonali-kumari1