Conv2DCustomBackpropInputOp only supports NHWC. #42
Comments
|
It looks like tensorflow is mapping some ops to the CPU. Unfortunately, tensorflow still does not support NCHW for some ops. |
|
Having the same problem, but changing batch size does not change anything. |
|
can't work on ubuntu16.04 and tf1.15 |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Hi, when I run training.py, I got such error message, any idea please.
by the way, the tensorflow version is 1.4.0, not compiled.
<module 'depthmotionnet.datareader' from '/home/kylin/demon/python/depthmotionnet/datareader/init.py'>
Tensor("tower_0/netFlow1/dense5/LeakyRelu:0", shape=(32, 4608), dtype=float32, device=/device:GPU:0)
nothing to restore. no checkpoint found.
train evolution "0_flow1" ...
start iteration 0
2018-05-17 19:37:59.029835: E tensorflow/core/common_runtime/executor.cc:643] Executor failed to create kernel. Invalid argument: Conv2DCustomBackpropInputOp only supports NHWC.
[[Node: tower_0/netFlow1/refine4/upconv/conv2d_transpose = Conv2DBackpropInput[T=DT_FLOAT, data_format="NCHW", padding="SAME", strides=[1, 1, 2, 2], use_cudnn_on_gpu=true, _device="/job:localhost/replica:0/task:0/device:CPU:0"](tower_0/gradients/tower_0/netFlow1/refine4/concat_grad/Shape, netFlow1/refine4/upconv/kernel/read, tower_0/netFlow1/concat)]]
stop iteration 0
Traceback (most recent call last):
File "/home/kylin/.local/share/virtualenvs/demon_venv/lib/python3.5/site-packages/tensorflow/python/client/session.py", line 1323, in _do_call
return fn(*args)
File "/home/kylin/.local/share/virtualenvs/demon_venv/lib/python3.5/site-packages/tensorflow/python/client/session.py", line 1302, in _run_fn
status, run_metadata)
File "/home/kylin/.local/share/virtualenvs/demon_venv/lib/python3.5/site-packages/tensorflow/python/framework/errors_impl.py", line 473, in exit
c_api.TF_GetCode(self.status.status))
tensorflow.python.framework.errors_impl.InvalidArgumentError: Conv2DCustomBackpropInputOp only supports NHWC.
[[Node: tower_0/netFlow1/refine4/upconv/conv2d_transpose = Conv2DBackpropInput[T=DT_FLOAT, data_format="NCHW", padding="SAME", strides=[1, 1, 2, 2], use_cudnn_on_gpu=true, _device="/job:localhost/replica:0/task:0/device:CPU:0"](tower_0/gradients/tower_0/netFlow1/refine4/concat_grad/Shape, netFlow1/refine4/upconv/kernel/read, tower_0/netFlow1/concat)]]
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "training.py", line 604, in
tf.app.run()
File "/home/kylin/.local/share/virtualenvs/demon_venv/lib/python3.5/site-packages/tensorflow/python/platform/app.py", line 48, in run
_sys.exit(main(_sys.argv[:1] + flags_passthrough))
File "training.py", line 598, in main
trainer.coordinator().raise_requested_exception()
File "/home/kylin/.local/share/virtualenvs/demon_venv/lib/python3.5/site-packages/tensorflow/python/training/coordinator.py", line 407, in raise_requested_exception
six.reraise(*self._exc_info_to_raise)
File "/home/kylin/.local/share/virtualenvs/demon_venv/lib/python3.5/site-packages/six.py", line 693, in reraise
raise value
File "/home/kylin/.local/share/virtualenvs/demon_venv/lib/python3.5/site-packages/tensorflow/python/training/coordinator.py", line 296, in stop_on_exception
yield
File "/home/kylin/Applications/tfutils/python/tfutils/simpletrainer.py", line 280, in mainloop
values = self._session.run(ops, options=run_options, run_metadata=run_metadata)
File "/home/kylin/.local/share/virtualenvs/demon_venv/lib/python3.5/site-packages/tensorflow/python/client/session.py", line 889, in run
run_metadata_ptr)
File "/home/kylin/.local/share/virtualenvs/demon_venv/lib/python3.5/site-packages/tensorflow/python/client/session.py", line 1120, in _run
feed_dict_tensor, options, run_metadata)
File "/home/kylin/.local/share/virtualenvs/demon_venv/lib/python3.5/site-packages/tensorflow/python/client/session.py", line 1317, in _do_run
options, run_metadata)
File "/home/kylin/.local/share/virtualenvs/demon_venv/lib/python3.5/site-packages/tensorflow/python/client/session.py", line 1336, in _do_call
raise type(e)(node_def, op, message)
tensorflow.python.framework.errors_impl.InvalidArgumentError: Conv2DCustomBackpropInputOp only supports NHWC.
[[Node: tower_0/netFlow1/refine4/upconv/conv2d_transpose = Conv2DBackpropInput[T=DT_FLOAT, data_format="NCHW", padding="SAME", strides=[1, 1, 2, 2], use_cudnn_on_gpu=true, _device="/job:localhost/replica:0/task:0/device:CPU:0"](tower_0/gradients/tower_0/netFlow1/refine4/concat_grad/Shape, netFlow1/refine4/upconv/kernel/read, tower_0/netFlow1/concat)]]
Caused by op 'tower_0/netFlow1/refine4/upconv/conv2d_transpose', defined at:
File "training.py", line 604, in
tf.app.run()
File "/home/kylin/.local/share/virtualenvs/demon_venv/lib/python3.5/site-packages/tensorflow/python/platform/app.py", line 48, in run
_sys.exit(main(_sys.argv[:1] + flags_passthrough))
File "training.py", line 193, in main
kernel_regularizer=_kernel_regularizer,
File "/home/kylin/demon/python/depthmotionnet/v2/blocks.py", line 231, in flow_block
**conv_params,
File "/home/kylin/demon/python/depthmotionnet/v2/blocks.py", line 108, in _refine
**kwargs,
File "/home/kylin/.local/share/virtualenvs/demon_venv/lib/python3.5/site-packages/tensorflow/python/layers/convolutional.py", line 1423, in conv2d_transpose
return layer.apply(inputs)
File "/home/kylin/.local/share/virtualenvs/demon_venv/lib/python3.5/site-packages/tensorflow/python/layers/base.py", line 671, in apply
return self.call(inputs, *args, **kwargs)
File "/home/kylin/.local/share/virtualenvs/demon_venv/lib/python3.5/site-packages/tensorflow/python/layers/base.py", line 575, in call
outputs = self.call(inputs, *args, **kwargs)
File "/home/kylin/.local/share/virtualenvs/demon_venv/lib/python3.5/site-packages/tensorflow/python/layers/convolutional.py", line 1289, in call
data_format=utils.convert_data_format(self.data_format, ndim=4))
File "/home/kylin/.local/share/virtualenvs/demon_venv/lib/python3.5/site-packages/tensorflow/python/ops/nn_ops.py", line 1211, in conv2d_transpose
name=name)
File "/home/kylin/.local/share/virtualenvs/demon_venv/lib/python3.5/site-packages/tensorflow/python/ops/gen_nn_ops.py", line 779, in conv2d_backprop_input
use_cudnn_on_gpu=use_cudnn_on_gpu, data_format=data_format, name=name)
File "/home/kylin/.local/share/virtualenvs/demon_venv/lib/python3.5/site-packages/tensorflow/python/framework/op_def_library.py", line 787, in _apply_op_helper
op_def=op_def)
File "/home/kylin/.local/share/virtualenvs/demon_venv/lib/python3.5/site-packages/tensorflow/python/framework/ops.py", line 2956, in create_op
op_def=op_def)
File "/home/kylin/.local/share/virtualenvs/demon_venv/lib/python3.5/site-packages/tensorflow/python/framework/ops.py", line 1470, in init
self._traceback = self._graph._extract_stack() # pylint: disable=protected-access
InvalidArgumentError (see above for traceback): Conv2DCustomBackpropInputOp only supports NHWC.
[[Node: tower_0/netFlow1/refine4/upconv/conv2d_transpose = Conv2DBackpropInput[T=DT_FLOAT, data_format="NCHW", padding="SAME", strides=[1, 1, 2, 2], use_cudnn_on_gpu=true, _device="/job:localhost/replica:0/task:0/device:CPU:0"](tower_0/gradients/tower_0/netFlow1/refine4/concat_grad/Shape, netFlow1/refine4/upconv/kernel/read, tower_0/netFlow1/concat)]]
The text was updated successfully, but these errors were encountered: