Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

new pipeline woes with tcp output #4445

Closed
jsvd opened this issue Jan 11, 2016 · 1 comment
Closed

new pipeline woes with tcp output #4445

jsvd opened this issue Jan 11, 2016 · 1 comment
Labels

Comments

@jsvd
Copy link
Member

jsvd commented Jan 11, 2016

input { generator { count => 100000 } }
#filter { ruby { code => 'sleep 5' } }
output {
  tcp {
    port => 3333
    host => localhost
    codec => plain { format => '%{sequence}
' }
  }
}

and nc -l 3333 I get:

/tmp/logstash-2.2.0.snapshot2 % bin/logstash -f config
{:pipeline_workers=>4, :pipeline_batch_size=>125, :pipeline_batch_delay=>5, :pipeline_id=>"base"}
Settings: User set pipeline workers: 4
Logstash startup completed
tcp output exception {:host=>"localhost", :port=>3333, :exception=>#<ConcurrencyError: can not set IO blocking after select; concurrent select detected?>, :backtrace=>["org/jruby/RubyIO.java:3682:in `select'", "/tmp/logstash-2.2.0.snapshot2/vendor/bundle/jruby/1.9/gems/logstash-output-tcp-2.0.2/lib/logstash/outputs/tcp.rb:101:in `register'", "org/jruby/RubyProc.java:281:in `call'", "/tmp/logstash-2.2.0.snapshot2/vendor/bundle/jruby/1.9/gems/logstash-codec-plain-2.0.2/lib/logstash/codecs/plain.rb:41:in `encode'", "/tmp/logstash-2.2.0.snapshot2/vendor/bundle/jruby/1.9/gems/logstash-output-tcp-2.0.2/lib/logstash/outputs/tcp.rb:143:in `receive'", "/tmp/logstash-2.2.0.snapshot2/vendor/bundle/jruby/1.9/gems/logstash-core-2.2.0.snapshot2-java/lib/logstash/outputs/base.rb:83:in `multi_receive'", "org/jruby/RubyArray.java:1613:in `each'", "/tmp/logstash-2.2.0.snapshot2/vendor/bundle/jruby/1.9/gems/logstash-core-2.2.0.snapshot2-java/lib/logstash/outputs/base.rb:83:in `multi_receive'", "/tmp/logstash-2.2.0.snapshot2/vendor/bundle/jruby/1.9/gems/logstash-core-2.2.0.snapshot2-java/lib/logstash/output_delegator.rb:119:in `worker_multi_receive'", "/tmp/logstash-2.2.0.snapshot2/vendor/bundle/jruby/1.9/gems/logstash-core-2.2.0.snapshot2-java/lib/logstash/output_delegator.rb:65:in `multi_receive'", "/tmp/logstash-2.2.0.snapshot2/vendor/bundle/jruby/1.9/gems/logstash-core-2.2.0.snapshot2-java/lib/logstash/pipeline.rb:277:in `output_batch'", "org/jruby/RubyHash.java:1342:in `each'", "/tmp/logstash-2.2.0.snapshot2/vendor/bundle/jruby/1.9/gems/logstash-core-2.2.0.snapshot2-java/lib/logstash/pipeline.rb:277:in `output_batch'", "/tmp/logstash-2.2.0.snapshot2/vendor/bundle/jruby/1.9/gems/logstash-core-2.2.0.snapshot2-java/lib/logstash/pipeline.rb:208:in `worker_loop'", "/tmp/logstash-2.2.0.snapshot2/vendor/bundle/jruby/1.9/gems/logstash-core-2.2.0.snapshot2-java/lib/logstash/pipeline.rb:177:in `start_workers'"], :level=>:warn}
tcp output exception {:host=>"localhost", :port=>3333, :exception=>#<Errno::EBADF: Bad file descriptor - Bad file descriptor>, :backtrace=>["org/jruby/RubyIO.java:3682:in `select'", "/tmp/logstash-2.2.0.snapshot2/vendor/bundle/jruby/1.9/gems/logstash-output-tcp-2.0.2/lib/logstash/outputs/tcp.rb:101:in `register'", "org/jruby/RubyProc.java:281:in `call'", "/tmp/logstash-2.2.0.snapshot2/vendor/bundle/jruby/1.9/gems/logstash-codec-plain-2.0.2/lib/logstash/codecs/plain.rb:41:in `encode'", "/tmp/logstash-2.2.0.snapshot2/vendor/bundle/jruby/1.9/gems/logstash-output-tcp-2.0.2/lib/logstash/outputs/tcp.rb:143:in `receive'", "/tmp/logstash-2.2.0.snapshot2/vendor/bundle/jruby/1.9/gems/logstash-core-2.2.0.snapshot2-java/lib/logstash/outputs/base.rb:83:in `multi_receive'", "org/jruby/RubyArray.java:1613:in `each'", "/tmp/logstash-2.2.0.snapshot2/vendor/bundle/jruby/1.9/gems/logstash-core-2.2.0.snapshot2-java/lib/logstash/outputs/base.rb:83:in `multi_receive'", "/tmp/logstash-2.2.0.snapshot2/vendor/bundle/jruby/1.9/gems/logstash-core-2.2.0.snapshot2-java/lib/logstash/output_delegator.rb:119:in `worker_multi_receive'", "/tmp/logstash-2.2.0.snapshot2/vendor/bundle/jruby/1.9/gems/logstash-core-2.2.0.snapshot2-java/lib/logstash/output_delegator.rb:65:in `multi_receive'", "/tmp/logstash-2.2.0.snapshot2/vendor/bundle/jruby/1.9/gems/logstash-core-2.2.0.snapshot2-java/lib/logstash/pipeline.rb:277:in `output_batch'", "org/jruby/RubyHash.java:1342:in `each'", "/tmp/logstash-2.2.0.snapshot2/vendor/bundle/jruby/1.9/gems/logstash-core-2.2.0.snapshot2-java/lib/logstash/pipeline.rb:277:in `output_batch'", "/tmp/logstash-2.2.0.snapshot2/vendor/bundle/jruby/1.9/gems/logstash-core-2.2.0.snapshot2-java/lib/logstash/pipeline.rb:208:in `worker_loop'", "/tmp/logstash-2.2.0.snapshot2/vendor/bundle/jruby/1.9/gems/logstash-core-2.2.0.snapshot2-java/lib/logstash/pipeline.rb:177:in `start_workers'"], :level=>:warn}
^CSIGINT received. Shutting down the pipeline. {:level=>:warn}
^CSIGINT received. Terminating immediately.. {:level=>:fatal}

on another run I got:

...
7477
7478
7479
7480
7481
7482
7483
7484
7485
7486
7487
7488
7489
7490
7491
7492
7493
7494
7495
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
185
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
@jsvd jsvd added the bug label Jan 11, 2016
@andrewvc
Copy link
Contributor

Thanks for finding this @jsvd . This looks to be a bug in the TCP output itself. It claims to support multiple workers for output, but in fact does not because IO#select on jruby is not threadsafe.

Moved to logstash-plugins/logstash-output-tcp#13

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

2 participants