Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Use pipe command sender mechanism of ServerEngine on Windows #1171

Merged
merged 1 commit into from
Aug 30, 2016
Merged

Use pipe command sender mechanism of ServerEngine on Windows #1171

merged 1 commit into from
Aug 30, 2016

Conversation

unak
Copy link
Contributor

@unak unak commented Aug 19, 2016

The pipe command sender mechanism has been merged into ServerEngine (treasure-data/serverengine#50).
This pull request makes fluentd to use it on Windows, to fix #1084.

@tagomoris
Copy link
Member

I'm checking this patch on Windows environment too... will paste all logs.

@tagomoris
Copy link
Member

tagomoris commented Aug 30, 2016 •
edited
Loading

I ran fluentd on cmd.exe, and Ctrl-c to stop it. (success: confirmed that both two processes exited)

C:\>fluentd -c C:\fluent_test.conf
2016-08-30 03:04:39 +0000 [info]: reading config file path="C:\\fluent_test.conf"
2016-08-30 03:04:39 +0000 [info]: starting fluentd-0.14.2
2016-08-30 03:04:39 +0000 [info]: spawn command to main: C:/Ruby22/bin/ruby.exe -Eascii-8bit:ascii-8bit  "C:/Ruby22/bin/fluentd" "-c" "C:\fluent_test.conf" --under-supervisor
2016-08-30 03:04:40 +0000 [info]: reading config file path="C:\\fluent_test.conf"
2016-08-30 03:04:40 +0000 [info]: starting fluentd-0.14.2 without supervision
2016-08-30 03:04:40 +0000 [info]: gem 'fluent-plugin-td' version '0.10.28'
2016-08-30 03:04:40 +0000 [info]: gem 'fluentd' version '0.14.2'
2016-08-30 03:04:40 +0000 [info]: gem 'fluentd' version '0.14.1'
2016-08-30 03:04:40 +0000 [info]: gem 'fluentd' version '0.14.0.pre.1'
2016-08-30 03:04:40 +0000 [info]: adding match in @dump pattern="**" type="file"
2016-08-30 03:04:40 +0000 [info]: adding source type="forward"
2016-08-30 03:04:41 +0000 [info]: using configuration file: <ROOT>
  <source>
    @type forward
    @label @dump
  </source>
  <label @dump>
    <match **>
      @type file
      path "C:/mydump.*.log"
      time_slice_format "%Y%m%d%H%M%S"
      time_slice_wait 1s
      buffer_path C:/mydump.*.log
      <buffer time>
        flush_mode lazy
        retry_type exponential_backoff
        path C:/mydump.*.log
        timekey_wait 1s
        timekey 1
      </buffer>
    </match>
  </label>
  <system>
    rpc_endpoint "127.0.0.1:24444"
  </system>
</ROOT>
2016-08-30 03:04:41 +0000 [info]: listening fluent socket on 0.0.0.0:24224
2016-08-30 03:04:43 +0000 [info]: Received graceful stop
2016-08-30 03:04:43 +0000 [info]: Worker 0 finished with
Terminate batch job (Y/N)? 2016-08-30 03:04:43 +0000 [info]: shutting down fluentd
2016-08-30 03:04:43 +0000 [info]: preparing shutdown input plugin type=:forward plugin_id="object:13ec4bc"
2016-08-30 03:04:43 +0000 [info]: preparing shutdown output plugin type=:file plugin_id="object:171036c"
2016-08-30 03:04:43 +0000 [info]: shutting down input plugin type=:forward plugin_id="object:13ec4bc"
2016-08-30 03:04:43 +0000 [info]: shutting down output plugin type=:file plugin_id="object:171036c"
2016-08-30 03:04:43 +0000 [info]: closing input plugin type=:forward plugin_id="object:13ec4bc"
2016-08-30 03:04:43 +0000 [info]: closing output plugin type=:file plugin_id="object:171036c"
Y

C:\>

@tagomoris
Copy link
Member

Windows service (success: 3 ruby process executed, and stopped correctly).

I registered Fluentd as a Windows service:

fluentd --reg-winsvc i
fluentd --reg-winsvc-fluentdopt '-c C:/fluent_test.conf -o C:/fluent.log'

Then, I started & stopped "Fluentd Windows Service". Logs:

C:\>more fluent.log
2016-08-30 03:15:36 +0000 [info]: reading config file path="C:/fluent_test.conf"
2016-08-30 03:15:36 +0000 [info]: starting fluentd-0.14.2
2016-08-30 03:15:36 +0000 [info]: spawn command to main: C:/Ruby22/bin/ruby.exe
-Eascii-8bit:ascii-8bit  "C:/Ruby22/bin/fluentd" "-c" "C:/fluent_test.conf" "-o"
 "C:/fluent.log" "-x" "fluentdwinsvc" --under-supervisor
2016-08-30 03:15:37 +0000 [info]: reading config file path="C:/fluent_test.conf"
2016-08-30 03:15:37 +0000 [info]: starting fluentd-0.14.2 without supervision
2016-08-30 03:15:37 +0000 [info]: gem 'fluent-plugin-td' version '0.10.28'
2016-08-30 03:15:37 +0000 [info]: gem 'fluentd' version '0.14.2'
2016-08-30 03:15:37 +0000 [info]: gem 'fluentd' version '0.14.1'
2016-08-30 03:15:37 +0000 [info]: gem 'fluentd' version '0.14.0.pre.1'
2016-08-30 03:15:37 +0000 [info]: adding match in @dump pattern="**" type="file"
2016-08-30 03:15:37 +0000 [info]: adding source type="forward"
2016-08-30 03:15:38 +0000 [info]: using configuration file: <ROOT>
  <source>
    @type forward
    @label @dump
  </source>
  <label @dump>
    <match **>
      @type file
      path "C:/mydump.*.log"
      time_slice_format "%Y%m%d%H%M%S"
      time_slice_wait 1s
      buffer_path C:/mydump.*.log
      <buffer time>
        flush_mode lazy
        retry_type exponential_backoff
        path C:/mydump.*.log
        timekey_wait 1s
        timekey 1
      </buffer>
    </match>
  </label>
  <system>
    rpc_endpoint "127.0.0.1:24444"
  </system>
</ROOT>
2016-08-30 03:15:38 +0000 [info]: listening fluent socket on 0.0.0.0:24224
2016-08-30 03:16:37 +0000 [info]: Received graceful stop
2016-08-30 03:16:37 +0000 [info]: Worker 0 finished with status 0

@tagomoris
Copy link
Member

@unak Just one thing I want to confirm:
When I launched Fluentd as a service, I couldn't see any logs on fluent.log file (its size was 0).
I could see whole logs after I stopped Fluentd service.

Can we fix this behavior to see logs as soon as possible? Or not?

@tagomoris
Copy link
Member

tagomoris commented Aug 30, 2016 •
edited
Loading

I retried the configuration, and confirmed that:

  • the size of log file is 0 bytes
  • but we can see logs actually (by type fluent.log or tail -f)

Note: fsync may help this case if needed https://github.com/ruby/ruby/blob/v2_2_0/NEWS#L110-L113

There are no problem to merge this change!

@tagomoris
Copy link
Member

tagomoris commented Aug 30, 2016 •
edited
Loading

I'll merge this branch besides CI red, because it is not related with this patch, and already known at #1115.

@tagomoris tagomoris merged commit 31f34c3 into fluent:master Aug 30, 2016
@unak unak deleted the win-piped-command branch August 30, 2016 05:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
v0.14 windows
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Stopping Fluentd service on Windows doesn't work well
2 participants
@unak @tagomoris