Asynchronous process reaping #1412

takase1121 · 2023-03-07T13:25:43Z

This PR allows process handles to be reaped asynchronously.
Previously, process reaping on __gc will stall the GC due to waiting for process to actually exit.
Now, this PR will allow the GC to run continuously while the process is sent to a background thread where it is terminated.

This also fixes an issue on Windows where console applications are improperly kept open, holding lite-xl in the background.

…ess_handle

src/api/process.c

Guldoman

I still need to test this, but code-wise lgtm.

src/api/process.c

Co-authored-by: Guldoman <giulio.lettieri@gmail.com>

takase1121 · 2023-03-22T09:20:01Z

Tested on macOS Monterey:

Skipping SIGTERM before adding to the kill list

Process 993 launched: '/Users/takase/gh/lite-xl/lite-xl' (x86_64)
(lldb)  jump 698
(lldb)  expr printf("Added (%ld)\n", self->pid)
(int) $132 = 13
Added (1005)
(lldb)  jump 698
(lldb)  expr printf("SIGTERM %d (%ld)\n", current_task->tries, current_task->handle)
(int) $133 = 17
SIGTERM 1 (1005)
(lldb)  expr printf("Added (%ld)\n", self->pid)
(int) $134 = 13
Added (1004)
(lldb)  jump 698
(lldb)  expr printf("Added (%ld)\n", self->pid)
(int) $135 = 13
Added (1003)
(lldb)  jump 698
(lldb)  expr printf("Free %d (%ld)\n", current_task->tries, current_task->handle)
(int) $136 = 14
Free 2 (1005)
(lldb)  expr printf("SIGTERM %d (%ld)\n", current_task->tries, current_task->handle)
(int) $137 = 17
SIGTERM 1 (1004)
(lldb)  expr printf("SIGTERM %d (%ld)\n", current_task->tries, current_task->handle)
(int) $138 = 17
SIGTERM 1 (1003)
(lldb)  expr printf("Added (%ld)\n", self->pid)
(int) $139 = 13
Added (1002)
(lldb)  jump 698
(lldb)  expr printf("Added (%ld)\n", self->pid)
(int) $140 = 13
Added (1001)
(lldb)  jump 698
(lldb)  expr printf("Free %d (%ld)\n", current_task->tries, current_task->handle)
(int) $141 = 14
Free 2 (1004)
(lldb)  expr printf("Free %d (%ld)\n", current_task->tries, current_task->handle)
(int) $142 = 14
Free 2 (1003)
(lldb)  expr printf("SIGTERM %d (%ld)\n", current_task->tries, current_task->handle)
(int) $143 = 17
SIGTERM 1 (1002)
(lldb)  expr printf("SIGTERM %d (%ld)\n", current_task->tries, current_task->handle)
(int) $144 = 17
SIGTERM 1 (1001)
(lldb)  expr printf("Added (%ld)\n", self->pid)
(int) $145 = 13
Added (1000)
(lldb)  jump 698
(lldb)  expr printf("Free %d (%ld)\n", current_task->tries, current_task->handle)
(int) $146 = 14
Free 2 (1002)
(lldb)  expr printf("Free %d (%ld)\n", current_task->tries, current_task->handle)
(int) $147 = 14
Free 2 (1001)
(lldb)  expr printf("SIGTERM %d (%ld)\n", current_task->tries, current_task->handle)
(int) $148 = 17
SIGTERM 1 (1000)
(lldb)  expr printf("Added (%ld)\n", self->pid)
(int) $149 = 12
Added (999)
(lldb)  jump 698
(lldb)  expr printf("Added (%ld)\n", self->pid)
(int) $150 = 12
Added (998)
(lldb)  jump 698
(lldb)  expr printf("Free %d (%ld)\n", current_task->tries, current_task->handle)
(int) $151 = 14
Free 2 (1000)
(lldb)  expr printf("SIGTERM %d (%ld)\n", current_task->tries, current_task->handle)
(int) $152 = 16
SIGTERM 1 (999)
(lldb)  expr printf("SIGTERM %d (%ld)\n", current_task->tries, current_task->handle)
(int) $153 = 16
SIGTERM 1 (998)
(lldb)  expr printf("Added (%ld)\n", self->pid)
(int) $154 = 12
Added (997)
(lldb)  jump 698
(lldb)  expr printf("Free %d (%ld)\n", current_task->tries, current_task->handle)
(int) $155 = 13
Free 2 (999)
(lldb)  expr printf("Free %d (%ld)\n", current_task->tries, current_task->handle)
(int) $156 = 13
Free 2 (998)
(lldb)  expr printf("SIGTERM %d (%ld)\n", current_task->tries, current_task->handle)
(int) $157 = 16
SIGTERM 1 (997)
(lldb)  expr printf("Added (%ld)\n", self->pid)
(int) $158 = 12
Added (996)
(lldb)  expr printf("Free %d (%ld)\n", current_task->tries, current_task->handle)
(int) $159 = 13
Free 2 (997)
(lldb)  expr printf("SIGTERM %d (%ld)\n", current_task->tries, current_task->handle)
(int) $160 = 16
SIGTERM 1 (996)
(lldb)  expr printf("Free %d (%ld)\n", current_task->tries, current_task->handle)
(int) $161 = 13
Free 2 (996)
(lldb)  expr printf("Kill list freed\n")
(int) $162 = 16
Kill list freed

Running a script that traps SIGTERM

Process 1219 launched: '/Users/takase/gh/lite-xl/build/src/lite-xl' (x86_64)
2023-03-22 16:40:15.400521+0800 lite-xl[1219:24678] SecTaskLoadEntitlements failed error=22 cs_flags=20, pid=1219
2023-03-22 16:40:15.400661+0800 lite-xl[1219:24678] SecTaskCopyDebugDescription: lite-xl[1219]/0#-1 LF=0
(lldb)  expr printf("Added (%ld)\n", self->pid)
(int) $346 = 13
Added (1239)
(lldb)  expr printf("Added (%ld)\n", self->pid)
(int) $347 = 13
Added (1236)
(lldb)  expr printf("SIGTERM %d (%ld)\n", current_task->tries, current_task->handle)
(int) $348 = 17
SIGTERM 1 (1239)
(lldb)  expr printf("SIGTERM %d (%ld)\n", current_task->tries, current_task->handle)
(int) $349 = 17
SIGTERM 1 (1236)
(lldb)  expr printf("Added (%ld)\n", self->pid)
(int) $350 = 13
Added (1235)
(lldb)  expr printf("SIGTERM %d (%ld)\n", current_task->tries, current_task->handle)
(int) $351 = 17
SIGTERM 2 (1239)
(lldb)  expr printf("SIGTERM %d (%ld)\n", current_task->tries, current_task->handle)
(int) $352 = 17
SIGTERM 2 (1236)
(lldb)  expr printf("SIGTERM %d (%ld)\n", current_task->tries, current_task->handle)
(int) $353 = 17
SIGTERM 1 (1235)
(lldb)  expr printf("SIGKILL %d (%ld)\n", current_task->tries, current_task->handle)
(int) $354 = 17
SIGKILL 3 (1239)
(lldb)  expr printf("SIGKILL %d (%ld)\n", current_task->tries, current_task->handle)
(int) $355 = 17
SIGKILL 3 (1236)
(lldb)  expr printf("SIGTERM %d (%ld)\n", current_task->tries, current_task->handle)
(int) $356 = 17
SIGTERM 2 (1235)
(lldb)  expr printf("Free %d (%ld)\n", current_task->tries, current_task->handle)
(int) $357 = 14
Free 4 (1239)
(lldb)  expr printf("Free %d (%ld)\n", current_task->tries, current_task->handle)
(int) $358 = 14
Free 4 (1236)
(lldb)  expr printf("SIGKILL %d (%ld)\n", current_task->tries, current_task->handle)
(int) $359 = 17
SIGKILL 3 (1235)
(lldb)  expr printf("Added (%ld)\n", self->pid)
(int) $360 = 13
Added (1231)
(lldb)  expr printf("Added (%ld)\n", self->pid)
(int) $361 = 13
Added (1230)
(lldb)  expr printf("Added (%ld)\n", self->pid)
(int) $362 = 13
Added (1228)
(lldb)  expr printf("Free %d (%ld)\n", current_task->tries, current_task->handle)
(int) $363 = 14
Free 4 (1235)
(lldb)  expr printf("SIGTERM %d (%ld)\n", current_task->tries, current_task->handle)
(int) $364 = 17
SIGTERM 1 (1231)
(lldb)  expr printf("SIGTERM %d (%ld)\n", current_task->tries, current_task->handle)
(int) $365 = 17
SIGTERM 1 (1230)
(lldb)  expr printf("SIGTERM %d (%ld)\n", current_task->tries, current_task->handle)
(int) $366 = 17
SIGTERM 1 (1228)
(lldb)  expr printf("SIGTERM %d (%ld)\n", current_task->tries, current_task->handle)
(int) $367 = 17
SIGTERM 2 (1231)
(lldb)  expr printf("SIGTERM %d (%ld)\n", current_task->tries, current_task->handle)
(int) $368 = 17
SIGTERM 2 (1230)
(lldb)  expr printf("SIGTERM %d (%ld)\n", current_task->tries, current_task->handle)
(int) $369 = 17
SIGTERM 2 (1228)
(lldb)  expr printf("SIGKILL %d (%ld)\n", current_task->tries, current_task->handle)
(int) $370 = 17
SIGKILL 3 (1231)
(lldb)  expr printf("SIGKILL %d (%ld)\n", current_task->tries, current_task->handle)
(int) $371 = 17
SIGKILL 3 (1230)
(lldb)  expr printf("SIGKILL %d (%ld)\n", current_task->tries, current_task->handle)
(int) $372 = 17
SIGKILL 3 (1228)
(lldb)  expr printf("Free %d (%ld)\n", current_task->tries, current_task->handle)
(int) $373 = 14
Free 4 (1231)
(lldb)  expr printf("Free %d (%ld)\n", current_task->tries, current_task->handle)
(int) $374 = 14
Free 4 (1230)
(lldb)  expr printf("Free %d (%ld)\n", current_task->tries, current_task->handle)
(int) $375 = 14
Free 4 (1228)
(lldb)  expr printf("Added (%ld)\n", self->pid)
(int) $376 = 13
Added (1226)
(lldb)  expr printf("SIGTERM %d (%ld)\n", current_task->tries, current_task->handle)
(int) $377 = 17
SIGTERM 1 (1226)
(lldb)  expr printf("Added (%ld)\n", self->pid)
(int) $378 = 13
Added (1224)
(lldb)  expr printf("Added (%ld)\n", self->pid)
(int) $379 = 13
Added (1223)
(lldb)  expr printf("Added (%ld)\n", self->pid)
(int) $380 = 13
Added (1222)
(lldb)  expr printf("SIGTERM %d (%ld)\n", current_task->tries, current_task->handle)
(int) $381 = 17
SIGTERM 2 (1226)
(lldb)  expr printf("SIGTERM %d (%ld)\n", current_task->tries, current_task->handle)
(int) $382 = 17
SIGTERM 1 (1224)
(lldb)  expr printf("SIGTERM %d (%ld)\n", current_task->tries, current_task->handle)
(int) $383 = 17
SIGTERM 1 (1223)
(lldb)  expr printf("SIGTERM %d (%ld)\n", current_task->tries, current_task->handle)
(int) $384 = 17
SIGTERM 1 (1222)
(lldb)  expr printf("SIGKILL %d (%ld)\n", current_task->tries, current_task->handle)
(int) $385 = 17
SIGKILL 3 (1226)
(lldb)  expr printf("SIGTERM %d (%ld)\n", current_task->tries, current_task->handle)
(int) $386 = 17
SIGTERM 2 (1224)
(lldb)  expr printf("SIGTERM %d (%ld)\n", current_task->tries, current_task->handle)
(int) $387 = 17
SIGTERM 2 (1223)
(lldb)  expr printf("SIGTERM %d (%ld)\n", current_task->tries, current_task->handle)
(int) $388 = 17
SIGTERM 2 (1222)
(lldb)  expr printf("Free %d (%ld)\n", current_task->tries, current_task->handle)
(int) $389 = 14
Free 4 (1226)
(lldb)  expr printf("SIGKILL %d (%ld)\n", current_task->tries, current_task->handle)
(int) $390 = 17
SIGKILL 3 (1224)
(lldb)  expr printf("SIGKILL %d (%ld)\n", current_task->tries, current_task->handle)
(int) $391 = 17
SIGKILL 3 (1223)
(lldb)  expr printf("SIGKILL %d (%ld)\n", current_task->tries, current_task->handle)
(int) $392 = 17
SIGKILL 3 (1222)
(lldb)  expr printf("Free %d (%ld)\n", current_task->tries, current_task->handle)
(int) $393 = 14
Free 4 (1224)
(lldb)  expr printf("Free %d (%ld)\n", current_task->tries, current_task->handle)
(int) $394 = 14
Free 4 (1223)
(lldb)  expr printf("Free %d (%ld)\n", current_task->tries, current_task->handle)
(int) $395 = 14
Free 4 (1222)
(lldb)  expr printf("Kill list freed\n")
(int) $596 = 16
Kill list freed

Disabling SIGKILL

(int) $10 = 20
Abandoning 4 (1720)
(lldb)  expr printf("Abandoning %d (%ld)\n", current_task->tries, current_task->handle)
(int) $11 = 20
Abandoning 4 (1718)
(lldb)  expr printf("Abandoning %d (%ld)\n", current_task->tries, current_task->handle)
(int) $12 = 20
Abandoning 4 (1716)
(lldb)  expr printf("Abandoning %d (%ld)\n", current_task->tries, current_task->handle)
(int) $13 = 20
Abandoning 4 (1715)
(lldb)  expr printf("Abandoning %d (%ld)\n", current_task->tries, current_task->handle)
(int) $14 = 20
Abandoning 4 (1713)
(lldb)  expr printf("Abandoning %d (%ld)\n", current_task->tries, current_task->handle)
(int) $15 = 20
Abandoning 4 (1710)
(lldb)  expr printf("Abandoning %d (%ld)\n", current_task->tries, current_task->handle)
(int) $16 = 20
Abandoning 4 (1708)
(lldb)  expr printf("Abandoning %d (%ld)\n", current_task->tries, current_task->handle)
(int) $17 = 20
Abandoning 4 (1706)
(lldb)  expr printf("Abandoning %d (%ld)\n", current_task->tries, current_task->handle)
(int) $18 = 20
Abandoning 4 (1705)
(lldb)  expr printf("Abandoning %d (%ld)\n", current_task->tries, current_task->handle)
(int) $19 = 20
Abandoning 4 (1704)
(lldb)  expr printf("Kill list freed\n")
(int) $20 = 16
Kill list freed

ps -ax | grep sleep

takase@Takases-MacBook-Pro lite-xl % ps -ax | grep sleep
 1652 ttys000    0:00.00 sleep 1000
 1654 ttys000    0:00.00 sleep 1000
 1655 ttys000    0:00.00 sleep 1000
 1657 ttys000    0:00.00 sleep 1000
 1660 ttys000    0:00.00 sleep 1000
 1662 ttys000    0:00.00 sleep 1000
 1664 ttys000    0:00.00 sleep 1000
 1666 ttys000    0:00.00 sleep 1000
 1667 ttys000    0:00.00 sleep 1000
 1680 ttys000    0:00.00 sleep 1000
 1683 ttys000    0:00.00 sleep 1000
 1684 ttys000    0:00.00 sleep 1000
 1686 ttys000    0:00.00 sleep 1000
 1688 ttys000    0:00.00 sleep 1000
 1690 ttys000    0:00.00 sleep 1000
 1692 ttys000    0:00.00 sleep 1000
 1694 ttys000    0:00.00 sleep 1000
 1695 ttys000    0:00.00 sleep 1000
 1696 ttys000    0:00.00 sleep 1000
 1707 ttys000    0:00.00 sleep 1000
 1709 ttys000    0:00.00 sleep 1000
 1711 ttys000    0:00.00 sleep 1000
 1712 ttys000    0:00.00 sleep 1000
 1714 ttys000    0:00.00 sleep 1000
 1717 ttys000    0:00.00 sleep 1000
 1719 ttys000    0:00.00 sleep 1000
 1721 ttys000    0:00.00 sleep 1000
 1722 ttys000    0:00.00 sleep 1000
 1723 ttys000    0:00.00 sleep 1000
 1729 ttys002    0:00.00 grep sleep

* refactor(process): introduce process_stream_handle separate from process_handle * feat(process): introduce process_handle helper functions * feat(process): add asynchronous process reaping * feat(process): wait for shorter period if possible * style(process): remove unecessary brackets * style(process): fix parentheses * refactor(process): remove useless setvbuf call * style(process): remove unecessary value * refactor(process): add size field into kill_list * refactor(process): use SDL_Delay for sleeping * style(process): remove trailing whitespace * fix(main): destroy window before closing lua * fix(process): check for timeout correctly * refactor(process): remove unecessary if check * refactor(process): remove size from the list * fix(process): fix invalid delay calculation Co-authored-by: Guldoman <giulio.lettieri@gmail.com> --------- Co-authored-by: Guldoman <giulio.lettieri@gmail.com>

takase1121 added 4 commits March 7, 2023 21:11

refactor(process): introduce process_stream_handle separate from proc…

a54ffc0

…ess_handle

feat(process): introduce process_handle helper functions

1a5357b

feat(process): add asynchronous process reaping

6e73a41

feat(process): wait for shorter period if possible

fd047bd

github-actions bot added the Category: C Core label Mar 7, 2023

takase1121 added 2 commits March 7, 2023 21:31

style(process): remove unecessary brackets

c80eb67

style(process): fix parentheses

3ea0f30

Guldoman reviewed Mar 7, 2023

View reviewed changes

takase1121 added 6 commits March 8, 2023 08:06

refactor(process): remove useless setvbuf call

7e69983

style(process): remove unecessary value

6b1ef98

refactor(process): add size field into kill_list

b54ce89

refactor(process): use SDL_Delay for sleeping

d0d07da

style(process): remove trailing whitespace

5cd0ed9

fix(main): destroy window before closing lua

e1f5704

takase1121 requested a review from Guldoman March 8, 2023 10:55

takase1121 added 2 commits March 9, 2023 10:22

fix(process): check for timeout correctly

0d1ce7d

refactor(process): remove unecessary if check

2c84e4f

Guldoman reviewed Mar 9, 2023

View reviewed changes

src/api/process.c Outdated Show resolved Hide resolved

src/api/process.c Outdated Show resolved Hide resolved

src/api/process.c Outdated Show resolved Hide resolved

refactor(process): remove size from the list

62d473e

Guldoman reviewed Mar 10, 2023

View reviewed changes

src/api/process.c Show resolved Hide resolved

Guldoman reviewed Mar 13, 2023

View reviewed changes

src/api/process.c Outdated Show resolved Hide resolved

fix(process): fix invalid delay calculation

b9467ca

Co-authored-by: Guldoman <giulio.lettieri@gmail.com>

Guldoman merged commit 0cacaf9 into lite-xl:master Mar 22, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Asynchronous process reaping #1412

Asynchronous process reaping #1412

takase1121 commented Mar 7, 2023

Guldoman left a comment

takase1121 commented Mar 22, 2023

Asynchronous process reaping #1412

Asynchronous process reaping #1412

Conversation

takase1121 commented Mar 7, 2023

Guldoman left a comment

Choose a reason for hiding this comment

takase1121 commented Mar 22, 2023