mjhjmtu

My lsof is hung so bad that I can't even kill -9 it.

Initially, it was owned by another process, foo. I tried killing foo, hoping that lsof would also go way. But while I was able to kill -9 foo, now lsof appears to be owned by `upstart.

root@box 0 L1 ~
$ ps -ef | grep 'lsof|upstart' | grep 3575
joe 1595 3575 0 12:43 pts/5 00:00:00 lsof /home/joe/tmp/a a/file.11
joe 1603 3575 0 12:43 pts/5 00:00:00 lsof /home/joe/tmp/a a/file.13
joe 1605 3575 0 12:43 pts/5 00:00:00 lsof /home/joe/tmp/a a/file.18
joe 1614 3575 0 12:43 pts/5 00:00:00 lsof /home/joe/tmp/a a/file.16
joe 3575 3331 0 Nov26 ? 00:00:01 /sbin/upstart --user
joe 4330 3575 0 12:53 pts/5 00:00:00 lsof /home/joe/tmp/a a/file.18
joe 4353 3575 0 12:54 pts/5 00:00:00 lsof /home/joe/tmp/a a/file.18
joe 4429 3575 0 12:54 pts/5 00:00:00 lsof /home/joe/tmp/a a/file.18
joe 4435 3575 0 12:54 pts/5 00:00:00 lsof /home/joe/tmp/a a/file.11
joe 4580 3575 0 12:55 pts/5 00:00:00 lsof bar

Next, I try to get a list of PIDs of lsof processes.

root@box 130 L1 ~
$ ps -ef | grep lsof | grep -v grep | gawk ' print $2 '
1595
1603
1605
1614
4330
4353
4429
4435
4580

I try to kill them all (as root).

root@box 0 L1 ~
$ ps -ef | grep lsof | grep -v grep | gawk ' print $2 ' | xargs -n1 kill -9

But, they are still "hanging in" there. (pun intended)

root@box 0 L1 ~
$ ps -ef | grep lsof | grep -v grep
joe 1595 3575 0 12:43 pts/5 00:00:00 lsof /home/joe/tmp/a a/file.11
joe 1603 3575 0 12:43 pts/5 00:00:00 lsof /home/joe/tmp/a a/file.13
joe 1605 3575 0 12:43 pts/5 00:00:00 lsof /home/joe/tmp/a a/file.18
joe 1614 3575 0 12:43 pts/5 00:00:00 lsof /home/joe/tmp/a a/file.16
joe 4330 3575 0 12:53 pts/5 00:00:00 lsof /home/joe/tmp/a a/file.18
joe 4353 3575 0 12:54 pts/5 00:00:00 lsof /home/joe/tmp/a a/file.18
joe 4429 3575 0 12:54 pts/5 00:00:00 lsof /home/joe/tmp/a a/file.18
joe 4435 3575 0 12:54 pts/5 00:00:00 lsof /home/joe/tmp/a a/file.11
joe 4580 3575 0 12:55 pts/5 00:00:00 lsof bar

Notes:

1. All of the above lsof commands were invoked by background tasks inside of foo.




2. The lsof bar was invoked manually on the shell, and even it hung! I can Ctrl-C out of it. But it continues to show in the ps -ef listing.

Questions:

1. Why is lsof hanging in the first place? All the files given as arguments to lsof are on a local ext4 filesystem of a Ubuntu 16.04 box.




2. Why even kill -9 isn't able to kill these processes - even when run as root?




3. Why I can Ctrl-C an interactively invoked lsof bar just fine but the lsof bar continues to show up in the ps -ef listing?