Suddenly I see a lot of involuntary context switches for multi threaded app

The name of the pictureThe name of the pictureThe name of the pictureClash Royale CLAN TAG#URR8PPP











up vote
1
down vote

favorite












I have a multi-threaded app, with 4 worker threads that until recently performed very well on my 4-core CPU.



The system is running Ubuntu 16.04.3 LTS.



Today I updated my OS with sudo apt-get dist-upgrade



Now, I suddenly see much lower performance in my app, and I have traced it down to the worker threads getting context-switched involuntarily.



Looking at /proc/xxx/status for a worker thread, I see that it is getting hit with 80 involuntary context switches per second or so, as the nonvoluntary_ctxt_switches field is increasing quickly.



Note that at least 30 times per second, there will be no work in the queue, and the workerthreads will voluntary go to sleep.



Thread NICE value is 0, and PRIORITY is 20.



Why did those involuntary context switches suddenly pop up?
Did Ubuntu recently change the scheduler in the kernel or something?
Are there any ways to find out why and who is bumping the threads?



Also: not only do the treads get switched out, it takes quite a bit of time to get serviced again.







share|improve this question






















  • Add an update with your previous OS version. Start through the Kernel Newbies site, which has great high level bullets of each Kernel version change. Start looking through each version between your old version + new version. You may find something. This might also relate to libc or libpthreads updates.
    – David Favor
    Oct 28 '17 at 13:16














up vote
1
down vote

favorite












I have a multi-threaded app, with 4 worker threads that until recently performed very well on my 4-core CPU.



The system is running Ubuntu 16.04.3 LTS.



Today I updated my OS with sudo apt-get dist-upgrade



Now, I suddenly see much lower performance in my app, and I have traced it down to the worker threads getting context-switched involuntarily.



Looking at /proc/xxx/status for a worker thread, I see that it is getting hit with 80 involuntary context switches per second or so, as the nonvoluntary_ctxt_switches field is increasing quickly.



Note that at least 30 times per second, there will be no work in the queue, and the workerthreads will voluntary go to sleep.



Thread NICE value is 0, and PRIORITY is 20.



Why did those involuntary context switches suddenly pop up?
Did Ubuntu recently change the scheduler in the kernel or something?
Are there any ways to find out why and who is bumping the threads?



Also: not only do the treads get switched out, it takes quite a bit of time to get serviced again.







share|improve this question






















  • Add an update with your previous OS version. Start through the Kernel Newbies site, which has great high level bullets of each Kernel version change. Start looking through each version between your old version + new version. You may find something. This might also relate to libc or libpthreads updates.
    – David Favor
    Oct 28 '17 at 13:16












up vote
1
down vote

favorite









up vote
1
down vote

favorite











I have a multi-threaded app, with 4 worker threads that until recently performed very well on my 4-core CPU.



The system is running Ubuntu 16.04.3 LTS.



Today I updated my OS with sudo apt-get dist-upgrade



Now, I suddenly see much lower performance in my app, and I have traced it down to the worker threads getting context-switched involuntarily.



Looking at /proc/xxx/status for a worker thread, I see that it is getting hit with 80 involuntary context switches per second or so, as the nonvoluntary_ctxt_switches field is increasing quickly.



Note that at least 30 times per second, there will be no work in the queue, and the workerthreads will voluntary go to sleep.



Thread NICE value is 0, and PRIORITY is 20.



Why did those involuntary context switches suddenly pop up?
Did Ubuntu recently change the scheduler in the kernel or something?
Are there any ways to find out why and who is bumping the threads?



Also: not only do the treads get switched out, it takes quite a bit of time to get serviced again.







share|improve this question














I have a multi-threaded app, with 4 worker threads that until recently performed very well on my 4-core CPU.



The system is running Ubuntu 16.04.3 LTS.



Today I updated my OS with sudo apt-get dist-upgrade



Now, I suddenly see much lower performance in my app, and I have traced it down to the worker threads getting context-switched involuntarily.



Looking at /proc/xxx/status for a worker thread, I see that it is getting hit with 80 involuntary context switches per second or so, as the nonvoluntary_ctxt_switches field is increasing quickly.



Note that at least 30 times per second, there will be no work in the queue, and the workerthreads will voluntary go to sleep.



Thread NICE value is 0, and PRIORITY is 20.



Why did those involuntary context switches suddenly pop up?
Did Ubuntu recently change the scheduler in the kernel or something?
Are there any ways to find out why and who is bumping the threads?



Also: not only do the treads get switched out, it takes quite a bit of time to get serviced again.









share|improve this question













share|improve this question




share|improve this question








edited Oct 28 '17 at 20:10

























asked Oct 28 '17 at 4:41









Bram

1245




1245











  • Add an update with your previous OS version. Start through the Kernel Newbies site, which has great high level bullets of each Kernel version change. Start looking through each version between your old version + new version. You may find something. This might also relate to libc or libpthreads updates.
    – David Favor
    Oct 28 '17 at 13:16
















  • Add an update with your previous OS version. Start through the Kernel Newbies site, which has great high level bullets of each Kernel version change. Start looking through each version between your old version + new version. You may find something. This might also relate to libc or libpthreads updates.
    – David Favor
    Oct 28 '17 at 13:16















Add an update with your previous OS version. Start through the Kernel Newbies site, which has great high level bullets of each Kernel version change. Start looking through each version between your old version + new version. You may find something. This might also relate to libc or libpthreads updates.
– David Favor
Oct 28 '17 at 13:16




Add an update with your previous OS version. Start through the Kernel Newbies site, which has great high level bullets of each Kernel version change. Start looking through each version between your old version + new version. You may find something. This might also relate to libc or libpthreads updates.
– David Favor
Oct 28 '17 at 13:16















active

oldest

votes











Your Answer







StackExchange.ready(function()
var channelOptions =
tags: "".split(" "),
id: "106"
;
initTagRenderer("".split(" "), "".split(" "), channelOptions);

StackExchange.using("externalEditor", function()
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled)
StackExchange.using("snippets", function()
createEditor();
);

else
createEditor();

);

function createEditor()
StackExchange.prepareEditor(
heartbeatType: 'answer',
convertImagesToLinks: false,
noModals: false,
showLowRepImageUploadWarning: true,
reputationToPostImages: null,
bindNavPrevention: true,
postfix: "",
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
);



);













 

draft saved


draft discarded


















StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2funix.stackexchange.com%2fquestions%2f401004%2fsuddenly-i-see-a-lot-of-involuntary-context-switches-for-multi-threaded-app%23new-answer', 'question_page');

);

Post as a guest



































active

oldest

votes













active

oldest

votes









active

oldest

votes






active

oldest

votes















 

draft saved


draft discarded















































 


draft saved


draft discarded














StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2funix.stackexchange.com%2fquestions%2f401004%2fsuddenly-i-see-a-lot-of-involuntary-context-switches-for-multi-threaded-app%23new-answer', 'question_page');

);

Post as a guest













































































Popular posts from this blog

How to check contact read email or not when send email to Individual?

Bahrain

Postfix configuration issue with fips on centos 7; mailgun relay