Prevent `wget` hanging when it encounters error 500

The name of the pictureThe name of the pictureThe name of the pictureClash Royale CLAN TAG#URR8PPP











up vote
1
down vote

favorite
1












I am downloading a couple of thousand files listed in a file using:



wget -i filename


Sometimes it encounters the following error reported by the server for a particular file:




HTTP request sent, awaiting response... 500 Internal Server Error




The problem is that wget then hangs. What I want for it to skip that file and continue downloading the rest of the list. How can I do that?






command-line wget







share|improve this question

























    up vote
    1
    down vote

    favorite
    1












    I am downloading a couple of thousand files listed in a file using:



    wget -i filename


    Sometimes it encounters the following error reported by the server for a particular file:




    HTTP request sent, awaiting response... 500 Internal Server Error




    The problem is that wget then hangs. What I want for it to skip that file and continue downloading the rest of the list. How can I do that?






    command-line wget







    share|improve this question























      up vote
      1
      down vote

      favorite
      1









      up vote
      1
      down vote

      favorite
      1






      1





      I am downloading a couple of thousand files listed in a file using:



      wget -i filename


      Sometimes it encounters the following error reported by the server for a particular file:




      HTTP request sent, awaiting response... 500 Internal Server Error




      The problem is that wget then hangs. What I want for it to skip that file and continue downloading the rest of the list. How can I do that?






      command-line wget







      share|improve this question













      I am downloading a couple of thousand files listed in a file using:



      wget -i filename


      Sometimes it encounters the following error reported by the server for a particular file:




      HTTP request sent, awaiting response... 500 Internal Server Error




      The problem is that wget then hangs. What I want for it to skip that file and continue downloading the rest of the list. How can I do that?






      command-line wget




      command-line wget






      share|improve this question













      share|improve this question











      share|improve this question




      share|improve this question










      asked Oct 3 '17 at 20:38









      JBentley

      1086




      1086




















          1 Answer
          1






          active

          oldest

          votes

















          up vote
          3
          down vote



          accepted










          Try using wget --tries=1 --waitretry=1 -i filename. This will try only once after a failure and only wait one second before doing so. It's also possible the server is not closing the socket after sending the 500 error. In this case, adding --read-timeout=30 will timeout the connection after 30 seconds of no data from the server. See the manual






          share|improve this answer


















          • 1




            --tries=0 will retry indefinitely. The server may be intentionally leaving the connection open - try setting --read-timeout=15
            – Outurnate
            Oct 3 '17 at 20:52










          • @JBentley On it
            – Outurnate
            Oct 3 '17 at 23:54










          Your Answer







          StackExchange.ready(function()
          var channelOptions =
          tags: "".split(" "),
          id: "106"
          ;
          initTagRenderer("".split(" "), "".split(" "), channelOptions);

          StackExchange.using("externalEditor", function()
          // Have to fire editor after snippets, if snippets enabled
          if (StackExchange.settings.snippets.snippetsEnabled)
          StackExchange.using("snippets", function()
          createEditor();
          );

          else
          createEditor();

          );

          function createEditor()
          StackExchange.prepareEditor(
          heartbeatType: 'answer',
          convertImagesToLinks: false,
          noModals: false,
          showLowRepImageUploadWarning: true,
          reputationToPostImages: null,
          bindNavPrevention: true,
          postfix: "",
          onDemand: true,
          discardSelector: ".discard-answer"
          ,immediatelyShowMarkdownHelp:true
          );



          );













           

          draft saved


          draft discarded


















          StackExchange.ready(
          function ()
          StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2funix.stackexchange.com%2fquestions%2f395917%2fprevent-wget-hanging-when-it-encounters-error-500%23new-answer', 'question_page');

          );

          Post as a guest

















          1 Answer
          1






          active

          oldest

          votes








          1 Answer
          1






          active

          oldest

          votes









          active

          oldest

          votes






          active

          oldest

          votes








          up vote
          3
          down vote



          accepted










          Try using wget --tries=1 --waitretry=1 -i filename. This will try only once after a failure and only wait one second before doing so. It's also possible the server is not closing the socket after sending the 500 error. In this case, adding --read-timeout=30 will timeout the connection after 30 seconds of no data from the server. See the manual






          share|improve this answer


















          • 1




            --tries=0 will retry indefinitely. The server may be intentionally leaving the connection open - try setting --read-timeout=15
            – Outurnate
            Oct 3 '17 at 20:52










          • @JBentley On it
            – Outurnate
            Oct 3 '17 at 23:54














          up vote
          3
          down vote



          accepted










          Try using wget --tries=1 --waitretry=1 -i filename. This will try only once after a failure and only wait one second before doing so. It's also possible the server is not closing the socket after sending the 500 error. In this case, adding --read-timeout=30 will timeout the connection after 30 seconds of no data from the server. See the manual






          share|improve this answer


















          • 1




            --tries=0 will retry indefinitely. The server may be intentionally leaving the connection open - try setting --read-timeout=15
            – Outurnate
            Oct 3 '17 at 20:52










          • @JBentley On it
            – Outurnate
            Oct 3 '17 at 23:54












          up vote
          3
          down vote



          accepted







          up vote
          3
          down vote



          accepted






          Try using wget --tries=1 --waitretry=1 -i filename. This will try only once after a failure and only wait one second before doing so. It's also possible the server is not closing the socket after sending the 500 error. In this case, adding --read-timeout=30 will timeout the connection after 30 seconds of no data from the server. See the manual






          share|improve this answer














          Try using wget --tries=1 --waitretry=1 -i filename. This will try only once after a failure and only wait one second before doing so. It's also possible the server is not closing the socket after sending the 500 error. In this case, adding --read-timeout=30 will timeout the connection after 30 seconds of no data from the server. See the manual







          share|improve this answer














          share|improve this answer



          share|improve this answer








          edited Oct 3 '17 at 23:56

























          answered Oct 3 '17 at 20:43









          Outurnate

          914416




          914416







          • 1




            --tries=0 will retry indefinitely. The server may be intentionally leaving the connection open - try setting --read-timeout=15
            – Outurnate
            Oct 3 '17 at 20:52










          • @JBentley On it
            – Outurnate
            Oct 3 '17 at 23:54












          • 1




            --tries=0 will retry indefinitely. The server may be intentionally leaving the connection open - try setting --read-timeout=15
            – Outurnate
            Oct 3 '17 at 20:52










          • @JBentley On it
            – Outurnate
            Oct 3 '17 at 23:54







          1




          1




          --tries=0 will retry indefinitely. The server may be intentionally leaving the connection open - try setting --read-timeout=15
          – Outurnate
          Oct 3 '17 at 20:52




          --tries=0 will retry indefinitely. The server may be intentionally leaving the connection open - try setting --read-timeout=15
          – Outurnate
          Oct 3 '17 at 20:52












          @JBentley On it
          – Outurnate
          Oct 3 '17 at 23:54




          @JBentley On it
          – Outurnate
          Oct 3 '17 at 23:54

















           

          draft saved


          draft discarded















































           


          draft saved


          draft discarded














          StackExchange.ready(
          function ()
          StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2funix.stackexchange.com%2fquestions%2f395917%2fprevent-wget-hanging-when-it-encounters-error-500%23new-answer', 'question_page');

          );

          Post as a guest
































































          -