mount.nfs: Stale file handle error - cannot umount

The name of the pictureThe name of the pictureThe name of the pictureClash Royale CLAN TAG#URR8PPP











up vote
2
down vote

favorite












Every time I try to mount a NFS share I get this:



>> mount -t nfs gitlab-replica-storage.blah.com:/export/registry-gitlab-prod-data-vol /mnt/test
mount.nfs: Stale file handle


The problem is that I cannot umount, as it says:



>> umount -f -l /mnt/test
umount: /mnt/test: not mounted


I tried checking if any process was using the mountpoint, but that is not the case.



Any other alternative to troubleshoot this?



As clarification:



  • I can mount it in another machine.

  • I cannot mount it in another mountpoint on the affected machine.






share|improve this question






















  • Are you positive the /export/registry-gitlab-prod-data-vol directory exists and has the correct permissions?
    – Jaken551
    Mar 23 at 12:56










  • @Jaken551 Yes, it is accessible by someone else. In fact I can mount it in another machine.
    – djuarez
    Mar 23 at 13:00











  • Try to add -v to mount -t command. See dmesg and /var/log/messages also. Maybe addition info will be issued. Are you trying reboot your machine?
    – Yurij Goncharuk
    Mar 23 at 15:00














up vote
2
down vote

favorite












Every time I try to mount a NFS share I get this:



>> mount -t nfs gitlab-replica-storage.blah.com:/export/registry-gitlab-prod-data-vol /mnt/test
mount.nfs: Stale file handle


The problem is that I cannot umount, as it says:



>> umount -f -l /mnt/test
umount: /mnt/test: not mounted


I tried checking if any process was using the mountpoint, but that is not the case.



Any other alternative to troubleshoot this?



As clarification:



  • I can mount it in another machine.

  • I cannot mount it in another mountpoint on the affected machine.






share|improve this question






















  • Are you positive the /export/registry-gitlab-prod-data-vol directory exists and has the correct permissions?
    – Jaken551
    Mar 23 at 12:56










  • @Jaken551 Yes, it is accessible by someone else. In fact I can mount it in another machine.
    – djuarez
    Mar 23 at 13:00











  • Try to add -v to mount -t command. See dmesg and /var/log/messages also. Maybe addition info will be issued. Are you trying reboot your machine?
    – Yurij Goncharuk
    Mar 23 at 15:00












up vote
2
down vote

favorite









up vote
2
down vote

favorite











Every time I try to mount a NFS share I get this:



>> mount -t nfs gitlab-replica-storage.blah.com:/export/registry-gitlab-prod-data-vol /mnt/test
mount.nfs: Stale file handle


The problem is that I cannot umount, as it says:



>> umount -f -l /mnt/test
umount: /mnt/test: not mounted


I tried checking if any process was using the mountpoint, but that is not the case.



Any other alternative to troubleshoot this?



As clarification:



  • I can mount it in another machine.

  • I cannot mount it in another mountpoint on the affected machine.






share|improve this question














Every time I try to mount a NFS share I get this:



>> mount -t nfs gitlab-replica-storage.blah.com:/export/registry-gitlab-prod-data-vol /mnt/test
mount.nfs: Stale file handle


The problem is that I cannot umount, as it says:



>> umount -f -l /mnt/test
umount: /mnt/test: not mounted


I tried checking if any process was using the mountpoint, but that is not the case.



Any other alternative to troubleshoot this?



As clarification:



  • I can mount it in another machine.

  • I cannot mount it in another mountpoint on the affected machine.








share|improve this question













share|improve this question




share|improve this question








edited Jul 19 at 19:34









Jeff Schaller

31.2k846105




31.2k846105










asked Mar 23 at 12:51









djuarez

1155




1155











  • Are you positive the /export/registry-gitlab-prod-data-vol directory exists and has the correct permissions?
    – Jaken551
    Mar 23 at 12:56










  • @Jaken551 Yes, it is accessible by someone else. In fact I can mount it in another machine.
    – djuarez
    Mar 23 at 13:00











  • Try to add -v to mount -t command. See dmesg and /var/log/messages also. Maybe addition info will be issued. Are you trying reboot your machine?
    – Yurij Goncharuk
    Mar 23 at 15:00
















  • Are you positive the /export/registry-gitlab-prod-data-vol directory exists and has the correct permissions?
    – Jaken551
    Mar 23 at 12:56










  • @Jaken551 Yes, it is accessible by someone else. In fact I can mount it in another machine.
    – djuarez
    Mar 23 at 13:00











  • Try to add -v to mount -t command. See dmesg and /var/log/messages also. Maybe addition info will be issued. Are you trying reboot your machine?
    – Yurij Goncharuk
    Mar 23 at 15:00















Are you positive the /export/registry-gitlab-prod-data-vol directory exists and has the correct permissions?
– Jaken551
Mar 23 at 12:56




Are you positive the /export/registry-gitlab-prod-data-vol directory exists and has the correct permissions?
– Jaken551
Mar 23 at 12:56












@Jaken551 Yes, it is accessible by someone else. In fact I can mount it in another machine.
– djuarez
Mar 23 at 13:00





@Jaken551 Yes, it is accessible by someone else. In fact I can mount it in another machine.
– djuarez
Mar 23 at 13:00













Try to add -v to mount -t command. See dmesg and /var/log/messages also. Maybe addition info will be issued. Are you trying reboot your machine?
– Yurij Goncharuk
Mar 23 at 15:00




Try to add -v to mount -t command. See dmesg and /var/log/messages also. Maybe addition info will be issued. Are you trying reboot your machine?
– Yurij Goncharuk
Mar 23 at 15:00










3 Answers
3






active

oldest

votes

















up vote
1
down vote



accepted











The error, ESTALE, was originally introduced to handle the situation
where a file handle, which NFS uses to uniquely identify a file on the
server, no longer refers to a valid file on the server. This can
happen when the file is removed on the server, either by an
application on the server, some other client accessing the server, or
sometimes even by another mounted file system from the same client.
The NFS server also returns this error when the file resides upon a
file system which is no longer exported. Additionally, some NFS
servers even change the file handle when a file is renamed, although
this practice is discouraged.



This error occurs even if a file or directory, with the same name, is
recreated on the server without the client being aware of it. The
file handle refers to a specific instance of a file and deleting the
file and then recreating it creates a new instance of the file.



The error, ESTALE, is usually seen when cached directory information
is used to convert a pathname to a dentry/inode pair. The information
is discovered to be out of date or stale when a subsequent operation
is sent to the NFS server. This can easily happen in system calls
such as stat(2) when the pathname is converted a dentry/inode pair
using cached information, but then a subsequent GETATTR call to the
server discovers that the file handle is no longer valid.



This error can also occur when a change is made on the server in
between looking up different components of the pathname to be looked
up or between a successful lookup and a subsequent operation.




Original link about ESTALE: ESTALE LWN .



I suggest to you check files and directories on NFS server or say to admin of NFS server to do this.



Maybe some old pagecache, inode, dentry cache entries are exists on NFS server. Please clean it:



# To free pagecache
echo 1 > /proc/sys/vm/drop_caches

# To free dentries and inodes
echo 2 > /proc/sys/vm/drop_caches

# To free pagecache, dentries and inodes
echo 3 > /proc/sys/vm/drop_caches





share|improve this answer





























    up vote
    0
    down vote













    Check whether the export is actually mounted:



    # cat /proc/mounts | grep nfs


    Stale file handle error means that the NFS server holds an old version of the files in his export path. An NFS server restart can sometimes help.
    But with older OSs (RHEL/CentOS 6.9) it is sometimes better to revert to NFS3 instead of NFS4. In my experience older NFS4 clients have sometimes difficulties with the newer NFS4.1 servers. This is especially true for file locking.






    share|improve this answer



























      up vote
      0
      down vote













      A mount -t nfs fails with Stale file handle if the server has some stale exports entries for that client.



      Example scenario: this might happen when the server reboots without the client umounting the nfs volumes first. When the server is back and the client then umounts and tries to mount the nfs volume the server might respond with:



      mount.nfs: Stale file handle


      You can check for this via looking at /proc/fs/nfs/exports or /proc/fs/nfsd/exports. If there is entry for the client it might be a stale one.



      You can fix this via explicitly un-exporting and re-exporting the relevant exports on the server. For example to do this with all exports:



      # exportfs -ua
      # cat /proc/fs/nfs/exports
      # exportfs -a


      After this the client's mount -t nfs ... should succeed.



      Note that mount yielding ESTALE is quite different from some other system call (like open/readdir/unlink/chdir ...) returning ESTALE. It's export being stale vs. a file handle being stale. A stale file handle easily happens with NFS (e.g. a client has a file handle but the file got deleted on the server).






      share|improve this answer




















        Your Answer







        StackExchange.ready(function()
        var channelOptions =
        tags: "".split(" "),
        id: "106"
        ;
        initTagRenderer("".split(" "), "".split(" "), channelOptions);

        StackExchange.using("externalEditor", function()
        // Have to fire editor after snippets, if snippets enabled
        if (StackExchange.settings.snippets.snippetsEnabled)
        StackExchange.using("snippets", function()
        createEditor();
        );

        else
        createEditor();

        );

        function createEditor()
        StackExchange.prepareEditor(
        heartbeatType: 'answer',
        convertImagesToLinks: false,
        noModals: false,
        showLowRepImageUploadWarning: true,
        reputationToPostImages: null,
        bindNavPrevention: true,
        postfix: "",
        onDemand: true,
        discardSelector: ".discard-answer"
        ,immediatelyShowMarkdownHelp:true
        );



        );








         

        draft saved


        draft discarded


















        StackExchange.ready(
        function ()
        StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2funix.stackexchange.com%2fquestions%2f433051%2fmount-nfs-stale-file-handle-error-cannot-umount%23new-answer', 'question_page');

        );

        Post as a guest






























        3 Answers
        3






        active

        oldest

        votes








        3 Answers
        3






        active

        oldest

        votes









        active

        oldest

        votes






        active

        oldest

        votes








        up vote
        1
        down vote



        accepted











        The error, ESTALE, was originally introduced to handle the situation
        where a file handle, which NFS uses to uniquely identify a file on the
        server, no longer refers to a valid file on the server. This can
        happen when the file is removed on the server, either by an
        application on the server, some other client accessing the server, or
        sometimes even by another mounted file system from the same client.
        The NFS server also returns this error when the file resides upon a
        file system which is no longer exported. Additionally, some NFS
        servers even change the file handle when a file is renamed, although
        this practice is discouraged.



        This error occurs even if a file or directory, with the same name, is
        recreated on the server without the client being aware of it. The
        file handle refers to a specific instance of a file and deleting the
        file and then recreating it creates a new instance of the file.



        The error, ESTALE, is usually seen when cached directory information
        is used to convert a pathname to a dentry/inode pair. The information
        is discovered to be out of date or stale when a subsequent operation
        is sent to the NFS server. This can easily happen in system calls
        such as stat(2) when the pathname is converted a dentry/inode pair
        using cached information, but then a subsequent GETATTR call to the
        server discovers that the file handle is no longer valid.



        This error can also occur when a change is made on the server in
        between looking up different components of the pathname to be looked
        up or between a successful lookup and a subsequent operation.




        Original link about ESTALE: ESTALE LWN .



        I suggest to you check files and directories on NFS server or say to admin of NFS server to do this.



        Maybe some old pagecache, inode, dentry cache entries are exists on NFS server. Please clean it:



        # To free pagecache
        echo 1 > /proc/sys/vm/drop_caches

        # To free dentries and inodes
        echo 2 > /proc/sys/vm/drop_caches

        # To free pagecache, dentries and inodes
        echo 3 > /proc/sys/vm/drop_caches





        share|improve this answer


























          up vote
          1
          down vote



          accepted











          The error, ESTALE, was originally introduced to handle the situation
          where a file handle, which NFS uses to uniquely identify a file on the
          server, no longer refers to a valid file on the server. This can
          happen when the file is removed on the server, either by an
          application on the server, some other client accessing the server, or
          sometimes even by another mounted file system from the same client.
          The NFS server also returns this error when the file resides upon a
          file system which is no longer exported. Additionally, some NFS
          servers even change the file handle when a file is renamed, although
          this practice is discouraged.



          This error occurs even if a file or directory, with the same name, is
          recreated on the server without the client being aware of it. The
          file handle refers to a specific instance of a file and deleting the
          file and then recreating it creates a new instance of the file.



          The error, ESTALE, is usually seen when cached directory information
          is used to convert a pathname to a dentry/inode pair. The information
          is discovered to be out of date or stale when a subsequent operation
          is sent to the NFS server. This can easily happen in system calls
          such as stat(2) when the pathname is converted a dentry/inode pair
          using cached information, but then a subsequent GETATTR call to the
          server discovers that the file handle is no longer valid.



          This error can also occur when a change is made on the server in
          between looking up different components of the pathname to be looked
          up or between a successful lookup and a subsequent operation.




          Original link about ESTALE: ESTALE LWN .



          I suggest to you check files and directories on NFS server or say to admin of NFS server to do this.



          Maybe some old pagecache, inode, dentry cache entries are exists on NFS server. Please clean it:



          # To free pagecache
          echo 1 > /proc/sys/vm/drop_caches

          # To free dentries and inodes
          echo 2 > /proc/sys/vm/drop_caches

          # To free pagecache, dentries and inodes
          echo 3 > /proc/sys/vm/drop_caches





          share|improve this answer
























            up vote
            1
            down vote



            accepted







            up vote
            1
            down vote



            accepted







            The error, ESTALE, was originally introduced to handle the situation
            where a file handle, which NFS uses to uniquely identify a file on the
            server, no longer refers to a valid file on the server. This can
            happen when the file is removed on the server, either by an
            application on the server, some other client accessing the server, or
            sometimes even by another mounted file system from the same client.
            The NFS server also returns this error when the file resides upon a
            file system which is no longer exported. Additionally, some NFS
            servers even change the file handle when a file is renamed, although
            this practice is discouraged.



            This error occurs even if a file or directory, with the same name, is
            recreated on the server without the client being aware of it. The
            file handle refers to a specific instance of a file and deleting the
            file and then recreating it creates a new instance of the file.



            The error, ESTALE, is usually seen when cached directory information
            is used to convert a pathname to a dentry/inode pair. The information
            is discovered to be out of date or stale when a subsequent operation
            is sent to the NFS server. This can easily happen in system calls
            such as stat(2) when the pathname is converted a dentry/inode pair
            using cached information, but then a subsequent GETATTR call to the
            server discovers that the file handle is no longer valid.



            This error can also occur when a change is made on the server in
            between looking up different components of the pathname to be looked
            up or between a successful lookup and a subsequent operation.




            Original link about ESTALE: ESTALE LWN .



            I suggest to you check files and directories on NFS server or say to admin of NFS server to do this.



            Maybe some old pagecache, inode, dentry cache entries are exists on NFS server. Please clean it:



            # To free pagecache
            echo 1 > /proc/sys/vm/drop_caches

            # To free dentries and inodes
            echo 2 > /proc/sys/vm/drop_caches

            # To free pagecache, dentries and inodes
            echo 3 > /proc/sys/vm/drop_caches





            share|improve this answer















            The error, ESTALE, was originally introduced to handle the situation
            where a file handle, which NFS uses to uniquely identify a file on the
            server, no longer refers to a valid file on the server. This can
            happen when the file is removed on the server, either by an
            application on the server, some other client accessing the server, or
            sometimes even by another mounted file system from the same client.
            The NFS server also returns this error when the file resides upon a
            file system which is no longer exported. Additionally, some NFS
            servers even change the file handle when a file is renamed, although
            this practice is discouraged.



            This error occurs even if a file or directory, with the same name, is
            recreated on the server without the client being aware of it. The
            file handle refers to a specific instance of a file and deleting the
            file and then recreating it creates a new instance of the file.



            The error, ESTALE, is usually seen when cached directory information
            is used to convert a pathname to a dentry/inode pair. The information
            is discovered to be out of date or stale when a subsequent operation
            is sent to the NFS server. This can easily happen in system calls
            such as stat(2) when the pathname is converted a dentry/inode pair
            using cached information, but then a subsequent GETATTR call to the
            server discovers that the file handle is no longer valid.



            This error can also occur when a change is made on the server in
            between looking up different components of the pathname to be looked
            up or between a successful lookup and a subsequent operation.




            Original link about ESTALE: ESTALE LWN .



            I suggest to you check files and directories on NFS server or say to admin of NFS server to do this.



            Maybe some old pagecache, inode, dentry cache entries are exists on NFS server. Please clean it:



            # To free pagecache
            echo 1 > /proc/sys/vm/drop_caches

            # To free dentries and inodes
            echo 2 > /proc/sys/vm/drop_caches

            # To free pagecache, dentries and inodes
            echo 3 > /proc/sys/vm/drop_caches






            share|improve this answer














            share|improve this answer



            share|improve this answer








            edited Mar 23 at 14:34

























            answered Mar 23 at 13:41









            Yurij Goncharuk

            2,2582521




            2,2582521






















                up vote
                0
                down vote













                Check whether the export is actually mounted:



                # cat /proc/mounts | grep nfs


                Stale file handle error means that the NFS server holds an old version of the files in his export path. An NFS server restart can sometimes help.
                But with older OSs (RHEL/CentOS 6.9) it is sometimes better to revert to NFS3 instead of NFS4. In my experience older NFS4 clients have sometimes difficulties with the newer NFS4.1 servers. This is especially true for file locking.






                share|improve this answer
























                  up vote
                  0
                  down vote













                  Check whether the export is actually mounted:



                  # cat /proc/mounts | grep nfs


                  Stale file handle error means that the NFS server holds an old version of the files in his export path. An NFS server restart can sometimes help.
                  But with older OSs (RHEL/CentOS 6.9) it is sometimes better to revert to NFS3 instead of NFS4. In my experience older NFS4 clients have sometimes difficulties with the newer NFS4.1 servers. This is especially true for file locking.






                  share|improve this answer






















                    up vote
                    0
                    down vote










                    up vote
                    0
                    down vote









                    Check whether the export is actually mounted:



                    # cat /proc/mounts | grep nfs


                    Stale file handle error means that the NFS server holds an old version of the files in his export path. An NFS server restart can sometimes help.
                    But with older OSs (RHEL/CentOS 6.9) it is sometimes better to revert to NFS3 instead of NFS4. In my experience older NFS4 clients have sometimes difficulties with the newer NFS4.1 servers. This is especially true for file locking.






                    share|improve this answer












                    Check whether the export is actually mounted:



                    # cat /proc/mounts | grep nfs


                    Stale file handle error means that the NFS server holds an old version of the files in his export path. An NFS server restart can sometimes help.
                    But with older OSs (RHEL/CentOS 6.9) it is sometimes better to revert to NFS3 instead of NFS4. In my experience older NFS4 clients have sometimes difficulties with the newer NFS4.1 servers. This is especially true for file locking.







                    share|improve this answer












                    share|improve this answer



                    share|improve this answer










                    answered Mar 23 at 13:41









                    monkeywrench

                    31




                    31




















                        up vote
                        0
                        down vote













                        A mount -t nfs fails with Stale file handle if the server has some stale exports entries for that client.



                        Example scenario: this might happen when the server reboots without the client umounting the nfs volumes first. When the server is back and the client then umounts and tries to mount the nfs volume the server might respond with:



                        mount.nfs: Stale file handle


                        You can check for this via looking at /proc/fs/nfs/exports or /proc/fs/nfsd/exports. If there is entry for the client it might be a stale one.



                        You can fix this via explicitly un-exporting and re-exporting the relevant exports on the server. For example to do this with all exports:



                        # exportfs -ua
                        # cat /proc/fs/nfs/exports
                        # exportfs -a


                        After this the client's mount -t nfs ... should succeed.



                        Note that mount yielding ESTALE is quite different from some other system call (like open/readdir/unlink/chdir ...) returning ESTALE. It's export being stale vs. a file handle being stale. A stale file handle easily happens with NFS (e.g. a client has a file handle but the file got deleted on the server).






                        share|improve this answer
























                          up vote
                          0
                          down vote













                          A mount -t nfs fails with Stale file handle if the server has some stale exports entries for that client.



                          Example scenario: this might happen when the server reboots without the client umounting the nfs volumes first. When the server is back and the client then umounts and tries to mount the nfs volume the server might respond with:



                          mount.nfs: Stale file handle


                          You can check for this via looking at /proc/fs/nfs/exports or /proc/fs/nfsd/exports. If there is entry for the client it might be a stale one.



                          You can fix this via explicitly un-exporting and re-exporting the relevant exports on the server. For example to do this with all exports:



                          # exportfs -ua
                          # cat /proc/fs/nfs/exports
                          # exportfs -a


                          After this the client's mount -t nfs ... should succeed.



                          Note that mount yielding ESTALE is quite different from some other system call (like open/readdir/unlink/chdir ...) returning ESTALE. It's export being stale vs. a file handle being stale. A stale file handle easily happens with NFS (e.g. a client has a file handle but the file got deleted on the server).






                          share|improve this answer






















                            up vote
                            0
                            down vote










                            up vote
                            0
                            down vote









                            A mount -t nfs fails with Stale file handle if the server has some stale exports entries for that client.



                            Example scenario: this might happen when the server reboots without the client umounting the nfs volumes first. When the server is back and the client then umounts and tries to mount the nfs volume the server might respond with:



                            mount.nfs: Stale file handle


                            You can check for this via looking at /proc/fs/nfs/exports or /proc/fs/nfsd/exports. If there is entry for the client it might be a stale one.



                            You can fix this via explicitly un-exporting and re-exporting the relevant exports on the server. For example to do this with all exports:



                            # exportfs -ua
                            # cat /proc/fs/nfs/exports
                            # exportfs -a


                            After this the client's mount -t nfs ... should succeed.



                            Note that mount yielding ESTALE is quite different from some other system call (like open/readdir/unlink/chdir ...) returning ESTALE. It's export being stale vs. a file handle being stale. A stale file handle easily happens with NFS (e.g. a client has a file handle but the file got deleted on the server).






                            share|improve this answer












                            A mount -t nfs fails with Stale file handle if the server has some stale exports entries for that client.



                            Example scenario: this might happen when the server reboots without the client umounting the nfs volumes first. When the server is back and the client then umounts and tries to mount the nfs volume the server might respond with:



                            mount.nfs: Stale file handle


                            You can check for this via looking at /proc/fs/nfs/exports or /proc/fs/nfsd/exports. If there is entry for the client it might be a stale one.



                            You can fix this via explicitly un-exporting and re-exporting the relevant exports on the server. For example to do this with all exports:



                            # exportfs -ua
                            # cat /proc/fs/nfs/exports
                            # exportfs -a


                            After this the client's mount -t nfs ... should succeed.



                            Note that mount yielding ESTALE is quite different from some other system call (like open/readdir/unlink/chdir ...) returning ESTALE. It's export being stale vs. a file handle being stale. A stale file handle easily happens with NFS (e.g. a client has a file handle but the file got deleted on the server).







                            share|improve this answer












                            share|improve this answer



                            share|improve this answer










                            answered Jun 3 at 7:56









                            maxschlepzig

                            31.9k29135202




                            31.9k29135202






















                                 

                                draft saved


                                draft discarded


























                                 


                                draft saved


                                draft discarded














                                StackExchange.ready(
                                function ()
                                StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2funix.stackexchange.com%2fquestions%2f433051%2fmount-nfs-stale-file-handle-error-cannot-umount%23new-answer', 'question_page');

                                );

                                Post as a guest













































































                                Popular posts from this blog

                                How to check contact read email or not when send email to Individual?

                                Bahrain

                                Postfix configuration issue with fips on centos 7; mailgun relay