Remove all parent directories from a list of paths

The name of the pictureThe name of the pictureThe name of the pictureClash Royale CLAN TAG#URR8PPP











up vote
1
down vote

favorite












I have a bash command that lists the directories in HDFS beginning with '/random'.



It goes as follows:



hdfs dfs -ls -R /random | grep '^d' | awk 'print $NF'


It generates the following output:



/random/custom
/random/custom/hive
/random/custom/hive/jars
/random/app
/random/app/nifi
/random/app/nifi/egispub
/random/app/nifi/empgis
/random/app/nifi/ods
/random/app/nifi/ptcsm
/random/app/nifi/tmds
/random/app/nifi/ucmdb
/random/app/oozie
/random/app/oozie/efdm
/random/app/oozie/efdm/ensco
/random/app/oozie/efdm/ensco/standardized_efdm_ensco_parser_5min
/random/app/oozie/efdm/ensco/standardized_efdm_ensco_parser_5min/lib
/random/app/oozie/efdm/mermec
/random/app/oozie/efdm/mermec/cleansed_efdm_mermec_5min
/random/app/oozie/efdm/mermec/standardized_efdm_mermec_5min
/random/app/oozie/gis
/random/app/oozie/gis/opk
/random/app/oozie/gis/opk/standardized_gis_opk_parser_10min
/random/app/oozie/gis/opk/standardized_gis_opk_parser_10min/lib
/random/app/oozie/gis/opk/standardized_gis_opk_parser_10min/schemas


I would like to modify my command so that only the full directory path is listed, as follows:



/random/custom/hive/jars
/random/app/nifi/egispub
/random/app/nifi/empgis
/random/app/nifi/ods
/random/app/nifi/ptcsm
/random/app/nifi/tmds
/random/app/nifi/ucmdb
/random/app/oozie/efdm/ensco/standardized_efdm_ensco_parser_5min/lib
/random/app/oozie/efdm/mermec/cleansed_efdm_mermec_5min
/random/app/oozie/efdm/mermec/standardized_efdm_mermec_5min
/random/app/oozie/gis/opk/standardized_gis_opk_parser_10min/lib
/random/app/oozie/gis/opk/standardized_gis_opk_parser_10min/schemas


How would you do it?






text-processing







share|improve this question









New contributor




user316697 is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.



















  • "I would like to modify my command so that only the full directory path is listed." What do you mean by this? Do you mean that only /random/custom/hive/jars should be output (assuming jars is a directory)? Do you mean that only the path to the /random/custom directory should be listed? The full directory path to which location?
    – code_dredd
    4 hours ago











  • Also, did you try anything? If so, what was it?
    – code_dredd
    4 hours ago










  • Hi, I updated my post to answer your question. I have not tried anything yet. I am looking into awk, sed and grep as options.
    – user316697
    4 hours ago














up vote
1
down vote

favorite












I have a bash command that lists the directories in HDFS beginning with '/random'.



It goes as follows:



hdfs dfs -ls -R /random | grep '^d' | awk 'print $NF'


It generates the following output:



/random/custom
/random/custom/hive
/random/custom/hive/jars
/random/app
/random/app/nifi
/random/app/nifi/egispub
/random/app/nifi/empgis
/random/app/nifi/ods
/random/app/nifi/ptcsm
/random/app/nifi/tmds
/random/app/nifi/ucmdb
/random/app/oozie
/random/app/oozie/efdm
/random/app/oozie/efdm/ensco
/random/app/oozie/efdm/ensco/standardized_efdm_ensco_parser_5min
/random/app/oozie/efdm/ensco/standardized_efdm_ensco_parser_5min/lib
/random/app/oozie/efdm/mermec
/random/app/oozie/efdm/mermec/cleansed_efdm_mermec_5min
/random/app/oozie/efdm/mermec/standardized_efdm_mermec_5min
/random/app/oozie/gis
/random/app/oozie/gis/opk
/random/app/oozie/gis/opk/standardized_gis_opk_parser_10min
/random/app/oozie/gis/opk/standardized_gis_opk_parser_10min/lib
/random/app/oozie/gis/opk/standardized_gis_opk_parser_10min/schemas


I would like to modify my command so that only the full directory path is listed, as follows:



/random/custom/hive/jars
/random/app/nifi/egispub
/random/app/nifi/empgis
/random/app/nifi/ods
/random/app/nifi/ptcsm
/random/app/nifi/tmds
/random/app/nifi/ucmdb
/random/app/oozie/efdm/ensco/standardized_efdm_ensco_parser_5min/lib
/random/app/oozie/efdm/mermec/cleansed_efdm_mermec_5min
/random/app/oozie/efdm/mermec/standardized_efdm_mermec_5min
/random/app/oozie/gis/opk/standardized_gis_opk_parser_10min/lib
/random/app/oozie/gis/opk/standardized_gis_opk_parser_10min/schemas


How would you do it?






text-processing







share|improve this question









New contributor




user316697 is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.



















  • "I would like to modify my command so that only the full directory path is listed." What do you mean by this? Do you mean that only /random/custom/hive/jars should be output (assuming jars is a directory)? Do you mean that only the path to the /random/custom directory should be listed? The full directory path to which location?
    – code_dredd
    4 hours ago











  • Also, did you try anything? If so, what was it?
    – code_dredd
    4 hours ago










  • Hi, I updated my post to answer your question. I have not tried anything yet. I am looking into awk, sed and grep as options.
    – user316697
    4 hours ago












up vote
1
down vote

favorite









up vote
1
down vote

favorite











I have a bash command that lists the directories in HDFS beginning with '/random'.



It goes as follows:



hdfs dfs -ls -R /random | grep '^d' | awk 'print $NF'


It generates the following output:



/random/custom
/random/custom/hive
/random/custom/hive/jars
/random/app
/random/app/nifi
/random/app/nifi/egispub
/random/app/nifi/empgis
/random/app/nifi/ods
/random/app/nifi/ptcsm
/random/app/nifi/tmds
/random/app/nifi/ucmdb
/random/app/oozie
/random/app/oozie/efdm
/random/app/oozie/efdm/ensco
/random/app/oozie/efdm/ensco/standardized_efdm_ensco_parser_5min
/random/app/oozie/efdm/ensco/standardized_efdm_ensco_parser_5min/lib
/random/app/oozie/efdm/mermec
/random/app/oozie/efdm/mermec/cleansed_efdm_mermec_5min
/random/app/oozie/efdm/mermec/standardized_efdm_mermec_5min
/random/app/oozie/gis
/random/app/oozie/gis/opk
/random/app/oozie/gis/opk/standardized_gis_opk_parser_10min
/random/app/oozie/gis/opk/standardized_gis_opk_parser_10min/lib
/random/app/oozie/gis/opk/standardized_gis_opk_parser_10min/schemas


I would like to modify my command so that only the full directory path is listed, as follows:



/random/custom/hive/jars
/random/app/nifi/egispub
/random/app/nifi/empgis
/random/app/nifi/ods
/random/app/nifi/ptcsm
/random/app/nifi/tmds
/random/app/nifi/ucmdb
/random/app/oozie/efdm/ensco/standardized_efdm_ensco_parser_5min/lib
/random/app/oozie/efdm/mermec/cleansed_efdm_mermec_5min
/random/app/oozie/efdm/mermec/standardized_efdm_mermec_5min
/random/app/oozie/gis/opk/standardized_gis_opk_parser_10min/lib
/random/app/oozie/gis/opk/standardized_gis_opk_parser_10min/schemas


How would you do it?






text-processing







share|improve this question









New contributor




user316697 is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.











I have a bash command that lists the directories in HDFS beginning with '/random'.



It goes as follows:



hdfs dfs -ls -R /random | grep '^d' | awk 'print $NF'


It generates the following output:



/random/custom
/random/custom/hive
/random/custom/hive/jars
/random/app
/random/app/nifi
/random/app/nifi/egispub
/random/app/nifi/empgis
/random/app/nifi/ods
/random/app/nifi/ptcsm
/random/app/nifi/tmds
/random/app/nifi/ucmdb
/random/app/oozie
/random/app/oozie/efdm
/random/app/oozie/efdm/ensco
/random/app/oozie/efdm/ensco/standardized_efdm_ensco_parser_5min
/random/app/oozie/efdm/ensco/standardized_efdm_ensco_parser_5min/lib
/random/app/oozie/efdm/mermec
/random/app/oozie/efdm/mermec/cleansed_efdm_mermec_5min
/random/app/oozie/efdm/mermec/standardized_efdm_mermec_5min
/random/app/oozie/gis
/random/app/oozie/gis/opk
/random/app/oozie/gis/opk/standardized_gis_opk_parser_10min
/random/app/oozie/gis/opk/standardized_gis_opk_parser_10min/lib
/random/app/oozie/gis/opk/standardized_gis_opk_parser_10min/schemas


I would like to modify my command so that only the full directory path is listed, as follows:



/random/custom/hive/jars
/random/app/nifi/egispub
/random/app/nifi/empgis
/random/app/nifi/ods
/random/app/nifi/ptcsm
/random/app/nifi/tmds
/random/app/nifi/ucmdb
/random/app/oozie/efdm/ensco/standardized_efdm_ensco_parser_5min/lib
/random/app/oozie/efdm/mermec/cleansed_efdm_mermec_5min
/random/app/oozie/efdm/mermec/standardized_efdm_mermec_5min
/random/app/oozie/gis/opk/standardized_gis_opk_parser_10min/lib
/random/app/oozie/gis/opk/standardized_gis_opk_parser_10min/schemas


How would you do it?






text-processing




text-processing






share|improve this question









New contributor




user316697 is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.











share|improve this question









New contributor




user316697 is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.









share|improve this question




share|improve this question








edited 2 mins ago









don_crissti

48k15126155




48k15126155






New contributor




user316697 is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.









asked 4 hours ago









user316697

62




62




New contributor




user316697 is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.





New contributor





user316697 is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.






user316697 is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.











  • "I would like to modify my command so that only the full directory path is listed." What do you mean by this? Do you mean that only /random/custom/hive/jars should be output (assuming jars is a directory)? Do you mean that only the path to the /random/custom directory should be listed? The full directory path to which location?
    – code_dredd
    4 hours ago











  • Also, did you try anything? If so, what was it?
    – code_dredd
    4 hours ago










  • Hi, I updated my post to answer your question. I have not tried anything yet. I am looking into awk, sed and grep as options.
    – user316697
    4 hours ago
















  • "I would like to modify my command so that only the full directory path is listed." What do you mean by this? Do you mean that only /random/custom/hive/jars should be output (assuming jars is a directory)? Do you mean that only the path to the /random/custom directory should be listed? The full directory path to which location?
    – code_dredd
    4 hours ago











  • Also, did you try anything? If so, what was it?
    – code_dredd
    4 hours ago










  • Hi, I updated my post to answer your question. I have not tried anything yet. I am looking into awk, sed and grep as options.
    – user316697
    4 hours ago















"I would like to modify my command so that only the full directory path is listed." What do you mean by this? Do you mean that only /random/custom/hive/jars should be output (assuming jars is a directory)? Do you mean that only the path to the /random/custom directory should be listed? The full directory path to which location?
– code_dredd
4 hours ago





"I would like to modify my command so that only the full directory path is listed." What do you mean by this? Do you mean that only /random/custom/hive/jars should be output (assuming jars is a directory)? Do you mean that only the path to the /random/custom directory should be listed? The full directory path to which location?
– code_dredd
4 hours ago













Also, did you try anything? If so, what was it?
– code_dredd
4 hours ago




Also, did you try anything? If so, what was it?
– code_dredd
4 hours ago












Hi, I updated my post to answer your question. I have not tried anything yet. I am looking into awk, sed and grep as options.
– user316697
4 hours ago




Hi, I updated my post to answer your question. I have not tried anything yet. I am looking into awk, sed and grep as options.
– user316697
4 hours ago










3 Answers
3






active

oldest

votes

















up vote
1
down vote













You're looking to printing non-directory files only? Try piping your result through



awk -F/ 'NF <= OLDNF print LAST; OLDNF = NF; LAST = $0 END print LAST'
/random/custom/hive/jars
/random/app/nifi/egispub
/random/app/nifi/empgis
/random/app/nifi/ods
/random/app/nifi/ptcsm
/random/app/nifi/tmds
/random/app/nifi/ucmdb
/random/app/oozie/efdm/ensco/standardized_efdm_ensco_parser_5min/lib
/random/app/oozie/efdm/mermec/cleansed_efdm_mermec_5min
/random/app/oozie/efdm/mermec/standardized_efdm_mermec_5min
/random/app/oozie/gis/opk/standardized_gis_opk_parser_10min/lib
/random/app/oozie/gis/opk/standardized_gis_opk_parser_10min/schemas


It looks for "growing depth" of the path read, and prints if that shrank (going back in dir depth) or stays equal (file on same hierarchy level).






share|improve this answer




















  • lines need to be feed to awk with sorted and this will fail if you had a lines like /c/dn/e/f/cn/a/b. since you are checking the number of fields
    – sddgob
    2 hours ago


















up vote
0
down vote













Another awk approach which is based on matching current line against the previous line. and you must sort the input first.



... |sort | awk '$0 ~ pre pre=$0; next preprint pre pre=$0 END print '


Input:



/random/custom
/c/d
/e/f/c
/a/b
/random/custom/hive/jars
/random/app
/random/app/nifi
/random/app/nifi/egispub
/random/app/nifi/empgis
/random/app/nifi/ods
/random/app/nifi/ptcsm
/random/app/nifi/tmds
/random/app/nifi/ucmdb
/random/app/oozie
/random/app/oozie/efdm
/random/app/oozie/efdm/ensco
/random/app/oozie/efdm/ensco/standardized_efdm_ensco_parser_5min
/random/app/oozie/efdm/ensco/standardized_efdm_ensco_parser_5min/lib
/random/app/oozie/efdm/mermec
/random/app/oozie/efdm/mermec/cleansed_efdm_mermec_5min
/random/app/oozie/efdm/mermec/standardized_efdm_mermec_5min
/random/app/oozie/gis
/random/app/oozie/gis/opk
/random/app/oozie/gis/opk/standardized_gis_opk_parser_10min
/random/app/oozie/gis/opk/standardized_gis_opk_parser_10min/lib
/random/app/oozie/gis/opk/standardized_gis_opk_parser_10min/schemas


Output:



$ awk '$0 ~ pre pre=$0; next preprint pre pre=$0 END print ' <(sort infile)
/a/b
/c/d
/e/f/c
/random/app/nifi/egispub
/random/app/nifi/empgis
/random/app/nifi/ods
/random/app/nifi/ptcsm
/random/app/nifi/tmds
/random/app/nifi/ucmdb
/random/app/oozie/efdm/ensco/standardized_efdm_ensco_parser_5min/lib
/random/app/oozie/efdm/mermec/cleansed_efdm_mermec_5min
/random/app/oozie/efdm/mermec/standardized_efdm_mermec_5min
/random/app/oozie/gis/opk/standardized_gis_opk_parser_10min/lib
/random/app/oozie/gis/opk/standardized_gis_opk_parser_10min/schemas
/random/custom/hive/jars





share|improve this answer



























    up vote
    0
    down vote













    With modern seds (and assuming sorted input):



    sed -E '$!N;/(.+)n1/!P;D' infile


    If your sed is anticuado try:



    sed '$!N;/(.1,)n1/!P;D' infile




    share




















      Your Answer







      StackExchange.ready(function()
      var channelOptions =
      tags: "".split(" "),
      id: "106"
      ;
      initTagRenderer("".split(" "), "".split(" "), channelOptions);

      StackExchange.using("externalEditor", function()
      // Have to fire editor after snippets, if snippets enabled
      if (StackExchange.settings.snippets.snippetsEnabled)
      StackExchange.using("snippets", function()
      createEditor();
      );

      else
      createEditor();

      );

      function createEditor()
      StackExchange.prepareEditor(
      heartbeatType: 'answer',
      convertImagesToLinks: false,
      noModals: false,
      showLowRepImageUploadWarning: true,
      reputationToPostImages: null,
      bindNavPrevention: true,
      postfix: "",
      onDemand: true,
      discardSelector: ".discard-answer"
      ,immediatelyShowMarkdownHelp:true
      );



      );






      user316697 is a new contributor. Be nice, and check out our Code of Conduct.









       

      draft saved


      draft discarded


















      StackExchange.ready(
      function ()
      StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2funix.stackexchange.com%2fquestions%2f476577%2fremove-all-parent-directories-from-a-list-of-paths%23new-answer', 'question_page');

      );

      Post as a guest

















      3 Answers
      3






      active

      oldest

      votes








      3 Answers
      3






      active

      oldest

      votes









      active

      oldest

      votes






      active

      oldest

      votes








      up vote
      1
      down vote













      You're looking to printing non-directory files only? Try piping your result through



      awk -F/ 'NF <= OLDNF print LAST; OLDNF = NF; LAST = $0 END print LAST'
      /random/custom/hive/jars
      /random/app/nifi/egispub
      /random/app/nifi/empgis
      /random/app/nifi/ods
      /random/app/nifi/ptcsm
      /random/app/nifi/tmds
      /random/app/nifi/ucmdb
      /random/app/oozie/efdm/ensco/standardized_efdm_ensco_parser_5min/lib
      /random/app/oozie/efdm/mermec/cleansed_efdm_mermec_5min
      /random/app/oozie/efdm/mermec/standardized_efdm_mermec_5min
      /random/app/oozie/gis/opk/standardized_gis_opk_parser_10min/lib
      /random/app/oozie/gis/opk/standardized_gis_opk_parser_10min/schemas


      It looks for "growing depth" of the path read, and prints if that shrank (going back in dir depth) or stays equal (file on same hierarchy level).






      share|improve this answer




















      • lines need to be feed to awk with sorted and this will fail if you had a lines like /c/dn/e/f/cn/a/b. since you are checking the number of fields
        – sddgob
        2 hours ago















      up vote
      1
      down vote













      You're looking to printing non-directory files only? Try piping your result through



      awk -F/ 'NF <= OLDNF print LAST; OLDNF = NF; LAST = $0 END print LAST'
      /random/custom/hive/jars
      /random/app/nifi/egispub
      /random/app/nifi/empgis
      /random/app/nifi/ods
      /random/app/nifi/ptcsm
      /random/app/nifi/tmds
      /random/app/nifi/ucmdb
      /random/app/oozie/efdm/ensco/standardized_efdm_ensco_parser_5min/lib
      /random/app/oozie/efdm/mermec/cleansed_efdm_mermec_5min
      /random/app/oozie/efdm/mermec/standardized_efdm_mermec_5min
      /random/app/oozie/gis/opk/standardized_gis_opk_parser_10min/lib
      /random/app/oozie/gis/opk/standardized_gis_opk_parser_10min/schemas


      It looks for "growing depth" of the path read, and prints if that shrank (going back in dir depth) or stays equal (file on same hierarchy level).






      share|improve this answer




















      • lines need to be feed to awk with sorted and this will fail if you had a lines like /c/dn/e/f/cn/a/b. since you are checking the number of fields
        – sddgob
        2 hours ago













      up vote
      1
      down vote










      up vote
      1
      down vote









      You're looking to printing non-directory files only? Try piping your result through



      awk -F/ 'NF <= OLDNF print LAST; OLDNF = NF; LAST = $0 END print LAST'
      /random/custom/hive/jars
      /random/app/nifi/egispub
      /random/app/nifi/empgis
      /random/app/nifi/ods
      /random/app/nifi/ptcsm
      /random/app/nifi/tmds
      /random/app/nifi/ucmdb
      /random/app/oozie/efdm/ensco/standardized_efdm_ensco_parser_5min/lib
      /random/app/oozie/efdm/mermec/cleansed_efdm_mermec_5min
      /random/app/oozie/efdm/mermec/standardized_efdm_mermec_5min
      /random/app/oozie/gis/opk/standardized_gis_opk_parser_10min/lib
      /random/app/oozie/gis/opk/standardized_gis_opk_parser_10min/schemas


      It looks for "growing depth" of the path read, and prints if that shrank (going back in dir depth) or stays equal (file on same hierarchy level).






      share|improve this answer












      You're looking to printing non-directory files only? Try piping your result through



      awk -F/ 'NF <= OLDNF print LAST; OLDNF = NF; LAST = $0 END print LAST'
      /random/custom/hive/jars
      /random/app/nifi/egispub
      /random/app/nifi/empgis
      /random/app/nifi/ods
      /random/app/nifi/ptcsm
      /random/app/nifi/tmds
      /random/app/nifi/ucmdb
      /random/app/oozie/efdm/ensco/standardized_efdm_ensco_parser_5min/lib
      /random/app/oozie/efdm/mermec/cleansed_efdm_mermec_5min
      /random/app/oozie/efdm/mermec/standardized_efdm_mermec_5min
      /random/app/oozie/gis/opk/standardized_gis_opk_parser_10min/lib
      /random/app/oozie/gis/opk/standardized_gis_opk_parser_10min/schemas


      It looks for "growing depth" of the path read, and prints if that shrank (going back in dir depth) or stays equal (file on same hierarchy level).







      share|improve this answer












      share|improve this answer



      share|improve this answer










      answered 3 hours ago









      RudiC

      2,201110




      2,201110











      • lines need to be feed to awk with sorted and this will fail if you had a lines like /c/dn/e/f/cn/a/b. since you are checking the number of fields
        – sddgob
        2 hours ago

















      • lines need to be feed to awk with sorted and this will fail if you had a lines like /c/dn/e/f/cn/a/b. since you are checking the number of fields
        – sddgob
        2 hours ago
















      lines need to be feed to awk with sorted and this will fail if you had a lines like /c/dn/e/f/cn/a/b. since you are checking the number of fields
      – sddgob
      2 hours ago





      lines need to be feed to awk with sorted and this will fail if you had a lines like /c/dn/e/f/cn/a/b. since you are checking the number of fields
      – sddgob
      2 hours ago













      up vote
      0
      down vote













      Another awk approach which is based on matching current line against the previous line. and you must sort the input first.



      ... |sort | awk '$0 ~ pre pre=$0; next preprint pre pre=$0 END print '


      Input:



      /random/custom
      /c/d
      /e/f/c
      /a/b
      /random/custom/hive/jars
      /random/app
      /random/app/nifi
      /random/app/nifi/egispub
      /random/app/nifi/empgis
      /random/app/nifi/ods
      /random/app/nifi/ptcsm
      /random/app/nifi/tmds
      /random/app/nifi/ucmdb
      /random/app/oozie
      /random/app/oozie/efdm
      /random/app/oozie/efdm/ensco
      /random/app/oozie/efdm/ensco/standardized_efdm_ensco_parser_5min
      /random/app/oozie/efdm/ensco/standardized_efdm_ensco_parser_5min/lib
      /random/app/oozie/efdm/mermec
      /random/app/oozie/efdm/mermec/cleansed_efdm_mermec_5min
      /random/app/oozie/efdm/mermec/standardized_efdm_mermec_5min
      /random/app/oozie/gis
      /random/app/oozie/gis/opk
      /random/app/oozie/gis/opk/standardized_gis_opk_parser_10min
      /random/app/oozie/gis/opk/standardized_gis_opk_parser_10min/lib
      /random/app/oozie/gis/opk/standardized_gis_opk_parser_10min/schemas


      Output:



      $ awk '$0 ~ pre pre=$0; next preprint pre pre=$0 END print ' <(sort infile)
      /a/b
      /c/d
      /e/f/c
      /random/app/nifi/egispub
      /random/app/nifi/empgis
      /random/app/nifi/ods
      /random/app/nifi/ptcsm
      /random/app/nifi/tmds
      /random/app/nifi/ucmdb
      /random/app/oozie/efdm/ensco/standardized_efdm_ensco_parser_5min/lib
      /random/app/oozie/efdm/mermec/cleansed_efdm_mermec_5min
      /random/app/oozie/efdm/mermec/standardized_efdm_mermec_5min
      /random/app/oozie/gis/opk/standardized_gis_opk_parser_10min/lib
      /random/app/oozie/gis/opk/standardized_gis_opk_parser_10min/schemas
      /random/custom/hive/jars





      share|improve this answer
























        up vote
        0
        down vote













        Another awk approach which is based on matching current line against the previous line. and you must sort the input first.



        ... |sort | awk '$0 ~ pre pre=$0; next preprint pre pre=$0 END print '


        Input:



        /random/custom
        /c/d
        /e/f/c
        /a/b
        /random/custom/hive/jars
        /random/app
        /random/app/nifi
        /random/app/nifi/egispub
        /random/app/nifi/empgis
        /random/app/nifi/ods
        /random/app/nifi/ptcsm
        /random/app/nifi/tmds
        /random/app/nifi/ucmdb
        /random/app/oozie
        /random/app/oozie/efdm
        /random/app/oozie/efdm/ensco
        /random/app/oozie/efdm/ensco/standardized_efdm_ensco_parser_5min
        /random/app/oozie/efdm/ensco/standardized_efdm_ensco_parser_5min/lib
        /random/app/oozie/efdm/mermec
        /random/app/oozie/efdm/mermec/cleansed_efdm_mermec_5min
        /random/app/oozie/efdm/mermec/standardized_efdm_mermec_5min
        /random/app/oozie/gis
        /random/app/oozie/gis/opk
        /random/app/oozie/gis/opk/standardized_gis_opk_parser_10min
        /random/app/oozie/gis/opk/standardized_gis_opk_parser_10min/lib
        /random/app/oozie/gis/opk/standardized_gis_opk_parser_10min/schemas


        Output:



        $ awk '$0 ~ pre pre=$0; next preprint pre pre=$0 END print ' <(sort infile)
        /a/b
        /c/d
        /e/f/c
        /random/app/nifi/egispub
        /random/app/nifi/empgis
        /random/app/nifi/ods
        /random/app/nifi/ptcsm
        /random/app/nifi/tmds
        /random/app/nifi/ucmdb
        /random/app/oozie/efdm/ensco/standardized_efdm_ensco_parser_5min/lib
        /random/app/oozie/efdm/mermec/cleansed_efdm_mermec_5min
        /random/app/oozie/efdm/mermec/standardized_efdm_mermec_5min
        /random/app/oozie/gis/opk/standardized_gis_opk_parser_10min/lib
        /random/app/oozie/gis/opk/standardized_gis_opk_parser_10min/schemas
        /random/custom/hive/jars





        share|improve this answer






















          up vote
          0
          down vote










          up vote
          0
          down vote









          Another awk approach which is based on matching current line against the previous line. and you must sort the input first.



          ... |sort | awk '$0 ~ pre pre=$0; next preprint pre pre=$0 END print '


          Input:



          /random/custom
          /c/d
          /e/f/c
          /a/b
          /random/custom/hive/jars
          /random/app
          /random/app/nifi
          /random/app/nifi/egispub
          /random/app/nifi/empgis
          /random/app/nifi/ods
          /random/app/nifi/ptcsm
          /random/app/nifi/tmds
          /random/app/nifi/ucmdb
          /random/app/oozie
          /random/app/oozie/efdm
          /random/app/oozie/efdm/ensco
          /random/app/oozie/efdm/ensco/standardized_efdm_ensco_parser_5min
          /random/app/oozie/efdm/ensco/standardized_efdm_ensco_parser_5min/lib
          /random/app/oozie/efdm/mermec
          /random/app/oozie/efdm/mermec/cleansed_efdm_mermec_5min
          /random/app/oozie/efdm/mermec/standardized_efdm_mermec_5min
          /random/app/oozie/gis
          /random/app/oozie/gis/opk
          /random/app/oozie/gis/opk/standardized_gis_opk_parser_10min
          /random/app/oozie/gis/opk/standardized_gis_opk_parser_10min/lib
          /random/app/oozie/gis/opk/standardized_gis_opk_parser_10min/schemas


          Output:



          $ awk '$0 ~ pre pre=$0; next preprint pre pre=$0 END print ' <(sort infile)
          /a/b
          /c/d
          /e/f/c
          /random/app/nifi/egispub
          /random/app/nifi/empgis
          /random/app/nifi/ods
          /random/app/nifi/ptcsm
          /random/app/nifi/tmds
          /random/app/nifi/ucmdb
          /random/app/oozie/efdm/ensco/standardized_efdm_ensco_parser_5min/lib
          /random/app/oozie/efdm/mermec/cleansed_efdm_mermec_5min
          /random/app/oozie/efdm/mermec/standardized_efdm_mermec_5min
          /random/app/oozie/gis/opk/standardized_gis_opk_parser_10min/lib
          /random/app/oozie/gis/opk/standardized_gis_opk_parser_10min/schemas
          /random/custom/hive/jars





          share|improve this answer












          Another awk approach which is based on matching current line against the previous line. and you must sort the input first.



          ... |sort | awk '$0 ~ pre pre=$0; next preprint pre pre=$0 END print '


          Input:



          /random/custom
          /c/d
          /e/f/c
          /a/b
          /random/custom/hive/jars
          /random/app
          /random/app/nifi
          /random/app/nifi/egispub
          /random/app/nifi/empgis
          /random/app/nifi/ods
          /random/app/nifi/ptcsm
          /random/app/nifi/tmds
          /random/app/nifi/ucmdb
          /random/app/oozie
          /random/app/oozie/efdm
          /random/app/oozie/efdm/ensco
          /random/app/oozie/efdm/ensco/standardized_efdm_ensco_parser_5min
          /random/app/oozie/efdm/ensco/standardized_efdm_ensco_parser_5min/lib
          /random/app/oozie/efdm/mermec
          /random/app/oozie/efdm/mermec/cleansed_efdm_mermec_5min
          /random/app/oozie/efdm/mermec/standardized_efdm_mermec_5min
          /random/app/oozie/gis
          /random/app/oozie/gis/opk
          /random/app/oozie/gis/opk/standardized_gis_opk_parser_10min
          /random/app/oozie/gis/opk/standardized_gis_opk_parser_10min/lib
          /random/app/oozie/gis/opk/standardized_gis_opk_parser_10min/schemas


          Output:



          $ awk '$0 ~ pre pre=$0; next preprint pre pre=$0 END print ' <(sort infile)
          /a/b
          /c/d
          /e/f/c
          /random/app/nifi/egispub
          /random/app/nifi/empgis
          /random/app/nifi/ods
          /random/app/nifi/ptcsm
          /random/app/nifi/tmds
          /random/app/nifi/ucmdb
          /random/app/oozie/efdm/ensco/standardized_efdm_ensco_parser_5min/lib
          /random/app/oozie/efdm/mermec/cleansed_efdm_mermec_5min
          /random/app/oozie/efdm/mermec/standardized_efdm_mermec_5min
          /random/app/oozie/gis/opk/standardized_gis_opk_parser_10min/lib
          /random/app/oozie/gis/opk/standardized_gis_opk_parser_10min/schemas
          /random/custom/hive/jars






          share|improve this answer












          share|improve this answer



          share|improve this answer










          answered 2 hours ago









          sddgob

          16.2k102564




          16.2k102564




















              up vote
              0
              down vote













              With modern seds (and assuming sorted input):



              sed -E '$!N;/(.+)n1/!P;D' infile


              If your sed is anticuado try:



              sed '$!N;/(.1,)n1/!P;D' infile




              share
























                up vote
                0
                down vote













                With modern seds (and assuming sorted input):



                sed -E '$!N;/(.+)n1/!P;D' infile


                If your sed is anticuado try:



                sed '$!N;/(.1,)n1/!P;D' infile




                share






















                  up vote
                  0
                  down vote










                  up vote
                  0
                  down vote









                  With modern seds (and assuming sorted input):



                  sed -E '$!N;/(.+)n1/!P;D' infile


                  If your sed is anticuado try:



                  sed '$!N;/(.1,)n1/!P;D' infile




                  share












                  With modern seds (and assuming sorted input):



                  sed -E '$!N;/(.+)n1/!P;D' infile


                  If your sed is anticuado try:



                  sed '$!N;/(.1,)n1/!P;D' infile





                  share











                  share


                  share










                  answered 4 mins ago









                  don_crissti

                  48k15126155




                  48k15126155




















                      user316697 is a new contributor. Be nice, and check out our Code of Conduct.









                       

                      draft saved


                      draft discarded


















                      user316697 is a new contributor. Be nice, and check out our Code of Conduct.












                      user316697 is a new contributor. Be nice, and check out our Code of Conduct.











                      user316697 is a new contributor. Be nice, and check out our Code of Conduct.













                       


                      draft saved


                      draft discarded














                      StackExchange.ready(
                      function ()
                      StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2funix.stackexchange.com%2fquestions%2f476577%2fremove-all-parent-directories-from-a-list-of-paths%23new-answer', 'question_page');

                      );

                      Post as a guest
































































                      -

                      Popular posts from this blog

                      How to check contact read email or not when send email to Individual?

                      Image
                      Clash Royale CLAN TAG #URR8PPP up vote 1 down vote favorite I'm using WordPress 4.9.8, CiviCRM to 5.5.1, I usually send email to contact by Search> Find contacts View contact details Action> Send email Send email ok, Contact received mail ok like picture But status only Email sent though contact read email or not. So, can CiviCRM can change status to Email read when contact read email? wordpress email share | improve this question asked Sep 26 at 0:12 ToanLuong 49 9 add a comment  |  up vote 1 down vote favorite I'm using WordPress 4.9.8, CiviCRM to 5.5.1, I usually send email to contact by Search> Find contacts View contact details Action> Send email Send email ok, Contact received mail ok like picture But status only Email sent though contact read email or not. So, can CiviCRM can change status to Email read when contact read email? wordpress email share | improve this question asked Se
                      Read more

                      Bahrain

                      Image
                      For other uses, see Bahrain (disambiguation). Sovereign island state in the Persian Gulf Kingdom of Bahrain مملكة البحرين   (Arabic) Mamlakat al-Baḥrayn Flag Coat of arms Anthem:  نشيد البحرين الوطني Bahrainona Our Bahrain Location of   Bahrain    (circled in red) Capital and largest city Manama 26°13′N 50°35′E  /  26.217°N 50.583°E  / 26.217; 50.583 Official languages Arabic Ethnic groups (2010 [1] ) 50.7% Arab (46% Bahrani, 4.7% other Arabs) 45.5% Asian 1% European 1.2% Others Religion Islam Demonym(s) Bahraini Government Unitary constitutional monarchy • King Hamad bin Isa Al Khalifa • Crown Prince Salman bin Hamad bin Isa Al Khalifa • Prime Minister Khalifa bin Salman Al Khalifa Legislature National Assembly • Upper house Consultative Council • Lower house Council of Representatives Independence • Declared Independence [2] 14 August 1971 • from United Kingdom [1] 15 August 1971 • Admitted to the United Nations 21 September 1971 • Kingdom of Bahrain 14 February 2002
                      Read more

                      Postfix configuration issue with fips on centos 7; mailgun relay

                      Image
                      Clash Royale CLAN TAG #URR8PPP up vote 0 down vote favorite I am trying to setup postfix to relay all mail generated on the local machine via SMTP to a mailgun relay. I have used the mailgun relay before with success on an ubuntu server, but I am migrating to a Centos 7 server which I will be running in FIPS mode. There error log is below, slightly sanitized. I have a small enough network that I choose to have each machine reach out to mailgun individually (this the loopback-only, 127.0.0.0/8 restrictions) and no firewall open port allowing smtp in to the machine. I assume the FIPS mode (and with it disabling of MD5) is causing problems, but I don't know how to overcome it or if it is even possible for tls_fprint to use some supported hash such as sha256 or sha512. However, the relay=none is slightly concerning since I have relayhost set, but perhaps that is because the smtp process is failing? Any help would be appreciated! postconf -n: alias_database = hash:/etc/alia
                      Read more