Pandas reset inner level of MultiIndex

The name of the pictureThe name of the pictureThe name of the pictureClash Royale CLAN TAG#URR8PPP











up vote
9
down vote

favorite












I have a DF in the following format:



 col1 col2
ID Date
1 1993-12-31 4 6
1994-12-31 8 5
1995-12-31 4 7
1996-12-31 3 3
2 2000-12-31 7 8
2001-12-31 5 9
2002-12-31 8 4


And I want to reset the 'Date' index giving the following:



 col1 col2
ID Date
1 0 4 6
1 8 5
2 4 7
3 3 3
2 0 7 8
1 5 9
2 8 4


I thought simply df.reset_index(level='Date', inplace=True, drop=True) would do it, but it does not.










share|improve this question

























    up vote
    9
    down vote

    favorite












    I have a DF in the following format:



     col1 col2
    ID Date
    1 1993-12-31 4 6
    1994-12-31 8 5
    1995-12-31 4 7
    1996-12-31 3 3
    2 2000-12-31 7 8
    2001-12-31 5 9
    2002-12-31 8 4


    And I want to reset the 'Date' index giving the following:



     col1 col2
    ID Date
    1 0 4 6
    1 8 5
    2 4 7
    3 3 3
    2 0 7 8
    1 5 9
    2 8 4


    I thought simply df.reset_index(level='Date', inplace=True, drop=True) would do it, but it does not.










    share|improve this question























      up vote
      9
      down vote

      favorite









      up vote
      9
      down vote

      favorite











      I have a DF in the following format:



       col1 col2
      ID Date
      1 1993-12-31 4 6
      1994-12-31 8 5
      1995-12-31 4 7
      1996-12-31 3 3
      2 2000-12-31 7 8
      2001-12-31 5 9
      2002-12-31 8 4


      And I want to reset the 'Date' index giving the following:



       col1 col2
      ID Date
      1 0 4 6
      1 8 5
      2 4 7
      3 3 3
      2 0 7 8
      1 5 9
      2 8 4


      I thought simply df.reset_index(level='Date', inplace=True, drop=True) would do it, but it does not.










      share|improve this question













      I have a DF in the following format:



       col1 col2
      ID Date
      1 1993-12-31 4 6
      1994-12-31 8 5
      1995-12-31 4 7
      1996-12-31 3 3
      2 2000-12-31 7 8
      2001-12-31 5 9
      2002-12-31 8 4


      And I want to reset the 'Date' index giving the following:



       col1 col2
      ID Date
      1 0 4 6
      1 8 5
      2 4 7
      3 3 3
      2 0 7 8
      1 5 9
      2 8 4


      I thought simply df.reset_index(level='Date', inplace=True, drop=True) would do it, but it does not.







      python pandas






      share|improve this question













      share|improve this question











      share|improve this question




      share|improve this question










      asked Aug 13 at 18:31









      KOB

      1,011617




      1,011617






















          4 Answers
          4






          active

          oldest

          votes

















          up vote
          4
          down vote



          accepted










          Using pd.MultiIndex.from_arrays and groupby + cumcount.



          df.index = pd.MultiIndex.from_arrays(
          [df.index.get_level_values(0), df.groupby(level=0).cumcount()],
          names=['ID', 'Date'])




          df
          col1 col2
          ID Date
          1 0 4 6
          1 8 5
          2 4 7
          3 3 3
          2 0 7 8
          1 5 9
          2 8 4


          This won't generalise to N levels, but there should be a df.index.set_levels equivalent I'm forgetting...






          share|improve this answer
















          • 2




            I find this solution easiest to follow.
            – KOB
            Aug 13 at 18:52

















          up vote
          4
          down vote













          You can groupby ID, then reset the index on each group using apply:



          new_df = (df.groupby(df.index.get_level_values('ID'))
          .apply(lambda x: x.reset_index()).drop(['ID','Date'],1))

          new_df.index = new_df.index.rename(['ID','Date'])

          >>> new_df
          col1 col2
          ID Date
          1 0 4 6
          1 8 5
          2 4 7
          3 3 3
          2 0 7 8
          1 5 9
          2 8 4





          share|improve this answer



























            up vote
            4
            down vote













            Using set_index and cumcount:



            tmp = df.reset_index('Date', drop=True)
            tmp.set_index(df.groupby(level=0).cumcount().rename('Date'), append=True)

            col1 col2
            ID Date
            1 0 4 6
            1 8 5
            2 4 7
            3 3 3
            2 0 7 8
            1 5 9
            2 8 4





            share|improve this answer





























              up vote
              3
              down vote













              New Answer



              Not as cool as the old answer but I'd rather be accurate than cool.



              from collections import defaultdict
              from itertools import count
              d = defaultdict(count)

              lbl =
              for a, *_ in df.index.values:
              lbl.append(next(d[a]))

              lvl = pd.RangeIndex(max(lbl) + 1)

              df.set_index(df.index.set_labels(lbl, 1).set_levels(lvl, 1))

              col1 col2
              ID Date
              1 0 4 6
              1 8 5
              2 4 7
              3 3 3
              2 0 7 8
              1 5 9
              2 8 4



              OLD ANSWER



              Do Not Use



              I misread the question. I didn't see that the new index needed to reset for every group.



              Hopefully useful to someone.



              You can use pandas.MultiIndex.set_levels



              n = 1
              lvl = df.index.levels[n]
              new_lvl = pd.RangeIndex(len(lvl))
              new_idx = df.index.set_levels(new_lvl, n)
              df.set_index(new_idx)

              col1 col2
              ID Date
              1 0 4 6
              1 8 5
              2 4 7
              3 3 3
              2 4 7 8
              5 5 9
              6 8 4



              One-line



              Yay! o/



              df.set_index(df.index.set_levels(pd.RangeIndex(len(df.index.levels[1])), 1))

              col1 col2
              ID Date
              1 0 4 6
              1 8 5
              2 4 7
              3 3 3
              2 4 7 8
              5 5 9
              6 8 4



              In place



              df.index.set_levels(pd.RangeIndex(len(df.index.levels[1])), 1, inplace=True)
              df

              col1 col2
              ID Date
              1 0 4 6
              1 8 5
              2 4 7
              3 3 3
              2 4 7 8
              5 5 9
              6 8 4





              share|improve this answer






















                Your Answer





                StackExchange.ifUsing("editor", function ()
                StackExchange.using("externalEditor", function ()
                StackExchange.using("snippets", function ()
                StackExchange.snippets.init();
                );
                );
                , "code-snippets");

                StackExchange.ready(function()
                var channelOptions =
                tags: "".split(" "),
                id: "1"
                ;
                initTagRenderer("".split(" "), "".split(" "), channelOptions);

                StackExchange.using("externalEditor", function()
                // Have to fire editor after snippets, if snippets enabled
                if (StackExchange.settings.snippets.snippetsEnabled)
                StackExchange.using("snippets", function()
                createEditor();
                );

                else
                createEditor();

                );

                function createEditor()
                StackExchange.prepareEditor(
                heartbeatType: 'answer',
                convertImagesToLinks: true,
                noModals: false,
                showLowRepImageUploadWarning: true,
                reputationToPostImages: 10,
                bindNavPrevention: true,
                postfix: "",
                onDemand: true,
                discardSelector: ".discard-answer"
                ,immediatelyShowMarkdownHelp:true
                );



                );













                 

                draft saved


                draft discarded


















                StackExchange.ready(
                function ()
                StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f51828518%2fpandas-reset-inner-level-of-multiindex%23new-answer', 'question_page');

                );

                Post as a guest






























                4 Answers
                4






                active

                oldest

                votes








                4 Answers
                4






                active

                oldest

                votes









                active

                oldest

                votes






                active

                oldest

                votes








                up vote
                4
                down vote



                accepted










                Using pd.MultiIndex.from_arrays and groupby + cumcount.



                df.index = pd.MultiIndex.from_arrays(
                [df.index.get_level_values(0), df.groupby(level=0).cumcount()],
                names=['ID', 'Date'])




                df
                col1 col2
                ID Date
                1 0 4 6
                1 8 5
                2 4 7
                3 3 3
                2 0 7 8
                1 5 9
                2 8 4


                This won't generalise to N levels, but there should be a df.index.set_levels equivalent I'm forgetting...






                share|improve this answer
















                • 2




                  I find this solution easiest to follow.
                  – KOB
                  Aug 13 at 18:52














                up vote
                4
                down vote



                accepted










                Using pd.MultiIndex.from_arrays and groupby + cumcount.



                df.index = pd.MultiIndex.from_arrays(
                [df.index.get_level_values(0), df.groupby(level=0).cumcount()],
                names=['ID', 'Date'])




                df
                col1 col2
                ID Date
                1 0 4 6
                1 8 5
                2 4 7
                3 3 3
                2 0 7 8
                1 5 9
                2 8 4


                This won't generalise to N levels, but there should be a df.index.set_levels equivalent I'm forgetting...






                share|improve this answer
















                • 2




                  I find this solution easiest to follow.
                  – KOB
                  Aug 13 at 18:52












                up vote
                4
                down vote



                accepted







                up vote
                4
                down vote



                accepted






                Using pd.MultiIndex.from_arrays and groupby + cumcount.



                df.index = pd.MultiIndex.from_arrays(
                [df.index.get_level_values(0), df.groupby(level=0).cumcount()],
                names=['ID', 'Date'])




                df
                col1 col2
                ID Date
                1 0 4 6
                1 8 5
                2 4 7
                3 3 3
                2 0 7 8
                1 5 9
                2 8 4


                This won't generalise to N levels, but there should be a df.index.set_levels equivalent I'm forgetting...






                share|improve this answer












                Using pd.MultiIndex.from_arrays and groupby + cumcount.



                df.index = pd.MultiIndex.from_arrays(
                [df.index.get_level_values(0), df.groupby(level=0).cumcount()],
                names=['ID', 'Date'])




                df
                col1 col2
                ID Date
                1 0 4 6
                1 8 5
                2 4 7
                3 3 3
                2 0 7 8
                1 5 9
                2 8 4


                This won't generalise to N levels, but there should be a df.index.set_levels equivalent I'm forgetting...







                share|improve this answer












                share|improve this answer



                share|improve this answer










                answered Aug 13 at 18:49









                coldspeed

                104k1789157




                104k1789157







                • 2




                  I find this solution easiest to follow.
                  – KOB
                  Aug 13 at 18:52












                • 2




                  I find this solution easiest to follow.
                  – KOB
                  Aug 13 at 18:52







                2




                2




                I find this solution easiest to follow.
                – KOB
                Aug 13 at 18:52




                I find this solution easiest to follow.
                – KOB
                Aug 13 at 18:52












                up vote
                4
                down vote













                You can groupby ID, then reset the index on each group using apply:



                new_df = (df.groupby(df.index.get_level_values('ID'))
                .apply(lambda x: x.reset_index()).drop(['ID','Date'],1))

                new_df.index = new_df.index.rename(['ID','Date'])

                >>> new_df
                col1 col2
                ID Date
                1 0 4 6
                1 8 5
                2 4 7
                3 3 3
                2 0 7 8
                1 5 9
                2 8 4





                share|improve this answer
























                  up vote
                  4
                  down vote













                  You can groupby ID, then reset the index on each group using apply:



                  new_df = (df.groupby(df.index.get_level_values('ID'))
                  .apply(lambda x: x.reset_index()).drop(['ID','Date'],1))

                  new_df.index = new_df.index.rename(['ID','Date'])

                  >>> new_df
                  col1 col2
                  ID Date
                  1 0 4 6
                  1 8 5
                  2 4 7
                  3 3 3
                  2 0 7 8
                  1 5 9
                  2 8 4





                  share|improve this answer






















                    up vote
                    4
                    down vote










                    up vote
                    4
                    down vote









                    You can groupby ID, then reset the index on each group using apply:



                    new_df = (df.groupby(df.index.get_level_values('ID'))
                    .apply(lambda x: x.reset_index()).drop(['ID','Date'],1))

                    new_df.index = new_df.index.rename(['ID','Date'])

                    >>> new_df
                    col1 col2
                    ID Date
                    1 0 4 6
                    1 8 5
                    2 4 7
                    3 3 3
                    2 0 7 8
                    1 5 9
                    2 8 4





                    share|improve this answer












                    You can groupby ID, then reset the index on each group using apply:



                    new_df = (df.groupby(df.index.get_level_values('ID'))
                    .apply(lambda x: x.reset_index()).drop(['ID','Date'],1))

                    new_df.index = new_df.index.rename(['ID','Date'])

                    >>> new_df
                    col1 col2
                    ID Date
                    1 0 4 6
                    1 8 5
                    2 4 7
                    3 3 3
                    2 0 7 8
                    1 5 9
                    2 8 4






                    share|improve this answer












                    share|improve this answer



                    share|improve this answer










                    answered Aug 13 at 18:37









                    sacul

                    18.4k31434




                    18.4k31434




















                        up vote
                        4
                        down vote













                        Using set_index and cumcount:



                        tmp = df.reset_index('Date', drop=True)
                        tmp.set_index(df.groupby(level=0).cumcount().rename('Date'), append=True)

                        col1 col2
                        ID Date
                        1 0 4 6
                        1 8 5
                        2 4 7
                        3 3 3
                        2 0 7 8
                        1 5 9
                        2 8 4





                        share|improve this answer


























                          up vote
                          4
                          down vote













                          Using set_index and cumcount:



                          tmp = df.reset_index('Date', drop=True)
                          tmp.set_index(df.groupby(level=0).cumcount().rename('Date'), append=True)

                          col1 col2
                          ID Date
                          1 0 4 6
                          1 8 5
                          2 4 7
                          3 3 3
                          2 0 7 8
                          1 5 9
                          2 8 4





                          share|improve this answer
























                            up vote
                            4
                            down vote










                            up vote
                            4
                            down vote









                            Using set_index and cumcount:



                            tmp = df.reset_index('Date', drop=True)
                            tmp.set_index(df.groupby(level=0).cumcount().rename('Date'), append=True)

                            col1 col2
                            ID Date
                            1 0 4 6
                            1 8 5
                            2 4 7
                            3 3 3
                            2 0 7 8
                            1 5 9
                            2 8 4





                            share|improve this answer














                            Using set_index and cumcount:



                            tmp = df.reset_index('Date', drop=True)
                            tmp.set_index(df.groupby(level=0).cumcount().rename('Date'), append=True)

                            col1 col2
                            ID Date
                            1 0 4 6
                            1 8 5
                            2 4 7
                            3 3 3
                            2 0 7 8
                            1 5 9
                            2 8 4






                            share|improve this answer














                            share|improve this answer



                            share|improve this answer








                            edited Aug 13 at 19:07

























                            answered Aug 13 at 18:48









                            user3483203

                            24.1k62147




                            24.1k62147




















                                up vote
                                3
                                down vote













                                New Answer



                                Not as cool as the old answer but I'd rather be accurate than cool.



                                from collections import defaultdict
                                from itertools import count
                                d = defaultdict(count)

                                lbl =
                                for a, *_ in df.index.values:
                                lbl.append(next(d[a]))

                                lvl = pd.RangeIndex(max(lbl) + 1)

                                df.set_index(df.index.set_labels(lbl, 1).set_levels(lvl, 1))

                                col1 col2
                                ID Date
                                1 0 4 6
                                1 8 5
                                2 4 7
                                3 3 3
                                2 0 7 8
                                1 5 9
                                2 8 4



                                OLD ANSWER



                                Do Not Use



                                I misread the question. I didn't see that the new index needed to reset for every group.



                                Hopefully useful to someone.



                                You can use pandas.MultiIndex.set_levels



                                n = 1
                                lvl = df.index.levels[n]
                                new_lvl = pd.RangeIndex(len(lvl))
                                new_idx = df.index.set_levels(new_lvl, n)
                                df.set_index(new_idx)

                                col1 col2
                                ID Date
                                1 0 4 6
                                1 8 5
                                2 4 7
                                3 3 3
                                2 4 7 8
                                5 5 9
                                6 8 4



                                One-line



                                Yay! o/



                                df.set_index(df.index.set_levels(pd.RangeIndex(len(df.index.levels[1])), 1))

                                col1 col2
                                ID Date
                                1 0 4 6
                                1 8 5
                                2 4 7
                                3 3 3
                                2 4 7 8
                                5 5 9
                                6 8 4



                                In place



                                df.index.set_levels(pd.RangeIndex(len(df.index.levels[1])), 1, inplace=True)
                                df

                                col1 col2
                                ID Date
                                1 0 4 6
                                1 8 5
                                2 4 7
                                3 3 3
                                2 4 7 8
                                5 5 9
                                6 8 4





                                share|improve this answer


























                                  up vote
                                  3
                                  down vote













                                  New Answer



                                  Not as cool as the old answer but I'd rather be accurate than cool.



                                  from collections import defaultdict
                                  from itertools import count
                                  d = defaultdict(count)

                                  lbl =
                                  for a, *_ in df.index.values:
                                  lbl.append(next(d[a]))

                                  lvl = pd.RangeIndex(max(lbl) + 1)

                                  df.set_index(df.index.set_labels(lbl, 1).set_levels(lvl, 1))

                                  col1 col2
                                  ID Date
                                  1 0 4 6
                                  1 8 5
                                  2 4 7
                                  3 3 3
                                  2 0 7 8
                                  1 5 9
                                  2 8 4



                                  OLD ANSWER



                                  Do Not Use



                                  I misread the question. I didn't see that the new index needed to reset for every group.



                                  Hopefully useful to someone.



                                  You can use pandas.MultiIndex.set_levels



                                  n = 1
                                  lvl = df.index.levels[n]
                                  new_lvl = pd.RangeIndex(len(lvl))
                                  new_idx = df.index.set_levels(new_lvl, n)
                                  df.set_index(new_idx)

                                  col1 col2
                                  ID Date
                                  1 0 4 6
                                  1 8 5
                                  2 4 7
                                  3 3 3
                                  2 4 7 8
                                  5 5 9
                                  6 8 4



                                  One-line



                                  Yay! o/



                                  df.set_index(df.index.set_levels(pd.RangeIndex(len(df.index.levels[1])), 1))

                                  col1 col2
                                  ID Date
                                  1 0 4 6
                                  1 8 5
                                  2 4 7
                                  3 3 3
                                  2 4 7 8
                                  5 5 9
                                  6 8 4



                                  In place



                                  df.index.set_levels(pd.RangeIndex(len(df.index.levels[1])), 1, inplace=True)
                                  df

                                  col1 col2
                                  ID Date
                                  1 0 4 6
                                  1 8 5
                                  2 4 7
                                  3 3 3
                                  2 4 7 8
                                  5 5 9
                                  6 8 4





                                  share|improve this answer
























                                    up vote
                                    3
                                    down vote










                                    up vote
                                    3
                                    down vote









                                    New Answer



                                    Not as cool as the old answer but I'd rather be accurate than cool.



                                    from collections import defaultdict
                                    from itertools import count
                                    d = defaultdict(count)

                                    lbl =
                                    for a, *_ in df.index.values:
                                    lbl.append(next(d[a]))

                                    lvl = pd.RangeIndex(max(lbl) + 1)

                                    df.set_index(df.index.set_labels(lbl, 1).set_levels(lvl, 1))

                                    col1 col2
                                    ID Date
                                    1 0 4 6
                                    1 8 5
                                    2 4 7
                                    3 3 3
                                    2 0 7 8
                                    1 5 9
                                    2 8 4



                                    OLD ANSWER



                                    Do Not Use



                                    I misread the question. I didn't see that the new index needed to reset for every group.



                                    Hopefully useful to someone.



                                    You can use pandas.MultiIndex.set_levels



                                    n = 1
                                    lvl = df.index.levels[n]
                                    new_lvl = pd.RangeIndex(len(lvl))
                                    new_idx = df.index.set_levels(new_lvl, n)
                                    df.set_index(new_idx)

                                    col1 col2
                                    ID Date
                                    1 0 4 6
                                    1 8 5
                                    2 4 7
                                    3 3 3
                                    2 4 7 8
                                    5 5 9
                                    6 8 4



                                    One-line



                                    Yay! o/



                                    df.set_index(df.index.set_levels(pd.RangeIndex(len(df.index.levels[1])), 1))

                                    col1 col2
                                    ID Date
                                    1 0 4 6
                                    1 8 5
                                    2 4 7
                                    3 3 3
                                    2 4 7 8
                                    5 5 9
                                    6 8 4



                                    In place



                                    df.index.set_levels(pd.RangeIndex(len(df.index.levels[1])), 1, inplace=True)
                                    df

                                    col1 col2
                                    ID Date
                                    1 0 4 6
                                    1 8 5
                                    2 4 7
                                    3 3 3
                                    2 4 7 8
                                    5 5 9
                                    6 8 4





                                    share|improve this answer














                                    New Answer



                                    Not as cool as the old answer but I'd rather be accurate than cool.



                                    from collections import defaultdict
                                    from itertools import count
                                    d = defaultdict(count)

                                    lbl =
                                    for a, *_ in df.index.values:
                                    lbl.append(next(d[a]))

                                    lvl = pd.RangeIndex(max(lbl) + 1)

                                    df.set_index(df.index.set_labels(lbl, 1).set_levels(lvl, 1))

                                    col1 col2
                                    ID Date
                                    1 0 4 6
                                    1 8 5
                                    2 4 7
                                    3 3 3
                                    2 0 7 8
                                    1 5 9
                                    2 8 4



                                    OLD ANSWER



                                    Do Not Use



                                    I misread the question. I didn't see that the new index needed to reset for every group.



                                    Hopefully useful to someone.



                                    You can use pandas.MultiIndex.set_levels



                                    n = 1
                                    lvl = df.index.levels[n]
                                    new_lvl = pd.RangeIndex(len(lvl))
                                    new_idx = df.index.set_levels(new_lvl, n)
                                    df.set_index(new_idx)

                                    col1 col2
                                    ID Date
                                    1 0 4 6
                                    1 8 5
                                    2 4 7
                                    3 3 3
                                    2 4 7 8
                                    5 5 9
                                    6 8 4



                                    One-line



                                    Yay! o/



                                    df.set_index(df.index.set_levels(pd.RangeIndex(len(df.index.levels[1])), 1))

                                    col1 col2
                                    ID Date
                                    1 0 4 6
                                    1 8 5
                                    2 4 7
                                    3 3 3
                                    2 4 7 8
                                    5 5 9
                                    6 8 4



                                    In place



                                    df.index.set_levels(pd.RangeIndex(len(df.index.levels[1])), 1, inplace=True)
                                    df

                                    col1 col2
                                    ID Date
                                    1 0 4 6
                                    1 8 5
                                    2 4 7
                                    3 3 3
                                    2 4 7 8
                                    5 5 9
                                    6 8 4






                                    share|improve this answer














                                    share|improve this answer



                                    share|improve this answer








                                    edited Aug 13 at 19:36

























                                    answered Aug 13 at 19:05









                                    piRSquared

                                    140k19118248




                                    140k19118248



























                                         

                                        draft saved


                                        draft discarded















































                                         


                                        draft saved


                                        draft discarded














                                        StackExchange.ready(
                                        function ()
                                        StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f51828518%2fpandas-reset-inner-level-of-multiindex%23new-answer', 'question_page');

                                        );

                                        Post as a guest













































































                                        Popular posts from this blog

                                        How to check contact read email or not when send email to Individual?

                                        Bahrain

                                        Postfix configuration issue with fips on centos 7; mailgun relay